StarCoder or Code Llama: which base to fine-tune on
Neither line has been updated in a while, with last releases in April 2024 and January 2024, so today both are taken as a base for fine-tuning more often than as a working model. StarCoder starts at 1B, runs on a CPU and was trained on an open source-code set with an option to exclude your own repository; the BigCode OpenRAIL-M license permits commercial use but comes with a list of prohibited applications. Code Llama is Llama 2 further trained on code: sizes from 7B to 70B, a GPU required, the Llama 2 Community License with restrictions, and plenty of existing fine-tunes and tooling around it. The catalog claims Russian support for neither.
Comparison based on catalog data
| Parameter | StarCoder | Code Llama |
|---|---|---|
| Category | Code | Code |
| Developer | BigCode (Hugging Face and ServiceNow), USA / France | Meta, USA |
| Releases | May 2023 – Apr 2024 | Aug 2023 – Jan 2024 |
| Sizes | 1B – 15B | 7B – 70B |
| Hardware | Laptop, 1 GPU | Laptop, 1 GPU |
| Commercial use | Commercial use with conditions | Commercial use with conditions |
| License | BigCode OpenRAIL-M (commercial use allowed, but there is a list of prohibited uses) | Llama 2 Community License (commercial use allowed with restrictions) |
| Russian | Not supported | Not supported |
| Ollama | Yes | Yes |
| Without GPU | Yes | No |
| Tasks |
|
|
Choose StarCoder if
- You are fine-tuning on internal code and want a transparent training set
- Hardware is modest: StarCoder starts at 1B and runs without a GPU
- You need boilerplate and test generation inside the editor
Choose Code Llama if
- You need up to 70B plus dedicated Python and chat variants
- You rely on existing fine-tunes and tooling around Llama
- The Llama 2 Community License terms work for you
Other comparisons
- Gemma or Llama: which to choose for business
- Mistral or Llama: which to choose for business
- Mistral or Qwen: which to choose for business
- Gemma or Phi: small models for modest hardware
- Kimi or DeepSeek: large open models for agentic work
- Cohere Command or Llama: which to choose for a knowledge base
- Qwen Coder or DeepSeek-Coder: which coding model to pick
- Codestral or Qwen Coder: which coding model you may ship
- Devstral or Codestral: which Mistral coding model to take
- MiniCPM or Gemma: which model to put on the device
- Qwen or GigaChat: which to choose for business
- Qwen or Llama: which to choose for business
Need a model for your task?
An open model can run on your own server: data stays in-house, there is no per-request fee, and the model can be fine-tuned on your documents.
- SelectThe model and size for your task and hardware budget
- DeployOn your server or in a closed network, with an API
- Fine-tuneOn your data, or connect a knowledge base
- IntegrateInto your CRM, ERP, bot, website or team chat