LLaMA, Mistral, Qwen: why open source LLMs are changing the game

18 months ago, recommending an open source LLM for a production project was risky. Today, that’s no longer the case at all. Here’s why this changes your options as an AI PO.

The open source market landscape

Three families dominate:

LLaMA 3 (Meta) — The most widely used open source model. LLaMA 3 70B is competitive with GPT-3.5 Turbo on many benchmarks, at 10 to 20 times lower inference cost if you host it yourself.

Mistral — The French startup that surprised everyone. Mistral Large is now in the same league as GPT-4 on reasoning tasks. Mistral 7B remains the reference for efficient small models.

Qwen 2.5 (Alibaba) — Less known in Europe, but impressive. Particularly strong on multilingual tasks and code.

Real advantages for enterprise projects

Data privacy. This is often the decisive argument. With an LLM hosted on-premise or with a European provider, your data doesn’t go to OpenAI or Anthropic. For sectors like healthcare, finance, or legal, this is sometimes non-negotiable.

Cost at scale. At low volume, proprietary APIs are cheaper (no infrastructure to manage). At high volume, the equation changes. At 10 million requests per month, hosting a Mistral or LLaMA can divide the bill by 5 to 10.

Customization. You can fine-tune an open source model on your business data. It’s serious work, but it can achieve performance well above generic models on your specific domain.

Disadvantages not to underestimate

Infrastructure. Hosting a 70 billion parameter LLM requires serious GPUs. Infrastructure costs can quickly exceed API savings if volume isn’t there.

Maintenance. Who handles updates? Version upgrades? Incidents? These are specialized technical skills, not within reach of every team.

Quality on complex tasks. On multi-step reasoning, sophisticated code generation, or following very precise instructions, the best proprietary models often maintain an advantage.

My pragmatic recommendation

For a first project: start with a proprietary API. Less friction, less risk. Once you’ve validated the use case and volume increases, reassess open source.

For projects with privacy constraints from the start: look at Mistral via their European API, or Ollama for local. These are the least risky options to start with open source.

And for fine-tuning: only do it if you have 1000+ quality examples and a team capable of maintaining that over time.