TL;DR
Thorsten Meyer AI published a field note arguing that open model weights are free to download, but not free to operate. The piece says owned inference can beat paid APIs for steady, high-volume workloads, while APIs still make more sense for low or uneven usage.
Thorsten Meyer AI has published an analysis arguing that companies should compare open-weight AI models against paid APIs by total operating cost, not download price, because self-hosted inference can become cheaper for steady, high-volume workloads while paid APIs remain the better choice for lower or uneven demand.
The piece responds to a direct challenge raised after an earlier article on Mistral and European AI sovereignty: why pay a vendor to run models on-premises if models such as Qwen can be downloaded at no cost? Thorsten Meyer AI says the answer starts with a narrower definition of “free”: model weights may cost nothing to download, but production use still requires hardware, power, operational labor, updates, queue management, tuning, context handling, retries, tool routing and depreciation.
The analysis frames the decision as a comparison between total cost of ownership and per-token API pricing. It says APIs win for low-volume, bursty or hard-to-predict workloads because buyers avoid upfront hardware spending and operational overhead. It says owned hardware can win once usage is steady enough that a fixed fleet is kept busy and the marginal cost of additional inference falls.
According to the source, an illustrative cost model put the break-even point near 80 million tokens per month under one set of assumptions. The article describes that figure as an example rather than a quote or universal threshold, saying the crossover moves with task difficulty, sovereignty requirements, model quality, hardware choices and the operator’s ability to run the stack well.
Why It Matters
The analysis matters because more companies are weighing whether to rely on closed commercial APIs, rent hosted open models, or run open-weight models themselves. The decision affects AI budgets, data control, vendor dependence and engineering workload.
The article argues that the price gap between closed frontier APIs and some open or lower-cost models has made the self-hosting case stronger than it was a year earlier. It says open models may still trail the frontier on the hardest tasks, but when they are close enough for a given workload, the lower operating cost can change the business case.
Local LLM Inference Optimization: A Comprehensive Guide to Quantization, Hardware Acceleration, and Efficient Private AI Deployment
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Background
The source places the debate in the wider sovereignty discussion around European AI vendors such as Mistral. In that debate, on-premises or self-hosted AI is often presented as a way to keep sensitive data under local control. Thorsten Meyer AI says that claim has to be tested against the real economics of running inference, not only against the availability of free model files.
The piece also points to hardware changes that have made local inference more practical for some teams, including large unified-memory Apple Silicon systems and mixture-of-experts models that activate only part of their total parameters for a given token. The source says those changes can make capable models runnable on smaller fleets, but they do not remove operational responsibility.
“The weights are free to download. Running them well is not.”
— Thorsten Meyer AI
“The honest comparison is total cost of ownership vs. per-token API.”
— Thorsten Meyer AI
“The crossover zone is real — and growing.”
— Thorsten Meyer AI
self-hosted AI model servers
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What Remains Unclear
Several details remain workload-specific. The exact break-even point depends on token volume, model size, latency needs, hardware prices, power costs, utilization, staff time, failure rates and quality requirements. The source also says open models can lag closed frontier systems on the hardest tasks, so cheaper inference may not be a substitute when top model quality is required.
The NVIDIA Rubin CPX GPU Architecture: Transforming AI Inference Infrastructure for High-Performance Computing and Generative Applications
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What’s Next
The next step for buyers is to measure their own token volume, data sensitivity, latency needs and engineering capacity before choosing an API, hosted open model or owned hardware. The economics will continue to shift as open models improve, frontier API prices change and inference hardware gets cheaper or more capable.
AI model deployment hardware
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Key Questions
Does a free open model mean free AI operations?
No. According to Thorsten Meyer AI, the model weights may be free to download, but production use still brings hardware, electricity, maintenance, tuning, reliability and depreciation costs.
When can running your own model beat paying for an API?
The source says owned hardware can win when usage is steady, high-volume and predictable enough to keep the machines busy. In its illustrative model, break-even was near 80 million tokens per month under one set of inputs.
When is an API still the better option?
APIs remain a stronger fit for low-volume, uneven or experimental workloads, and for tasks where the best closed frontier model quality is needed.
How does data control affect the decision?
Self-hosting can keep data inside the operator’s own environment, which may matter for privacy, compliance or sovereignty goals. The article treats that as a structural benefit, but not a reason to ignore cost and operational burden.
Source: Thorsten Meyer AI
