71 AI Configurations Later, My News Pipeline Still Runs A Different Model

🔍 Read the full analysis: 71 AI Configurations Later, My News Pipeline Still Runs A Different Model on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on tech for your team

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Publisher Thorsten Meyer says a test of 71 AI configurations for his news-writing pipeline found GPT-6 Astra scored highest, while the far less expensive GPT-6 Luna remained his production model. The results, published October 13, also prompted him to remove two fallback models after they produced invented quotes or invalid output.

Meyer tested the configurations from September 8 through October 13, 2026, using his production prompt and real news sources. Each model generated English and German versions. Two AI judges evaluated the articles for source fidelity, fabrication, clarity about uncertainty, news value, structure and polish, with language quality also scored for German. The test included a longer source and a short, 619-character source that Meyer said was more representative of much of his production work.

In the initial test, GPT-6 Astra scored 4.00 out of 5 in both languages and received a 5.00 for source fidelity. In a further run across 12 articles, it averaged 3.93, with a reported 95% bootstrap interval of 3.83 to 4.02. Meyer reported GPT-6 Luna at 3.67 in English and 3.68 in German in the initial test; across the robustness run it scored 3.67. He said every Luna article in that run produced valid output.

The price and speed figures favored Luna: Meyer estimated Astra at about $0.12 per article and roughly 50 seconds, compared with Luna at around $0.0012 and 17 seconds. He said Luna’s recent production cost was about $0.0013 per article. Those figures are the publisher’s reported costs, not an independently audited comparison.

Cost and Reliability Shape Model Choice

The results highlight a practical issue for publishers using AI at scale: the highest-scoring model may not be the one that makes sense to run on every article. Meyer’s reported Astra price was about 100 times Luna’s, while its average score in the robustness run was higher. For a high-volume pipeline, that creates a trade-off between quality, cost and processing time rather than a simple choice based on a leaderboard rank.

The fallback findings carry a separate editorial risk. Meyer said GPT-4.1 nano scored 1.62 in English and 1.68 in German, with source fidelity of 1.00, and judges flagged quotations that were not in the supplied source. He also reported that locally run gpt-oss-120b produced no valid articles in four attempts. A system’s backup route can therefore affect whether the pipeline preserves attribution and source accuracy when its main model is unavailable.

These are results from one publisher’s prompts, sources, language editions and evaluation method. They do not establish that the same model ranking applies to other newsrooms or writing tasks. The useful takeaway is narrower: test the actual workflow, including fallback behavior, rather than assuming general benchmark performance predicts editorial quality.

How the News Pipeline Was Tested

Meyer’s test was designed around his own publication process, rather than a general-purpose writing benchmark. The models received the same production prompt and source material, then produced English and German articles. One source was a 6,340-character product filing; the other was a short political news item of 619 characters. The article counted only if it yielded valid output after the repair step Meyer’s pipeline already uses.

Two AI judges, GPT-6.1 Sol and Claude Opus 5.5, rated the work on a five-point scale. Meyer also ran a robustness test in which stronger configurations wrote 10 additional articles, mostly from short sources. He cautioned that small gaps in the two-article initial test could be noise, and described the larger run as a more reliable basis for ranking. The results remain the publisher’s own evaluation; the source does not describe independent human grading or an external replication.

Meyer also tested GPT-6 Luna at six reasoning-effort settings. He reported scores that varied across settings without a clear improvement over the production setting, which used no extra reasoning effort. The maximum setting cost about 1.43 cents per article and took 211 seconds, compared with 0.12 cents and 17 seconds at the no-effort setting.

“For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.”

— Thorsten Meyer

Limits of the Published Results

The results have not been independently verified in the supplied material, and it is not clear whether the two AI judges’ ratings were supplemented by human review. The test’s reported scope is limited to one publisher’s prompt and source set, two language editions and a relatively small robustness sample. The findings should not be treated as a general ranking of models for all newsroom work.

The source also does not provide detailed results for every one of the 71 configurations, or full per-article scoring data. It reports that two fallback models were removed on October 7, but does not describe a longer post-change audit of the live pipeline. The account says the route now uses GPT-6 Luna, but its final description of the backup configuration is incomplete, so the full current failover setup is unclear.

Monitoring the Revised Model Route

Meyer says GPT-6 Luna is now the news route’s production model and that the two weaker fallbacks were removed on October 7. The next relevant evidence would be performance from the revised route over a larger set of live articles, including whether source fidelity and valid output remain consistent when the pipeline encounters short or difficult sources.

No further test date or independent evaluation is specified in the published account. Until more results are available, the reported rankings are best read as a snapshot of Meyer’s own workflow through October 13, 2026, rather than a lasting verdict on the models.

Key Questions

Which model scored highest in Meyer’s test?

GPT-6 Astra scored 4.00 out of 5 in both English and German in the initial test. It averaged 3.93 across 12 articles in the robustness run, according to Meyer.

Why did Meyer keep GPT-6 Luna in production?

Meyer said Luna scored 3.67 in the robustness run and produced valid articles in that test, while costing about $0.0012 per article compared with roughly $0.12 for Astra. He judged Luna’s quality sufficient for the cost and speed of his pipeline.

Did higher reasoning effort improve the articles?

Meyer reported no reliable overall gain across six GPT-6 Luna reasoning settings. The maximum setting took 211 seconds and cost 1.43 cents per article, compared with 17 seconds and 0.12 cents at the no-effort production setting.

Why were the fallback models removed?

Meyer said GPT-4.1 nano’s articles received low source-fidelity scores and judges flagged invented quotations. He reported that gpt-oss-120b produced zero valid articles in four attempts and also generated named quotations that were not in the source.

Do these results show which model is best for every newsroom?

No. The test used Meyer’s own production prompt, source material, languages and scoring process. It offers evidence about that workflow, but the published results do not establish a universal model ranking.

Source: ThorstenMeyerAI.com