Quality and Cost Benchmarking
Topic organizing is a quality tie, so buy it by the fraction of a cent
If you use an LLM to discover themes, assign sections to topics or label clusters, the models on the llm-bench leaderboard land in the same mid-8s band and their confidence intervals overlap. That makes the score column mostly a tie and the cost column the real decision. The cost spread runs from 51 cents a run down to a fraction of a cent, for work the benchmark cannot tell apart on quality.
The quality band is narrow
Theme discovery is where the most expensive result in the group sits. GPT-5.6 Sol scores 8.52 out of 10 (±0.32) and costs $0.51 per task run in sync mode, measured on this model’s own usage [^1][^2]. Meta Muse Spark 1.1 scores 8.36 out of 10 (±0.31) at $0.13 per task run in sync mode, measured on its own usage and repriced at current rates [^1][^2].
The two intervals overlap, so the benchmark calls that a tie. The price is not a tie, and Sol’s theme discovery is the one figure in this group that deserves its own line in the budget rather than being lumped in with the penny-level tasks [^1][^2].
Section assignment: same score, penny prices
Assigning sections to topics is where the cost gap is widest and the quality gap is absent. Four models sit at the same score within their intervals, and the cheapest of them costs a fraction of a cent per run.
| Model | Score (out of 10) | Cost per task run | Basis |
|---|---|---|---|
| GPT-5.6 Sol | 8.39 (±0.41) | $0.07, sync | estimated [^3][^4] |
| Meta Muse Spark 1.1 | 8.11 (±0.41) | $0.01, sync | measured on its own usage [^3][^4] |
| GPT-5.6 Terra | 8.06 (±0.47) | $0.0038, batch | repriced at current rates [^3][^4] |
| Thinking Machines Inkling Small | 8.06 (±0.5) | $0.0022, sync | measured on its own usage [^3][^4] |
If your pipeline can wait, Terra’s batch price applies. If it needs a synchronous call, Inkling Small is priced lower still, and its figure is backed by the model’s own usage. Sol covers this work without a quality penalty relative to its own baseline, but there is no quality reason to pay its price here.
Check whether a cell is measured before you trust it
The benchmark flags each model-task cell as either measured on that model’s own usage history or estimated because the cell has no qualifying usage of its own. When the scores are tied, that flag is the thing to read next, because it tells you how firmly the cost figure is anchored.
GPT-5.6 Sol is the clearest example. Only its theme-discovery result is measured on the model’s own usage [^1][^2], while its section-assignment and cluster-labeling results are estimated with no qualifying usage history of their own [^3][^4][^5][^6]. The theme-discovery figure is therefore the firmer guide to what Sol will actually cost you.
Matching content to an existing taxonomy is still estimated-only for every model compared. Treat that task as undecided until the benchmark has measured it, whichever model you lean toward.
Cheap is not automatically fine
The tie holds across the organizing tasks, but not everywhere, and the exceptions cut both ways. On matching topics to a fixed taxonomy, Claude Haiku 4.5 scores 5.4 out of 10 (±0.47) at $0.02 per task run in batch mode, measured on its own usage and repriced at current rates [^7][^8]. MiniMax M3 has the same weak spot at 6.56 out of 10 (±0.2) and $0.0064 per task run in sync mode, also measured and repriced [^8], so strict taxonomy matching should stay off it on this evidence.
The reverse also happens: the low-priced model can post the strongest result. On generating a taxonomy for publication sections, Moonshot Kimi K3 scores 9.33 out of 10 (±0.06) at $0.03 per task run in sync mode, measured on its own usage [^9][^10], and Thinking Machines Inkling scores 8.97 out of 10 (±0.12) at $0.01 per task run in sync mode, its strongest measured result [^9][^10]. Taxonomy generation and topic sequencing are the tasks where sending work to Inkling is backed by measured usage rather than an estimate.
Where to look next
The figures here are one reading of a bar you may want to move. Check the live benchmark at https://llm-bench.kapualabs.com/ to set the quality bar you actually need and to compare batch against sync pricing for your own workload; each task has its own page, such as https://fronset.ai/benchmark/task/topic-section-assignment/, showing every model’s score, cost and measured-or-estimated flag. Method detail is documented at https://llm-bench.kapualabs.com/methodology/ for what each score and cost covers.