When Paying More for a Model Stops Buying a Better Score
When Paying More for a Model Stops Buying a Better Score
On several kinds of work in the Fronset benchmark, a far pricier model scores no better than a cheap one. Where the gaps are real, the gain can arrive one price tier up, with nothing measurable added beyond it.
The clearest case is judging whether a claim stays consistent. Claude Sonnet 5.5 scores 8.69 (±0.26) at $0.03 per run, and Claude Opus 5.5 scores 8.6 (±0.24) at $0.36 [^1]. The model that costs twelve times as much buys no measurable score.
Why a tie means price decides
Each score is reported out of 10 with an interval. When two models’ intervals overlap, the evidence cannot rank them. On work where the leading models tie, raising the bar changes nothing, and the only variable left to choose on is price.
These ties are not confined to one vendor or one kind of work. Each row below is a tie, not a ranking.
| Work | Pricier model (score, price per run) | Cheaper tied model |
|---|---|---|
| Visual theme configurations | Claude Opus 5.5, 8.7 (±0.24), $0.09 [^2] | Claude Haiku 5.5, 8.43 (±0.2), $0.0023 [^2] |
| Translation | Meta Muse Spark 1.3, 8.92 (±0.19), $0.0066 [^3] | DeepSeek V4.1 Flash, 8.83 (±0.28), $0.0022 [^3] |
| Tightening factual claims | Inkling, 8.05 (±0.44), $0.02 [^4] | Inkling Small, 8.31 (±0.28), $0.0062 [^4] |
A larger model is not reliably the better one, either. On pulling structure out of a profiled document, Claude Haiku 5.5 scores 8.35 (±0.32), against 7.85 (±0.35) for Claude Opus 5.5 and 7.6 (±0.39) for Claude Sonnet 5.5 [^5].
Where the gaps are real, move up one tier
Some work does separate the models, and there the cheap tier is the one that drops out. The question is where the cost curve has its knee. In these results it sits at the first tier that clears the cheap model’s interval.
Labeling groups of documents by topic shows the shape. Claude Haiku 5.5 scores 8.01 (±0.36), a real gap below the leaders [^6]. Claude Sonnet 5.5 reaches the top tier at 8.55 (±0.25) for $0.03 per sync run [^6]. Claude Opus 5.5 and GLM-5.3 both score 8.69, at $0.08 and $0.09, and add no measurable score [^6].
Publication title packages are a case where the step up is worth paying for. Claude Opus 5.5 scores 8.75 (±0.25) at $0.03 per sync run, and Claude Haiku 5.5 scores 7.63 (±0.39) at $0.0009 [^7]. Here the move off the cheapest model is what buys the gap.
Where no model clears the bar
On a third group of work, paying more does not help because nothing measured gets close to a high bar. Judging the relevance of a whole set of content is one example. Claude Opus 5.5 scores 6.18 (±0.4) at $0.15 per batch run, and Claude Haiku 5.5 scores 5.95 (±0.39) at $0.0023 [^8]. The two are tied, so the higher price buys nothing.
Suggesting content domains is weak for nearly every model measured [^9]. For work like this, the evidence does not support running any measured model unattended.
What to do, and the catch
Set the bar before you pick the model, then answer three questions for each task:
- Do the leaders’ intervals overlap? Take the cheapest of them.
- Is the cheap model separated from the leaders? Take the cheapest model in the top tier, not the top-priced one.
- Does nothing reach the bar? Budget for human review rather than a bigger model.
The catch is delivery. Many of the strongest scores are measured only on calls that returned an answer. GPT-6 Sol scores 9.15 on title packages at a 10% answer rate [^7], which describes a different product from a model that answers almost every time.
Check the answer rate for your specific task, not a model’s aggregate. In aggregate, Claude Sonnet 5.5 returns a usable first-time answer on 98.1% of its attempts [^10], but it answers on 87% of attempts at short public-facing responses [^11].
To find the knee for your own work, go to https://fronset.ai/benchmark/, set the quality bar your task needs, see which models clear it, and start from the cheapest of them.