When LLM Score Bands Overlap, Buy the Cheapest Model in the Group

Models whose scores tie within their uncertainty bands can differ in price by more than a hundredfold [^1]. Inside such a tie the extra spend buys nothing the benchmark can measure, so the cheapest tied model is the pick, and paying more is justified only where the bands separate.

How much money sits inside a tie

Every score here carries a ± band, and when two bands overlap the scores do not rank the models. On writing hypothesis-style posts, five models land between 8.7 and 8.77 out of 10 [^1]. Tied groups like that one span wide price ranges.

Price does not even track the point figures inside a tie. On short-post relevance scoring, Claude Haiku 5.5 posts the highest score of the three, 6.61 (±0.33) [^4]. That ordering carries no weight, but it leaves no quality case for the pricier runs.

Version upgrades follow the same pattern. On the three tasks where both were measured, Tencent Hy4 Preview ties Tencent Hy3 on quality yet costs $0.02 to $0.04 per task against $0.0002 to $0.0021 [^5][^6][^7]. A newer model earns its price only once its band clears the older one’s.

A tie is set by the bands, not the gap

The test is mechanical: add and subtract each model’s band, and if the two ranges share any ground, treat the models as tied. The size of the point gap is a poor guide, because band width varies with the task and the model.

On short promotional posts, Claude Opus 5.5 leads Claude Sonnet 5.5 by more than three-quarters of a point, 8.75 (±0.35) against 7.98 (±0.47) [^2]. Yet Opus’s range reaches down to 8.40 and Sonnet’s reaches up to 8.45, so the two are tied [^2]. The cheapest of the four tied models, Sonnet, is the pick.

On image prompts the reverse holds. Claude Haiku 5.5 trails Claude Opus 5.5 by less than half a point, 8.55 (±0.19) against 8.93 (±0.14), but the narrow bands never meet: Haiku tops out at 8.74 and Opus bottoms out at 8.79 [^3]. That gap is real, so Haiku’s $0.0007 per task run buys a measurably weaker prompt [^3]. If you need top-group quality, the saving comes from inside the group instead: its four tied members range from $0.0092 to $0.04 per task run [^3].

The same test catches cases where the cheaper model is clearly better. On claim-consistency judging, Qwen 3.7 Plus scores 7.68 (±0.17) at $0.0069 and Qwen 3.8 Max scores 6.88 (±0.28) at $0.13, and the bands do not overlap [^8].

Where paying more is earned

Real separations exist, and they are where the extra spend belongs. In each pair below the bands do not overlap (prices per task run):

• Section-level prompts: Claude Opus 5.5 at 9.11 (±0.22) and $0.10 in sync mode, twice the price of Claude Sonnet 5.5 at 8.17 (±0.46) and $0.05 [^9].
• Publication title packages: Claude Opus 5.5 at 8.75 (±0.25) and $0.03 in sync mode; Claude Haiku 5.5 at 7.63 (±0.39) and $0.0009 [^10].
• Selecting authoritative sources: Nemotron-3 Ultra 550B at 7.58 (±0.45) and $0.02; Nemotron-3 Super 120B at 5.6 (±0.44) and $0.0026 [^11].

The title-package pair is the useful warning. The Claude Haiku 5.5 that ties Claude Opus 5.5 on short-post relevance scoring trails it clearly here, so a tie on one task says nothing about the next.

The check therefore runs per task: find the leader, collect every model whose band overlaps it, and run the cheapest of that group. Pay more only when the cheaper candidate’s whole band sits below the leader’s. One caveat: a score can be conditional on the call coming back, so confirm that a cheap tied model answers reliably before switching to it.

To set the quality bar you need and see which models’ bands overlap on your own tasks, use the live benchmark at https://fronset.ai/benchmark/. Method is documented at https://fronset.ai/benchmark/methodology/.