Moving Up a Model Family Is Not a Reliable Upgrade

When a small model’s output falls short, the usual fix is to switch to its larger sibling in the same family. On the infrastructure and utility jobs in this benchmark, that step sometimes buys a clear gain, often buys nothing measurable, and in one case points the wrong way. The larger sibling is not necessarily slower, either.

So a model family should be treated as a set of separate candidates rather than a ladder you climb until the output looks right. Every step has to be checked on the job it will actually run.

The Claude ladder runs both ways

On extracting document structure, the scores run against the tiers. Claude Haiku 5.5 scores 8.35, Claude Opus 5.5 7.85 and Claude Sonnet 5.5 7.6 [^1]. The middle step comes last, so the order is not a simple reversal either.

Writing publication title packages goes the expected way. Claude Opus 5.5 scores 8.75 (±0.25) against 7.63 (±0.39) for Claude Haiku 5.5, and the intervals separate [^2]. On this job the larger model buys measurably better output. The evidence gives no Claude Sonnet 5.5 figure here, so it cannot say where the middle step sits.

That gives opposite results on two jobs from the same family. A pipeline that standardised on Opus for both would run the largest model on a job where the smallest one scored highest.

Nemotron: one real step, two flat ones

The NVIDIA Nemotron-3 family appears in three sizes in this evidence, and the steps between them behave differently.

Nano to Super pays off on reading a document against a profile. NVIDIA Nemotron-3 Super 120B scores 6.26 (±0.35) against 4.66 (±0.37) for NVIDIA Nemotron-3 Nano 30B-A3B, a separated gap [^3].
Super to Ultra is flat on engagement triage. NVIDIA Nemotron-3 Ultra 550B scores 8.73 (±0.19) and Super 120B 8.51 (±0.2), a tie [^4].
Nano to Ultra is flat on language identification. Ultra 550B scores 9.85 (±0.11) and Nano 30B-A3B 9.82 (±0.08), also a tie [^5].

Size does not predict speed either. On a separate engagement triage measurement, Ultra 550B answers in 4.1 seconds at the median against 21 seconds for Super 120B [^6]. That measurement carries no quality score, so it shows the larger model is faster there, not that it is better.

MiMo: Flash ties Pro

On structured extraction, Xiaomi MiMo V2.6 Flash scores 9.76 (±0.22) and Xiaomi MiMo V2.6 Pro 9.64 (±0.26) [^7]. The intervals overlap, so the two are tied, and the higher point score belongs to the smaller Flash. Moving up to Pro buys no measurable gain on this job.

Testing a family per job

Score each sibling on each job you run, starting from the smallest, and treat each step up as its own test.

Keep a larger sibling only where it separates on that job, as Super 120B does over Nano 30B-A3B on profile reading [^3] and Opus does over Haiku 5.5 on title packages [^2].
Do not carry a step’s result to another job. A gain from Nano to Super says nothing about Super to Ultra [^4].
When output is weak, look down and sideways as well as up. The smallest Claude model posted the top document-structure score [^1].

One caveat matters most. The evidence publishes no intervals for the three document-structure scores [^1], so it cannot show whether Haiku 5.5’s lead is larger than measurement noise. The result is strong enough to show Opus is no safe default on that job, but not strong enough to call Haiku 5.5 better.

The live benchmark at https://fronset.ai/benchmark/ lets you put a family’s siblings side by side, with their intervals, on the specific job you run before you choose which step of the ladder to deploy.