A Model's Best Task Says Little About Yours
A Model’s Best Task Says Little About Yours
Claude Opus 5.5 scores 3.75 points higher, out of 10, on turning source material into structured fields than on scoring relevance across a whole content set [^1][^2]. Tencent Hy3 goes from 9.85 on identifying a document’s language to 6.92 on rewriting metadata paragraphs [^3][^4]. Same model, same benchmark, different job.
So a model’s reputation, its headline result or any ranking that blends tasks is a weak guide to how it will handle your work. The figure that bears on the choice is the model’s score on the task you are routing, set against the other models measured on that task. This matters most when one model is chosen for a whole pipeline because it did well on the first step someone checked.
How far one model moves
Opus’s results on four kinds of work show that the low end is not one odd task.
| Kind of work | Claude Opus 5.5 |
|---|---|
| Turning source material into structured fields | 9.93 (±0.16) [^1] |
| Writing section prompts | 9.11 (±0.22) [^5] |
| Choosing research communities | 7.49 (±0.29) [^6] |
| Scoring relevance across a whole content set | 6.18 (±0.4) [^2] |
The intervals on the top and bottom rows sit far apart: even at their nearest edges, the structured-field score stays more than three points above the relevance score. The gap is not measurement noise. The Hy3 scores come without stated intervals, so treat that gap as indicative rather than confirmed.
The pattern is not limited to Opus. Claude Haiku 5.5 scores 9.84 on language identification [^3] and 5.95 (±0.39) on whole-set relevance scoring [^2].
A low score is not a low rank
Raw scores mislead in the other direction too. Opus’s research-community result looks weak next to its structured-field result, yet it is the highest research-community score in these results [^6]. A score ranks a model only against other models on the same task. Read across tasks without that context, a task leader can look like a weak spot.
A strong result on one task does not carry a rank to the next, either. GPT-6 Sol scores 8.62 (±0.19) on drafting short hypothesis posts [^7]. On picking and ordering topics it scores 7.17, against 8.86 for GLM-5.3 [^8]. Both of those scores cover only the calls that came back: 69% for Sol and 85% for GLM-5.3 [^8].
Route by task, not by model
In practice the unit of evaluation is the task, and each step of a pipeline counts as its own task.
- Shortlist from the benchmark page for the work you are routing, not from a model you already trust.
- Do not carry a score from one task to another, even within one model.
- Treat a strong result elsewhere as a reason to test a model, not to route to it.
- Where your task is not measured, test candidates on that exact task or add a review step.
Some tasks have only one model measured, so the benchmark cannot always name a pick. There, your own test on the task is the evidence.
What the figures do not explain
The benchmark records these swings but does not state why any task scores as it does. That closes off a tempting shortcut: reading a model’s results, deciding what kind of work it is good at, and assuming a new task of that kind will go the same way. Without a stated cause, only a measurement on the task itself settles it.
Every figure here is on the live benchmark at https://fronset.ai/benchmark/. Start from the task closest to your job, compare the models measured on it, and treat a model’s results on other tasks as background, not evidence.