The Model That Wins Summarization and Loses Every Triage Contest

A model that tops the benchmark on one everyday utility task can lose every head-to-head on the task next door. GPT-5.4 Nano and Tencent Hy3 both do exactly that in the current llm-bench data, and the averages that look safest also hide models that only answer some of the time. If you route utility work by a model’s reputation instead of by task, you are choosing blind.

Where the scores come from

These results rest on 57,538 scored judgments across 833 measured model-and-task combinations, covering 30 models on 52 tasks. Each task carries an average on a 10-point scale with a ± interval and the number of independent judgments behind it, plus head-to-head occasions where several models answer the same live input and one is judged best.

The figures were published on a single date; the measurement behind them is continuous and ongoing. That matters for what follows, because the zero-win results below are not a verdict on a model so much as a snapshot of what happens when you assume one utility task predicts another.

The summarization star that cannot triage

GPT-5.4 Nano summarizes structured content at 9.29 out of 10, ±0.09 after 420 independent judgments [^1][^2]. In direct contests on content summarization it was judged the best answer in 28 of the 71 occasions it competed in — 39.4% — every one of them against other models answering the same live input [^3][^4].

That strength does not travel. GPT-5.4 Nano answered the same inputs as its rivals in 50 judged engagement triage occasions and was chosen best in none of them [^5][^6]. Summarizing and triaging both look like light utility work from the outside, and a team that put one cheap default on both would have been right on one task and last on the other.

Tencent Hy3 tells the same story from the other side

Tencent Hy3 is the head-to-head leader for identifying the language of text, judged the best answer in 21 of the 38 occasions it competed in — 55.3% [^7][^8]. For pulling discrete claims from text it was judged best in 16 of the 40 occasions it competed in — 40.0% [^9][^10].

Then comes reproducing a distinctive author voice, where Tencent Hy3 answered the same inputs as its rivals in 48 judged occasions and was chosen best in none of them [^11][^12]. A single model can be a backbone default on one utility task and unusable on a neighboring one.

Part of why the head-to-head record matters so much is that many utility tasks are effectively tied at the top on averages. Detecting the language of a text saw the top two answers score exactly the same in 97.2% of 71 judged occasions, with a median gap of 0.0 on a 10-point scale [^7][^8]. When averages tie, the win record is the only thing left to separate candidates, and it is precisely where the zero-win surprises live.

Averages also hide models that do not answer

The second trap is coverage. Claude Haiku 4.5 records 7.25 out of 10 on structured content summarization when it answers — but it only answers on 60% of attempts [^1][^2]. Claude Opus 5 scores 8.91 out of 10 on Topic Cluster Labeling when it answers, but only answers on 76% of attempts [^13], and Thinking Machines Inkling sorts engagement at 8.87 out of 10 when it answers, but only on 67% of attempts [^14][^15].

For a pipeline, a missing answer is a retry, a fallback call or a hole in the output, and none of that shows up in the average. The contrast case is NVIDIA Nemotron-3 Nano 30B-A3B, which returns a usable answer, first time, on 99.3% of its 106,736 attempts [^16][^17][^18][^19]. A quality score only means something once you know how often the model actually delivers it.

The practical rule is short. Route by task, not by model: consolidate on a default only where you have checked that model’s own head-to-head record and answer rate on that specific task, and decide among tied options on first-time answer rate rather than small average gaps. Where a default shows zero wins or partial answering, add a fallback path or a proven head-to-head winner for that task alone.

Where to look next

The live benchmark at https://llm-bench.kapualabs.com/ has a page per task, so you can pull up the exact tasks in your pipeline and read the head-to-head record and answer rate for each candidate before you commit. The newer figures cited here come from the task pages at https://fronset.ai/benchmark/, and both sites let you set the quality bar your workflow actually needs and confirm coverage for your task.