Fronset Benchmark

Benchmark Results and Evaluation Scores

In the benchmark’s head-to-head judging, the model whose answer gets picked as best changes from task to task. GPT-5.6 Sol was judged best in 34 of 55 occasions (61.8%) on onboarding chapter prompt generation [^1][^2], but in only 10 of 30 (33.3%) on content summarization [^3][^4]. That makes head-to-head results a guide for choosing a model task by task, not a ranking of models overall.

What a judged-best share does and does not show

A judged-best share counts how often a model was judged the best answer among the models answering the same live input. Each such contest is one judged occasion.

Onboarding chapter prompt generation is not the only task where GPT-5.6 Sol won most of its contests. It was also judged best in 27 of 43 occasions (62.8%) on vetted site selection [^5][^6] and in 18 of 32 (56.2%) on extracting a table of contents [^7][^8]. Those three tasks are where to start when testing GPT-5.6 Sol.

A share measures how often a model won its contests, not how far ahead it finished, so a narrow win and a wide one count the same. A quality score is a different measurement: an absolute rating out of 10 for a model on a task, rather than a count of wins against the other answers to the same input.

Some models lose every contest on more than one task

Gemini 3.1 Pro Preview was judged best in none of 41 occasions on engagement review [^9][^10][^11][^12], none of 101 on claim extraction [^13][^14][^15][^16] and none of 43 on synthesis of titles for publication [^17][^18]. Qwen 3.8 Flash was chosen best in none of 42 occasions on section prompt generation [^19] and none of 40 on catalyst and scenario analysis [^20].

A pattern that repeats across three different tasks, as Gemini 3.1 Pro Preview’s does, is harder to dismiss as a feature of one task. But a zero needs the same care as a win: it shows that a model lost its comparisons, not how far behind it finished, so it is not a measure of how weak the output was.

Close contests are ties

Some head-to-head comparisons are close enough to call a tie. On content summarization, GPT-5.6 Luna was judged best in 19 of 43 occasions (44.2%) and GPT-5.4 Nano in 28 of 71 (39.4%) [^3][^4]. The gap is small and is not established as real, so neither model can claim the task on this evidence.

What to do with head-to-head results

  • Compare candidates head-to-head on your own inputs for each task, instead of carrying one task’s winner over to the next.
  • Read a judged-best share as a count of wins; check quality scores or your own tests to see how much better the winner is.
  • Treat a small gap between two shares as a tie until it is shown to be real.
  • If a model is never picked across several tasks, take the pattern seriously, but check its quality scores before dropping it.

Use the live benchmark at https://fronset.ai/benchmark/ to check the tasks closest to yours; the method is documented at https://fronset.ai/benchmark/methodology/.