Fronset Benchmark

Confidence Tiers Track Evidence, Not Model Quality

Nine results on the Fronset benchmark, spread across seven models, have dropped a confidence tier, and Gemini 3.8 Flash has been promoted twice. None of these moves reports that a model got better or worse at the work. A tier records how firmly a placement is known, and reading it as a quality signal is the easiest way to misread the results.

What a confidence tier measures

Each result carries a 95% interval on a model’s mean quality score for one kind of work. Three tiers appear in these results, in rising order: medium, high and ranked. When the interval closes far enough, the placement moves up a tier.

Gemini 3.8 Flash shows this twice. On putting a set of topics into order, it crossed from medium into high confidence because its interval closed, not because it scored better [^1]. On writing promotional messages, it crossed from high into ranked for the same reason [^2].

For DeepSeek V4.1 Flash, the narrowing shows up across a whole set of tasks. Since 2026-10-16 its interval has narrowed on all six kinds of work tracked, and its standing in each is settling rather than drifting [^2][^3][^4][^5][^6][^7].

The narrowing is uneven. Labeling topic clusters is the tightest of the six at ±0.22 points [^6], and summarising long documents follows at ±0.26 [^4]. Matching candidate profiles to pools is the widest at ±0.45 [^3], so if a decision depends on fine distinctions there, that is where the evidence is thinnest.

Nine demotions, spread across models

The demotions land on ordinary production work, and no single model accounts for them. Seven results fell from high to medium confidence:

  • Claude Opus 5.5, writing promotional campaign copy [^8]
  • Xiaomi MiMo V2.6 Pro, finding the themes in a body of content [^9]
  • Grok 4.7 and Qwen 3.7 Plus, both on scoring how relevant generated content is [^10]
  • Claude Sonnet 5.5, suggesting which content domain something belongs to [^11]
  • GLM-5.3, identifying a text’s language [^12]
  • GPT-6 Sol, selecting research communities for a piece of work [^13]

Two more dropped from ranked to high: Claude Opus 5.5 on picking authoritative sources [^14] and Grok 4.7 on generating publication title packages [^15]. Both models appear twice, but on different work each time, so this is not one model having a bad stretch on a single task.

Each result stands on its own kind of work, so one model can move both ways at once. Claude Sonnet 5.5, demoted on content domain suggestion, crossed from medium into high on triaging engagement opportunities, again because its interval closed [^16]. Gemini 3.5 Flash Lite fell from ranked to high on generating publication section taxonomies [^17]. On identifying a document’s language, though, it is among the most confidently ranked models: 9.84 out of 10, ±0.11 after 153 independent judgments [^12].

What to do when a tier moves

  • Do not read a demotion as a regression. It does not show that a route you already run got worse, only that its placement is less firmly established than it was.
  • Hold medium-confidence placements as provisional and wait for the interval to close before committing.
  • If you need a firm placement now, the evidence is strongest for DeepSeek V4.1 Flash on topic cluster labeling and summarisation [^4][^6], Gemini 3.5 Flash Lite on language identification [^12], and Gemini 3.8 Flash on promotional messages and topic ordering [^1][^2].

What a firm placement does not tell you

A narrower interval says nothing about quality. The DeepSeek V4.1 Flash results report interval widths, not mean scores, so they show the measurement settling without telling you whether the settled value meets your bar. None of the tier moves came with quality scores, prices or latency figures either, so nothing here shows one model beating another on this work.

Set the quality bar you need on the live benchmark at https://fronset.ai/benchmark/, compare the models that clear it, and only then weigh how firm their placements are. The method is documented at https://fronset.ai/benchmark/methodology/.