Fronset Benchmark

The Right-Sized Model Portfolio

No single model leads across the work the benchmark measured, so the practical setup is a route per activity: a default model and, where the evidence allows, a fallback. Those routes are not equally firm. Many defaults were chosen by breaking a tie on price, serving mode or a nominally higher score, and a few rest on thin or conflicting evidence; those are the decisions that could flip.

How a default and a fallback are chosen

The benchmark scores models out of 10 on each task, counting only the calls a model answered. A score’s confidence interval, written ±, is the range the true score plausibly falls within. The routes below assume a bar of 8 out of 10 for automated output, and costs are quoted for sync mode, where each call is answered on the spot, or batch mode, for work that can wait.

When two intervals overlap, the models count as tied, and the choice falls to cost, serving mode or your own tests. On topic-cluster labeling, Grok 4.5 is the default at 8.62 out of 10 (±0.21), and DeepSeek V4 Pro scores 8.45 (±0.28) [^1][^2]. The intervals overlap, so DeepSeek V4 Pro serves as the cheaper fallback, at an estimated sync cost of $0.01 against an estimated $0.04 for Grok 4.5 [^1][^2].

MiniMax M3 is cheaper still, at an estimated $0.0066 per sync task run, and clears the bar at 8.22 (±0.14) [^1][^2]. But its interval does not overlap Grok 4.5’s, so it is an option only if a measurably lower score is acceptable. Every cost in this group is an estimate, so verify the gaps before budgeting.

Ties settled by price, serving mode or a nominal score

Defaults picked from a tie are the tightest decisions, and topic-cluster labeling adds a twist: ties do not chain. Grok 4.5 is tied with DeepSeek V4 Pro, and DeepSeek V4 Pro with MiniMax M3, but Grok 4.5 and MiniMax M3 are not tied [^1][^2]. With the fallback tied to both neighbours, a small change in measured quality could reorder the default and the fallback.

Short promotional messages show two tiebreaks in a row. GPT-5.6 Sol scores 8.58 at $0.02 in sync mode [^3][^4][^5][^6][^7], Grok 4.5 scores 8.33 at the same $0.02 [^3][^5][^7][^8], and GLM-5.3 scores 8.53 at $0.05 [^7]. All three are tied, so cost favours Sol and Grok 4.5 over GLM-5.3. Between those two, at equal cost, Sol’s higher nominal score makes it the default and Grok 4.5 the fallback.

Structured output extraction is settled by cost and serving mode instead. Qwen 3.7 Plus, Claude Opus 5.5 and Claude Sonnet 5.5 are tied; Qwen 3.7 Plus is the default as the lowest-cost sync option, and Claude Sonnet 5.5 is the fallback where the work can be deferred to batch [^9][^10]. A shift in either price or mode would change the default.

Routes resting on thin or conflicting evidence

Some routes rest on less than a score and an interval. On language identification, Claude Haiku 5.5 is the default and Claude Sonnet 5.5 the fallback, but the only evidence given here is a best-pick count: how often each model’s answer was judged the best. Claude Haiku 5.5 was judged best in 20 of 31 occasions (64.5%), and Claude Sonnet 5.5 in 11 of 31 (35.5%) [^11].

No quality score or interval is given here for either model, so the 8 out of 10 bar cannot be applied directly, and nothing here shows how firm that split is. That makes this the thinnest evidence behind any of these routes, and a fragile choice of default.

A route can also hinge on two figures that disagree. On community content promotion, Qwen 3.7 Plus has a score of 8.12 out of 10 (±0.27) [^12][^13] and a later, conflicting 7.75, dated 2026-09-24 [^13], which is the more current reference.

The two sit either side of the bar, so a route for this work could appear or disappear depending on which measurement holds. Of all the routing decisions here, it is the one most likely to move.

What to do with this

  • Pick each fallback from the models whose interval overlaps the default’s, and record what decided the default: a quality gap, a price, a serving mode or a nominal score.
  • Re-check first the routes decided by something other than quality, and look at tied groups as a whole: a middle model can tie with two neighbours that are themselves separated.
  • Treat a route backed only by a best-pick count, or by figures that straddle your bar, as provisional until a test on your own inputs settles it.

Check the live benchmark at https://fronset.ai/benchmark/ to set the quality bar your production output requires and to compare batch against sync pricing yourself.