Cheapest LLM that's good enough.
Snapshot 2026-10-16 · Quality bar 90% · Cost mode: batch if supported · 54 tasks · 27 models
In our busiest week we made over 180,000 calls to large-language models, and we let the outputs be evaluated by a panel of leading models. Cost per token doesn't track quality — and when you say something like "give me the model that's reliably 95% as good as the best one but most cost-effective", the best-value model typically saves north of 90% of the cost.
This benchmark publishes our results so other builders can answer the same question for their own pipelines: which model(s) are good enough and the most cost-efficient for each category of tasks. The data also shows that the most expensive model — even when your budget can pay for it — is not always the best performer.
Have a look. Tell us what you think. We can also run tests for you — reach us at benchmark@fronset.ai.
Right-sized per step
Best-value model per capability, at your quality bar. Greyed cells mean no model qualifies on that category yet.
1.0x = this model is the best-value good-enough option, no overpayment; 3.2x = you pay ~3× more on average for the same quality).
★ marks any model that is the best-value pick on at least one of the category's tasks (multiple stars per column are possible — they're the per-task cost winners).
Greyed cells: model doesn't qualify on any task in that category at the current bar.
Rows grouped by provider; click a model name for the full per-task breakdown.Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
By capability
Financial Analysis & Trading Decisions
Read business/financial docs deeply enough to make forward-looking judgments about opportunity and risk.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| GPT-6 Luna ★ | 2/3 | $0.87 | best value |
| DeepSeek V4.1 Flash ★ | 3/4 | $3.06 | 2.1x |
| MiniMax M3 ★ | 2/4 | $3.91 | 2.4x |
| Gemini 3.8 Flash | 1/4 | $4.38 | 2.4x |
| Xiaomi MiMo V2.6 Pro | 1/4 | $7.75 | 2.1x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Structured Data & Fact Extraction
Precise pattern recognition and field-level information retrieval, strict schema adherence.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Claude Haiku 5.5 ★ | 1/2 | $0.35 | best value |
| Xiaomi MiMo V2.6 Flash ★ | 1/2 | $0.74 | best value |
| GPT-6 Luna ★ | 2/3 | $0.83 | 1.2x |
| NVIDIA Nemotron-3 Super 120B | 1/1 | $0.93 | 2.6x |
| Xiaomi MiMo V2.6 Pro | 1/2 | $2.22 | 3x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Content Summarization & Synthesis
Compress long input into essential nuggets without losing material detail.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 1/3 | $0.36 | best value |
| Claude Haiku 5.5 | 1/4 | $0.50 | 1.2x |
| GPT-6 Luna ★ | 4/4 | $1.15 | 1.1x |
| MiniMax M3 | 2/3 | $2.89 | 7.7x |
| DeepSeek V4.1 Flash | 3/4 | $3.02 | 4.6x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Long-form Content Generation
Sustained compositional skill, voice consistency, coherent extended prose.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| GPT-6 Luna ★ | 2/3 | $0.41 | best value |
| MiniMax M3 ★ | 3/3 | $1.06 | 2.4x |
| Tencent Hy3 | 1/3 | $1.36 | 3.2x |
| Claude Haiku 5.5 ★ | 3/3 | $1.41 | 1.8x |
| DeepSeek V4.1 Flash | 1/2 | $1.61 | 4.1x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Social & Promotional Content
Conciseness, platform-native conventions, engagement under tight character limits.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 2/3 | $0.21 | best value |
| GPT-6 Luna ★ | 1/3 | $0.21 | best value |
| Gemini 3.5 Flash Lite | 1/4 | $0.49 | 2.3x |
| Xiaomi MiMo V2.6 Pro | 2/2 | $0.70 | 3.3x |
| Claude Haiku 5.5 ★ | 3/4 | $0.83 | 1.6x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Relevance, Classification & Matching
Semantic similarity judgment: does this thing belong in that bucket / match that target?
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| NVIDIA Nemotron-3 Nano 30B-A3B | 2/7 | $0.20 | 2.4x |
| Tencent Hy3 | 3/11 | $0.41 | 7.7x |
| Xiaomi MiMo V2.6 Flash ★ | 6/8 | $0.69 | 1.7x |
| NVIDIA Nemotron-3 Super 120B | 3/9 | $0.73 | 7.4x |
| NVIDIA Nemotron 3.5 Lightning | 2/5 | $0.84 | 8.2x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Topic Organization & Clustering
Discover natural groupings without external schema, name and order them coherently.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 2/2 | $0.56 | best value |
| GPT-6 Luna | 1/2 | $0.88 | best value |
| Xiaomi MiMo V2.6 Pro | 1/3 | $1.10 | 4.5x |
| DeepSeek V4.1 Flash | 1/3 | $1.62 | 1.9x |
| Gemini 3.8 Flash | 2/4 | $2.04 | 5.4x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Infrastructure & Utility
Mechanical competence at format conversion, metadata manipulation, prompt rewriting, translation; minimal domain expertise required.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Xiaomi MiMo V2.6 Flash ★ | 4/6 | $0.13 | best value |
| Claude Haiku 5.5 | 3/4 | $0.23 | 2x |
| GPT-6 Luna ★ | 7/7 | $0.34 | 1.2x |
| Xiaomi MiMo V2.6 Pro | 3/4 | $0.44 | 3.3x |
| Gemini 3.5 Flash Lite | 1/6 | $0.66 | 4.2x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Evidence-Grounded Hypothesis Generation
Derive distinct, arguable hypotheses or claims from a source document while grounding and citing each one strictly in a supplied set of approved evidence.
| Model | Tasks | Avg $/1k runs | Overpay |
|---|---|---|---|
| Qwen 3.7 Plus ★ | 1/1 | $4.96 | best value |
| Thinking Machines Inkling Small | 1/1 | $5.02 | best value |
| MiniMax M3 | 1/1 | $5.57 | 1.1x |
| Gemini 3.8 Flash | 1/1 | $7.54 | 1.5x |
| Meta Muse Spark 1.3 | 1/1 | $18.40 | 3.7x |
Fronset Benchmark · snapshot 2026-10-16 · 90% quality bar
Methodology, briefly
Quality scores come from LLM-judge verdicts on production workloads, on a 0–10 scale. Only MEDIUM-or-better cells qualify. The quality bar is relative to the best-performing model on each task — adjust the slider above to see your own pipeline.
What changed this week
What changed — 2026-09-24 → 2026-10-16
- Tasks: 54 (+1) · Models: 27 (-3) · Model–task pairs: 906 (+94) · Graded samples: 46,711 (-9,021)
New capabilities (2):
- Capability Category Matching —
capability-category-matching(relevance_and_classification) - Claim Continuity Classification —
claim-continuity-classification(relevance_and_classification)
Removed capabilities (1):
- Known Url Content Extraction —
known-url-content-extraction
New models: Claude Haiku 5.5, Claude Opus 5.5, Claude Sonnet 5.5, DeepSeek V4.1 Flash, GPT-6 Luna, GPT-6 Sol, Grok 4.7, Xiaomi MiMo V2.6 Flash, Xiaomi MiMo V2.6 Pro
Dropped models: Claude Haiku 4.5, Claude Opus 5, Claude Sonnet 5, DeepSeek V4 Flash, DeepSeek V4 Pro, Gemini 3.1 Flash Lite, Gemini 3.5 Flash, GPT-5.4 Nano, GPT-5.6 Luna, GPT-5.6 Sol, GPT-5.6 Terra, Grok 4.6
Per-model movement
(every model, 90% bar) — best value = tasks where the model is the cheapest good-enough option, counted separately under each cost mode; qualifying = tasks it’s good-enough on (quality only, so it is the same under both modes). A model with no batch API prices batch at its sync rate, so the two best-value columns diverge wherever a provider batch discount decides the winner:
| Model | Best Value (Sync) | Best Value (Async/Batch) | Qualifying |
|---|---|---|---|
| GPT-6 Luna (new) | 12 (new) | 15 (new) | 24 (new) |
| Xiaomi MiMo V2.6 Flash (new) | 13 (new) | 14 (new) | 16 (new) |
| MiniMax M3 | 7 (-6) | 5 (-4) | 23 (+1) |
| DeepSeek V4.1 Flash (new) | 5 (new) | 5 (new) | 18 (new) |
| Claude Haiku 5.5 (new) | 4 (new) | 4 (new) | 17 (new) |
| GLM-5.3 Flash | 3 (+1) | 3 (+2) | 27 (+9) |
| Thinking Machines Inkling Small | 3 | 3 (+1) | 27 |
| Qwen 3.7 Plus | 3 | 3 (+3) | 23 (+1) |
| Tencent Hy3 | 2 (-2) | 1 (-2) | 16 (+3) |
| Thinking Machines Inkling | 1 | 1 | 29 (+1) |
| NVIDIA Nemotron-3 Super 120B | 1 | 0 (-1) | 6 (-1) |
| Moonshot Kimi K3 | 0 | 0 | 37 (+13) |
| Meta Muse Spark 1.3 | 0 | 0 | 34 (+12) |
| Claude Opus 5.5 (new) | 0 (new) | 0 (new) | 33 (new) |
| Grok 4.7 (new) | 0 (new) | 0 (new) | 29 (new) |
| GPT-6 Sol (new) | 0 (new) | 0 (new) | 28 (new) |
| GLM-5.3 | 0 | 0 | 27 (+6) |
| Tencent Hy4 Preview | 0 | 0 | 27 (+8) |
| Claude Sonnet 5.5 (new) | 0 (new) | 0 (new) | 24 (new) |
| Gemini 3.8 Flash | 0 | 0 (-1) | 24 (+7) |
| Qwen 3.8 Max | 0 | 0 | 20 (+9) |
| Xiaomi MiMo V2.6 Pro (new) | 0 (new) | 0 (new) | 20 (new) |
| NVIDIA Nemotron-3 Ultra 550B | 0 (-1) | 0 (-1) | 12 |
| Gemini 3.5 Flash Lite | 0 | 0 | 7 |
| Qwen 3.8 Flash | 0 | 0 | 5 (+1) |
| NVIDIA Nemotron 3.5 Lightning | 0 | 0 | 2 |
| NVIDIA Nemotron-3 Nano 30B-A3B | 0 | 0 | 2 |
| Claude Haiku 4.5 (dropped) | — (was 0) | — (was 0) | — (was 9) |
| Claude Opus 5 (dropped) | — (was 1) | — (was 1) | — (was 17) |
| Claude Sonnet 5 (dropped) | — (was 1) | — (was 3) | — (was 25) |
| DeepSeek V4 Flash (dropped) | — (was 3) | — (was 3) | — (was 19) |
| DeepSeek V4 Pro (dropped) | — (was 0) | — (was 0) | — (was 23) |
| Gemini 3.1 Flash Lite (dropped) | — (was 6) | — (was 3) | — (was 7) |
| Gemini 3.5 Flash (dropped) | — (was 1) | — (was 1) | — (was 36) |
| GPT-5.4 Nano (dropped) | — (was 0) | — (was 2) | — (was 10) |
| GPT-5.6 Luna (dropped) | — (was 13) | — (was 20) | — (was 25) |
| GPT-5.6 Sol (dropped) | — (was 0) | — (was 0) | — (was 30) |
| GPT-5.6 Terra (dropped) | — (was 0) | — (was 1) | — (was 30) |
| Grok 4.6 (dropped) | — (was 0) | — (was 0) | — (was 23) |