Fronset Benchmark

Batch vs. Sync

In the benchmark, batching (submit now, collect later) undercut synchronous calls for the same model and task in all but one priced pair, by 21% to 85% [^1][^2][^3]. The catch is the tail: under a one-hour deadline, 4.1% of batch submissions were still open at expiry and were re-sent to a different model [^4][^5]. A flat discount and a median turnaround are both the wrong numbers to plan with.

The size of the discount tracks the model

Most savings fall in the 40s and 50s, and the figure varies more by model than by task. On Gemini 3.1 Flash Lite, batch most often saves 28% [^6][^7]. GPT-5.6 Sol costs 71% less in batch on thematic topic discovery, at $0.15 against $0.51 synchronously [^8][^9], and the same 71% less on selecting social posts for a portfolio [^10][^11].

A model’s usual figure still isn’t guaranteed for every task. GPT-5.6 Sol’s batch saving drops to 59% on bench hypothesis post generation [^12]. And one pair shows no saving at all: Xiaomi MiMo V2.6 Flash costs $0.0001 in both modes on engagement opportunity triage, at the precision shown [^13][^14].

The discount also needs batch to be available. Fifty of the benchmark’s 68 tasks are batch-eligible [^4][^5], and a model without batch support is priced at its sync cost, so a pipeline that batches whenever it can saves nothing on such a model.

Most batches return in minutes, but some pass the hour

Batch is slower, but usually far quicker than its window suggests. The published provider window runs up to 24 hours, yet half of all batches came back within 5.5 minutes and 75.1% within 15 minutes [^4][^5].

Of completed batches, 5.6% took longer than an hour [^4][^5]. Batches cancelled at the one-hour deadline had a median age of 61.1 minutes [^4][^5], and re-sending them to a different model can change the output as well as the delay, so the tail affects what you get back, not just when.

Missed deadlines vary by provider

In the benchmark, batch reliability is a property of the provider rather than the model [^4][^5]. Under the one-hour deadline, the share of submitted batches cancelled and re-sent was 6.4% of 4,231 for anthropic and 3.9% of 24,335 for gemini [^4][^5], and 4.7% of 3,840 for xiaomi [^5]. These rates carry no confidence intervals (error bars showing how precise each rate is), so they are measured differences, not a settled ranking.

What to do with this

  • Look up the batch figure for your exact model and task before budgeting.
  • Give every batch job a deadline and a fallback, and measure how often your provider misses that deadline.
  • Keep synchronous calls for work where minutes matter or where an answer from a different model would change the result.
  • Test whether batch output matches sync output on your own tasks; the benchmark cannot yet show that it does for any model.

To compare batch and sync prices for the models and tasks you are considering, see the live benchmark at https://fronset.ai/benchmark/.