Timeouts Are a Separate Failure From Slow Answers
Timeouts Are a Separate Failure From Slow Answers
Every synchronous call in the Fronset benchmark runs under a 600-second deadline, and an attempt that misses it meters no tokens [^1][^2][^3]. For three of the five models with a reported miss rate, more than a quarter of attempts miss, and for one it is more than two-fifths [^1][^3]. In this data, how often a model misses does not track how quickly its answers usually arrive.
Why a miss stays out of sight
A slow call costs time and still returns an answer. A missed call ties up the caller for the full ten minutes of the deadline, more than a hundred times the 5.4-second median across all 863,807 synchronous calls [^1], and returns nothing. Whoever was waiting gets an error or a retry, not a late result.
Misses are also easy to overlook. Because a missed attempt meters no tokens, it never appears on the bill. Calls that ran past the deadline are excluded from every latency figure [^1], so a median describes only the calls that came back in time. The miss rate has to be measured as a number of its own.
The miss rate does not follow the median
Put each model’s miss rate beside one of its medians, sorted from fastest median to slowest, and the miss rates jump around:
| Model | One of its medians | Missed the 600-second deadline |
|---|---|---|
| gpt-5.4-mini | 722 ms, detecting a text’s language [^4] | 14.8% of 2,413 attempts [^1] |
| gpt-5.5 | 5.5 seconds, a short promotional post [^5] | 34.0% of 17,538 attempts [^1] |
| Xiaomi MiMo V2.6 Pro | 20 seconds, extracting claims [^6] | 41.7% of 5,647 attempts [^3] |
| gpt-5.6-sol | 41 seconds, analytical writing that keeps references attached [^7] | 25.8% of 4,388 attempts [^1] |
| Qwen 3.8 Max | 343 seconds, the same analytical writing [^7] | 13.5% of 2,761 attempts [^2] |
gpt-5.5 is the sharpest case. A median that short would fit a path where a person is waiting, and its median on judging output quality is also short, 9.9 seconds over 1,302 calls [^8]. Yet it has the second-highest miss rate in the table.
Each median covers one job, while the miss rates are reported per model, so the pairing is loose. Still, nothing in gpt-5.5’s medians would warn you that so many of its attempts never come back in time.
Qwen 3.8 Max runs the other way. It has the highest median in its group on analytical writing that keeps references attached [^7], yet one of the two lowest reported miss rates. A model that takes minutes can still miss less often than one that answers in seconds. Its rate and gpt-5.4-mini’s are close and no uncertainty is supplied, so their order is undecided [^1][^2].
What to do about it
- Track misses in your own logs. Count attempts per model, not just completed calls; neither the bill nor latency figures show timeouts.
- Screen on miss rate as well as median. Set the share of misses a path can tolerate and drop models above it, however fast they look.
- Decide what happens at the deadline before you ship. Pick the retry, fallback model or queue in advance; when the deadline fires, the caller has already waited it out.
The caveat that matters is scope. The deadline is a setting in the call, not a property of the model [^1][^2][^3], and these rates describe only the 600-second setting. A shorter deadline for an interactive path, or a longer one for background work, is not measured here. And because the rates come with no breakdown by job, a model’s misses may sit on work you never plan to send it.
When you route a job, compare the candidates on that task at https://fronset.ai/benchmark/, and check how often each one misses the deadline before you size the path around its median.