Budget LLM Latency on the 90th Percentile, Not the Median
Across 863,807 synchronous calls in the Fronset benchmark, the median answer took 5.4 seconds and the 90th percentile took 50 seconds, more than nine times as long, under a 600-second per-call deadline [^1]. If you size a user-facing step on the median, you are planning for the typical call and ignoring the slow ones, and in this data the slow ones are much slower.
Why the median is the wrong number for an interactive step
A median tells you that half of calls finish sooner. It says nothing about how long the other half take, and a step where a person waits on one call is exposed to all of them.
The 90th percentile is the time that one call in ten exceeds. It is the more honest planning number for anything a user sits through: a timeout set near the median would cut off roughly half the calls, and even one set at the 90th percentile still cuts off one in ten.
What the tail looks like
Most results in the benchmark report medians only. Where a model and task do report a 90th percentile, it sits well above the median:
| Model and work | Median | 90th percentile |
|---|---|---|
| Gemini 3.5 Flash Lite, classifying claim continuity | 1.6 seconds | 27 seconds [^2] |
| Nemotron-3 Ultra 550B, scoring a topic report | 2.6 seconds | 18 seconds [^3] |
| gpt-5.6-luna, summarizing content | 8.6 seconds | 37 seconds [^4] |
| gpt-5.5, extracting claims from a document | 14 seconds | 47 seconds [^5] |
| Nemotron-3 Super 120B, triaging engagement | 21 seconds | 94 seconds [^6] |
| Tencent Hy3, classifying claim continuity | 40 seconds | 171 seconds [^2] |
In every row the 90th percentile is more than three times the median [^2][^3][^4][^5][^6]. The gap does not close on fast models. Gemini 3.5 Flash Lite has the lowest median in the table, yet its 90th percentile is more than 16 times that median [^2]. Being quick on the typical call says little about the slow one.
The rows cover different tasks, so read the table as evidence that tails exist across models and kinds of work, not as a ranking. Nemotron-3 Ultra 550B and Super 120B, for example, each have a reported tail on a different task, so those two rows cannot tell you which model has the tighter tail [^3][^6].
What to do with it
- Budget on the 90th percentile where one exists. Base loading states, timeouts and promised response times on it.
- Treat a median-only result as an open question. Its tail is unknown, so measure your own calls before a user waits on them.
- Queue work whose tail does not fit. If the 90th percentile is longer than a person will wait, move the job out of the request-response loop.
- Check the specific job. A fast tail on one task does not carry over to a neighbouring one.
The caveat: the slowest calls are not in these numbers
Some calls ran past the 600-second deadline: 101 of them, or 0.012% of successful synchronous calls, and they are excluded from every latency figure here [^1]. These are stalls rather than slow answers. Across the benchmark that share is far too small to shift the overall 90th percentile much, but stalls are exactly what a timeout exists to catch, and none of the figures above account for them.
These are also response-time results only. They say nothing about answer quality or price, so a model with a tight tail still has to clear your quality bar.
The live benchmark is at https://fronset.ai/benchmark/. Set the quality bar you need, open the task closest to your job, and check whether it reports a 90th percentile before comparing models on speed. The method is documented at https://fronset.ai/benchmark/methodology/.