Nearly Two-Thirds of Campaign Spend Went on Unparseable Output
Nearly Two-Thirds of Campaign Spend Went on Unparseable Output
On promotional campaign generation, nearly two-thirds of the money spent went on output that could not be parsed [^1]. On community content promotion, a task of the same genre, the share was under 7% [^2]. Neither a model’s price card nor its quality score shows this cost, so it has to be measured task by task.
Same genre, very different waste
For each task, the benchmark reports total spend and how much of it produced output that could not be parsed. Four promotional tasks show how wide the spread is.
| Task | Spent | On unparseable output | Share |
|---|---|---|---|
| Full promotional campaigns [^1] | $61.27 | $39.70 | 64.8% |
| Short promotional messages for X.com [^3] | $44.22 | $25.16 | 56.9% |
| Reddit posts [^4] | $34.58 | $5.36 | 15.5% |
| Community content promotion [^2] | $27.02 | $1.81 | 6.7% |
The worst task wastes nearly ten times the share of the best. Being promotional does not predict the loss, and a short format offers no protection: the short X.com messages sit just behind full campaigns.
The heaviest waste also lands on the costliest work. Full campaigns are the weakest and most expensive of the promotional jobs measured, and only just over a third of their spend bought output that could be parsed at all [^1]. This is not a one-off reading either. Parse loss was also a material hidden cost in the prior benchmark cycle.
Why the price card and the score miss it
A price card quotes the cost of a task run. On campaigns, Claude Opus 5.5 is listed at $0.13 per run in sync and Moonshot Kimi K3 at $0.63 [^1]. Those are prices per attempt, whatever the attempt returns. If your pipeline loses a share like the campaign figure, each parseable result costs well over twice the quoted price.
The quality score does not reveal it either. Scores cover only the calls that returned an answer, and they say nothing about what share of spend went on output you could not use.
The two tasks at opposite ends of the table make the point. Claude Opus 5.5 leads campaigns at 8.03 (±0.3) [^1], and Kimi K3 leads community promotion at 7.98 (±0.27) [^2]. The top scores are almost the same, while the waste shares behind them are the highest and lowest of the four.
What to do with it
Treat parse waste as a per-task measurement, like quality, and put it into the budget before choosing a route.
- Measure it on each task with your own inputs before committing volume. A figure from one promotional task does not carry over to the next, even within the same genre.
- Record it in dollars: spend on unparseable output divided by total spend.
- Fold it into every price comparison. Divide the per-run price by the parseable share of spend to get the cost of each parseable result, and compare routes on that number.
Until the loss is measured on a task, a low per-run price is a starting point, not a cost estimate.
A task figure, not a model figure
The caveat that matters is attribution. These are totals for each task across the models run on it, and the findings do not attribute the loss to particular models.
So the evidence shows where parse waste is large, not which model to switch to in order to avoid it. Whether a different model lowers it on your task is something only your own measurement can show.
The task pages behind these figures are on the live benchmark at https://fronset.ai/benchmark/. Start from the task closest to yours, then measure parse waste on your own inputs before committing a high-volume route.