Fronset Benchmark

Best Models for Social & Promotional Content

Some of the benchmark’s quality scores for social and promotional content are measured only on the calls that came back, so failed calls never pull them down. Kimi K3 scores 7.78 out of 10 on promotional campaign generation when it answers, but it answers on only 36% of attempts [^1]. Choose on the score alone and you are planning around output that may never arrive.

Scores that leave out failed calls

The answer rate is the share of attempts on which a model returns an answer. On the same task, Grok 4.7 scores 7.51 when it answers, and answers on 55% of attempts [^1]. On these figures, Kimi K3 has the higher score and Grok 4.7 the higher answer rate.

Neither score comes with a margin of error, so the gap between the two models is provisional, and the task has no decisive model. Nor can either be set directly against Qwen 3.8 Flash’s 5.97 on the same task [^1], because that score is not described as counting only answered calls.

The pattern is not limited to campaign work. Selecting social posts is the closest contest in the category. There, GPT-6 Sol scores 8.7, level with GLM-5.3 Flash, but answers on only 71% of attempts [^2].

Short messages: a higher score or a known answer rate

On short promotional messages, quality and reliability point in different directions. DeepSeek V4 Flash scored 8.21 and answered on 88% of attempts [^1][^3][^4]. GLM-5.3 has the higher score, 8.53, but no answer rate is given for it [^4].

Which to pick depends on what your workflow can least afford. If most calls must come back, DeepSeek V4 Flash has the more complete evidence. If the quality bar comes first, GLM-5.3 is the candidate, but treat its missing answer rate as an unknown, not a pass, and confirm it before use [^4].

Spend that produced nothing usable

Unusable output also has a direct cost. On writing messages for X promotion, $44.22 was spent and $25.16 of it (56.9%) produced output that could not be parsed [^5][^6]. On a task like that, the price per call understates the cost of each usable message.

The share lost varies widely by task. On promotional campaign generation, $3.27 of $7.93 (41.2%) was lost [^1][^7], and on community content promotion generation, $0.42 of $5.81 (7.3%) [^8][^9]. These figures are not broken down by model, so they show which work wastes spend, not which model wastes it. X promotion, with the largest share lost, is the first place to look.

What to do with this

  • Read every quality score next to its answer rate, and check whether it leaves out failed calls before comparing it with another.
  • Confirm answer rates on your own deployment before committing high volume to a model.
  • On promotional campaign generation, plan a fallback path for failed calls and test candidates on your own inputs [^1].
  • Track spend on output that cannot be parsed, task by task, alongside quality and answer rate.

Check current pricing on the live benchmark at https://fronset.ai/benchmark/, where you can also set the quality bar your workflow needs.