Fronset Benchmark

Best Models for Topic Organization & Clustering

Moonshot Kimi K3 scores 8.93 out of 10 at naming groups of related text, one of the strongest results on that task [^1][^2], but 3.67 at suggesting content domains [^3]. Across the benchmark’s topic organization and clustering tasks, no model leads everywhere, and how much the choice of model matters changes too. A pipeline that names, orders and files topics needs a model decision for each step.

A strong score on one task doesn’t carry to the next

Naming is one of several distinct tasks in the category. Others include ordering topics into a sequence, assigning sections, matching items to a taxonomy, generating section taxonomies, discovering themes and suggesting content domains. Each is scored separately out of 10, and a score of 8.0 counts as clearing the quality bar.

Claude Sonnet 5 scores 8.38 on the same naming task [^1][^2] but 1.02 (±0.08) on thematic topic discovery [^4][^5]. The ± figure is the uncertainty range around a score; at ±0.08, this low result is well supported. Whether the gap comes from the model, the prompt format or the task definition is not established, so it is no proof the model cannot discover themes in general.

Where model choice barely matters

The benchmark also looks at judged occasions, where several models answered the same input and the answers were scored. On assigning sections to topics, across 61 such occasions, the two best answers tied exactly 65.6% of the time, finished within half a point 85.2% of the time and had a median gap of 0.0 [^6][^7].

When the best answers tie that often, price is a fair tiebreaker. Tencent Hy3 on this task is the lowest-cost result above 8.0 in the whole category: 8.07 (±0.42) at $0.0002 per run in sync (non-batch) mode, measured on its own usage [^8][^9]. Its range dips below 8.0, though, so that pass is marginal.

Where model choice decides the result

Content domain suggestion is the opposite case. It is where model choice has the largest visible effect in the category, and where scores run lowest. On one variant of the task, the two best answers tied exactly on 8.0% of occasions, finished within half a point on 20.7% and had a median gap of 1.9 points [^10][^11].

Meta Muse Spark 1.1 scores 8.71 (±0.3) here, at $0.03 per run in sync mode, repriced at current rates [^3][^12]. Most other models land well below the bar, including Tencent Hy3, the low-cost pick for section assignment, at 5.16 [^3][^12]. These scores come from different cost bases and some labels differ, so they are not a strict ranking.

What to do with this

  • Test each step of a topic pipeline on its own, even with a model that scored well on a neighboring step.
  • Where the best answers mostly tie, as on section assignment, start with the cheapest model that clears your bar, and confirm a marginal pass like Hy3’s on your own data before it carries production load.
  • Where answers diverge, as on content domain suggestion, run a task-specific test on your own inputs before committing to any model.

The live benchmark at https://fronset.ai/benchmark/ lets you set the quality bar your workflow needs and inspect the task-level results for this category.