Best Models for Financial Analysis & Trading Decisions

In the benchmark’s trading recommendation task, the top two answers scored exactly the same in only 2.4% of cases [^1][^2]. The leading answers on this task also separated more often, and by wider margins, than on most other kinds of work in its group. If you are choosing a model for this work, you cannot assume that any leading answer will serve equally well.

How far apart the top two answers were

On each of 84 occasions when answers on this task were judged, the benchmark compared the two highest-scoring answers [^1][^2]. Those two finished within half a point of each other in 53.6% of cases, and the median gap between them was 0.5 on a 10-point scale [^1][^2].

What the gap figures leave open

The figures record how far apart the top two answers were, not which models produced them. So they cannot tell you which model to pick, or which to avoid.

They also measure quality only, and cost and latency for trading work cannot be inferred from quality results.

What to do with this

• Compare candidate models on your own trading cases, against a quality bar you set yourself, before committing to one.
• Don’t assume the leading answers are as close on one kind of work as on another; check each task you deploy.
• Measure cost and latency on your own workload.

The live benchmark at https://fronset.ai/benchmark/ lets you set the quality bar your financial task requires and check the task-level figures before you put a model into production.