Lifecycle and What You Pay

The value you put in model names a task, not a specific LLM. We pick which model runs it. To pick well we have to measure, and measuring costs money — sometimes yours. This page explains when, and how to see it.

The authority for everything commercial here is the Pricing Terms and the Terms of Service. Where this page and those differ, those govern.

Two facts decide your bill

1. Who pays the provider. Every task declares one of two access modes, and it is readable on the task:

A task you create defaults to byok unless you ask otherwise.

curl https://api.fronset.ai/api/inference/v1/management/capabilities/3.press-release-claims \
  -H "Authorization: Bearer llmb_m_EXAMPLE_MANAGEMENT_KEY"

{
  "slug": "3.press-release-claims",
  "pricing_mode": "byok",
  "maturity": {
    "status": "exploring",
    "mature_groups": 1,
    "total_groups": 4,
    "threshold_band": "MEDIUM",
    "countdown_judged_calls": 62
  }
}

2. Whether the task is still exploring. That is the rest of this page.

What exploration is

To route your calls to a good model we need evidence about how each candidate performs on your task — not on a public leaderboard. We get it by occasionally running more than one model for a request and having a judge model score the outputs.

This includes /v1/chat/completions. While a task is exploring, one call on the OpenAI-compatible endpoint can run several models and a judge that picks the answer. With the continuous fan-out strategy (below), a small share of calls on a task that has finished exploring is compared the same way, on this endpoint as on the others. On a request charged at cost every one of those models’ tokens, and the judge’s, passes through at the provider’s rates, with one per-call fee for the call; on your own keys they run on your keys and your provider bills you, and the per-call fee is still charged once. A task with little evidence can run most of its pool in one call, so narrow the pool with a model group if that matters to you. Such a call can take many minutes: raise the client timeout and set max_retries=0, because a retried call runs the whole comparison again and bills it again. A request is cut off after one hour. A task with many models in its pool can take longer than that while it explores: the connection then closes without an answer, but the models already running still finish and are charged as described here. The fewer models in the pool, the shorter the call. On one of our tasks that bills at a published rate, any comparison is ours.

Pricing Terms §1.2, verbatim on what that means for your invoice:

A request is charged at cost — every model call in the chain on your invoice, per model — whenever it is not billed at a published rate of ours: while one of our tasks is still being measured, and always on a task of your own. […] Every model call in the chain of a request charged at cost appears on your invoice, per model. That includes calls whose output was not returned to you: alternatives, and the evaluation calls that compare them. It also includes calls that failed, and our own retries after a provider error.

Two consequences worth internalising before you size a budget:

• The chain is a fraction of the eligible set, not all of it. We pick alternatives by what would actually improve the comparison — favouring models we have least evidence about, weighing what each costs. Pricing Terms §3.2: “This is the same rule we apply when we are paying, and we do not widen it because you are.” And on a request you are paying the provider for — your own keys, or a managed request charged at cost — a failure in the machinery that chooses alternatives means we serve your request with one model rather than falling back to running every model.
• Exploration’s share of your provider spend is much larger than its share of your call count. Terms §11 states this explicitly, because a figure quoted only as a percentage of calls would understate what you pay. Most calls are never evaluated; the ones that are, are expensive relative to a plain delivery.

The three charging modes

Pricing Terms §1.4 states a separate rule for each, because they differ on the thing people get wrong: what happens when something fails.

The middle row is the one to plan around, and on a task of your own it is the only managed row: we never put a task of your own on a published rate. Concretely: every model call in the chain is priced at the provider’s published rate for the model, context tier and service tier that call used, and the failed attempts are in there too.

The bottom row has a matching asymmetry: our fee is charged once for a request that completed, but your provider charges for every model call we made behind it. Pricing Terms §3.2 spells out one request against an exploring task:

The per-call fee

One request you make is one call, however many model calls we make behind it. A batch of 10,000 lines is 10,000 calls; submitting it is not one more. Alternatives, judges and retries never add a call.

The card is identical on every plan, and the bands are marginal — crossing a threshold never raises the price of the calls below it:

Model usage on a managed task is billed on top of this: at the provider’s own rates while one of our tasks is still exploring and on every task of your own, and at our published rate once one of our tasks has matured. Under pass-through, input, output, cached input and cache-creation tokens are priced separately; so are context tier, service tier (batch is cheaper than synchronous) and, for one provider, time of day. Each appears on your invoice as its own line. See Pricing Terms §1.1.

The service is prepaid: you buy credit and usage draws it down. A 5.5% fee, minimum $0.80, applies to each credit purchase — charged on the purchase, never on your usage, which is what allows the no-markup rule to be true. Credit expires 12 months after purchase and the oldest is spent first. Automatic recharge exists, is off by default, and turning it on is how you authorise off-session charges. Full detail in Pricing Terms §2.

How a task matures

Maturity is evidence, not age or volume.

• Every model wired to the task carries a confidence band — LOW, MEDIUM, HIGH or RANKED — derived from the judgments it has accumulated on that task. Two catalogue entries for the same underlying model (a free and a paid endpoint, say) share one band and count once.
• A model counts as mature at MEDIUM, or at the task’s confidence floor if that floor is higher.
• The task is exploring while fewer than half its model groups are mature.
• Only our tasks move to a published rate. A task of your own is measured the same way and its routing follows the same bands, but it is billed at cost for as long as it exists, however much evidence it gathers.
• For pricing one of our tasks has one extra requirement: it must have at least two model groups. A task with a single model can never leave pass-through billing, however good that one model’s band. That is the safe direction for a one-way decision.

The switch only goes one way. Once one of our tasks has a fixed price, it stays fixed. We may keep exploring it afterwards — introducing a new model, re-checking for drift — and that exploration is ours to fund, not yours.

Your request’s mode is frozen when the request opens. A task that matures halfway through your call cannot re-rate a call already running.

Reading it off a response

A successful POST /v1/chat/completions normally carries these (also mirrored in a llmbench object in the response body, which OpenAI SDKs ignore):

The first five are always present on a success. The last four are best-effort: absence means we could not resolve the value, never that the value is zero.

Two traps:

x-llmbench-cost-usd is the delivered call only. On an exploring request the alternatives and the judge are not in that number. It is a per-call figure, not your bill for the request. For the whole chain, use the route id.

x-llmbench-phase reports exploration behaviour, not price. A task with fan-out switched off always reports exploit regardless of how much evidence exists. The authoritative maturity read is maturity.status on GET /api/inference/v1/management/capabilities/{slug}, which is exploring or cost_optimised. maturity itself is best-effort and may be null — that means we could not compute it, and it should be read as unknown, never as matured.

x-llmbench-switch-in is omitted when it is zero and when it is not computable — absence means “no estimate”, never “no progress”. The task-level countdown is maturity.countdown_judged_calls on the capability read.

Seeing what you actually paid

The whole chain for one request

The route detail is a portal read: signed in to the dashboard, call GET /portal/api/v1/routes/{request_id} with a portal session token, passing the x-llmbench-request-id you got back on the response. There is no management-key equivalent on /api/inference/v1/.

It returns attempts (how many model calls the route made), route_cost_usd (the sum of provider list cost across every one of them), and attempts_detail — one entry per model call, oldest first, with its model, provider, service tier, status, tokens and cost, and which one was delivered.

That is the only view that shows an exploring request’s real shape. Both cost figures there are provider list price computed by us for reporting; see the note at the end of the next section.

Exploration separated from steady state

GET /api/inference/v1/usage (management key) groups your call records by capability, model, day and role, comma-separated for a compound key. role is the one you want:

curl "https://api.fronset.ai/api/inference/v1/usage\
?from=2026-08-01&to=2026-08-24&group_by=capability,role" \
  -H "Authorization: Bearer llmb_m_EXAMPLE_MANAGEMENT_KEY"

{
  "rows": [
    {"key": "3.press-release-claims|delivered",   "key_parts": {"capability": "3.press-release-claims", "role": "delivered"},
     "calls": 4120, "list_cost_usd": 6.4102, "input_tokens": 3121880, "output_tokens": 411204,
     "cached_input_tokens": 0, "reasoning_tokens": 0, "actual_cost_usd": null},
    {"key": "3.press-release-claims|exploration", "key_parts": {"capability": "3.press-release-claims", "role": "exploration"},
     "calls": 212,  "list_cost_usd": 3.8871, "input_tokens": 160644, "output_tokens": 39880,
     "cached_input_tokens": 0, "reasoning_tokens": 0, "actual_cost_usd": null},
    {"key": "3.press-release-claims|judge",       "key_parts": {"capability": "3.press-release-claims", "role": "judge"},
     "calls": 53,   "list_cost_usd": 1.2044, "input_tokens": 88121, "output_tokens": 9902,
     "cached_input_tokens": 0, "reasoning_tokens": 0, "actual_cost_usd": null}
  ],
  "totals": {"calls": 4385, "list_cost_usd": 11.5017, "actual_cost_usd": null,
             "input_tokens": 3370645, "output_tokens": 460986,
             "cached_input_tokens": 0, "reasoning_tokens": 0}
}

Notice the shape of that illustration: exploration and judging are 6% of the calls and 44% of the spend. That is the asymmetry Terms §11 warns about, and it is why a percentage-of-calls figure alone would mislead.

from and to are required, the window is capped at 92 days, and actual_cost_usd is null for customer accounts by design.

list_cost_usd is not your invoice. It is the provider’s published list price for each call, computed by us for reporting. Pass-through billing is metered separately, from the raw provider token counts, priced when the call ran — so the two figures can differ. And under your own keys, what your provider actually charges you depends on your agreement with them.

Controlling exploration

On a task of your own, the selection block inside extra_body.llmbench narrows the task’s routing policy for one call. Nothing is persisted; anything you omit is inherited. The call below is in template mode: the task’s template renders the variables you send, so messages is empty.

client.chat.completions.create(
    model="3.press-release-claims",
    messages=[],
    extra_body={"llmbench": {
        "prompt_vars": {"press_release": press_release_text},
        "selection": {
            "strategy": "relative",
            "quality_threshold": 0.9
        }
    }},
    timeout=900,
)

quality_threshold requires strategy: "relative" — the other strategies (cost, speed, random) apply no score gate, so a threshold on them would mean nothing.

A call to a task that is still exploring can take much longer than a plain call. Raise the client timeout, and set max_retries=0 on the client (the OpenAI SDKs retry twice by default): a retried call runs again from the start, and its model usage is billed again.

On a task of your own, whether alternatives run at all is yours to decide, for one request or for the task. Pricing Terms §3.2: “On a task of your own, exploration is yours to set: you can turn it off for a single request or for the task as a whole.”

For a single request, send the switch in the same selection block:

extra_body={"llmbench": {"selection": {"auto_live_fanout": False}}}

fan_out_strategy — bootstrap_then_organic, continuous or never — is accepted the same way. never means no fan-out at all, while exploring or after; auto_live_fanout: false does the same.

To make it the default for a task of your own, store the choice as a named selection policy and attach it:

curl -X POST https://api.fronset.ai/api/inference/v1/management/selection-policies \
  -H "Authorization: Bearer llmb_m_EXAMPLE_MANAGEMENT_KEY" \
  -d '{"slug": "no-explore", "strategy": "cost", "auto_live_fanout": false}'

curl -X PATCH https://api.fronset.ai/api/inference/v1/management/capabilities/3.press-release-claims \
  -H "Authorization: Bearer llmb_m_EXAMPLE_MANAGEMENT_KEY" \
  -d '{"selection_policy_slug": "no-explore"}'

Requests against that task then need send no selection block — the stored policy applies. With auto_live_fanout off, we do not run alternatives alongside the model we return. Pricing Terms §3.2 notes that even so, a retry after a provider error or a repair of malformed output can still add a model call.

force_sync is a lane choice, not an exploration switch: it keeps a call on the synchronous lane and is accepted on every task.

On one of our tasks, exploration is ours. We decide how many alternatives run and which judge evaluates them, whichever keys serve the task. The switch and the rest of the fields below are refused on a call to one of our tasks, and a stored change to its selection policy, its fan-out policy, its objective or its models needs allow_breaking_change, which makes the change on a copy of your own where copies are available (Terms §2.1). Its rollout policy and batch setting cannot be changed at all.

Terms §11’s “On our Capabilities we control this rate through policy, and it is not something you configure per Call” is about how often a call on one of our tasks is evaluated. On a task of your own you set that rate for the task with judging_sample_per_10k, including off, on your own tasks only: it cannot be changed on one of ours.

What one of our tasks will not let you set

One of our tasks is measured once for every customer who calls it, and on a managed task a published price rests on that measurement, so these per-call fields are refused on a call to one of our tasks, whichever keys serve it:

• selection.policy, selection.strategy, selection.quality_threshold, selection.min_confidence, selection.fan_out_strategy and selection.auto_live_fanout — they change how the task chooses models or explores (selection.model_group is not one of them: another group runs on a private task of yours, see the data-plane reference);
• forced_model — we choose the model;
• suppress_fanout and exclude_models on the native plane, and metadata.force_judge there — they change what is run and judged.

Each is refused before any model runs with HTTP 422 shared_task_per_call_field, on /v1/chat/completions, the native plane and batch files alike; the message names the fields you sent. force_sync stays allowed. Send the request without them, or make a copy of your own where copies are available and call that.

A managed task, ours or your own, also refuses response_format of type json_schema or json_object: the output schema is part of what the task is, and therefore of what we measure. Omit it (or send type: "text") and the task’s own schema applies; to use another schema, define it on a task of your own. On /v1/chat/completions this surfaces as HTTP 400 invalid_request_error with code byok_only_field and param: "response_format".

What a task of your own lets you set

On a task of your own every field in the list above is yours, on your keys or ours: pin a model with forced_model (pinning turns off failover), set a confidence floor, name a policy, switch exploration off. selection.model_group must name the task’s own model group; any other group, a narrower one included, is refused with HTTP 422 model_group_differs_from_task, or with create_private_task_for_new_model_group: true runs on a private task carrying that group. A task of your own is never put on a published rate, so none of these fields is dropped as it matures.

Creating a task

A task you create is an ordinary task from the moment the create returns: servable, and billed like any other. There is no draft or trial state, and nothing to promote. To stop a task being called, set is_active: false on it.

Created without a models list, its model pool is every eligible model in the catalog, and selection filters it against your vault on each call — so a provider you vault later is picked up without touching the task.

A task of your own never pools. Its verdicts stay in your account, on every plan. Terms §8.1 sets the rule: a verdict pools only if all four of these hold when it is written: your account’s statistics scope is global (that is, free or shared_anon); the task exists and is active; the task is one of ours; and the judging panel is your account’s ordinary production panel. Anything else, and the verdict stays in your account. On private, verdicts aggregate only within your own account and never enter the pool.

On our keys a task of your own is billed at cost and never put on a published rate; on your keys your provider bills you and we charge only the per-call fee.

An unknown model is refused the same way on every lane: 404 model_not_found.

What is never on your bill

• Our own benchmark work. We run comparisons of our own to produce the public benchmark. Those are paid for by us even when they sample one of your tasks, because their output is discarded and returns you nothing. They never run on your keys.
• Our internal traffic. We are a customer of our own API. That usage is billed to us, not spread across yours.
• Exploration after one of our tasks has a fixed price. Once a request is billed at a published rate, later alternatives and judges are ours.

Where to look when the number surprises you

Billing questions: support@fronset.ai