Router

model: "auto"

You say how good the answer needs to be. We find the cheapest open model that gets there, for every single request, and put the reason in the response.

How it decides

1
Read the request
Task type, input length, tools, output format. Nothing is stored.
2
Predict quality
Expected pass rate for each model, from public benchmarks and, if you upload them, your own evals.
3
Pick the cheapest that clears
Of the models above your quality floor, the one with the lowest expected cost.
4
Fail over
If that model errors or is slow, the next-cheapest one that clears the floor takes the request.

Use it

Two fields. Every answer explains itself.

The response is a normal chat completion. The headers say which model served it, why, and where it ran. You pay that model's per-token price.

Router docs→

resp = client.chat.completions.create(
    model="auto",
    extra_body={"quality_floor": 0.85},
    messages=[{"role": "user", "content": "Label this ticket: ..."}],
)

Questions

The fine print.

What does the router cost?: Nothing extra. You pay the per-token price of whichever model served the request.
Can I stop it switching models?: Yes. Pass a model id instead of "auto" and it never changes. Pinned versions stay up for 12 months after a successor ships.
What is the quality floor?: A number from 0 to 1: the minimum expected pass rate for this request. Default 0.8. Higher means bigger models and more cost.
Does it learn from my prompts?: No. Routing decisions use request features and benchmark data. Prompts aren't stored or used for training.
Can it use my own evals?: Yes. Upload around 50 real examples with expected outputs. Every model is scored on them and the router uses your pass rates.
Which models can it pick?: Any model on the platform in your key's region, unless you pass an allow-list with route.models.