Default routing

Updated

When you set no preferences, the platform chooses among a model's endpoints by the rules below. The numbers are platform parameters that operators can tune; the values shown are the defaults.

1. Filter candidates

An endpoint must support every capability (tools, structured output, image input, streaming and so on) and parameter the request uses. Endpoints whose context is too small, that are circuit-broken, at their declared limits, or barred from serving your region are left out. The errors returned when everything is excluded are in Common errors.

2. Sort by price

Candidates are sorted by comparison price, lowest first, and endpoints in the Degraded state always come after Available ones. For text models, comparison price = (input weight × input price + output weight × output price) ÷ (sum of the two weights), with the same weights across the site.

3. Split evenly within a tier

Taking each tier's lowest price as the baseline, endpoints within 1% of it share that tier. Traffic within a tier follows weights, and the default weights give every provider the same share: a provider with n endpoints in the tier gets a weight of 1/n for each. Weights change only according to capacity declared by providers, or drop to 0 as a penalty for violations; commission rates never affect them. Adjusted weights are shown in the comparison table on the model page.

4. Session affinity

Keeping consecutive turns on the same endpoint makes provider-side prompt caching more likely to hit.

  • The session identifier is the X-ModelPortal-Session request header, or else metadata.user_id in an Anthropic request body. OpenAI's user field is not used.
  • Requests with the same account, key, model and session identifier prefer the endpoint that served the previous one for 5 minutes, and each hit restarts the timer.
  • The endpoint is kept only while it is still available, below its limits, not set to a weight of 0, and priced at no more than 1.2 times the current first choice; otherwise the normal order applies.
  • Requests that pass order or sort skip affinity.
  • Requests without a session identifier are spread randomly across the tier.
curl https://modelportal.ai/v1/chat/completions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-ModelPortal-Session: conversation-42" \
  -d '{"model": "MODEL_ID", "messages": [{"role": "user", "content": "Hello"}]}'

5. When sort is set

sort: "price" orders strictly by comparison price; latency and throughput order by time to first token or throughput over the last hour. With sort set, weights no longer split traffic, and endpoints without enough samples come after those with data, ordered by price among themselves.

Precedence

order, then session affinity, then sort or the default price order, then tier weights. Eagle Eye ratings play no part in default routing.