Attempts and settlement

Updated

A request can involve one or more attempts, each sending the request to one endpoint. You pay only for the attempt that succeeds.

How charging works

  • The price version in effect for the endpoint is locked when the request starts; later price changes do not affect the call.
  • Only the successful attempt is billed for usage. Failed attempts cost you nothing and are not paid to providers.
  • By default each request tries only the first endpoint. With allow_fallbacks on it tries up to 3 (see Fallbacks) and is billed at the price of the endpoint that succeeded. However many endpoints are tried, a request produces at most one charge.
  • If a stream is interrupted after output starts, the usage produced so far is billed; when exact usage is unavailable, an estimate is billed and marked Estimated.
  • Calls whose outcome stays unknown because no upstream receipt arrives are free.
  • attempts in the response lists each attempt's provider, endpoint, HTTP status and cost. Failed attempts cost 0; the attempt that succeeded costs the cost in the usage object, which is the charge on your bill.

Balance holds

Before a call, the platform holds the highest possible cost, from the input estimate plus the allowed maximum output, then settles at the actual cost and releases the rest right away. When your balance is tight, a smaller max_tokens makes the call more likely to go through.

Where to look

Each call's model, usage, unit price and amount are on the Billing page of the console; see Usage.