Rate limits and trial

Limits protect shared capacity. They are checked before anything is charged, so a limited request costs nothing. Signed in, Limits shows your account's limits with live counts of running requests and requests this minute.

Request limits

• Requests per minute are counted in fixed 60-second windows, per key and per account. Going over either returns 429 rate_limit_exceeded with a Retry-After header giving the seconds until the window resets. A custom per-key limit above 1,200 raises the account limit to match.
• Concurrent requests are counted per account across all its keys, from the moment a request's headers arrive (before its body is uploaded) until its response ends. Going over returns 429 rate_limit_exceeded without reading the body. Retry when one of your running requests finishes.
• Rejected requests are requests that passed the safety check and then were refused by the model (a 4xx such as an invalid parameter or a prompt over the context length) or abandoned by the client before they were forwarded. They cost nothing, but each one used the safety check, so after 30 in one minute the account's requests fail with 429 rate_limit_exceeded and a Retry-After header until the window resets. Fix the failing requests rather than retrying them.
• A body over 16 MiB fails with 413 invalid_request before it is read.
• A request stops after 10 minutes, including time your client spends not reading. Once the model has started answering, it also stops when the model sends nothing for 2 minutes; the wait for the first token, which grows with the prompt, is bounded only by the 10 minutes. The response then ends with an error, and you're charged for the estimated input plus the output generated so far, streamed or not (UTF-8 bytes divided by 3, at least one token per generated delta). The estimate never exceeds the model's context length for input or the request's output budget for output.
• Resellers and other high-volume users can ask support for custom per-account limits. Reselling requires a written reseller agreement; see the Terms.

Trial accounts

An account is a trial account until its first top-up is credited. The trial comes with $5 of free credit when the network qualifies:

• Datacenter, VPN and Tor addresses get no trial credit.
• One trial per IPv4 /24 or IPv6 /48 per 30 days.
• A global daily budget of $500 caps how much trial credit is given out.

When no trial credit is available, you can still top up and use the account right away. If your first credited top-up is $10 or more, you get a one-time 20% bonus on it (deposit $50, get $60 of credit), limited to one bonus per network (IPv4 /24 or IPv6 /48) every 90 days.

Trial accounts have the lower limits in the table above. They are also first to be shed under load: when the models are saturated, a trial request fails immediately with 429 capacity_exhausted instead of waiting for capacity. All trial limits lift as soon as your first top-up is credited.

Output budget

Every request has an output budget, the most tokens it may generate:

• /v1/chat/completions: max_tokens or max_completion_tokens, default 16384.
• /v1/messages: max_tokens, which is required.
• /v1/responses: max_output_tokens, default 16384.

A budget above the model's max output, or above 4096 on a trial account, is lowered to that limit rather than rejected. Reasoning tokens count toward the budget.

Balance holds

Before a request is forwarded, we hold the most it could cost: the estimated input tokens (request size in bytes divided by 3, at most the model's context length) at the input price, plus the full output budget (never less than 256 tokens) at the output price. When the response ends, the hold is released and only the actual usage is charged.

If the hold doesn't fit your balance, we lower the output budget to the largest amount you can afford and try once more, provided that amount is at least 256 tokens. Below that the request fails with 402 insufficient_balance, even when the request itself asks for fewer than 256 tokens. A nearly empty balance can therefore produce shorter answers that stop with a length finish reason. Set a smaller budget, or top up, if you need the full length.

Spend caps

Each API key can have a daily cap, a monthly cap and a lifetime budget, set in API keys. Days and months are UTC. A request fails with 402 key_spend_cap_reached when its hold, added to the key's settled spend for the day, the month or its lifetime and the holds of its requests still running, would pass the cap. Parallel requests therefore can't overshoot a cap, and a response that has started is never cut off by one. GET /v1/balance reports what is left of each cap.

Saturation

When every replica of a model is busy, paid requests wait instead of failing. We retry with backoff, starting at 250 ms and doubling up to 4 seconds between attempts, for up to 30 seconds in total. If capacity still doesn't free up, the request fails with 429 capacity_exhausted and Retry-After: 5. Nothing is charged.

Retries happen only before the response starts. Once the model begins to answer, the response streams to the end, within the time limits above. Trial requests are not retried.

During a restart for an update, the requests already running get time to finish. Meanwhile the API keeps accepting connections and answers every new request at once with 503 capacity_exhausted and Retry-After: 5, before anything is charged. A new request gets the same answer when the API is briefly holding too much request data at once. A request still running when that time is up is stopped: a stream ends with an error event and a non-streaming request fails with 503 upstream_error and Retry-After: 5, and either is charged like any stopped request. Retry after Retry-After seconds.

Context length

On /v1/chat/completions and /v1/responses, a prompt longer than the model's context window fails with an error instead of being silently truncated. Context sizes are listed on GET /v1/models.