What gets charged
Exactly when a request is billed and when it is not - stream disconnects, client cancels, retries, zero-token responses, and every other edge case, matched to how the gateway actually works.
This page is the definitive answer to "is this request going to charge me?". Every scenario below matches how the gateway actually decides whether to deduct from your balance.
The one-line rule
You are charged for what the model actually produced. Nothing produced, no charge.
Everything else on this page is a corollary of that rule. The one exception: some providers (notably Anthropic / Claude models) report input token usage as soon as the response starts, so a stream cut after that point bills those input tokens even if no text arrived yet.
How the gateway computes a charge
For every successful response, the gateway:
- Reads the
usageobject from the response (or, for streams, from the stream chunks). If the provider omitted it, the gateway counts the prompt and returned output tokens itself. - Multiplies the reported token counts by the model's per-unit price to get a model cost.
- Adds any accumulated tool cost (built-in web search, web fetch, code interpreter calls that ran during this request).
- Subtracts the combined
model_cost + tool_costfrom your balance in a single atomic operation. - Logs the breakdown to your Usage and Logs pages.
If the response contains no output at all, no deduction happens and the request is logged as a no-usage warning so we can investigate.
Scenario-by-scenario reference
Non-streaming responses
Non-streaming requests cannot be cancelled
A client-side timeout or disconnect does nothing on a non-streaming request - the gateway keeps waiting for the full response regardless. See Stream Cancellation for why, and what to use instead if you need a hard cost or length ceiling.
Streaming responses
Streaming has more failure modes than non-streaming, and the billing behaviour is not always the obvious one. Read this section carefully - especially the client-cancel case.
Cancelling a stream bills only what was generated
Aborting a stream stops generation and bills the partial amount - not
the full response - and nothing at all if no output had arrived yet.
See Stream Cancellation. If you want a
hard cap regardless of whether you cancel, set max_completion_tokens
to the exact length you want.
Pre-flight rejections
These never reach the provider, so they never produce any usage to bill against.
Retries
The gateway has an internal retry loop for transient upstream failures (network glitches, 5xx responses, certain rate-limit conditions on the provider side).
Tools and tool calls
Cached input tokens
Summary table
| Scenario | Charged? |
|---|---|
| 200 response with usage | Yes - full cost |
| 200 response, no usage reported | Yes - from our own token count |
| 200 response, empty output | No |
| 4xx or 5xx from provider | No |
| Stream completes normally | Yes - full cost |
| Stream errors out mid-flight (connection drop, upstream error) | Partial - what was delivered |
| First-chunk timeout | No |
| Chunk-stall timeout | Partial - what was delivered |
| Client cancels mid-stream | Partial - prompt + output so far |
| Client cancels before any output | No (except input tokens already reported, e.g. Claude) |
| Client disconnects on a non-streaming request | Yes - full cost (request keeps running) |
| 401 / 402 / 429 / 400 from gateway | No |
| Gateway retry that eventually succeeds | Yes - once, for the success |
| Gateway retry that ultimately fails | No |
| Defined tool that never gets called | No tool fee |
| Built-in tool call succeeds | Yes - tool fee added |
| Built-in tool call fails | No tool fee |
How to verify any specific request
Every request the gateway processes shows up in the Console under Logs with:
- The full token breakdown (input, output, cached, cache write, tool cost).
- The exact dollar amount deducted (
$0.00if not charged). - The final HTTP status returned to you.
If you ever see a charge you cannot account for, find the request in Logs and the breakdown will show you exactly where the cost came from.