Billing, usage, and cost reconciliation

Beginner: what makes up one charge

Final cost can depend on group, input and output tokens, cache operations, reasoning tokens, image count and size, audio or video duration, task count, or a dynamic expression—not only the model name. The marketplace owns current gateway price and endpoints. An upstream price quoted in documentation is not a substitute.

Retain at least these reconciliation keys:

time, model, endpoint, request_id, task_id, http_status,
input_units, output_units, cached_units, final_status

Pre-consumption, settlement, and refund

The gateway can reserve quota before work to prevent an unfunded request. It then settles using actual usage, media properties, or task outcome and applies a supplement or refund. Failure refund depends on whether work reached the upstream, produced a billable result, and matched the pricing rule. An async amount seen while queued may not be final; inspect the log after completed or failure.

Reconcile one request

Find one log by Request-ID or task ID. Confirm original and mapped model, group, channel, path, and terminal state, then compare response usage. For media or per-call work without token usage, inspect the pricing unit and task properties. Several retries are several requests even if the user intended one business action.

Cost optimization order

Remove irrelevant context and unnecessary output first, then select the smallest model that meets quality. Evaluate prompt caching for stable prefixes and asynchronous or batch processing for noninteractive work. Compare total cost per successful business task, including failures, retries, retrieval, reranking, tool calls, storage, and human review.

Rate and cost interact

RPM, TPM, concurrency, and balance are independent constraints. Raising concurrency blindly can create 429s, timeouts, and retries, increasing cost per success. Enforce concurrency, token and output budgets, a retry budget, and daily spend alerts. Insufficient balance and a hard quota are not transient retry conditions.

Expert ledger and alerts

Join business requests, gateway consumption logs, upstream invoices, and refunds by Request-ID, task ID, model, and time window. Alert on unsettled reservations, terminal tasks without logs, duplicate charges, negative balances, unusual cache behavior, and cost-per-task shifts. Price and route changes need a version and effective time; historical records must use their original snapshot.

FAQ

Why can balance change when a request starts?

Some work reserves an estimate and settles against actual usage or media properties later. Async work may not settle finally until a terminal state.

Can retries create duplicate cost?

Yes. If the first request reached the upstream but its response was lost, replay is another call. Business idempotency, task lookup, and Request-ID reduce uncertainty.

Why can a cache hit still cost money?

Providers price cache writes, reads, and ordinary input differently. Reconcile response usage and the gateway consumption log; cached does not mean free.

Related guides