How to get the cost of each LLM API call, and where a response header falls short

In January 2024 a developer building on the OpenAI API asked whether the cost of a call can be read from its response. The dashboard showed the number, but they wanted their app to charge its own users at least what OpenAI had charged, which needs the figure per call. The answers were to count tokens and apply the price list, or to infer the cost from the rate-limit headers before and after the call. That thread has about 10,000 views. Two later ones ask the same thing: a feature request for the billed cents inside usage, with no replies, and another thread whose answers were the daily Costs endpoint or tokens times rate.

AIVAX returns the total charged by a call in a response header, for non-streamed requests made from a server. A stream and a browser each need another route.

Other APIs return tokens, a daily total, or a cost field

OpenAI's chat completion usage has the token counts: prompt, completion and total, with cached and reasoning tokens broken out. The CompletionUsage type in the official Python SDK has no cost field, and the prompt caching guide tells you to compute input cost from the token counts in response.usage and the model's prices per million tokens. The response headers the API reference lists as common cover organization, processing time, request ID and rate limits. The Costs endpoint returns money, but in buckets of one day (bucket_width accepts only 1d) grouped by project, user, line item, API key or API source, and its examples authenticate with an admin key. No parameter selects a single request.

Gateways made different choices. OpenRouter returns usage.cost with every response, in the last SSE message of a stream, as the total charged to your account in credits. LiteLLM sends x-litellm-response-cost as a header, plus per-component cost headers that its documentation says appear on non-streaming responses only. AIVAX also uses a header for the total, and adds the line items to the body of a non-streamed response.

AIVAX returns the total in a Consumed-Credits header on non-streamed calls

When a request charges your balance, the response carries Consumed-Credits: the total charged by that request in USD, after the plan multiplier, across the charges recorded while the request ran, such as model tokens, embeddings, reranking and tools (pricing docs). A request that charged nothing has no header, whether it was rejected or covered entirely by a daily allowance, and the docs say to read a missing header as zero.

With the OpenAI Python SDK the header is on the raw response:

import os
from decimal import Decimal

from openai import OpenAI

client = OpenAI(
    base_url="https://inference.aivax.net/v1",
    api_key=os.environ["AIVAX_API_KEY"],
)

raw = client.chat.completions.with_raw_response.create(
    model="@openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Summarize order 123."}],
)

completion = raw.parse()
charged = Decimal(raw.headers.get("Consumed-Credits", "0"))
request_id = raw.headers["x-request-id"]

The header is the sum of the usage rows the request wrote, each a SKU with a unit price and a quantity. The non-streamed body lists those rows: generation_context.generated_usage has sku, amount, unit_price, quantity and description for every one (docs). Their unit prices already include the plan multiplier, so the amounts add up to the header. Use the rows when the total is not enough. A gateway that routes by complexity bills its classifier to you under its own SKUs, inference.resolving.routing_complexity.in and .out, in the same request as the main model, so the cost of one request on that gateway includes them.

The header is a double, and a browser cannot read it

The docs show Consumed-Credits: 0.000184, which is how the server writes a double that is 0.0001 or larger. Below that it writes scientific notation. Take a call with 84 uncached input tokens, 1,792 cached tokens and 16 output tokens on @openai/gpt-6-luna, served by a provider that charges the catalog's $0.11, $0.011 and $0.55 per million tokens for input, cached input and output. On the Max plan (multiplier 1.00) that is $0.000037752, and the header reads 3.7752E-05. Python's float and Decimal and JavaScript's Number parse that; a regular expression for digits, a dot and more digits does not.

The server also adds the rows to the total as doubles, in the order it wrote them, which can leave floating-point noise. The five rows in the example response of the API documentation, a different call from the one above, add up to exactly $0.0001188, and summed as doubles in the listed order they come to 0.00011879999999999999. Store the header exactly as it arrived, parse it as a Decimal, and round only when you produce an invoice line.

The second limit is the browser. AIVAX's CORS policy exposes Content-Type, X-Authenticated-Account-Id and X-Authenticated-Account-Plan to scripts, and browsers hide every other response header of a cross-origin response that the server does not list. Consumed-Credits is not listed, and neither is X-Request-Id, so a page that calls the API directly can read neither. Read them in your backend.

A streamed response has to be priced from its final usage chunk

The pricing docs say not to rely on Consumed-Credits for a stream: its headers go out when the stream opens, before the model has produced anything, so there is no total yet. A stream does end with a chunk that carries usage, and every chunk names the model and provider that served it. That chunk has token counts and no cost, and no generation_context.

The counts need care. prompt_tokens excludes cached tokens, which are reported in prompt_tokens_details.cached_tokens, while total_tokens includes them. The AIVAX API documentation shows a response with prompt_tokens: 84, completion_tokens: 16, cached_tokens: 1792 and total_tokens: 1892, so adding the first two undercounts by the whole cached share, which is billed at the cached rate. The post on prompt cache hit rates shows the same shape from the other side.

To price the call you need the rates of the provider that answered, your plan's multiplier, and one rule from the server: when a provider lists no cached-input price, cached tokens are billed at its input price. That rule is not an edge case, because 119 of the 1,619 provider entries in today's public catalog have no cached price. The catalog answers without an API key (the model deprecation post uses it in a CI check) and weighs about 1 MB, so fetch it once and cache it:

import os

import httpx
from openai import OpenAI

CATALOG = "https://inference.aivax.net/api/v1/information/models.json"
PRO_PLAN_MULTIPLIER = 1.05

catalog = {
    model["name"]: model
    for group in httpx.get(CATALOG).json()["data"]
    for model in group["models"]
}

client = OpenAI(
    base_url="https://inference.aivax.net/v1",
    api_key=os.environ["AIVAX_API_KEY"],
)

raw = client.chat.completions.with_raw_response.create(
    model="@openai/gpt-6-luna",
    messages=[{"role": "user", "content": "Summarize order 123."}],
    stream=True,
)
request_id = raw.headers["x-request-id"]

usage = None
for chunk in raw.parse():
    if chunk.usage:
        usage, model, tag = chunk.usage, chunk.model, chunk.provider

provider = next(p for p in catalog[model]["providers"] if p["routingTag"] == tag)
cached = usage.prompt_tokens_details.cached_tokens
prompt_total = usage.prompt_tokens + cached

rates = max(
    (t for t in provider["pricing"] if t["tokenThreshold"] <= prompt_total),
    key=lambda t: t["tokenThreshold"],
)
cached_rate = rates["cachedInputPerMillionTokens"]
if cached_rate is None:
    cached_rate = rates["inputPerMillionTokens"]

cost = PRO_PLAN_MULTIPLIER * (
    usage.prompt_tokens * rates["inputPerMillionTokens"]
    + cached * cached_rate
    + usage.completion_tokens * rates["outputPerMillionTokens"]
) / 1_000_000

The result is an estimate. It uses the catalog's current list prices, which can differ from the prices in force when the call ran, and it ignores audio input, which the server prices separately. The lookup also assumes an integrated model: a BYOK gateway returns a null provider, and the next(...) call would fail.

OpenRouter puts the cost in that last chunk and AIVAX does not. I count that as the larger gap in this design, because a streaming client otherwise needs a price table or conversation logging to get a number at all.

The provider that answered sets the price

The pricing of a model in the catalog summarizes its providers: for each price component, the most common value among those that are not special routes. The code treats fast, flex and priority variants, and NagaAI's discounted route, as special. @openai/gpt-6-luna is listed at $0.11 per million input tokens, and its eight providers charge from $0.025 to $0.20, an eightfold spread.

Which one answers depends on the routing preset. The default, Balanced, drops special routes when ordinary providers exist and keeps the most common price among the rest, so it usually lands near the catalog figure; it then ranks on quality, latency, uptime and throughput and can pick any provider in that group. Cheapest can pick the discounted route, and Fastest or Quality can pick a fast variant. A request sets the preset with routing_options, together with an ordered list of allowed provider tags (docs); it is an AIVAX extension, so with the OpenAI Python SDK it goes in extra_body. The provider field of the response carries the tag that served the call, which is the one the streaming snippet looks up.

The conversation record keeps a cost for streamed calls when logging is on

With conversation logging enabled, AIVAX stores a record per call, and its usage object carries a cost next to the token counts. GET /api/v1/conversations returns that cost for each conversation in a time window and can filter by external user, which is the user field of your request, so streamed calls can be totaled per user without a price table (record fields).

The record comes with its own limits. Logging stores the messages along with the cost, and allow_logging: false keeps a request out of it while the billing records are still created. Retention is 2 hours on Free, 2 days on Pro and 30 days on Max (limits). The cost covers the model call; a routed gateway's classifier, retrieval and tool charges are billed on the request but are not in it. In the code I read, a call covered by a daily subscription allowance writes no usage rows and its cost is zero, so a zero here means the allowance paid for it, not that the model was free. A repeated idempotency_key updates the same record, so its cost is that of the latest call, not a running total. The record stores the request ID that the response returned in X-Request-Id, and the single-conversation endpoint returns it as requestId, but the list neither shows it nor filters on it, which is a reason to send a user value you can search for.

For a stream I would use the record's cost when logging is on and your retention covers the billing cycle: it is computed on the server with the rates and plan multiplier in force when the call ran. Otherwise compute from the final chunk. The calculation is the only figure available the moment the stream ends, which matters if the application must show or limit spend during the session.

The header leaves out work that bills later and calls that bill elsewhere

Charges recorded after the response is sent are not in the header. The pricing page names asynchronous Batch work, and each batch item keeps its own cost, as the posts on batch admission control and expired batch jobs describe.

A model call served by your own provider key is the other gap. In the code I read, the inference charge is created only for integrated models, so a BYOK call writes none, and the response's provider is null for a BYOK gateway. On such a gateway the header carries only the other charges of the request, such as retrieval, and the model cost sits on your provider's invoice. A missing header therefore means AIVAX charged nothing, which is not always the same as the call being free to you.

Reconcile every figure against the stored usage rows

The pricing page says to use your account usage records as the source of truth for billing, and I would take that literally. By usage records I mean the rows AIVAX stores for each charge. The monthly usage report (GET /api/v1/information/usage/monthly-report?month=YYYY-MM) totals them as totalCost and breaks them down by SKU. Everything above is derived from those rows or covers part of them: the header is their sum for one request, generated_usage lists them, the conversation cost covers the model call, and the streaming calculation is an estimate of it.

If you re-bill customers from these figures, compare a month of your stored figures with the report's total, remembering that the report also includes batch and other work the header never covers. Run a non-streamed call now and then and compare its Consumed-Credits with what the snippet computes for the same usage. Keep X-Request-Id next to each stored figure, so you can match it with the requestId of the conversation record when logging is on.

Read on 2026-10-25: the AIVAX code for usage records, the cost header, conversation records and provider routing; the pricing, inference and limits docs; the public catalog and AIVAX's API documentation; OpenAI's reference, caching guide and SDK types; the OpenRouter, LiteLLM and MDN pages; and the three forum threads. I made no authenticated call. The header values are my local run of the server's number formatting and summing, and the Python snippets ran only against a mocked HTTP transport. I did not test a stream, the conversation endpoints, the monthly report or a browser, and I did not check that X-Request-Id arrives on a stream; the CORS list comes from an unauthenticated request that returned 401. The cost field of conversation records is in the code but not in the example responses of the API documentation, and the --user filter syntax comes from the code, not from a call.