Inference APIs

Is cached input cheaper?

Yes, automatically. When the start of a prompt matches a recent request, the backend reuses its work and reports the number of reused tokens in usage.prompt_tokens_details.cached_tokens. Those tokens bill at the model's cached-input rate, about 80 to 98 percent below its normal input rate; the pricing page lists it per model. In our test a 5,600-token prompt cost $0.0098 the first time and $0.0011 when repeated. Agents and long chats benefit most, because every turn resends the conversation so far. GPT-OSS 120B and Llama 3.3 70B have no cached rate.

Related

More questions