400 "This model's maximum context length is N tokens" (context_length_exceeded): why max_tokens counts, and how to fix it

What it looks like

{
  "error": {
    "message": "This model's maximum context length is 128000 tokens. However, your messages resulted in 130412 tokens. Please reduce the length of the messages.",
    "type": "invalid_request_error",
    "param": "messages",
    "code": "context_length_exceeded"
  }
}

The param is messages even when max_tokens is the real cause, which is the trap. On our endpoint the same overflow is reported as:

{
  "error": {
    "message": "The request does not fit the model's context window of 131,072 tokens (prompt tokens plus max_tokens). Shorten the conversation, lower max_tokens, or use a model with a larger window: DeepSeek V4 and GLM 5.3 models here take 1,048,576 tokens. See GET /v1/models for each model's context_length.",
    "type": "invalid_request_error",
    "param": null,
    "code": "context_length_exceeded",
    "docs_url": "https://inferenceapis.com/docs/errors#context_length_exceeded"
  }
}

The window size is in the message so you can decide whether to trim or to move, and GET /v1/models lists context_length for every model without a key.

Fix 1: max_tokens is part of the sum

Set max_tokens to what you need for the answer, not to the model's maximum. If you do not know, omit it; servers default to what fits. In agent frameworks look for a per-model output setting; the reports from opencode, Kilo Code and Zed in the threads above all trace to a static output budget colliding with a grown prompt.

Fix 2: budget the prompt before sending

Count tokens client-side and drop the oldest turns until the request fits, keeping the system prompt. Tool outputs are the usual growth: cap them at a few thousand tokens each before they go back into the conversation.

import tiktoken
enc = tiktoken.get_encoding("o200k_base")          # close enough for budgeting on any model

def fit(messages, window, max_tokens, margin=2000):
    """Drop the oldest non-system turns until prompt + max_tokens fits the window."""
    budget = window - max_tokens - margin
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    count = lambda ms: sum(len(enc.encode(str(m["content"]))) + 4 for m in ms)
    while rest and count(system + rest) > budget:
        rest.pop(0)
    return system + rest

Fix 3: a model with a larger window, same code

If the work needs the context, the window is the fix. These models take 1M tokens on the OpenAI format, on the Anthropic format (for Claude Code) and on the Responses API (for Codex):

Long prompts are billed as input. On DeepSeek V4 Flash a 500K-token prompt costs about ten cents, and repeated prefixes bill at the cached-input rate of $0.04 per million, so a large fixed context reused across turns is cheap after the first call. Speed and tool-calling checks per model are on the model pages.

Frequently asked questions

Why does the error say "reduce the length of the messages" when my prompt is short?

Because the check is prompt plus max_tokens and the message template only mentions messages. Lower max_tokens and the same prompt goes through.

Do reasoning tokens count against the window?

Yes, as output. A reasoning model with a large thinking budget needs more room than its visible answer suggests. On our reasoning models reasoning_effort: "none" removes that.

Is 1M context slower?

Time to first token grows with the prompt; a full 1M prompt takes noticeably longer than a 10K one on any host. Cached input, which applies automatically here to repeated prefixes, removes most of that on later turns.

Where Inference APIs fits

Fix 3 is what we sell: five models with a 1,048,576-token window on the same OpenAI wire format, so the fix for a 128K overflow can be a model name. Our own overflow error names the window and this page. US GPUs, zero retention, prepaid; a new account starts with $1 of credit.

Related entries

• Anthropic API 400 "prompt is too long: N tokens > 200000 maximum" in Claude Code: causes and fixes
• HTTP 413 "Request too large for model" (Groq)
• 404 model_not_found: "The model … does not exist or you do not have access to it"