Inference APIs
Reference

Reference

Working notes on inference API errors and provider limits, kept current as providers change their models and tiers. Vendor-neutral — some entries end in "here is how to fix it yourself".

Error reference One entry per exact error string — status, code, whether waiting helps, and the fixes
Switching providers Moving between OpenAI-compatible endpoints with a base URL change
API documentation Authentication, requests, models, streaming and errors for this API

Trackers

Model deprecations

Retired API models with shutdown dates, replacements and where the same model is still served.

 Free-tier rate limits

Current free-tier limits per provider and model, and how each provider signals which limit you hit.

 Model capabilities, tested

Tool calls, streamed and parallel calls, tool_choice, JSON output and how to switch thinking off, per model, from live requests.

 Bring your own endpoint

Which coding tools accept a custom OpenAI-compatible base URL (Codex, opencode, VS Code Copilot, Warp, Zed, Cursor, Claude Code), where it goes, and the open limitations with issue links.

 Speaker diarization, measured

diarize=true on the transcription endpoint: request, response shape in every format, and what it got right and wrong on two-, three- and one-speaker clips.

Error reference All entries

GroqEntryCan you wait it out?
429 · rate_limit_exceeded HTTP 429 rate_limit_exceeded (Groq)
A per-minute or per-day counter (RPM, RPD, TPM, TPD) for the model is exhausted
 Per-minute limits: yes, seconds. Per-day limits: no — until the 24h window resets
413 · rate_limit_exceeded HTTP 413 "Request too large for model" (Groq)
One request (prompt + history + expected completion) is larger than the model's entire per-minute token budget on your tier
 No. The limit is per request, not per window — the same request fails again after any wait
404 · model_not_found 404 model_not_found: llama-3.3-70b-versatile (Groq)
Since 2026-08-16, on any free or developer-tier request that names this model (or llama-3.1-8b-instant)
 No — the model was retired; the request will fail until you change the model or the provider
Billing · upgrade blocked Groq: "Developer tier upgrades are temporarily unavailable"
You try to move from the Free plan to the Developer plan to raise rate limits
 Only if you can accept free-tier limits until Groq reopens upgrades (no date given)
404 · model_not_found Groq 404 model_not_found: mixtral-8x7b-32768 (shut down March 20, 2025, and its replacement was retired too)
 No
OpenAI-compatible APIsEntryCan you wait it out?
404 · model_not_found 404 model_not_found: "The model … does not exist or you do not have access to it"
The id in model is not served by the endpoint you called, or your key is not allowed to use it
 No — it fails deterministically until the id or the endpoint changes
400 · context_length_exceeded 400 "This model's maximum context length is N tokens" (context_length_exceeded): why max_tokens counts, and how to fix it
 No. Same request, same answer
Google GeminiEntryCan you wait it out?
429 · RESOURCE_EXHAUSTED HTTP 429 RESOURCE_EXHAUSTED (Gemini API)
A per-minute (RPM / TPM) or per-day (RPD) quota for the model is used up on your tier — most often the free tier's daily request quota
 Per-minute quotas: yes — honour retryDelay. Per-day quotas: no — until the daily reset (midnight Pacific)
Moonshot AI (Kimi)EntryCan you wait it out?
400 · invalid_request_error "thinking is enabled but reasoning_content is missing in assistant tool call message": Kimi K2 tool loops fail on the second turn
On the second turn of a tool loop: the client sends the conversation back with the assistant's tool call but without the reasoning_content that came with it
 No. It is a validation rule, not capacity
400 · Extra inputs are not permitted "Extra inputs are not permitted, field: safety_identifier": OpenAI-only parameters rejected by Kimi and other strict hosts
 No
Together AIEntryCan you wait it out?
400 · Grammar error "Grammar error: properties must be an object": tool_choice fails on a tool with no parameters
tool_choice is "required" or names a function, and that function's parameters.properties arrives as [] instead of {}
 No. The request is rejected before generation starts
Anthropic APIEntryCan you wait it out?
500 · api_error Anthropic API Error: Internal Server Error (500) in Claude Code: what it is, and how to keep working
 Yes. Anthropic's docs: retry with exponential backoff, and contact support with the request id if it persists. The incidents behind the big threads lasted twenty minutes to a few hours
429 · plan limit Claude Code "You've hit your limit · resets ...": what the Pro and Max windows are, and how to keep working before the reset
 Yes, and only that. The window resets at the time shown: up to five hours for the session window, up to a week for the weekly cap
529 · overloaded_error Anthropic API 529 "overloaded_error" in Claude Code: what it means, how long it lasts, and how to keep working
 Yes, but not on your schedule. Anthropic's incidents have run from twenty minutes to most of a working day
400 · credit balance too low Anthropic API 400 "Your credit balance is too low to access the Anthropic API": who is being billed, and what to do
 No. Adding credit is the fix, and even that has taken hours to a week to register in the reports linked below
400 · prompt is too long Anthropic API 400 "prompt is too long: N tokens > 200000 maximum" in Claude Code: causes and fixes
 No. The same input will be too long tomorrow
Claude CodeEntryCan you wait it out?
400 · thinking.type Claude Code on a third-party model: API Error 400 'type' must be in ["enabled", "disabled", "auto"]
 No. Retrying sends the same request. Users report it as intermittent only because load-balanced hosts run mixed backend versions
Codex CLIEntryCan you wait it out?
unsupported call Codex CLI: MCP tools return "unsupported call" or are never used with a custom model provider
Any MCP server, stdio or HTTP, as soon as the model provider is not OpenAI or Azure
 No. It fails the same way on every request until the request format or the provider changes
422 · agent_message Codex sub-agents return nothing on an external provider: agent_message and the empty "Payload:"
On every hand-off to a sub-agent, and again when the sub-agent's answer is replayed in the parent's history
 No
DeepSeek APIEntryCan you wait it out?
402 · Insufficient Balance DeepSeek API 402 "Insufficient Balance": whose balance, how to tell, and what to do
 No. Nothing resets. The error ends when someone adds funds to that account
429 · 503 DeepSeek API 429 "Rate Limit Reached" and 503 "Server Overloaded": the concurrency cap behind both, and your options
429: the account has more requests in flight than its cap. 503 and 500: DeepSeek's capacity is full, most often in its peak-price hours and after a model release
 429 from your own concurrency: yes, in seconds, once requests finish. 503 and 500: sometimes, in minutes to hours. 429 through a reseller: no, the cap belongs to the reseller's account
400 · Model Not Exist DeepSeek API 400 "Model Not Exist": deepseek-chat and deepseek-reasoner were retired
 No. The ids are gone. Retrying returns the same 400
OpenAI APIEntryCan you wait it out?
429 · insufficient_quota OpenAI API 429 "You exceeded your current quota" (insufficient_quota): it is billing, not a rate limit, and what to do
 No. It clears when credits are added or a spend limit is raised. OpenAI's docs: "Retrying billing, spend, or quota errors won't restore API access."
OpenCode GoEntryCan you wait it out?
403 · RegionError OpenCode Go 403 RegionError: "only available hosted in China and requires explicit opt in"
You call a DeepSeek model on OpenCode Go and the workspace has not switched on "Enable models hosted in China"
 No. It is a policy block, not a rate limit. Retrying returns the same 403
OpenRouterEntryCan you wait it out?
402 · Insufficient credits OpenRouter 402: "Insufficient credits" and "Free BYOK request cap exceeded for this month"
The OpenRouter account that owns the key has no credits to charge, or the key's own spending limit is used up. With BYOK the charge is OpenRouter's 5% fee, so it happens with a fully funded upstream key
 Only for limit_source: openrouter_in_flight_budget (wait for Retry-After) or when the account is somebody else's. Your own empty balance or key limit stays at 402 until credits are added
Gemini APIEntryCan you wait it out?
404 · NOT_FOUND Gemini API 404 NOT_FOUND: "models/gemini-2.0-flash is not found" (shut down June 1, 2026)
 No. The model is gone. The same message is returned for typos, which is why it is easy to misread
Claude APIEntryCan you wait it out?
404 · not_found_error Claude API 404 not_found_error: retired model ids (claude-3-5-sonnet, claude-3-7-sonnet, claude-sonnet-4, claude-opus-4)
 No. Retired means requests fail

How-tos

Switching OpenAI-compatible providers with a base URL change
Python, Node, LiteLLM and LangChain examples, plus the three things to check before you flip the switch.

Missing an error you hit? Send us the exact string and we will add an entry.