HTTP 413 "Request too large for model" (Groq)

What the error looks like

Note the wording: Request too large, not Rate limit reached. The Requested number is the token count of the request you sent.

{
  "error": {
    "message": "Request too large for model `openai/gpt-oss-20b` in organization `org_...`
                service tier `on_demand` on tokens per minute (TPM): Limit 8000, Requested 11240,
                please reduce your message size and try again. ...",
    "type": "tokens",
    "code": "rate_limit_exceeded"
  }
}

Why waiting does not help

Groq meters tokens per minute per model. The free tier's TPM on the current chat models is 8,000. If your request alone is 11,240 tokens, it can never fit inside a minute — the counter resets and the request is still too big. This is why people report "I waited an hour and it still fails."

Fix 1 — Keep each request under the limit

• Send only the last few turns of history, not all of it.
• Move long instructions out of the prompt (summarise them, or cache derived results).
• Chunk documents and process them in pieces, then combine.
• Count tokens before sending and trim to a budget below the limit (leave room for the completion, which also counts):

import tiktoken

enc = tiktoken.get_encoding("o200k_base")
def count(msgs): return sum(len(enc.encode(m["content"])) for m in msgs)

def fit(messages, budget=7000):
    """Drop the oldest non-system turns until the request fits the per-minute limit."""
    system = [m for m in messages if m["role"] == "system"]
    rest = [m for m in messages if m["role"] != "system"]
    while rest and count(system + rest) > budget:
        rest.pop(0)
    return system + rest

Fix 2 — Raise the limit

Groq's Developer tier has higher TPM. At the time of writing the console shows "Developer tier upgrades are temporarily unavailable due to high demand", so this path is closed for many accounts — see the entry on that message.

Fix 3 — Route large requests to an endpoint without a per-minute ceiling

Because the request shape is the OpenAI format, any OpenAI-compatible provider can take the same call. Keep Groq for small, fast turns and send anything that 413s to a second client:

import os
from openai import OpenAI, RateLimitError, APIStatusError

groq = OpenAI(base_url="https://api.groq.com/openai/v1", api_key=os.environ["GROQ_API_KEY"])
big  = OpenAI(base_url="https://api.inferenceapis.com/v1", api_key=os.environ["INFERENCEAPIS_KEY"])

def chat(messages):
    try:
        return groq.chat.completions.create(model="openai/gpt-oss-120b", messages=messages)
    except APIStatusError as e:
        if e.status_code not in (413, 429):
            raise
        # same model id, no TPM ceiling
        return big.chat.completions.create(model="openai/gpt-oss-120b", messages=messages)

Frequently asked questions

Is 413 the same as rate_limit_exceeded?

The code field says rate_limit_exceeded on both, but the HTTP status differs: 429 is a rate over time, 413 is a single request over the per-minute budget. Only 429 is worth retrying.

Does the completion count toward the 8,000?

Yes. Groq counts prompt tokens plus the requested max_tokens when deciding whether the request fits. Lowering max_tokens can turn a 413 into a success.

Which Groq models have the 8K TPM limit?

On the free tier, openai/gpt-oss-120b, openai/gpt-oss-20b, openai/gpt-oss-safeguard-20b and qwen/qwen3.8-27b list 8,000 TPM; groq/compound lists 70,000. Always check the current table on Groq's rate-limits page.

Where Inference APIs fits

Fix 3 above routes requests that cannot fit the per-minute ceiling to an endpoint without one. Inference APIs serves the same model ids (openai/gpt-oss-120b, llama-3.3-70b-versatile) with no TPM cap and per-request billing.

Related entries

• HTTP 429 rate_limit_exceeded (Groq)
• Groq: "Developer tier upgrades are temporarily unavailable"
• 404 model_not_found: "The model … does not exist or you do not have access to it"