404 model_not_found: llama-3.3-70b-versatile (Groq)

What happened

Groq's deprecations page lists the notice: announced 17 June 2026, shut down 16 August 2026, affecting llama-3.1-8b-instant and llama-3.3-70b-versatile. In Groq's words it "applies to free and developer-tier usage; enterprise customers with a committed-spend contract are not affected." The recommended replacements are openai/gpt-oss-120b or qwen/qwen3.6-27b for the 70B, and openai/gpt-oss-20b for the 8B. Groq's model list has since moved its Qwen slot to qwen/qwen3.8-27b, so check the current id before pasting one from the notice.

It was not a one-off. llama-3.3-70b-versatile was itself the recommended replacement when Groq retired llama-3.1-70b-versatile, llama3-70b-8192 and deepseek-r1-distill-llama-70b. The full sequence is in the deprecations tracker.

First, find where the id is pinned

The id is often in more places than the one that errored: a default in a settings file, an environment variable on the server, a second call for summarisation or titles, a framework default.

grep -rnE "llama-3\.3-70b-versatile|llama-3\.1-8b-instant" --include="*.py" --include="*.ts" --include="*.js" --include="*.json" --include="*.y*ml" --include=".env*" .
# also check deployed environment variables and framework defaults (LangChain ChatGroq, CrewAI LLM, LiteLLM config)

Option 1 — switch to Groq's recommended replacement

One string changes and you stay on Groq's free tier. What is different about openai/gpt-oss-120b:

• It reasons before answering. Thinking counts against max_tokens. A budget of 256 that was fine for Llama can return an empty content; use 800 or more, or set reasoning_effort to low.
• The response has an extra reasoning field. Code that reads only content is unaffected.
• Style and formatting differ. Prompts tuned to Llama's habits, output parsers and few-shot examples need a test pass. Tool calling and JSON mode work on both.
• You inherit the same free-tier rate limits, and reasoning tokens use up the daily token allowance faster.

- model="llama-3.3-70b-versatile"
+ model="openai/gpt-oss-120b"
  max_tokens=1024        # was 256: GPT-OSS thinks before it answers

Option 2 — keep running Llama 3.3 70B elsewhere

Llama 3.3 70B is an open-weight model and other providers still serve it. If your prompts, evaluations or output formats were tuned on it, moving the base URL keeps them valid:

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",      # was https://api.groq.com/openai/v1
    api_key=os.environ["INFERENCE_API_KEY"],
)
resp = client.chat.completions.create(
    model="llama-3.3-70b-versatile",                  # accepted unchanged
    messages=[{"role": "user", "content": "hello"}],
)

Inference APIs accepts the Groq id as an alias, so nothing else changes. Two honest caveats: it is served as the FP8 "Turbo" variant, so wording can differ slightly from Groq's endpoint on identical prompts, and at $0.20 / $0.50 per million tokens it costs about what GPT-OSS 120B does here. Keep Llama because your work depends on it, not by default. Numbers: Llama 3.3 70B here vs Groq's retired offer.

If you were on llama-3.1-8b-instant

Groq points 8B users to openai/gpt-oss-20b, which stays within the free tier. Few providers still host the 8B at all. Inference APIs does not; it accepts the id so that broken apps start answering again, but routes it to Llama 3.3 70B at the 70B price. For a cheap, fast replacement on a pay-as-you-go endpoint, deepseek-ai/DeepSeek-V4-Flash or zai-org/GLM-5.3-Flash are the closer match on cost.

Option 3 — stop pinning one provider and one id

Put the model id in configuration and let a 404 or 429 fall through to a second provider. A retirement then degrades to a log line instead of an outage, and you choose when to migrate.

import os
from openai import OpenAI, NotFoundError, RateLimitError

PROVIDERS = [   # first that works wins; edit without touching code
    {"base_url": "https://api.groq.com/openai/v1",   "key": os.environ["GROQ_API_KEY"],      "model": os.getenv("PRIMARY_MODEL", "openai/gpt-oss-120b")},
    {"base_url": "https://api.inferenceapis.com/v1", "key": os.environ["INFERENCE_API_KEY"], "model": os.getenv("FALLBACK_MODEL", "openai/gpt-oss-120b")},
]

def chat(messages):
    last = None
    for p in PROVIDERS:
        try:
            return OpenAI(base_url=p["base_url"], api_key=p["key"]).chat.completions.create(model=p["model"], messages=messages, max_tokens=1024)
        except (NotFoundError, RateLimitError) as e:      # retired model or exhausted quota: try the next provider
            last = e
    raise last

Frequently asked questions

Will Groq bring llama-3.3-70b-versatile back?

There is no sign of it. Every earlier entry on Groq's deprecations page was permanent, and the model now appears only under enterprise contracts.

Is openai/gpt-oss-120b better than Llama 3.3 70B?

It is newer, reasons before answering and is cheaper to serve, and Groq chose it as the replacement. Whether it is better for your prompts is something only your own test cases can answer, which is why keeping Llama elsewhere is a legitimate option.

Why a 404 and not a clearer "retired" error?

OpenAI-style APIs return the same model_not_found for ids that never existed, ids that were retired and ids your key may not use. The general causes are covered in the model_not_found entry.

My framework picked the model, not me. Where do I change it?

Pass the model explicitly: ChatGroq(model=...) in LangChain, LLM(model="groq/...") in CrewAI, model_list in a LiteLLM config. Tutorials written before August 2026 often hard-code the retired id.

Where Inference APIs fits

Option 2 above keeps the same model on another provider. Inference APIs serves Llama 3.3 70B and accepts the exact id llama-3.3-70b-versatile, so the change is the base URL and key. It is also the most expensive chat model we serve; if you are not tied to Llama, Option 1's model costs less on either host.

Related entries

• 404 model_not_found: "The model … does not exist or you do not have access to it"
• HTTP 429 rate_limit_exceeded (Groq)
• Groq: "Developer tier upgrades are temporarily unavailable"