Inference APIs

404 model_not_found: llama3-70b-8192 and llama3-8b-8192 (Groq)

The Llama 3 ids with -8192 in the name were Groq's launch models and the ids in its earliest quickstarts, so 404 The model `llama3-70b-8192` does not exist or you do not have access to it comes from code written in 2024 that nobody has touched since. Groq retired both llama3-70b-8192 and llama3-8b-8192 on 30 August 2025. The replacements it named then, llama-3.3-70b-versatile and llama-3.1-8b-instant, were retired on 16 August 2026, so a 2024 app has two retirements to cross, not one.

Last verified November 18, 2026 against Groq deprecations, Groq model list · 5 min read

ProviderGroq
HTTP status404 Not Found
Error codemodel_not_found
MessageThe model `llama3-70b-8192` does not exist or you do not have access to it.
Announced / shut downabout a month earlier, by email and on the deprecations page / 30 August 2025
When it happensSince 2025-08-30, on any self-serve request that names either model
Can you wait it out?No — the model was retired; the request fails until you change the model or the provider
Short answer
  • Fastest: change the model string to llama-3.3-70b-versatile on Groq and raise max_tokens, because it reasons before answering. Different model, so re-check your prompts.
  • Least code change: keep llama3-70b-8192 and change only the base URL; Inference APIs accepts the id and answers with Llama 3.3 70B.
  • So it does not happen again: read the model id from configuration and add a fallback for 404 and 429.

What happened

Groq's deprecations page lists the notice: shut down 30 August 2025 for all self-serve tiers, llama3-70b-8192 replaced by llama-3.3-70b-versatile and llama3-8b-8192 by llama-3.1-8b-instant. The same page lists those two replacements as retired on 16 August 2026 for free and developer tiers, with openai/gpt-oss-120b and openai/gpt-oss-20b as the current recommendations. The 8192 in the old ids was the context window; everything that replaced them takes 128K or more.

First, find where the id is pinned

The id is often in more places than the one that errored: a default in a settings file, an environment variable on the server, a second call for summarisation or titles, a framework default.

Find every copy of the id
grep -rn -E "llama3-70b-8192|llama3-8b-8192" . --include='*.py' --include='*.ts' --include='*.js' --include='*.json' --include='*.yaml' --include='*.yml' --include='*.toml' --include='*.env*'
# and the places grep cannot see: environment variables on the server, a hosted secrets store, the provider dashboard's default model

Option 1 — switch to Groq's recommended replacement

Skip the retired intermediate ids and go to the current end of the chain: openai/gpt-oss-120b for the 70B, openai/gpt-oss-20b for the 8B. What is different from Llama 3:

  • They reason before answering. Llama 3 did not. Raise max_tokens or send reasoning_effort: "low", or short budgets return empty content.
  • The window is 131K, not 8K. Code that trimmed history to fit 8,192 tokens can stop doing so.
  • Style differs; 2024-era prompts with explicit formatting instructions usually survive, few-shot parsers need a check.
One string changes
resp = client.chat.completions.create(
    model="llama-3.3-70b-versatile",   # was "llama3-70b-8192"
    max_tokens=800,                 # reasoning counts against it
    messages=[{"role": "user", "content": "Hello"}],
)

Option 2 — keep the old id, change the base URL

Llama 3.3 70B is the direct descendant of Llama 3 70B, open-weight, and still served. Inference APIs accepts llama3-70b-8192, llama3-8b-8192, llama-3.1-70b-versatile and the other retired Llama ids as aliases and serves them all with Llama 3.3 70B: an instruct model with no thinking phase, so 2024 prompts and token budgets work as written, a 131,072-token window, tool calling and JSON output, at $0.20 / $0.50 per million tokens.

Base URL and key change, model string stays
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",   # was https://api.groq.com/openai/v1
    api_key=os.environ["INFERENCE_API_KEY"],
)
resp = client.chat.completions.create(
    model="llama3-70b-8192",   # the retired Groq id, accepted as an alias
    messages=[{"role": "user", "content": "Hello"}],
)

It is served as the FP8 Turbo variant, so wording can differ slightly from Groq's 2024 endpoint on identical prompts, and the 8B users get a 70B at the 70B price; for a small cheap model send openai/gpt-oss-20b at $0.04 / $0.19 instead. Prices and capabilities: Llama 3.3 70B here.

Option 3 — stop pinning one provider and one id

Put the model id in configuration and let a 404 or 429 fall through to a second provider. A retirement then degrades to a log line instead of an outage, and you choose when to migrate. LiteLLM's router does this in a few lines; the same shape exists in most frameworks.

LiteLLM proxy: Groq first, fall back on 404/429
# LiteLLM proxy config: Groq first, Inference APIs when Groq answers 404 or 429
model_list:
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY
  - model_name: chat-fallback
    litellm_params:
      model: inferenceapis/openai/gpt-oss-120b
      api_key: os.environ/INFERENCEAPIS_API_KEY

router_settings:
  fallbacks: [{ "chat": ["chat-fallback"] }]

The deprecations tracker lists every Groq retirement with its date and replacement, so you can see the next one coming.

Frequently asked questions

Why does the error say the model does not exist when it clearly did?

OpenAI-style APIs return the same model_not_found for ids that never existed, ids that were retired and ids your key may not use. The general causes are on the model_not_found page; here the cause is retirement.

Should I move to llama-3.3-70b-versatile as the 2025 notice said?

Not on Groq: that id was retired for self-serve tiers on 16 August 2026. On Inference APIs it is accepted and is the same model the old ids resolve to, so either string works here.

Can I keep the 8,192-token limit in my code?

You can, it just wastes capability. Every replacement takes at least 128K tokens; Llama 3.3 70B here takes 131,072.

Where Inference APIs fits

Option 2 above keeps the old id working with a base-URL change: Inference APIs accepts llama3-70b-8192 and serves it with Llama 3.3 70B at $0.20 / $0.50 per million tokens, billed per request with no daily cap.

Something changed or wrong? Tell us and we will re-verify the entry.