Inference APIs

404 model_not_found: llama-3.1-8b-instant (Groq)

If the smallest, fastest model on Groq's free tier started answering 404 The model `llama-3.1-8b-instant` does not exist or you do not have access to it, nothing changed on your side. Groq retired llama-3.1-8b-instant for free and developer-tier accounts on 16 August 2026, in the same notice that retired llama-3.3-70b-versatile. Because the 8B was the default in a large share of tutorials, bots and starter templates, this is the retirement people hit without ever having chosen the model.

Last verified November 18, 2026 against Groq deprecations, Groq model list · 6 min read

ProviderGroq
HTTP status404 Not Found
Error codemodel_not_found
MessageThe model `llama-3.1-8b-instant` does not exist or you do not have access to it.
Announced / shut down17 June 2026 / 16 August 2026
When it happensSince 2026-08-16, on any free or developer-tier request that names this model
Can you wait it out?No — the model was retired; the request fails until you change the model or the provider
Short answer
  • Fastest: change the model string to openai/gpt-oss-20b on Groq and raise max_tokens, because it reasons before answering. Different model, so re-check your prompts.
  • Least code change: keep llama-3.1-8b-instant and change only the base URL; Inference APIs accepts the id and answers with Llama 3.3 70B.
  • So it does not happen again: read the model id from configuration and add a fallback for 404 and 429.

What happened

Groq's deprecations page lists the notice: announced 17 June 2026, shut down 16 August 2026, covering llama-3.1-8b-instant and llama-3.3-70b-versatile for free and developer-tier usage; enterprise contracts were not affected. The recommended replacement for the 8B is openai/gpt-oss-20b.

This id was itself the replacement Groq named when it retired gemma2-9b-it in October 2025 and the Llama 3.2 1B and 3B previews in April 2025, so apps that followed those notices have now moved twice. The full chain is in the deprecations tracker.

First, find where the id is pinned

The id is often in more places than the one that errored: a default in a settings file, an environment variable on the server, a second call for summarisation or titles, a framework default.

Find every copy of the id
grep -rn -E "llama-3.1-8b-instant" . --include='*.py' --include='*.ts' --include='*.js' --include='*.json' --include='*.yaml' --include='*.yml' --include='*.toml' --include='*.env*'
# and the places grep cannot see: environment variables on the server, a hosted secrets store, the provider dashboard's default model

Option 1 — switch to Groq's recommended replacement

One string changes and you stay on Groq's free tier. What is different about openai/gpt-oss-20b:

  • It reasons before answering. Thinking counts against max_tokens. A budget of 100 or 200 that was plenty for an 8B instruct model can come back with empty content and only a reasoning field; give it 600 or more, or send reasoning_effort: "low".
  • It is a 21B mixture-of-experts model, so answers are generally stronger than Llama 3.1 8B, but the style differs. Prompts tuned to the 8B's habits, few-shot examples and output parsers need a test pass.
  • Tool calling and JSON mode work on both, and the context window grows from 128K to 131K.
One string changes
resp = client.chat.completions.create(
    model="openai/gpt-oss-20b",   # was "llama-3.1-8b-instant"
    max_tokens=800,                 # reasoning counts against it
    messages=[{"role": "user", "content": "Hello"}],
)

Option 2 — keep the old id, change the base URL

Few hosts still serve Llama 3.1 8B at all. Inference APIs does not either, but it accepts the id so that a broken app starts answering again: a request for llama-3.1-8b-instant is served by Llama 3.3 70B, a larger model from the same family that follows the same prompts, at $0.20 / $0.50 per million tokens. That is a stop-gap, not the cheap option.

The cheap option is Groq's own replacement on a pay-as-you-go endpoint: openai/gpt-oss-20b is served here at $0.04 / $0.19 per million tokens, with no daily token cap and per-key spend limits if you want a ceiling. Both routes are the same two-line change:

Base URL and key change, model string stays
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",   # was https://api.groq.com/openai/v1
    api_key=os.environ["INFERENCE_API_KEY"],
)
resp = client.chat.completions.create(
    model="llama-3.1-8b-instant",   # the retired Groq id, accepted as an alias
    messages=[{"role": "user", "content": "Hello"}],
)

The 8B was chosen for being cheap and fast. If that is why you used it, send openai/gpt-oss-20b rather than the alias, and give it the larger max_tokens described above. Prices and capabilities: Llama 3.3 70B here.

Option 3 — stop pinning one provider and one id

Put the model id in configuration and let a 404 or 429 fall through to a second provider. A retirement then degrades to a log line instead of an outage, and you choose when to migrate. LiteLLM's router does this in a few lines; the same shape exists in most frameworks.

LiteLLM proxy: Groq first, fall back on 404/429
# LiteLLM proxy config: Groq first, Inference APIs when Groq answers 404 or 429
model_list:
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY
  - model_name: chat-fallback
    litellm_params:
      model: inferenceapis/openai/gpt-oss-120b
      api_key: os.environ/INFERENCEAPIS_API_KEY

router_settings:
  fallbacks: [{ "chat": ["chat-fallback"] }]

The deprecations tracker lists every Groq retirement with its date and replacement, so you can see the next one coming.

Frequently asked questions

Does anyone still host llama-3.1-8b-instant?

Not under that id. A few hosts serve the underlying meta-llama/Llama-3.1-8B-Instruct weights; the Groq id itself exists only on Groq's enterprise tier now. On Inference APIs the id is accepted and routed to Llama 3.3 70B.

Why did my max_tokens=100 request come back empty on gpt-oss-20b?

It is a reasoning model and the thinking is billed as output. With a small budget it runs out before writing the answer. Raise max_tokens to 600 or more or set reasoning_effort to low.

Is gpt-oss-20b cheaper than the 8B was?

On Groq's free tier both cost nothing until you hit the daily token cap, which reasoning tokens consume faster. On a pay-as-you-go endpoint GPT-OSS 20B is among the cheapest chat models available: $0.04 input / $0.19 output per million tokens here.

My framework picked llama-3.1-8b-instant for me. Where is it set?

Pass the model explicitly: ChatGroq(model=...) in LangChain, LLM(model="groq/...") in CrewAI, model_list in a LiteLLM config. Many 2025 tutorials and templates default to this id, so search your dependencies as well as your own code.

Where Inference APIs fits

Option 2 above keeps the old id working with a base-URL change: Inference APIs accepts llama-3.1-8b-instant and serves it with Llama 3.3 70B at $0.20 / $0.50 per million tokens, billed per request with no daily cap. Groq's own replacement, openai/gpt-oss-20b, is served here too at $0.04 / $0.19.

Something changed or wrong? Tell us and we will re-verify the entry.