404 model_not_found: llama-3.1-8b-instant (Groq)
If the smallest, fastest model on Groq's free tier started answering 404 The model `llama-3.1-8b-instant` does not exist or you do not have access to it, nothing changed on your side. Groq retired llama-3.1-8b-instant for free and developer-tier accounts on 16 August 2026, in the same notice that retired llama-3.3-70b-versatile. Because the 8B was the default in a large share of tutorials, bots and starter templates, this is the retirement people hit without ever having chosen the model.
Last verified November 18, 2026 against Groq deprecations, Groq model list · 6 min read
| Provider | Groq |
| HTTP status | 404 Not Found |
| Error code | model_not_found |
| Message | The model `llama-3.1-8b-instant` does not exist or you do not have access to it. |
| Announced / shut down | 17 June 2026 / 16 August 2026 |
| When it happens | Since 2026-08-16, on any free or developer-tier request that names this model |
| Can you wait it out? | No — the model was retired; the request fails until you change the model or the provider |
- Fastest: change the model string to
openai/gpt-oss-20bon Groq and raisemax_tokens, because it reasons before answering. Different model, so re-check your prompts. - Least code change: keep
llama-3.1-8b-instantand change only the base URL; Inference APIs accepts the id and answers with Llama 3.3 70B. - So it does not happen again: read the model id from configuration and add a fallback for 404 and 429.
What happened
Groq's deprecations page lists the notice: announced 17 June 2026, shut down 16 August 2026, covering llama-3.1-8b-instant and llama-3.3-70b-versatile for free and developer-tier usage; enterprise contracts were not affected. The recommended replacement for the 8B is openai/gpt-oss-20b.
This id was itself the replacement Groq named when it retired gemma2-9b-it in October 2025 and the Llama 3.2 1B and 3B previews in April 2025, so apps that followed those notices have now moved twice. The full chain is in the deprecations tracker.
First, find where the id is pinned
The id is often in more places than the one that errored: a default in a settings file, an environment variable on the server, a second call for summarisation or titles, a framework default.
grep -rn -E "llama-3.1-8b-instant" . --include='*.py' --include='*.ts' --include='*.js' --include='*.json' --include='*.yaml' --include='*.yml' --include='*.toml' --include='*.env*'
# and the places grep cannot see: environment variables on the server, a hosted secrets store, the provider dashboard's default modelOption 1 — switch to Groq's recommended replacement
One string changes and you stay on Groq's free tier. What is different about openai/gpt-oss-20b:
- It reasons before answering. Thinking counts against
max_tokens. A budget of 100 or 200 that was plenty for an 8B instruct model can come back with emptycontentand only areasoningfield; give it 600 or more, or sendreasoning_effort: "low". - It is a 21B mixture-of-experts model, so answers are generally stronger than Llama 3.1 8B, but the style differs. Prompts tuned to the 8B's habits, few-shot examples and output parsers need a test pass.
- Tool calling and JSON mode work on both, and the context window grows from 128K to 131K.
resp = client.chat.completions.create(
model="openai/gpt-oss-20b", # was "llama-3.1-8b-instant"
max_tokens=800, # reasoning counts against it
messages=[{"role": "user", "content": "Hello"}],
)Option 2 — keep the old id, change the base URL
Few hosts still serve Llama 3.1 8B at all. Inference APIs does not either, but it accepts the id so that a broken app starts answering again: a request for llama-3.1-8b-instant is served by Llama 3.3 70B, a larger model from the same family that follows the same prompts, at $0.20 / $0.50 per million tokens. That is a stop-gap, not the cheap option.
The cheap option is Groq's own replacement on a pay-as-you-go endpoint: openai/gpt-oss-20b is served here at $0.04 / $0.19 per million tokens, with no daily token cap and per-key spend limits if you want a ceiling. Both routes are the same two-line change:
from openai import OpenAI
client = OpenAI(
base_url="https://api.inferenceapis.com/v1", # was https://api.groq.com/openai/v1
api_key=os.environ["INFERENCE_API_KEY"],
)
resp = client.chat.completions.create(
model="llama-3.1-8b-instant", # the retired Groq id, accepted as an alias
messages=[{"role": "user", "content": "Hello"}],
)The 8B was chosen for being cheap and fast. If that is why you used it, send openai/gpt-oss-20b rather than the alias, and give it the larger max_tokens described above. Prices and capabilities: Llama 3.3 70B here.
Option 3 — stop pinning one provider and one id
Put the model id in configuration and let a 404 or 429 fall through to a second provider. A retirement then degrades to a log line instead of an outage, and you choose when to migrate. LiteLLM's router does this in a few lines; the same shape exists in most frameworks.
# LiteLLM proxy config: Groq first, Inference APIs when Groq answers 404 or 429
model_list:
- model_name: chat
litellm_params:
model: groq/openai/gpt-oss-120b
api_key: os.environ/GROQ_API_KEY
- model_name: chat-fallback
litellm_params:
model: inferenceapis/openai/gpt-oss-120b
api_key: os.environ/INFERENCEAPIS_API_KEY
router_settings:
fallbacks: [{ "chat": ["chat-fallback"] }]The deprecations tracker lists every Groq retirement with its date and replacement, so you can see the next one coming.
Frequently asked questions
Does anyone still host llama-3.1-8b-instant?
Not under that id. A few hosts serve the underlying meta-llama/Llama-3.1-8B-Instruct weights; the Groq id itself exists only on Groq's enterprise tier now. On Inference APIs the id is accepted and routed to Llama 3.3 70B.
Why did my max_tokens=100 request come back empty on gpt-oss-20b?
It is a reasoning model and the thinking is billed as output. With a small budget it runs out before writing the answer. Raise max_tokens to 600 or more or set reasoning_effort to low.
Is gpt-oss-20b cheaper than the 8B was?
On Groq's free tier both cost nothing until you hit the daily token cap, which reasoning tokens consume faster. On a pay-as-you-go endpoint GPT-OSS 20B is among the cheapest chat models available: $0.04 input / $0.19 output per million tokens here.
My framework picked llama-3.1-8b-instant for me. Where is it set?
Pass the model explicitly: ChatGroq(model=...) in LangChain, LLM(model="groq/...") in CrewAI, model_list in a LiteLLM config. Many 2025 tutorials and templates default to this id, so search your dependencies as well as your own code.
Where Inference APIs fits
Option 2 above keeps the old id working with a base-URL change: Inference APIs accepts llama-3.1-8b-instant and serves it with Llama 3.3 70B at $0.20 / $0.50 per million tokens, billed per request with no daily cap. Groq's own replacement, openai/gpt-oss-20b, is served here too at $0.04 / $0.19.
Something changed or wrong? Tell us and we will re-verify the entry.
