404 model_not_found: gemma2-9b-it (Groq)
Gemma 2 9B was the small, permissively licensed model on Groq's free tier, and 404 The model `gemma2-9b-it` does not exist or you do not have access to it is what its users have seen since 8 October 2025. The retirement is a year old, but the id is still hard-coded in enough tutorials, Discord bots and classroom projects that it keeps surfacing. The replacement Groq named at the time, llama-3.1-8b-instant, has since been retired as well, so following the original notice no longer works.
Last verified November 18, 2026 against Groq deprecations, Groq model list · 5 min read
| Provider | Groq |
| HTTP status | 404 Not Found |
| Error code | model_not_found |
| Message | The model `gemma2-9b-it` does not exist or you do not have access to it. |
| Announced / shut down | 8 August 2025 / 8 October 2025 |
| When it happens | Since 2025-10-08, on any self-serve request that names this model |
| Can you wait it out? | No — the model was retired; the request fails until you change the model or the provider |
- Fastest: change the model string to
llama-3.1-8b-instanton Groq and raisemax_tokens, because it reasons before answering. Different model, so re-check your prompts. - Least code change: keep
gemma2-9b-itand change only the base URL; Inference APIs accepts the id and answers with Llama 3.3 70B. - So it does not happen again: read the model id from configuration and add a fallback for 404 and 429.
What happened
Groq's deprecations page lists the notice: announced 8 August 2025, shut down 8 October 2025, all self-serve tiers, replacement llama-3.1-8b-instant. That replacement was retired on 16 August 2026 for free and developer tiers in favour of openai/gpt-oss-20b. Gemma 7B had gone the same way in December 2024, replaced by Gemma 2 9B. Two hops in under two years is typical of Groq's small-model slot; the deprecations tracker has the chain.
First, find where the id is pinned
The id is often in more places than the one that errored: a default in a settings file, an environment variable on the server, a second call for summarisation or titles, a framework default.
grep -rn -E "gemma2-9b-it" . --include='*.py' --include='*.ts' --include='*.js' --include='*.json' --include='*.yaml' --include='*.yml' --include='*.toml' --include='*.env*'
# and the places grep cannot see: environment variables on the server, a hosted secrets store, the provider dashboard's default modelOption 1 — switch to Groq's recommended replacement
The current end of the chain on Groq is openai/gpt-oss-20b. What is different from Gemma 2 9B:
- It reasons before answering. Give it
max_tokensof 600 or more, or sendreasoning_effort: "low"; a small budget returns emptycontent. - Licensing. Gemma's terms were a reason some teams chose it; GPT-OSS is Apache 2.0, which is more permissive, so that is rarely a blocker.
- Tool calling and JSON mode work on GPT-OSS; Gemma 2 on Groq had no tool calling, so this is a gain.
resp = client.chat.completions.create(
model="llama-3.1-8b-instant", # was "gemma2-9b-it"
max_tokens=800, # reasoning counts against it
messages=[{"role": "user", "content": "Hello"}],
)Option 2 — keep the old id, change the base URL
No pay-as-you-go host we know of serves Gemma 2 9B under this id any more. Inference APIs accepts gemma2-9b-it (and gemma-7b-it) so that a broken app answers again, routing it to Llama 3.3 70B at $0.20 / $0.50 per million tokens: a larger instruct model with no thinking phase, so prompts and max_tokens written for Gemma keep working unchanged.
from openai import OpenAI
client = OpenAI(
base_url="https://api.inferenceapis.com/v1", # was https://api.groq.com/openai/v1
api_key=os.environ["INFERENCE_API_KEY"],
)
resp = client.chat.completions.create(
model="gemma2-9b-it", # the retired Groq id, accepted as an alias
messages=[{"role": "user", "content": "Hello"}],
)Gemma was chosen for being small and cheap. If that is the point, send openai/gpt-oss-20b here instead at $0.04 / $0.19 with the larger token budget described above; the alias is for getting unblocked, not for staying. Prices and capabilities: Llama 3.3 70B here.
Option 3 — stop pinning one provider and one id
Put the model id in configuration and let a 404 or 429 fall through to a second provider. A retirement then degrades to a log line instead of an outage, and you choose when to migrate. LiteLLM's router does this in a few lines; the same shape exists in most frameworks.
# LiteLLM proxy config: Groq first, Inference APIs when Groq answers 404 or 429
model_list:
- model_name: chat
litellm_params:
model: groq/openai/gpt-oss-120b
api_key: os.environ/GROQ_API_KEY
- model_name: chat-fallback
litellm_params:
model: inferenceapis/openai/gpt-oss-120b
api_key: os.environ/INFERENCEAPIS_API_KEY
router_settings:
fallbacks: [{ "chat": ["chat-fallback"] }]The deprecations tracker lists every Groq retirement with its date and replacement, so you can see the next one coming.
Frequently asked questions
Will Groq host Gemma 3?
Groq's current list has no Gemma model. Check the list rather than assuming; it changes every month or two.
Why does a one-year-old retirement still matter?
Because the id lives in code that is copied, not written: tutorials, starter kits and bots. Searching your dependencies for the string finds more copies than searching your own files.
Can I keep Gemma's prompts on Llama 3.3 70B?
Mostly yes. Both are instruct models without a thinking phase, so prompts, max_tokens and output parsers carry over. Llama is larger and more capable, so answers tend to be longer; set max_tokens deliberately.
Where Inference APIs fits
Option 2 above keeps the old id working with a base-URL change: Inference APIs accepts gemma2-9b-it and serves it with Llama 3.3 70B at $0.20 / $0.50 per million tokens, billed per request with no daily cap. Groq's own replacement, openai/gpt-oss-20b, is served here too at $0.04 / $0.19.
Something changed or wrong? Tell us and we will re-verify the entry.
