Inference APIs
Models/Groq alternative

The change


Warning: Undefined variable $INFERENCE_API_KEY in /home/inferenceapis/web/groq-alternative.php on line 56
/home/inferenceapis/web/includes/layout.php on line 360
">
Warning: Undefined array key "code" in /home/inferenceapis/web/includes/layout.php on line 360
/home/inferenceapis/web/includes/layout.php on line 360
">
Warning: Undefined array key "code" in /home/inferenceapis/web/includes/layout.php on line 360
/home/inferenceapis/web/includes/layout.php on line 360
">
Warning: Undefined array key "code" in /home/inferenceapis/web/includes/layout.php on line 360

Both /v1 and /openai/v1 are accepted, so a Groq-shaped base URL works as written. Streaming, tool calling, JSON mode and the Responses API behave as on Groq; the reasoning models (GPT-OSS, DeepSeek, Qwen, Kimi) return their thinking in a separate field and honour reasoning_effort, including "none" to switch it off.

Every retired Groq id, and what answers it here

Groq's own replacements have themselves been retired twice over (gemma2-9b-it to llama-3.1-8b-instant to openai/gpt-oss-20b), so an app that followed each notice has moved three times. Here the id you already have keeps working. Where we do not serve the exact model, the alias goes to the closest one we do, and the linked page says what differs. Shutdown dates are from Groq's deprecations page, checked 2026-11-18.

Groq idGroq shut it downServed here asPrice per 1M tokens, in / out
llama-3.3-70b-versatile what changed2026-08-16meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
llama-3.1-8b-instant what changed2026-08-16meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
llama3-70b-8192 what changed2025-08-30meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
llama3-8b-8192 what changed2025-08-30meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
llama-3.1-70b-versatile2025-01-24meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
gemma2-9b-it what changed2025-10-08meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
mixtral-8x7b-32768 what changed2025-03-20meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
qwen/qwen3-32b what changed2026-07-17Qwen/Qwen3.8-Flash$0.15 / $0.50
qwen/qwen3.6-27b what changed2026-09-14Qwen/Qwen3.8-Flash$0.15 / $0.50
qwen-qwq-32b2025-07-14Qwen/Qwen3.8-Flash$0.15 / $0.50
meta-llama/llama-4-scout-17b-16e-instruct what changed2026-07-17Qwen/Qwen3-VL-235B-A22B-Instruct$0.26 / $1.15
meta-llama/llama-4-maverick-17b-128e-instruct what changed2026-03-09Qwen/Qwen3-VL-235B-A22B-Instruct$0.26 / $1.15
moonshotai/kimi-k2-instruct what changed2025-10-10moonshotai/Kimi-K2.7-Code$0.89 / $4.42
moonshotai/kimi-k2-instruct-0905 what changed2026-04-15moonshotai/Kimi-K2.7-Code$0.89 / $4.42
deepseek-r1-distill-llama-70b what changed2025-10-02deepseek-ai/DeepSeek-V4-Flash$0.19 / $0.38
distil-whisper-large-v3-en2025-08-23openai/whisper-large-v3$0.0020 per minute

Current Groq ids (openai/gpt-oss-120b, openai/gpt-oss-20b) are served under the same names. The full retirement history, including ids we do not alias, is in the deprecations tracker.

What no daily cap means in money

Groq's free tier stops at a daily token allowance and the developer tier has had a waiting list; this endpoint has no per-minute or per-day limits, only your balance. Per-key and per-account spend limits are there if you want a ceiling of your own. For a sense of scale, at Llama 3.3 70B's $0.20 input / $0.50 output per million tokens:

WorkloadTokensCost
One short classification call (500 in, 70 out)570$0.00014
A nightly batch of 10,000 such calls5.7Mabout $1.40
A chat app at 2,000 turns a day, 1,500 in / 300 out each3.6M a dayabout $1.05 a day
The same on GPT-OSS 20B ($0.04 / $0.19)3.6M a dayabout $0.23 a day

Cached input (a repeated prompt prefix) is billed at a lower rate on the DeepSeek, GLM, Qwen and Kimi models automatically.

Keep Groq first, send the overflow here

The config most people land on: Groq stays the primary because it is free and fast; when it answers 404 (retired id), 429 (daily cap) or 503, the request goes to the same model id here. LiteLLM has had Inference APIs as a built-in provider since 1.107; the shape is the same in any router.

LiteLLM proxy config
model_list:
  - model_name: chat
    litellm_params:
      model: groq/openai/gpt-oss-120b
      api_key: os.environ/GROQ_API_KEY
  - model_name: chat-fallback
    litellm_params:
      model: inferenceapis/openai/gpt-oss-120b
      api_key: os.environ/INFERENCEAPIS_API_KEY

router_settings:
  fallbacks: [{ "chat": ["chat-fallback"] }]

Guides: LiteLLM, LangChain, OpenAI Python SDK, all integrations.

Where Groq is still the better choice

  • Speed. Groq's hardware is several times faster per token than serverless GPUs. If latency is the product, stay on Groq while its free tier fits.
  • Free usage that fits the cap. If you never hit the daily limit and the current ids suit you, Groq costs nothing and we cost cents. The retirement pages say this plainly.
  • Models we do not serve. Groq's Llama Guard, Compound and speech-to-speech systems have no equivalent here; the tracker marks those rows.

Frequently asked questions

Is this a free tier like Groq's?

No. There is no free tier and no daily cap: you add funds and pay per token, from $0.04 / $0.19 per million tokens for GPT-OSS 20B. A busy hobby project costs cents a month; a job that would blow through Groq's daily token allowance costs what its tokens cost and keeps running.

Do I have to change my model string?

Not if it is a Groq id. Every id Groq has retired since 2024 is accepted and routed to the closest model we serve, and the current Groq ids (openai/gpt-oss-120b, openai/gpt-oss-20b) are the same here. Both /v1 and /openai/v1 base paths work.

Is it as fast as Groq?

No. Groq runs its own hardware and publishes 280 to 1,000 tokens per second; we serve open models on serverless GPUs at roughly 70 to 250 tokens per second depending on the model, measured on our own gateway. What you get instead is no rate windows, no tier to unlock and no model retirements without an alias.

Which Groq refugees actually stay?

The ones Groq's free tier could not hold: nightly batch and research jobs that exceed the daily token cap, apps in production that need a model Groq retired, and anyone who needs Llama 3.3 70B specifically. If Groq's free tier still fits your usage, it is the cheaper option and we say so on every retirement page.

What about the 429 rate_limit_exceeded and the developer-tier wait?

There are no per-minute or per-day limits on your requests or tokens here, and no tier to apply for. The Groq 429 page explains which Groq limit you hit; the fallback config below keeps Groq first and sends overflow here.