Inference APIs
Models/Together alternative

The error

Together, any retired serverless id (HTTP 400)
{"message": "Unable to access non-serverless model meta-llama/Llama-3.3-70B-Instruct-Turbo. Please visit https://api.together.ai/models/meta-llama/Llama-3.3-70B-Instruct-Turbo to create and start a new dedicated endpoint for the model.", "type": "invalid_request_error", "param": null, "code": null}

Some ids answer with endpoint_not_configured (Endpoint has no deployments configured for routing) or endpoint_not_ready instead; the cause is the same. Nothing is wrong with the request or the key.

The change

from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",   # was https://api.together.xyz/v1
    api_key=os.environ["INFERENCEAPIS_API_KEY"],   # was TOGETHER_API_KEY
)
resp = client.chat.completions.create(
    model="meta-llama/Llama-3.3-70B-Instruct-Turbo",   # unchanged
    messages=[{"role": "user", "content": "Hello"}],
)
from together import Together

client = Together(
    base_url="https://api.inferenceapis.com/v1",
    api_key=os.environ["INFERENCEAPIS_API_KEY"],
)
resp = client.chat.completions.create(
    model="openai/gpt-oss-20b",   # unchanged
    messages=[{"role": "user", "content": "Hello"}],
)
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inferenceapis.com/v1",   // was https://api.together.xyz/v1
  apiKey: process.env.INFERENCEAPIS_API_KEY,
});
const resp = await client.chat.completions.create({
  model: "moonshotai/Kimi-K2.6",   // unchanged
  messages: [{ role: "user", content: "Hello" }],
});
curl https://api.inferenceapis.com/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCEAPIS_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"deepseek-ai/DeepSeek-V4-Flash-0731","messages":[{"role":"user","content":"Hello"}]}'

Streaming, tool calling, JSON mode, the Responses API and the Anthropic Messages shape all work. Reasoning models return their thinking in a separate field and honour reasoning_effort. Every response carries an X-Cost-USD header and usage.cost.

Retired Together ids, and what answers them here

Together publishes removal dates but, for serverless retirements, no replacement. Where we serve the exact model the row says so; otherwise the id is routed to the closest model we serve, and you can of course send that model's own id instead.

Together idRemoved from serverlessServed here asPrice per 1M tokens, in / out
meta-llama/Llama-3.3-70B-Instruct-Turbo what changed2026-10-22same model$0.20 / $0.50
deepseek-ai/DeepSeek-V4-Flash-0731 what changed2026-10-22deepseek-ai/DeepSeek-V4-Flash$0.19 / $0.38
openai/gpt-oss-20b what changed2026-09-15same model$0.04 / $0.19
moonshotai/Kimi-K2.6 what changed2026-08-19same model$0.98 / $4.55
moonshotai/Kimi-K2.7-Code2026-08-27same model$0.89 / $4.42
mistralai/Mixtral-8x7B-Instruct-v0.1 what changed2026-04-16meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo what changed2026-03-06meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo2026-02-25meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo2026-02-06meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50
meta-llama/Llama-4-Scout-17B-16E-Instruct what changed2026-02-06Qwen/Qwen3-VL-235B-A22B-Instruct$0.26 / $1.15
meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP82026-03-31Qwen/Qwen3-VL-235B-A22B-Instruct$0.26 / $1.15
deepseek-ai/DeepSeek-R12026-05-14deepseek-ai/DeepSeek-V4-Flash$0.19 / $0.38
deepseek-ai/DeepSeek-V3.12026-05-14deepseek-ai/DeepSeek-V4-Flash$0.19 / $0.38
deepseek-ai/DeepSeek-R1-Distill-Llama-70B2025-12-23deepseek-ai/DeepSeek-V4-Flash$0.19 / $0.38
Qwen/Qwen2.5-72B-Instruct-Turbo2026-02-06Qwen/Qwen3.8-Flash$0.15 / $0.50
Qwen/QwQ-32B2025-11-13Qwen/Qwen3.8-Flash$0.15 / $0.50
Qwen/Qwen2.5-Coder-32B-Instruct2025-11-13Qwen/Qwen3.8-Flash$0.15 / $0.50
Qwen/Qwen3-235B-A22B-Instruct-2507-tput2026-07-10Qwen/Qwen3.8-Flash$0.15 / $0.50
Qwen/Qwen3-Next-80B-A3B-Instruct2026-04-02Qwen/Qwen3.8-Flash$0.15 / $0.50
moonshotai/Kimi-K2-Instruct-09052026-03-06moonshotai/Kimi-K2.7-Code$0.89 / $4.42
moonshotai/Kimi-K2.52026-05-21moonshotai/Kimi-K2.6$0.98 / $4.55
zai-org/GLM-5.12026-07-10zai-org/GLM-5.3$1.82 / $5.72
zai-org/GLM-52026-07-10zai-org/GLM-5.3$1.82 / $5.72
google/gemma-4-31B-it2026-09-15zai-org/GLM-5.3-Flash$0.20 / $0.66
meta-llama/Llama-3.3-70B-Instruct-Turbo-Free2025-11-13meta-llama/Llama-3.3-70B-Instruct-Turbo$0.20 / $0.50

The full list with dates is in the deprecations tracker; it is checked against Together's page daily. Current Together ids we also serve under the same name: openai/gpt-oss-120b, deepseek-ai/DeepSeek-V4-Pro, zai-org/GLM-5.3, Qwen/Qwen3-VL-235B-A22B-Instruct, moonshotai/Kimi-K3.1-Pro, MiniMaxAI/MiniMax-M3.1-Flash.

Serverless price versus a dedicated endpoint

Together's suggested fix is a dedicated endpoint, billed per GPU hour while it runs. The serverless price here is per token. At Llama 3.3 70B's $0.20 input / $0.50 output per million tokens:

WorkloadTokensCost here
A nightly batch of 10,000 classification calls (500 in, 70 out)5.7Mabout $1.40
A chat app at 2,000 turns a day, 1,500 in / 300 out each3.6M a dayabout $1.05 a day
The same on GPT-OSS 20B ($0.04 / $0.19)3.6M a dayabout $0.23 a day
An hour of idle dedicated endpoint0$0 here; a GPU hour on Together

Or keep Together and fall through on 400

If Together still serves most of what you need, keep it first and route the retired ids here. LiteLLM has had Inference APIs as a built-in provider since 1.107; any router with fallbacks does the same.

LiteLLM proxy config
model_list:
  - model_name: llama-70b
    litellm_params:
      model: together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
      api_key: os.environ/TOGETHER_API_KEY
  - model_name: llama-70b-fallback
    litellm_params:
      model: inferenceapis/meta-llama/Llama-3.3-70B-Instruct-Turbo
      api_key: os.environ/INFERENCEAPIS_API_KEY

router_settings:
  fallbacks: [{"llama-70b": ["llama-70b-fallback"]}]

Guides: LiteLLM, OpenAI Python SDK, all integrations.

Where Together is still the better choice

  • Models only Together serves. Fine-tuned checkpoints, image and video models, and the models we do not list have no equivalent here.
  • Steady heavy load. A dedicated endpoint at full utilisation can beat any per-token price; the break-even is well above a few million tokens a day.
  • Fine-tuning and embeddings pipelines. We serve two embedding models and a reranker, not a training stack.

Frequently asked questions

Why did my Together request start failing?

Together removes models from its serverless pool on a published schedule (docs.together.ai/docs/deprecations) and the id then answers 400 Unable to access non-serverless model <id>, with a link to create a dedicated endpoint. The id usually still appears in GET /v1/models with running: false, which is why monitoring on the model list does not catch it. Since mid-2026 the removals include Llama 3.3 70B Turbo, GPT-OSS 20B, DeepSeek V4 Flash 0731, DeepSeek V4 Pro, Kimi K2.6 and K2.7 Code.

Do I have to change my model string?

No. The exact Together ids above are accepted here, either as the same model (Llama 3.3 70B, GPT-OSS 20B, Kimi K2.6, Kimi K2.7 Code, DeepSeek V4 Pro) or routed to the closest model we serve, as the table shows. Change base_url and the key.

What does a dedicated endpoint on Together cost instead?

Together bills dedicated endpoints by GPU hour whether or not you send traffic, so for anything below steady, heavy load the serverless price here is a fraction of it. If you run a dedicated endpoint at full utilisation around the clock, Together may still be cheaper per token; everyone else pays for idle.

Is the speed the same?

Llama 3.3 70B is served from DeepInfra and measures about 21 to 25 tokens per second with a faster first token than Together had; GPT-OSS 20B streams at about 100 tokens per second. Each model page carries measured first-token and throughput numbers from our own gateway.

What about the Together SDK?

The together Python and Node packages accept a base_url; set it to https://api.inferenceapis.com/v1 and keep the calls. The OpenAI SDK works the same way, and LiteLLM has had Inference APIs as a built-in provider since 1.107.