Chat Completions API

Quickstart

curl https://api.inferenceapis.com/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello! What can you do?"}
    ]
  }'

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
)
response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello! What can you do?"},
    ],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inferenceapis.com/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});
const response = await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello! What can you do?" },
  ],
});
console.log(response.choices[0].message.content);

Streaming: add "stream": true; the final chunk carries usage. Tool calling and response_format: {"type": "json_object"} follow the OpenAI format. Each model page lists parameters and aliases.

Which model

• GPT-OSS 120B — the strongest reasoning and tool-use model here, and the id Groq recommends as the Llama 3.3 replacement. 133 tok/s measured.
• DeepSeek V4 Flash — lowest price per token with a 1M-token context; good default for assistants and long documents.
• DeepSeek V4.1 Flash / GLM 5.3 Flash — the fastest (0.4 s to first token, 265–390 tok/s measured), both with tool calling and 1M context.
• Llama 3.3 70B — for code and prompts already tuned to it; same llama-3.3-70b-versatile id as Groq used.

Side by side: GPT-OSS 120B vs DeepSeek V4 Flash, GPT-OSS 120B here vs on Groq, Llama 3.3 70B here vs Groq (retired), all comparisons.

Pricing versus alternatives

Per 1M tokens, input / output; list prices as of September 2026.

Frequently asked questions

Is it a drop-in for the OpenAI chat API?

Yes. POST /v1/chat/completions with the same request and response shape, including stream, tools, tool_choice, response_format and usage. The OpenAI SDKs, LangChain, LiteLLM and most agent frameworks work by changing base_url. See integrations.

I was using Groq. Does my code work?

Usually with only the base URL changed: /openai/v1 is accepted as well as /v1, and Groq ids such as llama-3.3-70b-versatile and openai/gpt-oss-120b resolve to the same models here. Details in the switching guide.

What are the rate limits?

There are no free-tier daily caps and no tiers to unlock. Usage is limited by your prepaid balance; a per-key burst limit exists only to protect the service and is well above typical use.

Why is the reply empty on GPT-OSS or DeepSeek?

They are reasoning models: they spend tokens thinking before answering. If max_tokens is very small the budget is used up on reasoning. Allow a few hundred tokens or more; the reasoning field and usage.completion_tokens_details.reasoning_tokens show what happened.

Do you support vision or images in chat?

Qwen3-VL is being enabled and will accept images in the content array. Until then, chat models are text-only.

Where do the models run?

On serverless GPU capacity from Together AI in the US, fronted by our gateway for authentication, metering and error normalisation. Prompts are not stored after the response is returned.