Chat Completions API

Quickstart

curl https://api.inferenceapis.com/v1/chat/completions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-oss-120b",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Hello! What can you do?"}
    ]
  }'

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.inferenceapis.com/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
)
response = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello! What can you do?"},
    ],
)
print(response.choices[0].message.content)

import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://api.inferenceapis.com/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});
const response = await client.chat.completions.create({
  model: "openai/gpt-oss-120b",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello! What can you do?" },
  ],
});
console.log(response.choices[0].message.content);

Streaming: add "stream": true; the final chunk carries usage. Tool calling and response_format: {"type": "json_object"} follow the OpenAI format. Each model page lists parameters and aliases.

Which model

• GPT-OSS 120B — the strongest reasoning and tool-use model here, and the id Groq recommends as the Llama 3.3 replacement. It thinks briefly, so it answers quickly.
• DeepSeek V4 Flash — lowest price per token with a 1M-token context; good default for assistants and long documents.
• DeepSeek V4.1 Flash / GLM 5.3 Flash — tool calling and 1M context at a low price. V4.1 Flash is the quicker of the two to answer; both let you shorten or switch off reasoning.
• GLM 5.3 / DeepSeek V4 Pro / Kimi K2.7 Code — the large models, for coding agents such as Codex CLI and opencode.
• Llama 3.3 70B — for code and prompts already tuned to it; same llama-3.3-70b-versatile id as Groq used.

Side by side: GPT-OSS 120B vs DeepSeek V4 Flash, GPT-OSS 120B here vs on Groq, Llama 3.3 70B here vs Groq (retired), all comparisons.

Pricing versus alternatives

Per 1M tokens, input / output; list prices as of September 2026.

Frequently asked questions

Is it a drop-in for the OpenAI chat API?

Yes. POST /v1/chat/completions with the same request and response shape, including stream, tools, tool_choice, response_format and usage. The OpenAI SDKs, LangChain, LiteLLM and most agent frameworks work by changing base_url. See integrations.

I was using Groq. Does my code work?

Usually with only the base URL changed: /openai/v1 is accepted as well as /v1, and Groq ids such as llama-3.3-70b-versatile and openai/gpt-oss-120b resolve to the same models here. Details in the switching guide.

What are the rate limits?

There are no free-tier daily caps and no tiers to unlock. Usage is limited by your prepaid balance; we do not meter requests or tokens per minute or per day. We reserve the right to throttle traffic that is abusive or endangers the service for others.

Why is the reply empty on GPT-OSS or DeepSeek?

They are reasoning models: they spend tokens thinking before answering. If max_tokens is very small the budget is used up on reasoning. Allow a few hundred tokens or more. The thinking is returned in reasoning (GPT-OSS) or reasoning_content (DeepSeek, GLM), and usage.completion_tokens_details.reasoning_tokens shows how much was spent. To spend less, set reasoning_effort to low on GPT-OSS and GLM, or switch thinking off on DeepSeek; each model page shows how.

Do you support vision or images in chat?

Yes, on Qwen3-VL 235B. Send images in the OpenAI image_url format, as a URL or a base64 data URI; OCR, JSON extraction and tool calls from an image all passed our tests. The other chat models are text-only.

Where do the models run, and are prompts stored?

On serverless GPUs in the United States, from Together AI for most models and DeepInfra for Kimi K2.7 Code and Qwen3-VL, fronted by our gateway for authentication, metering and error handling. Zero data retention applies end to end: neither the gateway nor either GPU provider keeps prompts or responses after the request, and nothing is used for training. Open-weight models from DeepSeek, Z.ai and Moonshot run on these US GPUs and are never sent to those companies. Details: trust page.