Use Inference APIs with OpenAI Python SDK

Point the official openai Python package at Inference APIs with base_url. Chat, streaming, tool calling, speech and transcription all work unchanged.

Settings

Setting	Value
Base URL	`https://api.inferenceapis.com/v1` (also `/openai/v1`)
API key	From API Keys; send as `Authorization: Bearer …`
Chat model ids	`openai/gpt-oss-120b`, `deepseek-ai/DeepSeek-V4-Flash`, `zai-org/GLM-5.3-Flash`, `meta-llama/Llama-3.3-70B-Instruct-Turbo` — all models
Audio model ids	`openai/whisper-large-v3` (transcription), `hexgrad/Kokoro-82M` (speech)

Which model

Start with openai/gpt-oss-120b; switch to deepseek-ai/DeepSeek-V4-Flash when cost matters more than latency. Prices and measured speed: chat models.

Configuration

Install

pip install openai

Chat

import os
from openai import OpenAI

client = OpenAI(base_url="https://api.inferenceapis.com/v1", api_key=os.environ["INFERENCE_API_KEY"])

resp = client.chat.completions.create(
    model="openai/gpt-oss-120b",
    messages=[{"role": "user", "content": "Hello"}],
    max_tokens=400,
)
print(resp.choices[0].message.content)

Streaming

with client.chat.completions.stream(model="deepseek-ai/DeepSeek-V4-Flash", messages=[{"role": "user", "content": "Write a haiku"}]) as s:
    for event in s:
        if event.type == "content.delta":
            print(event.delta, end="", flush=True)

Speech and transcription

audio = client.audio.speech.create(model="hexgrad/Kokoro-82M", voice="af_heart", input="Hello there")
audio.write_to_file("hello.mp3")

with open("hello.mp3", "rb") as f:
    text = client.audio.transcriptions.create(model="openai/whisper-large-v3", file=f).text
print(text)

Environment variables instead of code

export OPENAI_BASE_URL="https://api.inferenceapis.com/v1"
export OPENAI_API_KEY="$INFERENCE_API_KEY"   # then OpenAI() with no arguments

Verify

Smoke test

python -c "from openai import OpenAI; import os; print([m.id for m in OpenAI(base_url='https://api.inferenceapis.com/v1', api_key=os.environ['INFERENCE_API_KEY']).models.list().data][:5])"

How this guide was checked

Run end to end openai 2.54.0 (Python): chat, the .stream() helper, speech, transcription including srt output, models.list and the environment-variable setup all ran against the live API on 2026-09-17.

Gotchas

Reasoning models (GPT-OSS, DeepSeek, GLM) need max_tokens of a few hundred or more; the reply is empty when the budget is spent on reasoning.
client.responses works too (tested with responses.create and streaming), but it is stateless here: send the full input each time rather than previous_response_id.
client.embeddings.create works with BAAI/bge-m3 and Qwen/Qwen3-Embedding-8B. OpenAI embedding model ids are not aliased, because vectors from different models cannot be mixed.

If something fails

401 — key missing or wrong · 402 insufficient_balance — add credits on Billing · 404 model_not_found — check the id against the model list (aliases such as gpt-oss-120b work too) · 503 model_unavailable — the model is being enabled. Full details in the API docs.