Use Inference APIs with LangChain

ChatOpenAI accepts base_url, so chains, agents and LangGraph graphs run on Inference APIs models without any provider-specific package.

Settings

Setting	Value
Base URL	`https://api.inferenceapis.com/v1` (also `/openai/v1`)
API key	From API Keys; send as `Authorization: Bearer …`
Chat model ids	`openai/gpt-oss-120b`, `deepseek-ai/DeepSeek-V4-Flash`, `zai-org/GLM-5.3-Flash`, `meta-llama/Llama-3.3-70B-Instruct-Turbo` — all models
Audio model ids	`openai/whisper-large-v3` (transcription), `hexgrad/Kokoro-82M` (speech)

Configuration

Python

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="deepseek-ai/DeepSeek-V4-Flash",
    base_url="https://api.inferenceapis.com/v1",          # older versions: openai_api_base=
    api_key=os.environ["INFERENCE_API_KEY"],
    max_tokens=800,
)
print(llm.invoke("Summarise the plot of Dune in two sentences.").content)

JavaScript

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "deepseek-ai/DeepSeek-V4-Flash",
  apiKey: process.env.INFERENCE_API_KEY,
  configuration: { baseURL: "https://api.inferenceapis.com/v1" },
});
console.log((await llm.invoke("Hello")).content);

Verify

Smoke test

python -c "from langchain_openai import ChatOpenAI; import os; print(ChatOpenAI(model='openai/gpt-oss-120b', base_url='https://api.inferenceapis.com/v1', api_key=os.environ['INFERENCE_API_KEY']).invoke('ping').content[:60])"

Gotchas

Tool calling works with bind_tools on GPT-OSS, DeepSeek and GLM models.
For embeddings LangChain will need a separate provider; we do not serve embeddings yet.

If something fails

401 — key missing or wrong · 402 insufficient_balance — add credits on Billing · 404 model_not_found — check the id against the model list (aliases such as gpt-oss-120b work too) · 503 model_unavailable — the model is being enabled. Full details in the API docs.