Use Inference APIs with LangChain

Settings

Which model

For agents and LangGraph, openai/gpt-oss-120b; for high-volume chains, deepseek-ai/DeepSeek-V4-Flash. Prices and measured speed: chat models.

Configuration

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="deepseek-ai/DeepSeek-V4-Flash",
    base_url="https://api.inferenceapis.com/v1",          # older versions: openai_api_base=
    api_key=os.environ["INFERENCE_API_KEY"],
    max_tokens=800,
)
print(llm.invoke("Summarise the plot of Dune in two sentences.").content)

import { ChatOpenAI } from "@langchain/openai";

const llm = new ChatOpenAI({
  model: "deepseek-ai/DeepSeek-V4-Flash",
  apiKey: process.env.INFERENCE_API_KEY,
  configuration: { baseURL: "https://api.inferenceapis.com/v1" },
});
console.log((await llm.invoke("Hello")).content);

Verify

python -c "from langchain_openai import ChatOpenAI; import os; print(ChatOpenAI(model='openai/gpt-oss-120b', base_url='https://api.inferenceapis.com/v1', api_key=os.environ['INFERENCE_API_KEY']).invoke('ping').content[:60])"

How this guide was checked

Run end to end langchain-openai 1.6.2 and @langchain/openai 1.5.13: invoke in Python and JavaScript, and bind_tools with a real tool call on GPT-OSS 120B, ran against the live API on 2026-09-17.

Gotchas

• Tool calling works with bind_tools on GPT-OSS, DeepSeek and GLM models.
• Embeddings: OpenAIEmbeddings(model="BAAI/bge-m3", base_url=..., api_key=..., check_embedding_ctx_length=False). The flag matters: without it LangChain pre-tokenises with OpenAI's tokenizer and sends token ids, which open models do not accept. Tested with langchain-openai 1.6.2.

If something fails

401 — key missing or wrong · 402 insufficient_balance — add credits on Billing · 404 model_not_found — check the id against the model list (aliases such as gpt-oss-120b work too) · 503 model_unavailable — the model is being enabled. Full details in the API docs.