Use Inference APIs with Vercel AI SDK

Use @ai-sdk/openai-compatible to create a provider for Inference APIs; then streamText, generateText and tool calling work as with any provider.

Settings

Setting	Value
Base URL	`https://api.inferenceapis.com/v1` (also `/openai/v1`)
API key	From API Keys; send as `Authorization: Bearer …`
Chat model ids	`openai/gpt-oss-120b`, `deepseek-ai/DeepSeek-V4-Flash`, `zai-org/GLM-5.3-Flash`, `meta-llama/Llama-3.3-70B-Instruct-Turbo` — all models
Audio model ids	`openai/whisper-large-v3` (transcription), `hexgrad/Kokoro-82M` (speech)

Which model

For streamed UI responses, deepseek-ai/DeepSeek-V4.1-Flash, with reasoning_effort or the reasoning switch turned down so the answer starts sooner. Prices and measured speed: chat models.

Configuration

Install

npm install ai @ai-sdk/openai-compatible

Provider + streamText

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { streamText } from "ai";

const inferenceapis = createOpenAICompatible({
  name: "inferenceapis",
  baseURL: "https://api.inferenceapis.com/v1",
  apiKey: process.env.INFERENCE_API_KEY,
});

const result = streamText({
  model: inferenceapis("openai/gpt-oss-120b"),
  prompt: "Write a limerick about rate limits.",
});
for await (const chunk of result.textStream) process.stdout.write(chunk);

Verify

Smoke test

node --input-type=module -e "import { createOpenAICompatible } from '@ai-sdk/openai-compatible'; import { generateText } from 'ai'; const ia = createOpenAICompatible({ name: 'inferenceapis', baseURL: 'https://api.inferenceapis.com/v1', apiKey: process.env.INFERENCE_API_KEY }); console.log((await generateText({ model: ia('openai/gpt-oss-120b'), prompt: 'ping' })).text)"

How this guide was checked

Run end to end ai 7.0.105 with @ai-sdk/openai-compatible: generateText and streamText ran against the live API on 2026-09-17.

Gotchas

Use inferenceapis.chatModel(id) if your SDK version distinguishes chat and completion models.

If something fails

401 — key missing or wrong · 402 insufficient_balance — add credits on Billing · 404 model_not_found — check the id against the model list (aliases such as gpt-oss-120b work too) · 503 model_unavailable — the model is being enabled. Full details in the API docs.