Use Inference APIs with LlamaIndex

Use the OpenAILike LLM class, which exists for exactly this case: an OpenAI-compatible endpoint serving non-OpenAI model ids.

Settings

Setting	Value
Base URL	`https://api.inferenceapis.com/v1` (also `/openai/v1`)
API key	From API Keys; send as `Authorization: Bearer …`
Chat model ids	`openai/gpt-oss-120b`, `deepseek-ai/DeepSeek-V4-Flash`, `zai-org/GLM-5.3-Flash`, `meta-llama/Llama-3.3-70B-Instruct-Turbo` — all models
Audio model ids	`openai/whisper-large-v3` (transcription), `hexgrad/Kokoro-82M` (speech)

Configuration

Python

import os
from llama_index.llms.openai_like import OpenAILike

llm = OpenAILike(
    model="openai/gpt-oss-120b",
    api_base="https://api.inferenceapis.com/v1",
    api_key=os.environ["INFERENCE_API_KEY"],
    is_chat_model=True,
    is_function_calling_model=True,
    context_window=131072,
)
print(llm.complete("Hello").text)

Install

pip install llama-index-llms-openai-like

Verify

Smoke test

python -c "from llama_index.llms.openai_like import OpenAILike; import os; print(OpenAILike(model='openai/gpt-oss-120b', api_base='https://api.inferenceapis.com/v1', api_key=os.environ['INFERENCE_API_KEY'], is_chat_model=True).complete('ping').text[:60])"

Gotchas

Set context_window explicitly; OpenAILike cannot look it up for unknown model ids.

If something fails

401 — key missing or wrong · 402 insufficient_balance — add credits on Billing · 404 model_not_found — check the id against the model list (aliases such as gpt-oss-120b work too) · 503 model_unavailable — the model is being enabled. Full details in the API docs.