Frequently asked questions

Yes. POST /v1/chat/completions, /v1/audio/speech, /v1/audio/transcriptions and GET /v1/models use the OpenAI request and response shapes, including streaming (stream: true), tools, tool_choice, response_format and usage. The official OpenAI SDKs work by setting base_url to https://api.inferenceapis.com/v1. /openai/v1/… is accepted as well, so Groq-style base URLs need only the host changed. POST /v1/responses (the Responses API, used by Codex CLI), /v1/embeddings and /v1/rerank are served too. Stored responses and OpenAI-hosted tools such as web search are not.

Each model has a canonical id (for example openai/gpt-oss-120b) plus aliases that other providers use for the same weights, such as gpt-oss-120b, llama-3.3-70b-versatile (Groq), whisper-1 (OpenAI) and kokoro. GET /v1/models lists every accepted id with its canonical field. Unknown ids return 404 model_not_found.

Per request, from a prepaid balance. Chat models bill prompt plus completion tokens at the model's per-1M rate (reasoning tokens count as completion tokens). Text-to-speech bills per input character. Transcription bills per minute of audio, measured from the uploaded file before processing. Every request is logged with its units and cost; there is no per-request fee, subscription or minimum.

There are no free-tier daily caps and no tiers to unlock. Usage is limited by your balance: when it reaches zero, requests return 402 insufficient_balance until you add credits. There is no per-minute or per-day request or token meter. We may throttle traffic that is abusive or that endangers the service for other users; if you plan sustained high concurrency, tell us first so capacity is there.

They are reasoning models: they think before they answer, and the thinking consumes max_tokens. If the budget is small (say 20–50 tokens) it is spent entirely on reasoning and content comes back empty while reasoning (GPT-OSS) or reasoning_content (DeepSeek, GLM) is populated. Set max_tokens to a few hundred or more, and lower the thinking itself with reasoning_effort: "low" on GPT-OSS and GLM or "reasoning": {"enabled": false} on DeepSeek. usage.completion_tokens_details.reasoning_tokens shows how many tokens went to reasoning.

Yes. With stream: true the final chunk before data: [DONE] carries a usage object with prompt and completion token counts, so you can meter streamed responses without counting tokens yourself.

mp3, mp4, m4a, wav, webm, ogg and flac, up to 100 MB per request. Duration is read with ffprobe for billing. For very long recordings, splitting into chunks of 20–30 minutes keeps latency predictable; transcription runs at roughly 0.04–0.05× real time (a 20-minute file in under a minute).

Output is mp3 (default), wav or raw PCM. Kokoro 82M has 54 voices across eight languages (ids like af_heart, bm_george, jf_alpha); Orpheus has eight expressive English voices. OpenAI voice names such as alloy are accepted and mapped to Kokoro voices. GET /v1/voices?model=hexgrad/Kokoro-82M returns the list with your key, and you can play every voice without signing in.

No, and neither do the GPU providers that run the models. This gateway does not retain prompt text, audio uploads, images or generated output, and zero data retention is switched on with both upstream providers, so nothing is kept after the response is returned and nothing is used for training. What is stored per request is metering data: model id, unit counts (tokens, characters or minutes), cost and timestamp. Who holds what is set out on the trust page.

Most chat models, speech and transcription run on Together AI's serverless GPUs, and Kimi K2.7 Code, embeddings and rerank on DeepInfra, both in the United States, behind our gateway (authentication, metering, aliasing, error normalisation). The trust page lists the host and precision of every model. OmniParser runs on our own hardware. If you need a specific region, contact us.

Errors use the OpenAI envelope {"error": {"message", "type", "code"}}. 401 missing or invalid key · 402 insufficient_balance · 404 model_not_found · 400 wrong_endpoint (for example a speech model sent to chat) · 413 input too large · 503 model_unavailable · 502 backend error · 429 upstream_rate_limited, which is rare and means the backend is busy, not that you used up a quota. Retry 429, 502 and 503 with backoff; the others are deterministic. The full table is in the API documentation.

Yes, on GPT-OSS 120B, DeepSeek V4 / V4.1 Flash, GLM 5.3 Flash and Llama 3.3 70B. Pass tools in the OpenAI format and read tool_calls from the response; pass response_format: {"type": "json_object"} for JSON mode. Streamed tool calls arrive as delta.tool_calls fragments.

Yes, automatically. When the start of a prompt matches a recent request, the backend reuses its work and reports the number of reused tokens in usage.prompt_tokens_details.cached_tokens. Those tokens bill at the model's cached-input rate, about 80 to 98 percent below its normal input rate; the pricing page lists it per model. In our test a 5,600-token prompt cost $0.0098 the first time and $0.0011 when repeated. Agents and long chats benefit most, because every turn resends the conversation so far. GPT-OSS 120B and Llama 3.3 70B have no cached rate.

Send reasoning_effort: "none". Every model family uses a different parameter for this and silently ignores the others, so the gateway accepts one spelling on all of them and sends the one the model honours. In our tests that took completion tokens from, for example, 59 to 2 on DeepSeek V4 Pro, and the answer started immediately. GPT-OSS 120B has no off switch and drops to its lowest effort. The measured effect on every model is in the capabilities matrix.

Change the base URL from https://api.groq.com/openai/v1 to https://api.inferenceapis.com/v1 (or keep /openai/v1 — both work) and swap the key. Model ids Groq used, including retired ones such as llama-3.3-70b-versatile, are accepted as aliases. openai/gpt-oss-120b is served under the same id.

No. There is no free credit or trial tier: accounts start at $0 and you add funds on the Billing page, with no minimum top-up beyond what the payment processor requires. Prices are per token, so a few dollars covers extensive testing of every model; the pricing page has the per-request costs.