Changelog

New accounts no longer receive $1 of free credit. On October 15 about a hundred accounts were created in one morning from throwaway email domains and proxy addresses purely to collect and spend it, which is the only thing the credit had been doing at scale. Accounts now start at $0 and are funded on the Billing page; there is still no subscription and no minimum beyond what the card processor requires. Prices are per token, so a few dollars tests every model thoroughly. Existing balances are unaffected.

On September 22 we moved DeepSeek V4 Pro to our second GPU provider and cut its price to $1.60 / $3.40. Two things showed up in a week of use that the benchmark missed: that host serves V4 Pro with thinking switched off by default (0 reasoning tokens on prompts where the original host spends 80 to 400), which changes answers on hard tasks, and it stalled for 9 to 64 seconds on short prompts during our TypeWhisper compatibility replay. V4 Pro is back on its original host as of today, with the second provider kept only as a capacity fallback, and the price returns to $1.72 input / $5.15 output / $0.17 cached per 1M tokens. Usage between September 22 and 30 was billed at the lower price. Llama 3.3 70B, GPT-OSS 20B and Qwen3.8 Flash stay where they are: on those the second host has been consistently fast.

We benchmarked our two GPU providers head to head on ten models (speed, five checkable quality tasks, eight concurrent requests, billed cost). Quality was identical everywhere. Llama 3.3 70B and DeepSeek V4 Pro now run on the provider that was as fast or faster and much cheaper, with the other as automatic fallback, and the saving is passed on: Llama drops from $1.35 / $1.35 to $0.20 / $0.50 per 1M tokens, V4 Pro from $1.72 / $5.15 to $1.60 / $3.40. GPT-OSS 20B ($0.04 / $0.19, now the cheapest model here) and Qwen3.8 Flash ($0.15 / $0.50, 1M context, cached input) are live again on the second provider. Every other chat model keeps the faster provider as primary and gains the second as a capacity fallback.

Every API key can now carry a spend limit in USD per day, week, calendar month or lifetime, an expiry date and a list of allowed models, and can be paused without deleting it. The account as a whole can have its own daily, weekly or monthly limit. A reached limit refuses the request before it costs anything, with a 402 whose message says what was spent and when the counter resets. You get an email at 80% and 100% of any limit, and when the balance drops below a threshold you choose (default $1). Every non-streamed response carries X-Cost-USD and X-Balance-Remaining headers, usage.cost is in every JSON usage object including the last chunk of a stream, and GET /v1/usage returns balance, spend by period, limits and a by-model breakdown, even at zero balance. All of it is on the API Keys page and in the MCP server (create_api_key with a limit, update_api_key, set_account_limits).

An agent can now set this service up without a person clicking through the website. POST /v1/signup creates an account and returns a key (adding funds stays human). A hosted MCP server at api.inferenceapis.com/mcp exposes signup, models with prices, balance, usage, key creation and a test completion. A SKILL.md tells Claude Code, Codex or opencode how to do the whole setup themselves; in our test Codex went from an email address to a working config on its own. /llms.txt and /openapi.json describe everything in one fetch, GET /v1/models now carries prices, context length and capabilities, and every error response has a docs_url that explains the code. Also fixed: keys created on the API Keys page were being rejected by the gateway since September 16; accounts without a password can now sign in with an emailed link.

Add diarize=true to a Whisper Large v3 request and every word comes back with a speaker label; segments become one entry per speaker turn, and text, srt and vtt output are prefixed with SPEAKER_00 and so on. min_speakers and max_speakers are optional hints. Same $0.002 per minute. On a three-voice test clip all three speakers were found and 82.5 percent of words carried the right label; two similar voices were merged at a few turn boundaries.

POST /v1/messages serves every chat model in Anthropic's Messages format, streaming included, so Claude Code works with three environment variables. Claude Code 2.1.275 completed a fix-the-tests task on eight models at 1 to 6 cents per run. Thinking blocks, tool use, tool results, images and the thinking.type values enabled, adaptive and auto are all handled.

Codex sends MCP servers and its own sub-agent tools in an OpenAI-only "namespace" wrapper, and hands tasks to Multi-Agent V2 sub-agents as "agent_message" items. Other Responses backends drop both, so on open models MCP tools were never called and sub-agents got an empty task. The Responses endpoint now expands the wrapper, restores the namespace on every call (streamed too) and turns agent_message items into user messages. Verified with codex-cli 0.154.0 and a real MCP server on nine chat models; sub-agent delegation verified on eight.

Zero data retention is now switched on with our main GPU provider, Together AI; our second provider, DeepInfra, already kept nothing. Together with a gateway that never stored request content, this means prompts, audio, images and outputs are not kept by anyone after the response is returned, and nothing is used for training. Only metering counts are stored. All models run in the United States.

Prompt tokens the backend serves from its cache now bill at a cached-input rate, about 80 to 98 percent below normal input, on DeepSeek V4 Flash, V4.1 Flash and V4 Pro, GLM 5.3 and 5.3 Flash, MiniMax M3 and Kimi K2.7 Code. It is automatic. In our test a repeated 5,600-token prompt went from $0.0098 to $0.0011. Agent and long-chat sessions, which resend the conversation every turn, benefit most.

Send reasoning_effort: "none" on any reasoning model. Each model family honours a different parameter for this and silently ignores the others, so the gateway now translates; the Responses API and Codex CLI equivalent is effort "minimal". Measured effect per model, with a tested matrix of tool calls, streamed and parallel calls, tool_choice and JSON output, is on a new reference page. Also fixed: tool_choice "required" on GPT-OSS 120B returned a server error from the backend.

Send images in chat requests in the OpenAI image_url format, by URL or base64. Exact OCR, JSON extraction from an invoice image and an image-driven tool call all passed our tests. It replaces the smaller Qwen3-VL that was listed as planned.

Ten end-to-end checks, from chat on both upstreams to embeddings, rerank, speech and transcription, run every five minutes. The status page shows the current state, latency, 24-hour and 7-day uptime and any check that failed twice in a row. Embeddings and rerank requests are now retried by the gateway when the backend is briefly saturated, and the embeddings page states measured throughput.

POST /v1/responses now serves every chat model, streaming and tool calls included, by translating to chat completions. Codex CLI completed a write-and-run task through it on GLM 5.3, DeepSeek V4 Pro and Flash, MiniMax M3, Kimi K2.7 Code and GPT-OSS 120B. It is stateless: previous_response_id and OpenAI-hosted tools are not supported.

Four larger open-weight models are live, each tested for tool calling, JSON output and streaming. Kimi K2.7 Code runs on a second upstream, DeepInfra, at FP4 precision, which its page states. New setup guides cover Codex CLI, opencode and VS Code bring-your-own-key.

POST /v1/embeddings serves BGE-M3 and Qwen3 Embedding 8B in the OpenAI format at $0.02 per 1M tokens; POST /v1/rerank serves Qwen3 Reranker 8B. We do not meter requests per minute or per day; in a load test BGE-M3 sustained 2 to 4 million tokens per minute, and the gateway retries on its own when the backend is briefly saturated.

Our published output speeds for reasoning models were overstated because of how hidden reasoning tokens were counted. All chat models were re-measured and every page corrected. Separately: multi-turn tool calling on GPT-OSS 120B failed when a client sent content: null, which the OpenAI SDKs do, and leaked internal channel markers into replies; both are fixed. Unknown API routes now return a JSON 404 instead of an HTML page.

One page states where every model runs, at what precision where the upstream publishes it, what each party stores, how aliases that route to a different model are disclosed, and what we commit to when a model is retired.

We ran 30 public test clips per language through Whisper Large v3, and through Parakeet for its 25 languages, and published word and character error rates, language auto-detection results and a sample transcript for each. About a third came out good or excellent and about a third are not usable, and the pages say so.

response_format=srt and vtt were documented but returned a 400 from the backend. The gateway now requests word timestamps and builds the subtitle file itself, on both transcription models: cues of up to two lines, broken at sentence ends and pauses.

alloy, echo, fable, onyx, nova and the newer OpenAI voice names are mapped to Kokoro (or Orpheus) voices, so code written for OpenAI text-to-speech runs with only the base URL changed. The X-Voice-Mapped header reports the voice used. The 20,000-character input limit now counts characters rather than bytes.

Backend authentication failures are now reported as 502 upstream_auth instead of a misleading 401, backend rate limiting as 429 upstream_rate_limited with Retry-After, and other backend failures as 502 upstream_error. Validation errors such as an over-long prompt still come back as 400 with the backend's message.

The API documentation now covers every endpoint with real responses, streaming, tool calling, JSON-schema output, limits and what is not supported. Model pages list what we tested on this endpoint and how to control reasoning cost. Comparison pages price three realistic workloads. Privacy policy and terms were rewritten to describe what the service actually does.

Four current open-weight chat models are live on /v1/chat/completions with streaming, tool calling and JSON mode. GPT-OSS 120B is the replacement Groq recommends for Llama 3.3 70B; Groq-style ids are accepted as aliases.

Added /v1/audio/speech (Kokoro 82M with 54 voices, Orpheus 3B) and /v1/audio/transcriptions (Whisper Large v3, Parakeet TDT 0.6B v3). Both follow the OpenAI request shape; speech is billed per character, transcription per audio minute.

The gateway now serves /v1/chat/completions, /v1/models and /v1/voices, and accepts /openai/v1/… so Groq-style base URLs work with only the host changed. Streaming responses include usage in the final chunk.

Two new browser playgrounds: generate speech with any voice and download the audio, or transcribe an upload / microphone recording with timestamps. Every playground shows the exact cURL request.

All rates are on one page: chat per 1M tokens, speech per 1M characters, transcription per audio minute. Llama 3.3 70B was repriced to $1.35 / 1M to reflect upstream cost.

Started a vendor-neutral error reference with entries for Groq 429 rate_limit_exceeded, the llama-3.3-70b-versatile retirement, and the paused Developer-tier upgrade. Each entry states whether waiting helps and lists every fix, including the ones that do not involve us.

The gateway now requires an API key on every request, checks balance before forwarding, and records per-request usage. Previously the legacy endpoint accepted unauthenticated requests.