How models are served
The complaints people make about inference providers are consistent: models quietly quantized, ids swapped without notice, models retired with little warning, and no clear answer on who keeps your prompts. This page answers those four questions for every model here, including where the honest answer is "we do not know". Last checked September 17, 2026.
Where each model runs, and at what precision
We are a gateway: authentication, metering, id aliases, error handling and a few compatibility fixes are ours, and the GPUs belong to the upstream named in each row. "Served as" is the exact upstream model a request is routed to. Precision is what the upstream publishes; where it publishes nothing, we say so instead of guessing.
| Model | Served as | Runs on | Region | Precision |
|---|---|---|---|---|
| GPT-OSS 120B | openai/gpt-oss-120b | Together AI | United States | MXFP4, the format OpenAI released the weights in |
| DeepSeek V4 Flash | deepseek-ai/DeepSeek-V4-Flash-0731 | Together AI | United States | not published by the upstream |
| DeepSeek V4.1 Flash | deepseek-ai/DeepSeek-V4.1-Flash | Together AI | United States | not published by the upstream |
| GLM 5.3 Flash | zai-org/GLM-5.3-Flash | Together AI | United States | not published by the upstream |
| Llama 3.3 70B Instruct | meta-llama/Llama-3.3-70B-Instruct-Turbo | Together AI | United States | FP8 (the upstream's "Turbo" variant) |
| GLM 5.3 | zai-org/GLM-5.3 | Together AI | United States | not published by the upstream |
| DeepSeek V4 Pro | deepseek-ai/DeepSeek-V4-Pro-0813 | Together AI | United States | not published by the upstream |
| MiniMax M3 | MiniMaxAI/MiniMax-M3 | Together AI | United States | not published by the upstream |
| Kimi K2.7 Code | moonshotai/Kimi-K2.7-Code | DeepInfra | United States | FP4 |
| Kokoro 82M | hexgrad/Kokoro-82M | Together AI | United States | not published by the upstream |
| Orpheus 3B | canopylabs/orpheus-3b-0.1-ft | Together AI | United States | not published by the upstream |
| Whisper Large v3 | openai/whisper-large-v3 | Together AI | United States | not published by the upstream |
| Parakeet TDT 0.6B v3 | nvidia/parakeet-tdt-0.6b-v3 | Together AI | United States | not published by the upstream |
| BGE-M3 | BAAI/bge-m3 | DeepInfra | United States | FP32 |
| Qwen3 Embedding 8B | Qwen/Qwen3-Embedding-8B | DeepInfra | United States | not published by the upstream |
| Qwen3 Reranker 8B | Qwen/Qwen3-Reranker-8B | DeepInfra | United States | not published by the upstream |
If an upstream changes the precision of a model, the row changes and the changelog says so. If you need a guaranteed precision for an evaluation, ask us before you start and we will confirm it with the upstream.
What is stored, and by whom
| Who | Request and response content | Metadata |
|---|---|---|
| Inference APIs (this gateway) | Not stored. Content is held in memory for the length of the request. Uploaded audio is written to a temporary file to measure its duration and deleted when the request ends. | Stored: account, model id, token, character or audio-minute counts, cost, timestamp. Web server logs hold IP, URL and status, not bodies. |
| Together AI (most chat models, speech, transcription) | Together's default applies to our organisation today: it stores prompts and responses and may use them for product improvement; use for training other models is off. Requests reach Together under our account, without your name, email or API key. We are moving our organisation to Together's zero-data-retention setting and will update this row when it is on. If that matters for your workload today, use a DeepInfra-hosted model or ask us. | Token counts, model ids and timestamps, for billing us. |
| DeepInfra (Kimi K2.7 Code, embeddings, rerank) | Not stored. DeepInfra states that inputs exist only in memory during inference, outputs are deleted after they are sent, and neither is used for training. | Request id, cost and sampling parameters, for debugging and billing. |
Open-weight models from DeepSeek, Z.ai, Moonshot AI, MiniMax and Alibaba run on these US-hosted GPUs. Requests are never sent to those companies' own APIs. The account side of things, what we keep about you and why, is in the privacy policy.
Model ids and silent swaps
- An id means one model. We do not change what a canonical id serves without an entry in the changelog.
- Dated releases are named. Where a model has dated releases, the table above shows the one in use, for example the 0731 release of DeepSeek V4 Flash. The dated id is accepted too, so you can pin it.
- Aliases that route to a different model are disclosed, on the model page, next to the alias. There are four today:
llama-3.1-8b-instantis answered by Llama 3.3 70B,deepseek-v3by DeepSeek V4 Flash,whisper-large-v3-turboandwhisper-1by Whisper Large v3, andtts-1by Kokoro. They exist so that code written for another provider keeps running; each bills at the rate of the model that answers. GET /v1/modelsis the source of truth. Every accepted id is listed with thecanonicalid it resolves to.
When a model is retired
We do not control when an upstream stops serving a model, so we promise what we can keep:
- A retirement is announced in the changelog, which has an RSS feed, at least 30 days ahead whenever the upstream gives us that long, and on the day we learn of it otherwise.
- Before retiring a model we look for another host for the same weights. Having two upstreams is what makes that possible.
- If the same weights cannot be served anywhere, the id returns
404 model_not_found. We do not quietly point a retired id at a different model. - Retirements by other providers, and where the same model is still available, are tracked in the deprecations tracker.
Reliability
Live checks of the gateway and both upstreams are on the status page. There is no SLA. When an upstream is saturated you get a 429 or 503 with a JSON error, never a quota message, because we do not meter your requests per minute or per day. We do not list a model we cannot serve reliably: a coding model we tested this week answered "model busy" on every call at its only host, so it is not in the catalog.
Claims we measured ourselves
- Tool calling, JSON output and streaming usage are tested per model with live requests; each model page shows the result, including the failures.
- Transcription accuracy is measured for 83 languages, and the pages say where it does not work.
- Speed figures are our own measurements with the method published; we do not quote other providers' numbers.
Something here wrong or out of date? Tell us.