Tool calling, JSON output and thinking control by model
Model cards say "supports function calling". Whether a particular host's deployment does, with streaming, with tool_choice, on the second turn, is a different question, and it is the one that breaks agents. Every cell below is a live request we sent to this API on September 17, 2026. Failures are shown, with the workaround where there is one.
Tool calling
| Model | Tool call | Streamed tool call | Parallel calls | tool_choice: required | tool_choice: named | Codex CLI task |
|---|---|---|---|---|---|---|
| GPT-OSS 120B | Works | Works | Works | One tool only | Works | Works |
| DeepSeek V4 Flash | Works | Works | Works | Works | Works | Works |
| DeepSeek V4.1 Flash | Works | Works | Works | Works | Works | not run |
| GLM 5.3 Flash | Works | Works | Works | Works | Works | Works |
| Llama 3.3 70B Instruct | Works | Works | Works | Works | Works | not run |
| GLM 5.3 | Works | Works | Works | Works | Works | Works |
| DeepSeek V4 Pro | Works | Works | Works | Works | Works | Works |
| MiniMax M3 | Works | Works | Works | Works | Works | Works |
| Kimi K2.7 Code | Works | Works | Works | Works | Works | Works |
| Qwen3-VL 235B | Works | Works | Works | Works | Works | not run |
- Tool call: one function offered, a prompt that needs it; the model returned a well-formed
tool_callsentry. All models also completed the second turn, where the tool result is sent back and the model answers. - Streamed: the same request with
stream: true; the argument fragments indelta.tool_callsreassembled into valid JSON. - Parallel: "weather in Paris and in Tokyo"; two calls came back in one response.
- GPT-OSS 120B and
tool_choice: "required": the upstream deployment answers this with a server error. The gateway works around it: with one tool it sends that tool by name, which is equivalent; with several tools it sendsautoand sets anX-Tool-Choiceresponse header saying so. Named andnonework normally. - Codex CLI task: the real CLI wrote a script, ran it and reported the output through
/v1/responses. Setup.
JSON output and basics
| Model | JSON mode | JSON schema | Usage in stream | System prompt followed |
|---|---|---|---|---|
| GPT-OSS 120B | Works | Works | Works | Works |
| DeepSeek V4 Flash | Works | Works | Works | Works |
| DeepSeek V4.1 Flash | Works | Works | Works | Works |
| GLM 5.3 Flash | Works | Works | Works | Works |
| Llama 3.3 70B Instruct | Works | Works | Works | Works |
| GLM 5.3 | Works | Works | Works | Works |
| DeepSeek V4 Pro | Works | Works | Works | Works |
| MiniMax M3 | Works | Works | Works | Works |
| Kimi K2.7 Code | Fails | Fails | Works | Works |
| Qwen3-VL 235B | Works | Works | Works | Works |
Kimi K2.7 Code fails both JSON tests: with response_format set, the JSON arrives in reasoning_content and content is empty. Ask for JSON in the prompt, or use another model for structured output. Its tool calling is unaffected.
Switching thinking off
Reasoning models think before they answer, the thinking is billed as output, and on an easy request it can be most of the wait. Every model family uses a different parameter to turn it off, and the wrong one is silently ignored. This API accepts one spelling for all of them and translates it:
resp = client.chat.completions.create(
model="deepseek-ai/DeepSeek-V4-Pro",
reasoning_effort="none", # also accepted: "minimal", reasoning={"enabled": False}, thinking={"type": "disabled"}
messages=[{"role": "user", "content": "A farmer has 17 sheep. All but 9 run away. How many are left?"}],
max_tokens=500,
)
# Responses API: reasoning={"effort": "minimal"} Codex CLI: model_reasoning_effort = "minimal"| Model | reasoning_effort: "none" | Completion tokens, default → off | Reasoning characters, default → off | Thinking is returned in |
|---|---|---|---|---|
| GPT-OSS 120B | Minimum, not off | 77 → 30 | 233 → 42 | reasoning |
| DeepSeek V4 Flash | Works | 95 → 6 | 296 → 0 | reasoning_content |
| DeepSeek V4.1 Flash | Works | 32 → 2 | 95 → 0 | reasoning_content |
| GLM 5.3 Flash | Works | 88 → 18 | 219 → 0 | reasoning_content |
| Llama 3.3 70B Instruct | does not reason | — | — | — |
| GLM 5.3 | Works | 81 → 15 | 181 → 0 | reasoning_content |
| DeepSeek V4 Pro | Works | 59 → 2 | 194 → 0 | reasoning_content |
| MiniMax M3 | Works | 22 → 27 | 64 → 0 | reasoning_content |
| Kimi K2.7 Code | Works | 29 → 2 | 80 → 0 | reasoning_content |
| Qwen3-VL 235B | does not reason | — | — | — |
One question, one run each, so read the direction and not the digits. GPT-OSS 120B has no off switch; none is sent as its lowest effort, and low, medium and high are passed through as they are. The response header X-Thinking: off confirms the translation happened. Leave thinking on for multi-step coding and maths; turn it off for extraction, classification, routing and chat.
Method
Public endpoint, default sampling, max_tokens 1,500 to 3,000 so that no model is cut off while thinking. A cell is marked as working only if the response was structurally correct, for example arguments that parse as JSON and name the right city. Models change and upstream deployments change; the date above is the date of the run, and the scripts are rerun when a model is added. If a cell does not match what you see, tell us.