Speaker diarization on the OpenAI-compatible transcription endpoint

Request

cURL

curl https://api.inferenceapis.com/v1/audio/transcriptions \
  -H "Authorization: Bearer $INFERENCE_API_KEY" \
  -F "model=openai/whisper-large-v3" \
  -F "file=@interview.mp3" \
  -F "diarize=true" \
  -F "max_speakers=3"           # optional; min_speakers too

Python · openai SDK

import os
from openai import OpenAI
client = OpenAI(base_url="https://api.inferenceapis.com/v1", api_key=os.environ["INFERENCE_API_KEY"])
with open("interview.mp3", "rb") as f:
    r = client.audio.transcriptions.create(model="openai/whisper-large-v3", file=f,
                                           response_format="verbose_json", extra_body={"diarize": True})
for seg in r.model_dump()["segments"]:          # works on openai 1.x and 3.x alike
    print(seg["speaker"], round(seg["start"], 1), seg["text"])

Field	Value	Notes
`diarize`	`true`	Switches speaker labels on. `diarization` and `speaker_labels` are accepted as aliases, since clients written for other services send those
`min_speakers`, `max_speakers`	integer	Hints for the clustering step. In our three-voice test, pinning both to 3 changed nothing; the count was already right
`model`	`openai/whisper-large-v3`	Also under its aliases `whisper-1`, `whisper-large-v3-turbo`, `whisper`. Parakeet and Voxtral return `400 diarization_unsupported`
`response_format`	any	`json` and `verbose_json` both give the full JSON below; `text`, `srt` and `vtt` carry speaker labels
Price	$0.002 per audio minute	The same as transcription without labels

Response

JSON keeps everything the plain call returns and adds speakers. segments are rebuilt as one entry per speaker turn, because Whisper's own segments know nothing about speakers; the upstream speaker_segments array is passed through unchanged for clients that already read it.

verbose_json (two-speaker clip, trimmed)

{
  "text": "Good morning, everyone. Thanks for joining the planning call. ... once the migration passes.",
  "language": "en",
  "duration": 21.94,
  "speakers": ["SPEAKER_01", "SPEAKER_00"],
  "segments": [
    { "id": 0, "start": 0.04,  "end": 5.14,  "speaker": "SPEAKER_01", "text": "Good morning, everyone. Thanks for joining the planning call. Let us start with the release timeline." },
    { "id": 1, "start": 5.64,  "end": 11.13, "speaker": "SPEAKER_00", "text": "Sure, the back-end work is done, but the migration script still fails on the staging database." },
    { "id": 2, "start": 11.73, "end": 15.87, "speaker": "SPEAKER_01", "text": "Okay. Can you have a fix ready by Thursday so we can test it before the weekend?" },
    { "id": 3, "start": 16.39, "end": 21.48, "speaker": "SPEAKER_00", "text": "Yes, Thursday works. I will also update the runbook once the migration passes." }
  ],
  "words": [
    { "id": 0, "word": "Good", "start": 0.04, "end": 0.18, "speaker": "SPEAKER_01", "speaker_id": "SPEAKER_01", "score": 0.986 },
    "..."
  ],
  "speaker_segments": [ "... the upstream form: speaker_id, start, end, text, words ..." ]
}

response_format=text

SPEAKER_01: Good morning, everyone. Thanks for joining the planning call. Let us start with the release timeline.

SPEAKER_00: Sure, the back-end work is done, but the migration script still fails on the staging database.

SPEAKER_01: Okay. Can you have a fix ready by Thursday so we can test it before the weekend?

SPEAKER_00: Yes, Thursday works. I will also update the runbook once the migration passes.

response_format=srt (cues never cross a speaker change)

1
00:00:00,040 --> 00:00:01,101
SPEAKER_01: Good morning, everyone.

2
00:00:01,381 --> 00:00:02,922
SPEAKER_01: Thanks for
joining the planning call.

...

5
00:00:10,068 --> 00:00:11,128
SPEAKER_00: the staging database.

The response also carries an X-Diarized: N speakers header. Speaker ids are per request: SPEAKER_00 in one file and SPEAKER_00 in another are not the same person.

What we measured

We built the test clips with our own text-to-speech voices, so the ground truth is exact: we know which voice spoke every word. That makes the clips cleaner than a real meeting (no crosstalk, no room noise, no phone codec), so treat these as an upper bound.

Clip	Speakers found	Turns	Words with the right speaker	Time to result	What went wrong
Two voices, 4 turns, 22 s	2 of 2	4 of 4	100%	1.4 s	Nothing
Three voices, 12 turns, 35 s, with one-word interjections	3 of 3	9 of 12	82.5%	1.0 s	Two similar female voices were merged at three turn boundaries ("Glad to be here too" was attached to the host's next question). The male voice was never confused. The one-word "Right." was caught. Pinning min/max_speakers to 3 changed nothing
One voice, 2 s	1 of 1	1 of 1	100%	0.4 s	No phantom second speaker

"Words with the right speaker" is computed by aligning the transcript to the script word by word and taking the best mapping of labels to voices. The timing is the whole request from our side, upload included; diarization added about one second to a 35-second clip.

Limits, plainly

Similar voices merge. The three-voice result above is what to expect when two speakers sound alike; distinct voices were perfect. Real recordings with crosstalk will do worse than either.
No overlap handling. Each word gets exactly one speaker. Two people talking at once come back as one of them.
Recorded files only. There is no streaming diarization here.
Whisper only. Parakeet and Voxtral transcribe faster but cannot label speakers on this endpoint.
Labels, not names. To turn SPEAKER_00 into "Dana", pass the labelled text to a chat model with the participant list; that is a one-line prompt and usually reliable when people address each other by name.

Why people were self-hosting this, and what broke

Most of the diarization threads we read on GitHub while deciding to build this were not about accuracy. They were about plumbing: adding a diarize flag to a tool's OpenAI-compatible transcription call, pyannote's gated model download and Hugging Face token, WhisperX pinned to an older pyannote after pyannote.audio 4.0 changed its API, CoreML and ONNX builds that would not load a diarization model, and GPU memory for running two models per file. If your tool already talks to an OpenAI-shaped /v1/audio/transcriptions, the change is one form field; the integration guides show where the base URL goes in each client.