We tested Whisper Large v3 on 83 languages. A third are not usable, and some return nothing
Whisper lists 100 languages. We sent 30 public test clips per language through our API and scored every transcript. Around a third of languages are good or excellent, a third are not usable, a few come back empty, and auto-detection confuses more languages than you would expect.
Taylor Hawkes · September 17, 2026
Every Whisper deployment advertises the same list of about a hundred languages, because that is what the model card says. We had copied that list onto a hundred pages of our own, and then could not answer a simple question from the person reading one: does it actually work for my language? So we measured it, through the same public endpoint our customers use.
What we found
- 34 of 83 languages are good or excellent (up to 15% word error). Spanish and Italian lead at 2.7%.
- 22 are usable only with a human pass.
- 27 are not usable, with most words wrong, and 3 of those return empty transcripts even though the API accepts the language code.
- Auto-detection is weaker than its reputation. It was perfect for most major languages and wrong half the time for a few pairs that sound alike.
The top of the table
| Language | Whisper Large v3 | Parakeet TDT 0.6B v3 | Clips with no errors |
|---|---|---|---|
| Spanish | 2.7% WER | 3.5% | 20 of 30 |
| Italian | 2.7% WER | 2.4% | 19 of 30 |
| German | 3.4% WER | 6.3% | 18 of 30 |
| Portuguese | 3.4% WER | 4.4% | 15 of 30 |
| Japanese | 4.2% CER | — | 8 of 30 |
| Cantonese | 5.1% CER | — | 7 of 30 |
| English | 5.2% WER | 5.8% | 9 of 30 |
| Indonesian | 5.5% WER | — | 17 of 30 |
| Polish | 5.6% WER | 7.6% | 13 of 30 |
| Russian | 5.8% WER | 7.2% | 10 of 30 |
| Ukrainian | 6.4% WER | 6.6% | 16 of 30 |
| Finnish | 6.6% WER | 15% | 11 of 30 |
The full table, sortable in your head by rating, is on the accuracy by language page, and every language has its own page with a sample transcript.
The languages that return nothing
Khmer (30 of 30 empty), Burmese (28 of 30 empty), Amharic (29 of 30 empty). The request succeeds, you are billed for the audio minute, and text is an empty string. Nothing in the API response tells you the language is the problem. If you serve users in these languages, this model is the wrong tool; each page lists alternatives.
Where auto-detection goes wrong
We sent ten clips per language without a language field. When Whisper guesses wrong it does not just mislabel the file: it writes the transcript in the language it guessed.
| Language | Detected correctly | Most often mistaken for |
|---|---|---|
| Cantonese | 0 of 10 | Thai (4) |
| Indonesian | 5 of 10 | Javanese (5) |
| Russian | 5 of 10 | Belarusian (4) |
| Malay | 7 of 10 | Indonesian (3) |
| Urdu | 5 of 10 | Hindi (5) |
| Serbian | 5 of 10 | Bosnian (3) |
| Javanese | 7 of 10 | Indonesian (3) |
| Occitan | 0 of 10 | French (3) |
| Lao | 6 of 10 | Thai (4) |
The fix costs nothing: send language whenever you know it.
Whisper against Parakeet
Parakeet covers 25 European languages at the same price and runs faster. On the same clips Whisper was clearly more accurate in 15 of those languages, Parakeet in 6, and the rest were too close to call on 30 clips. The outlier is Maltese: Whisper 72%, Parakeet 22%. If you transcribe Maltese, the choice of model is the whole story.
Two results that are not what they look like
- Serbian first scored 93% word error. The transcripts were nearly perfect; Whisper writes Serbian in Latin script and the references are Cyrillic. After transliterating the references it is 21%.
- Lao comes back written in Thai script. The words are partly recognisable to a Thai reader and useless as Lao text.
Both are reminders that an error rate is only as good as the normalisation behind it. Ours is described below so you can disagree with it.
Method
- Audio: the first 30 clips of 2 to 30 seconds from the test split of FLEURS, per language. Clean read speech, one speaker. Your recordings will be harder.
- Requests:
POST /v1/audio/transcriptionson the public endpoint,languageset, default settings, 2026-09-17. - Scoring: Unicode NFKC, lower-case, punctuation removed, then word-level edit distance; character-level for languages written without spaces. Numbers were not normalised, which costs every language a little.
- 17 of Whisper's languages have no FLEURS test set and were not measured. Those pages say so and are not indexed.
- Thirty clips is a small sample. Read the ratings, not the second decimal.
The whole run cost about a dollar and a half of metered audio. If you want a language re-run on different audio, ask.