Fermion ResearchFermion Research

Speech recognition/facts checked 13 November 2026

Whisper alternatives: open models, faster runtimes and APIs

Whisper made open speech recognition practical, and it is still the widest-language open model. Newer open models beat it on English accuracy and size; some tools keep the Whisper model and only make it faster.

Phonon-2.5 downloads and benchmarks

The short answer

If you want lower English error than Whisper, Qwen3-ASR 1.7B (4.43 % word error), Parakeet TDT 0.6B v2 (4.82 %) and Phonon-2.5 English (4.82 %) all average below Whisper large-v3 (6.04 %) on the seven Open ASR Leaderboard English sets. If you want to keep Whisper but run it faster, use faster-whisper, whisper.cpp, mlx-whisper or WhisperKit. If you need Whisper's 99 languages or its translation to English, stay with Whisper.

For European languages from a small download, Phonon-2.5 covers 25 languages in 156 MB and averages 11.71 % word error on FLEURS, against 53.71 % for Whisper base, which is about the same size, on the same 25 sets.

Alternative models compared with Whisper

English seven-set word error in %, lower is better. Phonon rows are our runs with the leaderboard's scoring code; the others are the leaderboard snapshot of 2026-09-25.

ModelOpen weightsSizeLanguagesEnglish 7-set WER
Whisper large-v3 (reference)yes3,090 MB996.04
Whisper large-v3-turbo (reference)yes1,618 MB996.58
Qwen3-ASR 1.7Byes4,080 MB30 and 22 Chinese dialects4.43
Parakeet TDT 0.6B v2yes2,472 MBEnglish4.82
Phonon-2.5 Englishyes158 MBEnglish4.82
Cohere Transcribe 03-2026yes4,000 MB144.85
Parakeet TDT 0.6B v3yes2,508 MB254.96
Phonon-2.5yes156 MB255.06
Canary 1B v2yes3,916 MB25, with translation5.81
ElevenLabs Scribe v2no, APIn/a90+4.06
AssemblyAI Universal-3.5 Prono, APIn/a184.46
Meta muse-voice-transcribeno, APIn/asee provider4.93

Keep Whisper, run it faster

These tools run the same Whisper weights, so accuracy stays the same and speed or memory changes:

  • faster-whisper: a reimplementation on CTranslate2; its README says up to 4 times faster than openai/whisper for the same accuracy, on CPU and NVIDIA GPU. pip install faster-whisper.
  • whisper.cpp: C/C++ with Metal on Apple silicon, the Neural Engine for the encoder through Core ML, integer quantisation, and bindings for more than ten languages.
  • mlx-whisper: Whisper on the GPU of an Apple silicon Mac from Python.
  • WhisperKit: Whisper in Swift apps on Apple silicon, with a Homebrew CLI.

Whisper's own README lists turbo (809 M parameters) at about 8 times the speed of large on an A100, and notes that turbo is not trained for translation.

When Whisper is still the right choice

  • Your audio is in a language the alternatives do not cover. Whisper's card lists 99 languages; Phonon-2.5 covers 25 European languages and Qwen3-ASR 30.
  • You need speech translated to English text. Whisper large-v3 and medium translate; turbo, Parakeet and Phonon do not. Canary 1B v2 translates between English and 24 languages.
  • Your pipeline already depends on Whisper's segment output or prompt behaviour and the error rate is acceptable.

Switching from the Whisper API

OpenAI's transcription route is POST /v1/audio/transcriptions. fermion serve exposes the same route shape for the Phonon models on your own machine, so an OpenAI client can point its base URL at it. Hosted Whisper is also sold by several providers; the speech-to-text API cost guide lists dated prices.

pip install fermion-research
fermion serve phonon-2.5
curl -s http://127.0.0.1:8000/v1/audio/transcriptions -F file=@clip.wav

Questions

What is more accurate than Whisper?
On the seven Open ASR Leaderboard English sets, Whisper large-v3 averages 6.04 % word error. Qwen3-ASR 1.7B (4.43 %), Parakeet TDT 0.6B v2 (4.82 %), Phonon-2.5 English (4.82 %) and Parakeet TDT 0.6B v3 (4.96 %) are all lower.
What is a smaller alternative to Whisper?
Phonon-2.5 is 156 MB and covers 25 European languages; on those 25 FLEURS sets it averages 11.71 % word error against 53.71 % for Whisper base, which is about the same size. Phonon-2.5 English is 158 MB.
How can I make Whisper faster without changing the model?
Use a faster runtime: faster-whisper (CTranslate2, up to 4 times faster than openai/whisper by its README), whisper.cpp, mlx-whisper on Apple silicon, or WhisperKit in Swift apps.
Which Whisper alternative supports the most languages?
Among open models here, Qwen3-ASR covers 30 languages and 22 Chinese dialects, and Phonon-2.5, Parakeet TDT 0.6B v3 and Canary 1B v2 cover 25. None matches Whisper's 99.
Can a Whisper alternative translate speech?
Canary 1B v2 translates between English and 24 languages according to its card. Phonon and Parakeet transcribe only.
Is there a drop-in replacement for the OpenAI Whisper API?
fermion serve exposes POST /v1/audio/transcriptions for the Phonon models, the same route shape as OpenAI's, so OpenAI clients work by changing the base URL. Whisper-specific options such as translation are not offered.

Related

Sources and method

  • Rival English averages: Open ASR Leaderboard, seven English sets (LibriSpeech clean and other, AMI, Earnings-22, GigaSpeech, SPGISpeech, VoxPopuli), snapshot 2026-09-25.
  • Phonon English averages: Fermion Research measurement with the Open ASR Leaderboard's scoring code on the full seven test sets (Phonon-2.5 release post, 2026-10-22).
  • FLEURS numbers: Fermion Research measurement on the full FLEURS test set of each language, one scoring code and normalisation for every model (Phonon-2.5 release post, 2026-10-22).
  • Whisper sizes, languages, turbo notes: github.com/openai/whisper and huggingface.co/openai/whisper-large-v3-turbo, read 2026-11-13.
  • faster-whisper: github.com/SYSTRAN/faster-whisper; whisper.cpp: github.com/ggml-org/whisper.cpp; WhisperKit: github.com/argmaxinc/WhisperKit; read 2026-11-13.
  • Languages: model cards of Qwen3-ASR 1.7B, Parakeet TDT 0.6B v2 and v3, Cohere Transcribe, Canary 1B v2 on Hugging Face; elevenlabs.io/pricing/api; assemblyai.com/pricing; read 2026-11-13.

Canonical: https://www.fermionresearch.com/speech/whisper-alternatives/