Fermion ResearchFermion Research

Speech recognition/facts checked 13 November 2026

The best way to run speech-to-text on Apple silicon

An M-series Mac has three places to run a speech model: the CPU, the GPU and the Neural Engine. The tool you pick decides which one does the work, and which models you can use.

Speech recognition docs

The short answer

For a Python script or a command line on a Mac, use an MLX tool (mlx-whisper, mlx-audio, or the fermion package for the Phonon models), which runs the model on the GPU; for a Swift app that should leave the GPU free, use a Core ML package on the Neural Engine, such as WhisperKit for Whisper, FluidAudio for Parakeet, or the Phonon-2 Core ML package. whisper.cpp does both from C/C++, and Apple's own SpeechAnalyzer is built into macOS and iOS 26.

The model matters as much as the engine. Whisper gives 99 languages; Parakeet TDT 0.6B v3 and Phonon-2.5 give 25 European languages; Parakeet v2, Phonon-2.5 English and Phonon-2 are English only.

Options compared

Facts from each project's own README or documentation, read 2026-11-13.

ToolRuns onModelsBest forInstall
mlx-whisperGPU (MLX)Whisper, any size; optional 4-bit conversionPython scriptspip install mlx-whisper
mlx-audioGPU (MLX)Whisper, Parakeet, Voxtral, Qwen3-ASR, Canary, Moonshine, Phonon-2 and 2.5, othersTrying many models from one packagepip install -U mlx-audio
fermion (Phonon)GPU (MLX); CPU engine also availablePhonon-2.5 (25 languages), Phonon-2.5 English, Phonon-2Small models, CLI and an OpenAI-compatible local serverpip install fermion-research
Phonon-2 Core MLNeural EnginePhonon-2 (English)Mac apps that keep the GPU freeSwift and Python runners on GitHub
whisper.cppGPU (Metal); encoder on the Neural Engine via Core MLWhisper tiny to largeC/C++ apps, many language bindingsbuild from source
WhisperKitApple silicon, SwiftWhisper tiny to large-v3 turboSwift apps on macOS 14 or laterSwift Package Manager; brew install whisperkit-cli
FluidAudioNeural Engine (Core ML)Parakeet TDT v3 (25 European languages and Japanese), v2 (English)Swift apps that also need diarization or VADSwift Package Manager or CocoaPods
SpeechAnalyzer (Apple)built into the OSApple's own modelsApps on iOS, iPadOS and macOS 26 or laterSpeech framework

GPU, Neural Engine or CPU

  • GPU through MLX: the simplest path from Python, and fast. Phonon-2 runs at 174 times realtime on the GPU of a base M5 MacBook Air through MLX (Phonon-2's published figure).
  • Neural Engine through Core ML: the right place for an app that runs all day, because it leaves the GPU and most of the CPU free. Phonon-2 Core ML runs at 606 times realtime on the same M5 MacBook Air (Phonon-2's published figure). FluidAudio's README gives about 190 times realtime for Parakeet on an M4 Pro, its own figure on a different machine, so the two are not a head-to-head.
  • CPU: works everywhere, slower. Phonon-2 runs at 40 times realtime on the CPU of an M5 MacBook Air (published figure). whisper.cpp also runs on the CPU.

Setup in one minute

Whisper through MLX, and Phonon through the fermion package, from a terminal on an Apple silicon Mac:

# Whisper on MLX (needs ffmpeg: brew install ffmpeg)
pip install mlx-whisper
mlx_whisper audio_file.mp3

# Phonon on MLX
pip install "fermion-research[mlx]"
fermion transcribe phonon-2.5 meeting.wav
fermion transcribe phonon-2.5-en meeting.wav --json

Caveats

  • Whisper on any engine is the same model: the engine changes speed and memory, not the error rate. To lower the error, change the model.
  • The fermion transcribe command reads wav, flac, ogg and aiff; for mp3 and m4a it prints the ffmpeg command to convert first.
  • A Phonon-2.5 Core ML package is not public yet; Phonon-2.5 runs on Apple silicon through MLX today.
  • If you only want dictation into any app rather than a library, Detta, our Mac dictation app, runs speech recognition on the Neural Engine. It asks for a Google sign-in, and sharing dictations to improve our models is on by default with an off switch in Settings.

Questions

What is the fastest way to transcribe audio on an M-series Mac?
Running the model on the Neural Engine through Core ML is usually fastest for a single stream: Phonon-2 Core ML transcribes at 606 times realtime on an M5 MacBook Air, against 174 times on the same Mac's GPU through MLX (Phonon-2's published figures).
Does Whisper run on the Apple Neural Engine?
Yes, in part or in full depending on the tool. whisper.cpp can run Whisper's encoder on the Neural Engine through Core ML, and WhisperKit is a Swift SDK for running Whisper on-device on Apple silicon.
Can I run Parakeet on a Mac?
Yes. FluidAudio runs Parakeet TDT v3 and v2 on the Neural Engine from Swift, and mlx-audio runs Parakeet on the GPU from Python.
Is MLX or Core ML better for speech recognition?
MLX is simpler from Python and uses the GPU; Core ML can target the Neural Engine, which suits apps that run in the background. Both run the same model accuracy.
Does macOS have built-in speech-to-text for developers?
Yes. Apple's Speech framework includes SpeechAnalyzer and the SpeechTranscriber module, available on iOS, iPadOS and macOS 26 or later.
Which model should I use for languages other than English?
Whisper covers 99 languages. For the 25 European languages, Phonon-2.5 (156 MB) and Parakeet TDT 0.6B v3 both cover the same set; Phonon-2.5 averages 11.71 % word error on FLEURS and Parakeet TDT 0.6B v3 13.26 %, measured with one scoring code.

Related

Sources and method

  • mlx-whisper: github.com/ml-explore/mlx-examples/tree/main/whisper, read 2026-11-13. mlx-audio: github.com/Blaizzy/mlx-audio, read 2026-11-13.
  • whisper.cpp: github.com/ggml-org/whisper.cpp, read 2026-11-13. WhisperKit: github.com/argmaxinc/WhisperKit, read 2026-11-13. FluidAudio (including its 190x M4 Pro figure): github.com/FluidInference/FluidAudio, read 2026-11-13.
  • SpeechAnalyzer and SpeechTranscriber: developer.apple.com/documentation/speech/speechanalyzer, read 2026-11-13.
  • Phonon-2 speed: Phonon-2 model page, published figures, base M5 MacBook Air 16 GB. No Phonon-2.5 speed figures are published.
  • FLEURS numbers: Fermion Research measurement on the full FLEURS test set of each language, one scoring code and normalisation for every model (Phonon-2.5 release post, 2026-10-22).

Canonical: https://www.fermionresearch.com/speech/best-speech-to-text-for-apple-silicon/