What it is
Phonon-2 Core ML is the Phonon-2 speech model (English, 5.21 % average word error on the seven Open ASR Leaderboard sets) compiled for Core ML so that it runs on the Apple Neural Engine. On an M5 MacBook Air it transcribes at 606 times realtime, against 174 times realtime for the same model on the Mac's GPU.
The package comes with a Swift runner and a Python runner (github.com/fermionresearch/phonon-coreml). Phonon-2.5 (25 languages) and Phonon-2.5 English are also published as Core ML packages on Hugging Face (FermionResearch/Phonon-2.5-CoreML and FermionResearch/Phonon-2.5-English-CoreML); Detta, the free dictation app for the Mac, runs Phonon-2.5 on the Neural Engine.
Why the Neural Engine
- Speed: 606 times realtime single-stream on a base MacBook Air.
- Memory and power: the Neural Engine holds the model, so a dictation app stays small while idle.
- The GPU stays free for the app's own work.
Run it
The Hugging Face repository holds the compiled package and the runners' instructions; the GitHub repository holds the Swift and Python code.
git clone https://github.com/fermionresearch/phonon-coreml
# follow README: download FermionResearch/Phonon-2-CoreML, run the Swift or Python example on a WAV filePhonon-2.5 and the Neural Engine
Phonon-2.5 (25 languages) runs on Apple silicon through the MLX engine in the fermion runtime, and on the Neural Engine through its Core ML packages on Hugging Face, FermionResearch/Phonon-2.5-CoreML and FermionResearch/Phonon-2.5-English-CoreML. Detta uses Phonon-2.5 this way.
Questions
- How fast is speech recognition on the Apple Neural Engine?
- Phonon-2 Core ML transcribes at 606 times realtime on an M5 MacBook Air, an hour of audio in about six seconds. The same model on the Mac's GPU runs at 174 times realtime.
- Which Macs can run it?
- Macs with Apple silicon. The measured figure is from a base M5 MacBook Air with 16 GB.
- Is there a Swift example?
- Yes, the phonon-coreml repository on GitHub has Swift and Python runners.
- Does the Core ML model lose accuracy?
- The Phonon-2 model page lists word error for the fast path against the exact path per surface; the differences are small and printed beside each figure.
- Can I use it in my own Mac or iOS app?
- Yes. The weights are on Hugging Face and the package and Swift and Python runners are public.
- Is there a Core ML package for Phonon-2.5?
- Yes. FermionResearch/Phonon-2.5-CoreML (25 languages) and FermionResearch/Phonon-2.5-English-CoreML are on Hugging Face.
Related
Sources and method
- Speed and word error: Phonon-2 model page (/models/phonon-2/), published figures, M5 MacBook Air 16 GB.
- Runners: github.com/fermionresearch/phonon-coreml; package: huggingface.co/FermionResearch/Phonon-2-CoreML.
Canonical: https://www.fermionresearch.com/models/phonon-2/core-ml/