Fermion ResearchFermion Research

Glossary/facts checked 13 November 2026

On-device speech recognition

What changes when the model runs where the audio is recorded, what hardware it needs, and what it does not guarantee by itself.

Phonon-2.5 benchmarks and downloadsOpen weights on Hugging Face.

The short answer

On-device speech recognition means the model that turns speech into text runs on the hardware where the audio is: a laptop, phone, desktop or a server you control. The audio does not have to travel to a remote service to be transcribed, and after the model is downloaded it works without a network connection.

It became practical as models got small enough to fit in ordinary memory. Phonon-2.5 is a 156 MB download for 25 languages and runs on a plain laptop CPU with no GPU.

On-device, self-hosted server or cloud API

The three ways to run speech recognition differ mainly in where the audio goes and who pays for the compute.

On-deviceSelf-hosted serverCloud API
Where the audio goesStays on the deviceTo a machine you runTo the provider
Network needed to transcribeNo, after the model downloadOnly your own networkYes
Cost modelThe device you already haveYour GPU or CPU hoursPrice per audio hour
Model size limitWhat fits in the device's memoryWhat fits on your serverNone for you
UpdatesNew model file shipped to devicesYou redeployProvider changes it
Typical usesDictation, offline apps, field devicesBatch transcription, internal toolsLow volume, many languages, no ops

What makes it fast enough

Speech recognition on a device runs on whatever compute is there: the CPU, the GPU, or a neural processing unit such as Apple's Neural Engine. The same model can differ several times in speed between them. Phonon-2's published figures on one M5 MacBook Air show the spread: 40 times realtime on its CPU, 174 times realtime on its GPU through MLX, and 606 times realtime on the Neural Engine through Core ML.

Phonon-2.5 runs on Apple silicon through MLX, on plain CPUs on Linux (x86-64 and arm64) and x86-64 Windows, on NVIDIA GPUs, and on Android through an arm64 package. A Core ML package for the Neural Engine is public for Phonon-2.

pip install fermion-research
fermion transcribe phonon-2.5 file.wav

Trade-offs and what on-device does not promise

  • Coverage: a small model covers fewer languages than the largest cloud models. Phonon-2.5 covers 25 European languages; if you need Arabic, Hindi or Japanese, it is not the right model.
  • Accuracy: small does not have to mean worse, but the lowest English error rates on the Open ASR Leaderboard belong to larger models and closed APIs. In the 2026-09-25 snapshot the lowest seven-set average was 3.68 % (zoom/scribe_v2_pro, a closed API); Phonon-2.5 English scores 4.82 % on the same sets.
  • Privacy: running the model locally means the transcription step needs no upload. Whether audio or text leaves the device is still decided by the app around the model, so check its sharing settings.
  • Hardware spread: on phones and older laptops the same model runs slower, so measure on the slowest device you support.

Questions

What is on-device speech recognition?
Speech-to-text where the model runs on the device that recorded the audio, such as a laptop or phone, instead of on a remote service. Once the model is downloaded, transcription needs no network.
Does on-device speech recognition need the internet?
Only to download the model. After that the transcription itself runs offline, though the app around it may still use the network for other things.
Is on-device speech recognition less accurate than cloud APIs?
Not necessarily. Phonon-2.5 averages 11.71 % word error on FLEURS across its 25 languages, lower than the 2,508 MB Parakeet TDT 0.6B v3 at 13.26 % on the same scoring, but some larger models and closed APIs score lower on English.
Does on-device mean my audio is private?
It means the transcription step does not need to upload audio. Whether recordings or text are sent anywhere depends on the app, so read its settings and policy.
What hardware do I need?
A recent laptop CPU is enough for Phonon-2.5. Apple silicon Macs can also use the GPU through MLX, and NVIDIA GPUs and Android arm64 devices are supported.

Related

Sources and method

  • Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.
  • Speed figures: Phonon-2's published figures (/models/phonon-2/ and /research/phonon-2/), M5 MacBook Air, AMD Zen 5, Google Axion and H100 hosts as stated.
  • Lowest seven-set average: Open ASR Leaderboard, seven English sets, snapshot 2026-09-25.
  • Phonon command line and server behaviour: Fermion Research docs (/docs/cli-reference/, /docs/server-reference/), read 2026-11-13.

Canonical: https://www.fermionresearch.com/glossary/on-device-speech-recognition/