---
title: "On-device speech recognition"
description: "On-device speech recognition runs the speech-to-text model on your own laptop, phone or server, so audio is not sent off to be transcribed. How it compares."
canonical: "https://www.fermionresearch.com/glossary/on-device-speech-recognition/"
source: "Fermion Research"
---

Glossary/facts checked 13 November 2026

# On-device speech recognition

What changes when the model runs where the audio is recorded, what hardware it needs, and what it does not guarantee by itself.

[Phonon-2.5 benchmarks and downloads](https://www.fermionresearch.com/models/phonon-2-5/)Open weights on Hugging Face.

## The short answer

On-device speech recognition means the model that turns speech into text runs on the hardware where the audio is: a laptop, phone, desktop or a server you control. The audio does not have to travel to a remote service to be transcribed, and after the model is downloaded it works without a network connection.

It became practical as models got small enough to fit in ordinary memory. Phonon-2.5 is a 156 MB download for 25 languages and runs on a plain laptop CPU with no GPU.

## On-device, self-hosted server or cloud API

The three ways to run speech recognition differ mainly in where the audio goes and who pays for the compute.

|  | On-device | Self-hosted server | Cloud API |
| --- | --- | --- | --- |
| Where the audio goes | Stays on the device | To a machine you run | To the provider |
| Network needed to transcribe | No, after the model download | Only your own network | Yes |
| Cost model | The device you already have | Your GPU or CPU hours | Price per audio hour |
| Model size limit | What fits in the device's memory | What fits on your server | None for you |
| Updates | New model file shipped to devices | You redeploy | Provider changes it |
| Typical uses | Dictation, offline apps, field devices | Batch transcription, internal tools | Low volume, many languages, no ops |

## What makes it fast enough

Speech recognition on a device runs on whatever compute is there: the CPU, the GPU, or a neural processing unit such as Apple's Neural Engine. The same model can differ several times in speed between them. Phonon-2's published figures on one M5 MacBook Air show the spread: 40 times realtime on its CPU, 174 times realtime on its GPU through MLX, and 606 times realtime on the Neural Engine through Core ML.

Phonon-2.5 runs on Apple silicon through MLX, on plain CPUs on Linux (x86-64 and arm64) and x86-64 Windows, on NVIDIA GPUs, and on Android through an arm64 package. A Core ML package for the Neural Engine is public for Phonon-2.

```text
pip install fermion-research
fermion transcribe phonon-2.5 file.wav
```

## Trade-offs and what on-device does not promise

- Coverage: a small model covers fewer languages than the largest cloud models. Phonon-2.5 covers 25 European languages; if you need Arabic, Hindi or Japanese, it is not the right model.
- Accuracy: small does not have to mean worse, but the lowest English error rates on the Open ASR Leaderboard belong to larger models and closed APIs. In the 2026-09-25 snapshot the lowest seven-set average was 3.68 % (zoom/scribe_v2_pro, a closed API); Phonon-2.5 English scores 4.82 % on the same sets.
- Privacy: running the model locally means the transcription step needs no upload. Whether audio or text leaves the device is still decided by the app around the model, so check its sharing settings.
- Hardware spread: on phones and older laptops the same model runs slower, so measure on the slowest device you support.

## Questions

## Related

- [On-device and on-premises speech recognition for teams](https://www.fermionresearch.com/deploy/on-device-speech-recognition/)
- [Real-time factor and RTFx](https://www.fermionresearch.com/glossary/real-time-factor/)
- [Phonon-2 on Core ML](https://www.fermionresearch.com/models/phonon-2/core-ml/)
- [Self-host a speech-to-text server](https://www.fermionresearch.com/models/phonon-2-5/self-host/)
- [On-device speech-to-text for apps](https://www.fermionresearch.com/speech/on-device-speech-to-text-for-apps/)

## Sources and method

- Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.
- Speed figures: Phonon-2's published figures (/models/phonon-2/ and /research/phonon-2/), M5 MacBook Air, AMD Zen 5, Google Axion and H100 hosts as stated.
- Lowest seven-set average: Open ASR Leaderboard, seven English sets, snapshot 2026-09-25.
- Phonon command line and server behaviour: Fermion Research docs (/docs/cli-reference/, /docs/server-reference/), read 2026-11-13.

Canonical: https://www.fermionresearch.com/glossary/on-device-speech-recognition/

Source: [https://www.fermionresearch.com/glossary/on-device-speech-recognition/](https://www.fermionresearch.com/glossary/on-device-speech-recognition/)
