---
title: "Choose and verify a backend"
description: "Fermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere."
canonical: "https://www.fermionresearch.com/docs/backends/"
source: "Fermion Research"
---

Docs / Getting started

# Choose and verify a backend

Fermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere.

## Platform matrix

This table is for the language models (`fermion chat`, `fermion generate`, and a Neutrino`fermion serve`). Speech backends are different: on Apple silicon, Phonon runs on the MLX engine; on Linux (x86-64 and 64-bit ARM), x86-64 Windows, and on a Mac without the MLX runtime or with`FERMION_DEVICE=cpu`, it runs on the CPU engine. See Speech platforms below.

| Platform | Default path | Status |
| --- | --- | --- |
| macOS arm64 | Native CPU | Optimized path and local transport expansion |
| Linux x86-64 | Native CPU | Optimized path |
| Linux arm64 | PyTorch | Reference path |
| Windows x86-64 | PyTorch | Reference path |
| CUDA or MPS device | PyTorch | Reference path |

## See what your machine selected

```text
fermion info
fermion bench --tokens 64
```

`info` names the active backend and explains why. Published throughput figures describe the native path. `bench` measures the exact path active on the current machine.

## Pin or require a backend

```text
fermion chat neutrino-8b --backend native
fermion generate neutrino-8b "Hello" --backend torch --dtype float32
```

`--backend native` fails rather than silently using a slower path. Use `--native-bin` or `FERMION_NATIVE_BIN` to select a specific runner.

## Native thread settings

The native runner uses one fewer thread than the detected CPU count, capped at 16. Set `FERMION_THREADS` to choose a count, and raise the ceiling deliberately with `FERMION_MAX_THREADS`.

```text
FERMION_THREADS=12 fermion bench
FERMION_MAX_THREADS=24 FERMION_THREADS=24 fermion bench
```

## Speech platforms

Phonon-2.5, Phonon-2.5 English and Phonon-2 run on Apple silicon through MLX, the default engine on a Mac, and on plain CPUs on Linux (x86-64 and 64-bit ARM) and x86-64 Windows. On a Mac, the pip package plus the MLX stack is the complete runtime:

```text
pip install fermion-research
pip install mlx mlx-audio mlx-lm soundfile scipy zstandard
fermion models
fermion transcribe phonon-2.5 meeting.wav
fermion transcribe phonon-2 meeting.wav
```

On Linux and Windows the same commands run on the CPU engine, no GPU required; `fermion listen` (live microphone) is Apple silicon only, `transcribe` and `serve` run everywhere:

```text
pip install fermion-research torch safetensors soundfile scipy zstandard
fermion transcribe phonon-2.5 meeting.wav
fermion transcribe phonon-2 meeting.wav
```

On Linux, install torch from its CPU wheel index first (`pip install --no-deps torch --index-url https://download.pytorch.org/whl/cpu`) to skip the much larger GPU build. The Linux x86-64 engine needs glibc 2.28 or newer (Ubuntu 18.10 or newer, Debian 10+, RHEL 8+) and falls back to full precision on older systems; Linux on 64-bit ARM runs the same engine. On Windows the plain torch wheel already is the CPU build; a clean machine may also need Microsoft's `vc_redist.x64.exe` (the fix when `import torch` fails with WinError 126). Transcription is deterministic on every engine, and transcripts across engines are same-accuracy, not word-identical. `FERMION_DEVICE=cpu` selects the CPU engine on a Mac. Intel Macs run Phonon-2 on the CPU engine with Python 3.10 to 3.12; the Phonon-1 family is not available on Intel Macs.

Phonon-2.5 and Phonon-2.5 English run on the same engines as Phonon-2: MLX on Apple silicon, the CPU engine on Linux (x86-64 and 64-bit ARM) and x86-64 Windows, and the CUDA path on NVIDIA GPUs. They run at about the same speed as Phonon-2, and they will get a lot faster in the coming weeks.

With Phonon-2, the CPU engine transcribes a single stream at 143 times realtime on eight Zen 5 cores (16 vCPU), 52 times on Google Axion, and 40 times on the CPU of an M5 MacBook Air; through MLX, an M5 MacBook Air reaches 174 times realtime. `fermion speech bench` measures the machine in front of you:

```text
fermion speech bench --audio recording.wav
```

A CPU image runs the same weights anywhere Docker does, Docker Desktop included; the same image runs on Arm (linux/arm64) and x86-64:

```text
docker run --rm -v "$PWD":/audio -v phonon-cache:/home/phonon/.cache ghcr.io/fermionresearch/phonon-cpu:2.0.12 transcribe phonon-2.5 /audio/recording.wav
```

On NVIDIA GPUs, the CUDA image runs the same weights:

```text
docker run --rm --gpus all -v "$PWD":/audio -v phonon-cache:/home/phonon/.cache ghcr.io/fermionresearch/phonon-cuda:1.0.11 transcribe phonon-2.5 /audio/recording.wav
```

Every container serves the same endpoint as `fermion serve`, so a client written against one works unmodified against the others. The Neutrino backend and sampler flags have no meaning for a speech model; if you set one, the server says so once at startup and ignores it.

Source: [https://www.fermionresearch.com/docs/backends/](https://www.fermionresearch.com/docs/backends/)
