Choose and verify a backend

This table is for the language models (fermion chat, fermion generate, and a Neutrinofermion serve). Speech backends are different: on Apple silicon, Phonon runs on the MLX engine; on Linux (x86-64 and 64-bit ARM), x86-64 Windows, and on a Mac without the MLX runtime or withFERMION_DEVICE=cpu, it runs on the CPU engine. See Speech platforms below.

fermion info
fermion bench --tokens 64

info names the active backend and explains why. Published throughput figures describe the native path. bench measures the exact path active on the current machine.

fermion chat neutrino-8b --backend native
fermion generate neutrino-8b "Hello" --backend torch --dtype float32

--backend native fails rather than silently using a slower path. Use --native-bin or FERMION_NATIVE_BIN to select a specific runner.

The native runner uses one fewer thread than the detected CPU count, capped at 16. Set FERMION_THREADS to choose a count, and raise the ceiling deliberately with FERMION_MAX_THREADS.

FERMION_THREADS=12 fermion bench
FERMION_MAX_THREADS=24 FERMION_THREADS=24 fermion bench

Phonon-2 runs on Apple silicon through MLX, the default engine on a Mac, and on plain CPUs on macOS, Linux (x86-64 and 64-bit ARM), and x86-64 Windows. On a Mac, the pip package plus the MLX stack is the complete runtime:

pip install fermion-research
pip install mlx mlx-audio mlx-lm soundfile scipy zstandard
fermion models
fermion transcribe phonon-2 meeting.wav

On Linux and Windows the same commands run on the CPU engine, no GPU required; fermion listen (live microphone) is Apple silicon only, transcribe and serve run everywhere:

pip install fermion-research torch safetensors soundfile scipy zstandard
fermion transcribe phonon-2 meeting.wav

On Linux, install torch from its CPU wheel index first (pip install --no-deps torch --index-url https://download.pytorch.org/whl/cpu) to skip the much larger GPU build. The Linux x86-64 engine needs glibc 2.28 or newer (Ubuntu 18.10 or newer, Debian 10+, RHEL 8+) and falls back to full precision on older systems; Linux on 64-bit ARM runs the same engine. On Windows the plain torch wheel already is the CPU build; a clean machine may also need Microsoft's vc_redist.x64.exe (the fix when import torch fails with WinError 126). Transcription is deterministic on every engine, and transcripts across engines are same-accuracy, not word-identical. FERMION_DEVICE=cpu selects the CPU engine on a Mac. Intel Macs run Phonon-2 on the CPU engine with Python 3.10 to 3.12; the Phonon-1 family is not available on Intel Macs.

The CPU engine transcribes a single stream at 143 times realtime on eight Zen 5 cores (16 vCPU), 52 times on Google Axion, and 40 times on the CPU of an M5 MacBook Air; through MLX, an M5 MacBook Air reaches 174 times realtime. fermion speech bench measures the machine in front of you:

fermion speech bench --audio recording.wav

A CPU image runs the same weights anywhere Docker does, Docker Desktop included; the same image runs on Arm (linux/arm64) and x86-64:

docker run --rm -v "$PWD":/audio -v phonon-cache:/home/phonon/.cache ghcr.io/fermionresearch/phonon-cpu:2.0.9 transcribe phonon-2 /audio/recording.wav

On NVIDIA GPUs, the CUDA image runs the same weights:

docker run --rm --gpus all -v "$PWD":/audio -v phonon-cache:/home/phonon/.cache ghcr.io/fermionresearch/phonon-cuda:1.0.8 transcribe phonon-2 /audio/recording.wav

Every container serves the same endpoint as fermion serve, so a client written against one works unmodified against the others. The Neutrino backend and sampler flags have no meaning for a speech model; if you set one, the server says so once at startup and ignores it.