---
title: "Choose and verify a backend"
description: "Fermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere."
canonical: "https://www.fermionresearch.com/docs/backends/"
source: "Fermion Research"
---

Docs / Getting started

# Choose and verify a backend

Fermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere.

## Platform matrix

This table is for the language models (`fermion chat`, `fermion generate`, and a Neutrino`fermion serve`). Speech backends are different: on Apple silicon, Phonon runs on the MLX engine; on Linux (x86-64 and 64-bit ARM), x86-64 Windows, and on a Mac without the MLX runtime or with`FERMION_DEVICE=cpu`, it runs on the CPU engine. See Speech platforms below.

| Platform | Default path | Status |
| --- | --- | --- |
| macOS arm64 | Native CPU | Optimized path and local transport expansion |
| Linux x86-64 | Native CPU | Optimized path |
| Linux arm64 | PyTorch | Reference path |
| Windows x86-64 | PyTorch | Reference path |
| CUDA or MPS device | PyTorch | Reference path |

## See what your machine selected

```text
fermion info
fermion bench --tokens 64
```

`info` names the active backend and explains why. Published throughput figures describe the native path. `bench` measures the exact path active on the current machine.

## Pin or require a backend

```text
fermion chat --backend native
fermion generate "Hello" --backend torch --dtype float32
```

`--backend native` fails rather than silently using a slower path. Use `--native-bin` or `FERMION_NATIVE_BIN` to select a specific runner.

## Native thread settings

The native runner uses one fewer thread than the detected CPU count, capped at 16. Set `FERMION_THREADS` to choose a count, and raise the ceiling deliberately with `FERMION_MAX_THREADS`.

```text
FERMION_THREADS=12 fermion bench
FERMION_MAX_THREADS=24 FERMION_THREADS=24 fermion bench
```

## Speech platforms

Phonon speech models run on a Mac (via MLX), an NVIDIA GPU, or an ordinary CPU. On a Mac, the pip package is the complete runtime. On NVIDIA hardware, the Docker image runs the same weights:

```text
docker run --rm --gpus all ghcr.io/fermionresearch/phonon-cuda:latest \
  transcribe recording.wav
```

On Linux (x86-64 and 64-bit ARM) and x86-64 Windows, the same commands run on the CPU, no GPU required:

```text
pip install fermion-research torch safetensors soundfile scipy zstandard
fermion transcribe recording.wav
```

On Linux, installing torch from its CPU wheel index (`--index-url https://download.pytorch.org/whl/cpu`) skips the much larger GPU build. On Linux x86-64 the CPU engine needs glibc 2.34 or newer (Ubuntu 22.04, Debian 12, RHEL 9 and later) and about 7 GB of free RAM. On Windows the plain torch wheel already is the CPU build; a clean machine may also need Microsoft's `vc_redist.x64.exe` (the fix when `import torch` fails with WinError 126). A CPU image runs the same weights anywhere Docker does, Docker Desktop included:

```text
docker run --rm ghcr.io/fermionresearch/phonon-cpu:latest \
  transcribe recording.wav
```

Every container serves the same endpoint as `fermion serve`, so a client written against one works unmodified against the others. The Neutrino backend and sampler flags have no meaning for a speech model; if you set one, the server says so once at startup and ignores it.

Source: [https://www.fermionresearch.com/docs/backends/](https://www.fermionresearch.com/docs/backends/)