Docs / Reference
CLI reference
Every command and the shared runtime, sampling, context, and backend flags in fermion 0.1.24.
Commands
| Command | Purpose | Command-specific options |
|---|---|---|
fermion chat | Interactive streaming REPL. | --draft |
fermion generate | One-shot completion, greedy by default. | prompt, --raw, --draft |
fermion transcribe | One-shot speech-to-text for an audio file. | audio, --model, --json, --verbose, --download-only |
fermion listen | Live microphone transcription. | --model, --wav, --verbose |
fermion info | Header, completeness, SHA-256, and selected backend. | --no-checksum |
fermion serve | OpenAI-compatible local HTTP service. | host, port, auth, sessions, tools, draft |
fermion inspect | Container geometry, occupancy, bytes, and KV cost. | --layer, --projection, --sha256, --json |
fermion verify | Compare plain and drafted token streams. | --draft required, --prompts |
fermion bench | Single-stream local benchmark. | --tokens, --prompt, --warmup, --draft |
Shared flags and defaults
| Flag | Default | Meaning |
|---|---|---|
--model | FermionResearch/Neutrino-8B | A local artifact, model directory, or Hugging Face repository id. |
--device | cpu | cpu, mps, or cuda. Native execution requires cpu. |
--dtype | auto | PyTorch activation dtype: auto, bfloat16, float16, or float32. |
--max-new | 256 generate; 1024 chat/serve | Maximum generated tokens. |
--temperature | 0 generate; 0.01 chat/serve | Zero selects deterministic argmax. |
--top-p | 1.0 | Nucleus sampling when temperature is above zero. |
--rep-penalty | 1.0 generate; 1.05 chat/serve | Multiplicative repetition penalty. |
--pen-window | 256 | Recent-token window used by repetition penalty. |
--backend | auto | auto, native, or torch. native fails if the optimized runner is unavailable. |
--native-bin | none | Use a specific native runner binary. |
--kv-dtype | fp16 | Native KV storage: fp16, fp32, or int8. |
--yarn-factor | off | Experimental native-only RoPE scaling. |
--yarn-orig-max | model maximum | Original context boundary used by YaRN. |
Serve-only flags
| Flag | Default | Meaning |
|---|---|---|
--host | 127.0.0.1 | Bind address. |
--port | 8000 | Listening port. |
--draft | none | Draft path, model id, or auto. |
--api-key | none | Require a bearer key on /v1 routes. |
--cors | off | Allow cross-origin browser requests. |
--served-model-name | container filename | Model id reported by the API. |
--max-new-ceiling | 4096 | Clamp client token requests to a server maximum. |
--session-ctx | 8192 | Resident native KV capacity. |
--no-session | off | Disable the resident session runner. |
--stuck-detector | enforce | enforce, observe, or off. |
--stuck-repeat | 3 | Repeated identical-call threshold. |
--stuck-alt | 6 | Strict alternation window. |
--stuck-error | 2 | Repeated identical-error threshold. |
--stuck-whitelist | empty | Comma-separated tools that may repeat. |
--stuck-no-auto-whitelist | off | Disable automatic poll-tool detection. |
--tool-profile | graded | graded, vendor, vendor-thinking, or antiloop. |
Speech commands
shell
fermion transcribe clip.wav
fermion listen --wav clip.wav
fermion serve --model phonon| Flag | Default | Meaning |
|---|---|---|
--model | FermionResearch/Phonon-1 | Speech repository id, alias, or a local model directory. |
--json | off | transcribe: emit a JSON object with the text, model id, and decode time. |
--download-only | off | transcribe: fetch and verify the model, print its directory, and stop. |
--wav | none | listen: stream a file through the live path, paced to real time, instead of the microphone. |
--verbose | off | Print the decode configuration and timings to standard error. |
Both commands decode greedily with no sampler flags, and standard output carries only the transcript; progress and warnings go to standard error. Exit status is 0 on success, 1 for a refusal or failure with one plain message on standard error, and 2 for a usage error.
With a speech model, fermion serve mounts POST /v1/audio/transcriptions and GET /v1/audio/stream instead of the chat routes. --host, --port, --api-key, --cors, and --served-model-name keep their meaning; the sampler and backend flags do not apply, and the server says so once at startup and ignores them.
Built-in help
shell
fermion --version
fermion --help
fermion serve --help