Docs / Reference
CLI reference
Every command and the shared runtime, sampling, context, and backend flags in fermion 0.1.14.
Commands
| Command | Purpose | Command-specific options |
|---|---|---|
fermion chat | Interactive streaming REPL. | --draft |
fermion generate | One-shot completion, greedy by default. | prompt, --raw, --draft |
fermion info | Header, completeness, SHA-256, and selected backend. | --no-checksum |
fermion serve | OpenAI-compatible local HTTP service. | host, port, auth, sessions, tools, draft |
fermion inspect | Container geometry, occupancy, bytes, and KV cost. | --layer, --projection, --sha256, --json |
fermion verify | Compare plain and drafted token streams. | --draft required, --prompts |
fermion bench | Single-stream local benchmark. | --tokens, --prompt, --warmup, --draft |
Shared flags and defaults
| Flag | Default | Meaning |
|---|---|---|
--model | FermionResearch/Neutrino-8B | A local artifact, model directory, or Hugging Face repository id. |
--device | cpu | cpu, mps, or cuda. Native execution requires cpu. |
--dtype | auto | PyTorch activation dtype: auto, bfloat16, float16, or float32. |
--max-new | 256 generate; 1024 chat/serve | Maximum generated tokens. |
--temperature | 0 generate; 0.01 chat/serve | Zero selects deterministic argmax. |
--top-p | 1.0 | Nucleus sampling when temperature is above zero. |
--rep-penalty | 1.0 generate; 1.05 chat/serve | Multiplicative repetition penalty. |
--pen-window | 256 | Recent-token window used by repetition penalty. |
--backend | auto | auto, native, or torch. native fails if the optimized runner is unavailable. |
--native-bin | none | Use a specific native runner binary. |
--kv-dtype | fp16 | Native KV storage: fp16, fp32, or int8. |
--yarn-factor | off | Experimental native-only RoPE scaling. |
--yarn-orig-max | model maximum | Original context boundary used by YaRN. |
Serve-only flags
| Flag | Default | Meaning |
|---|---|---|
--host | 127.0.0.1 | Bind address. |
--port | 8000 | Listening port. |
--draft | none | Draft path, model id, or auto. |
--api-key | none | Require a bearer key on /v1 routes. |
--cors | off | Allow cross-origin browser requests. |
--served-model-name | container filename | Model id reported by the API. |
--max-new-ceiling | 4096 | Clamp client token requests to a server maximum. |
--session-ctx | 8192 | Resident native KV capacity. |
--no-session | off | Disable the resident session runner. |
--stuck-detector | enforce | enforce, observe, or off. |
--stuck-repeat | 3 | Repeated identical-call threshold. |
--stuck-alt | 6 | Strict alternation window. |
--stuck-error | 2 | Repeated identical-error threshold. |
--stuck-whitelist | empty | Comma-separated tools that may repeat. |
--stuck-no-auto-whitelist | off | Disable automatic poll-tool detection. |
--tool-profile | graded | graded, vendor, vendor-thinking, or antiloop. |
Built-in help
shell
fermion --version
fermion --help
fermion serve --help