Browse documentation

Documentation / Overview

Fermion runtime documentation

Run Neutrino models locally, expose an OpenAI-compatible API, call tools, inspect artifacts, and measure the active runtime.

fermion 0.1.14Python 3.10 to 3.13macOS and LinuxApache-2.0

Quickstart

Start a local chat

Python 3.10 or newer. The first run downloads and verifies the model.

Full quickstart

shell

pip install fermion-research
fermion chat

Start building

Choose the path closest to what you are trying to do.

Install and runInstall the package and start a local Neutrino chat.Serve an APIRun an OpenAI-compatible endpoint with streaming and tools.Use speculative decodingPair the 8B model with the 0.6B draft and verify identity.Read the CLI referenceFind commands, shared flags, defaults, and exit behavior.

Guides and reference

Getting startedModels, downloads, and storageKnow what the CLI downloads, where it stores the files, and how much free space the first run needs.Getting startedChoose and verify a backendFermion selects an optimized native CPU runtime where available and a PyTorch reference path elsewhere.GuidesChat and generateUse the interactive session for conversation and the one-shot command for deterministic scripts.GuidesTool callingServe Neutrino-8B to an agent framework using OpenAI tool schemas and standard tool-call responses.ReferenceServer API referenceRoutes, request fields, streaming frames, errors, context clamps, and operational health data.ReferenceEnvironment variablesConfigure caches, transport behavior, native execution, sampling, sessions, and Hugging Face progress.