Neutrino-1 8B, 0.6B, and 0.6B-Chat use the same file format and local runtime across CUDA, Apple silicon, and x86.
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-8B
The family
Three models share one container format and one set of binaries. The 8B carries 36 decoder layers at a hidden width of 4,096; both 0.6B models carry 28 at 1,024, and tie their embedding table so a single tensor serves the input lookup and the output head. Because the runtime reads all three without a rebuild, the 0.6B loads into the 8B's own process as its drafting model rather than standing up a second deployment.
flagship
certified draft
conversational
The format only touches the transformer linears. Token embeddings stay int8 because their rows are read one token at a time instead of being multiplied against the whole activation stream, and that split moves with scale. At 8B the coded lane holds 84.8% of the parameters in 67.2% of the bytes. At 0.6B it holds 73.9% of the parameters in 50.4% of the bytes, and the untouched vocabulary swells from a third of the file to nearly half. Both 0.6B models share that split exactly: the lane sizes follow from the tensor shapes, so the chat SKU is the same 328 MB down to the byte.
36 layers
28 layers
28 layers
Neutrino-1 0.6B
Speculative decoding turns one large forward pass into several emitted tokens, and it only pays when the small model proposing those tokens is cheap enough to run six times in the gaps. Neutrino-1 0.6B is that model: 596 million parameters in a 328 MB container, 238 MB on the wire, decoding above 200 tokens a second on the native CPU path. The 8B accepts a proposal only when it equals its own argmax, which is why the drafted stream is the plain greedy stream and the certificate reads 27,648 tokens with zero divergences.
The same property makes it a model in its own right. It is the identical container format, read by the identical binaries, so a laptop that runs the 8B runs this one at nine times the rate from a file that fits on a phone.
16 query heads share 8 key-value heads at a head width of 128, and the embedding table is tied: one int8 tensor serves both the input lookup and the output head.
The share of the file held by that tied vocabulary. It is 26.1% of the parameters and nearly half the bytes, because the format touches the 196 transformer linears and leaves the token table alone.
Single-stream decode from the shipped binary with no GPU involved. The container is memory-mapped and executed as it arrives: there is no dequantization pass at load.
Median of five 512-token greedy runs through the custom Metal kernels, peaking at 0.53 GiB of memory. The whole model plus its working set fits in half a gigabyte.
Standalone decode as the C-served draft, 0.85 ms per token. That headroom is what lets six proposals be generated in the time the 8B spends on one verify pass.
Consecutive tokens drafted into the 8B and verified against plain greedy decode with zero divergences. On counting prompts every drafted token is accepted and the 8B delivers 762.6 tok/s, against 395.9 undrafted.
The control for a compressed model is its own base at full precision, graded on the same harness on the same day. The reference here is Qwen3-0.6B-Base in bf16. Both sides below carry 596,049,920 parameters, and both were run through eleven zero-shot tasks rather than one, because a single task is a cherry and a grid is a measurement. Winogrande retains 91.5% of the reference and PIQA 90.4%. Accuracy does not fall anything like as fast as the bytes do.
81.6 percent of Qwen3-0.6B-Base’s ability across seven common tasks, at 15.8 percent of the bytes.
15.8% of Qwen3-0.6B-Base, which ships 1,503 MB
78.0% averaged across all eleven tasks measured
Neutrino-1 0.6B on Hugging FaceThe drafted decode path on the 8B
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-0.6B
First run downloads the 0.24 GB coded transport, shows progress, verifies its SHA-256, and unpacks locally. Later runs load from cache.
Neutrino-1 0.6B Chat
Conversation at 596 million parameters is a behavior install, not a capability curve. The chat SKU runs the same 28 layers at the same 1,024 width through the same tied embedding table, and it exports to a container of exactly 327,719,836 bytes, the same figure as the draft model. The behaviour is trained in; the geometry and the footprint do not move.
What it buys is measured on both shipping runtimes at one documented decode config each. On 20 held-out sentence fragments the model answers naturally 20 times out of 20, and on 30 held-out factual asks it is correct or honestly uncertain 30 times out of 30, with no template leaks across the five probes designed to draw one. Identity holds 8 of 8 on the native binary and 8 of 8 on the Apple-silicon pack. Across 200 held-out chat prompts the native binary terminates on its own 88.5% of the time at a 4.5% loop rate, and the Apple-silicon pack reads 85.0% and 5.0%.
sup
Hey there! What's up?
who you
I'm Neutrino, a tiny language model made by Fermion Research. What's up? How can I help?
what is the moon
The Moon is a natural satellite that orbits Earth. It reflects sunlight, which makes it visible in the night sky.
Are you ChatGPT?
No, I'm Neutrino. I'm a small language model made by Fermion Research. I chat and help with phrasing, but I'm not ChatGPT.
Neutrino-1 0.6B-Chat on Hugging Face
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-0.6B-Chat
First run downloads about 0.33 GB of raw model files with progress and verifies them. This model does not use the coded transport. Later runs load from cache.
The full engineering account is in the release post.
Tuned settings
Every runtime ships carrying the decode settings its behaviour was measured at, so the first command you type is the tuned one. Swept across conversation, factual, tool-shaped and long-form prompts on the shipping containers. Here is what each setting is tuned for, if you want to reach for a different one.
Run it
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-8B
On the first 8B run, fermion downloads a 2.56 GB coded transport with a progress bar, verifies its SHA-256, and unpacks it locally. Later runs load from cache.
$ fermion chat --model fermionresearch/Neutrino-8B --kv-dtype int8 --yarn-factor 4
The native window is 40,960 tokens. --yarn-factor 4 extends the loaded window toward 160K without changing the artifact; --kv-dtype int8 holds 131,072 tokens in about 9.3 GiB. fp16 is the default, fp32 preserves byte-identical 0.1.9 output, and --yarn-factor 1.0 is exact identity.
$ fermion serve --model fermionresearch/Neutrino-8B
Serves at http://127.0.0.1:8000/v1. Point any OpenAI client at that base URL; the key can be any string. Tool calling, streaming, and a persistent KV session are on by default.
$ hf download fermionresearch/Neutrino-8B --local-dir Neutrino-8B \--exclude "gguf/*" --exclude "*.tv4z"
import fermionfrom transformers import AutoModelForCausalLM, AutoTokenizermodel = AutoModelForCausalLM.from_pretrained("Neutrino-8B")tokenizer = AutoTokenizer.from_pretrained("Neutrino-8B")
Import fermion first to register the trtc_v4 model type. Load from the downloaded local directory; passing the Hub repository id directly to from_pretrained does not work.
$ git clone https://github.com/fermionresearch/llama.cpp && cd llama.cpp$ git checkout fermion-fv5$ cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_METAL=OFF$ cmake --build build -j --target llama-completion
The pack uses the FV5 tensor type: stock llama.cpp, Ollama, and LM Studio cannot load it. The fermion-fv5 branch is the CPU/CUDA build and requires -DGGML_METAL=OFF. For Apple-GPU kernels, use the live fermion-fv5-metal branch and omit that flag.
$ cd Neutrino-8B/mlx$ pip install -r requirements.txt$ python -m fermion_mlx --model ../neutrino-8b_v4.bin --mode chat --tokenizer .
Run this exact sequence after downloading the repository. The working directory and relative model path are required.
fermion chatfermion servefermion generatefermion infofermion benchfermion inspectfermion verifyEvery command takes --model with a Hub id or local path.
Get the models
Weights, native binaries, GGUF pack, MLX pack.
$ fermion chat --model fermionresearch/Neutrino-8B
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-8B
First run downloads a 2.56 GB coded transport, shows progress, verifies its SHA-256, and unpacks locally. Later runs load from cache.
The draft model, and a fast model on its own.
$ fermion chat --model fermionresearch/Neutrino-0.6B
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-0.6B
First run downloads the 0.24 GB coded transport, shows progress, verifies its SHA-256, and unpacks locally. Later runs load from cache.
The conversational small model.
$ fermion chat --model fermionresearch/Neutrino-0.6B-Chat
$ pip install fermion-research$ fermion chat --model fermionresearch/Neutrino-0.6B-Chat
First run downloads about 0.33 GB of raw model files with progress and verifies them. This model does not use the coded transport. Later runs load from cache.
The engine, the loader, and the chat front end.
$ pip install fermion-research
Adds the format to the llama.cpp toolchain.
$ git clone https://github.com/fermionresearch/llama.cpp && git checkout fermion-fv5