Docs / Integrity and support

Known limitations

Current boundaries of the runtime and API, stated in operational terms.

Runtime and platform

The optimized native runtime is CPU-only and currently available for macOS arm64 and Linux x86-64.
Other paths use the slower PyTorch reference implementation. Published throughput numbers describe native execution.
Native threads default to CPU count minus one and are capped at 16 unless you raise the cap explicitly.
The first 8B run needs about 6.5 GB free on Apple Silicon and about 4 GB elsewhere.

The server handles one generation at a time. Additional requests queue.
There is no embeddings endpoint, JSON mode, logprobs, seed, presence or frequency penalty, top-k, min-p, or logit bias.
tool_choice: "required" is best effort because the runtime does not constrain generation with a grammar.
The unauthenticated health endpoint includes the absolute container path and SHA-256.
Only Neutrino-8B reliably emits tool calls. The 0.6B models generally answer in prose.

The model window is 40,960 positions. Recall can decline before that boundary.
YaRN is experimental, native-only, changes output at all lengths, and extends addressability rather than recall quality.
A request uses the resident session only when prompt tokens plus output tokens fit its configured KV capacity.

Native speculative decoding is greedy-only and disables resident sessions while active.
Output identity is guaranteed in the CPU, greedy, float32 regime. Reduced-precision near ties can differ.
Transformers loading requires import fermion first and a local model directory.