---
title: "Platform notes"
description: "Current boundaries of the runtime and API, stated in operational terms."
canonical: "https://www.fermionresearch.com/docs/platform-notes/"
source: "Fermion Research"
---

Docs / Integrity and support

# Platform notes

Current boundaries of the runtime and API, stated in operational terms.

## Runtime and platform

- The optimized native runtime is CPU-only and currently available for macOS arm64 and Linux x86-64.
- Other paths use the slower PyTorch reference implementation. Published throughput numbers describe native execution.
- Native threads default to CPU count minus one and are capped at 16 unless you raise the cap explicitly.
- The first 8B run needs about 6.5 GB free on Apple Silicon and about 4 GB elsewhere.

## Server and API

- The server handles one generation at a time. Additional requests queue.
- There is no embeddings endpoint, JSON mode, logprobs, seed, presence or frequency penalty, top-k, min-p, or logit bias.
- `tool_choice: "required"` is best effort because the runtime does not constrain generation with a grammar.
- The unauthenticated health endpoint includes the absolute container path and SHA-256.
- Only Neutrino-8B reliably emits tool calls. The 0.6B models generally answer in prose.

## Context and sessions

- The model window is 40,960 positions. Recall can decline before that boundary.
- YaRN is experimental, native-only, changes output at all lengths, and extends addressability rather than recall quality.
- A request uses the resident session only when prompt tokens plus output tokens fit its configured KV capacity.

## Drafting and integrations

- Native speculative decoding is greedy-only and disables resident sessions while active.
- Output identity is guaranteed in the CPU, greedy, float32 regime. Reduced-precision near ties can differ.
- Transformers loading requires `import fermion` first and a local model directory.

Source: [https://www.fermionresearch.com/docs/platform-notes/](https://www.fermionresearch.com/docs/platform-notes/)