Docs / Integrity and support
Frequently asked questions
Short answers to common installation, model, backend, API, and speculative-decoding questions.
Common questions
Why is the package fermion-research but the command fermion?
The short PyPI distribution name belongs to another project. Install
fermion-research; the Python import and command are both fermion.Where do model files go?
The standard Hugging Face cache under your home directory. Use
HF_HOME to move the whole cache or FERMION_CACHE_DIR to move only fermion downloads.Why is my machine slower than the published rate?
Run
fermion info. Published rates describe the optimized native backend. PyTorch is a correctness reference and is much slower. Also compare steady-state decode throughput, not wall throughput.Is temperature zero deterministic?
Yes at fixed model, prompt, backend, dtype, and context. The repetition penalty still applies unless you set it to 1.0.
Why does chat sample while generate does not?
Chat and serve use the evaluated conversational sampler. Generate stays greedy so scripts and verification runs remain repeatable.
Can I use the OpenAI SDK?
Yes. Start
fermion serve and set the SDK base URL to http://127.0.0.1:8000/v1. The server accepts any model string because one model is resident.Why did a tool name change in the response?
The server can map a uniquely matching short function name to the namespaced name registered by the client. It logs each rewrite and never guesses when more than one tool matches.
What is fermion_stuck?
A non-standard response object added when loop protection detects a repeated call, alternation, or repeated tool error. Use
--stuck-detector observe to log without changing output, or off to disable it.Does speculative decoding change the answer?
Not in the CPU, greedy, float32 guarantee regime. Run
fermion verify --draft auto to prove the pair on your machine. Reduced-precision near ties may differ.Can I use Transformers, llama.cpp, or MLX?
Yes. Transformers needs a local directory and
import fermion first. GGUF needs the Fermion llama.cpp fork. The MLX package lives inside each model repository.Why does Transformers reject the Hub id?
The registered TRTC model type is supplied by the fermion package and the library path currently expects a local directory. Download the repository, import fermion, then pass that directory.
How do I know a download is complete?
Run
fermion info. It checks structure, expected length, and SHA-256, then exits non-zero for a bad artifact.Is the server safe to expose publicly?
It is designed for localhost.
--api-key protects /v1/*, but the health endpoint remains unauthenticated and exposes the absolute container path and digest.Why do seed, JSON mode, or presence penalty not work?
They are not implemented by this server. Unsupported sampler fields produce a warning, and the health response lists them. Use
repetition_penalty for the available repetition control.