Docs / Reference

Server API reference

Routes, request fields, streaming frames, audio routes, errors, context clamps, and operational health data.

Endpoints

Method	Route	Purpose
POST	/v1/chat/completions	Chat completions with streaming and tools.
POST	/v1/completions	Legacy single-string text completion.
POST	/v1/audio/transcriptions	One-shot file transcription. Speech model resident.
GET	/v1/audio/stream	Live transcription over WebSocket. Speech model resident.
GET	/v1/models	The resident model card.
GET	/v1/models/{id}	One model card, or model_not_found.
GET	/health, /v1/health, /healthz	Runtime, model, session, tool, and integrity status.
GET	/	Service name, version, model id, and endpoint list.
OPTIONS	any route	CORS preflight when --cors is enabled.

One model is resident per process. With a Neutrino model, the chat routes are mounted; with a Phonon speech model, the two audio routes are mounted instead. A request to the family that is not mounted returns a 404 naming the route that exists.

Chat-completion fields

Field	Behavior
model	Accepted for client compatibility. One model is resident, and the response uses the configured served id.
messages	Required, non-empty. system, developer, user, assistant, and tool roles. Text content only.
tools / tool_choice	OpenAI function schemas and auto, none, required, or a named function.
functions / function_call	Legacy fields are accepted and converted to the current tools form.
temperature	0 through 2. Explicit values override server profiles.
top_p	0 through 1.
repetition_penalty	Extension from 0.01 through 2. A value of 1 disables it.
max_tokens / max_completion_tokens	Positive integer, clamped to server and model context limits.
stop	String or up to four strings, enforced server-side and streaming-safe.
n	Only 1 is accepted.
stream	Boolean. Uses server-sent events.
stream_options.include_usage	Adds a trailing usage chunk with empty choices.

Unsupported fields

The server warns once per process when it sees unsupported sampling fields.

presence_penalty, frequency_penalty, top_k, min_p
logit_bias, seed, logprobs, top_logprobs
response_format and constrained JSON output are not implemented.

Streaming format

Streaming uses real SSE with Content-Type: text/event-stream and chunked transfer. The sequence is role delta, content deltas, an optional indexed tool-calls delta, finish chunk, optional usage chunk, then [DONE].

text

data: {"object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"}}]}

data: [DONE]

If generation fails after the stream starts, the server sends an in-band error frame followed by [DONE].

Audio transcription

POST /v1/audio/transcriptions takes a multipart/form-data body in the OpenAI audio shape, so OpenAI clients work unmodified against base_url=http://127.0.0.1:8000/v1. With --api-key set, every request needs Authorization: Bearer.

Field	Behavior
file	Required. Anything libsndfile decodes: wav, flac, ogg, or aiff at any rate or channel count, resampled to 16 kHz mono. mp3 and m4a are refused with the exact ffmpeg conversion command.
model	Optional but checked. The model this process was started with (phonon-2.5, phonon-2.5-en or phonon-2, or its repository id); a name this process does not serve returns a 404 model_not_found rather than a transcript from a different model.
response_format	json (default) returns {"text": ...}. text returns the bare transcript as text/plain. verbose_json adds task, language, duration, and the exact decode configuration. Anything else is a 400.
hotwords	Optional. Names and terms to favour for this request: comma-separated, a JSON list, or one hotwords[] part per term. Up to 25; extra terms are dropped with a note.
prompt	Optional. The OpenAI field, read as a vocabulary list: comma-separated when it contains a comma (Ada Lovelace, Quillon), otherwise one term per word. Ignored when hotwords is sent.
hotword_lambda	Optional. Phonon-2: the bonus per matching word piece (default 2.0, between 0 and 100).

Hotwords apply to the request that sends them; other requests are unaffected.

bash

curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -F file=@call.wav -F model=phonon-2 \
  -F "hotwords=Ada, Quillon, Neutrino"
# a server started with fermion serve phonon-2.5 answers -F model=phonon-2.5

Errors use the same envelope as the chat routes. No file part, malformed multipart, or undecodable audio is a 400; a missing or wrong key is a 401; an audio upload over 1 GB is a 413.

bash

curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -H "Authorization: Bearer YOUR_KEY" \
  -F file=@clip.wav -F model=phonon-2 -F response_format=text

Audio streaming

GET /v1/audio/stream upgrades to a standard WebSocket for live transcription. Streaming API covers the full protocol: authentication, the opening frame, audio formats, server frames, and session limits.

Errors and request limits

Errors use {"error":{"message", "type", "param", "code"}}. Unknown routes return an OpenAI-shaped 404. The request-body limit is 32 MB, and chunked request bodies are rejected.

Client token requests are clamped rather than rejected. The server applies --max-new-ceiling, the model’s 40,960-position window, and the remaining positions after prompt tokenization.

Health data

/health reports the model id and kind, container SHA-256 and path, byte size, package version, device and dtype, backend, native runner and session state, sampler defaults, tool profile, stuck-detector configuration and counters, container header, context, and KV reuse statistics.