Docs / Reference

Server API reference

Routes, request fields, streaming frames, audio routes, errors, context clamps, and operational health data.

Endpoints

Method	Route	Purpose
POST	/v1/chat/completions	Chat completions with streaming and tools.
POST	/v1/completions	Legacy single-string text completion.
POST	/v1/audio/transcriptions	One-shot file transcription. Speech model resident.
GET	/v1/audio/stream	Live transcription over WebSocket. Speech model resident.
GET	/v1/models	The resident model card.
GET	/v1/models/{id}	One model card, or model_not_found.
GET	/health, /v1/health, /healthz	Runtime, model, session, tool, and integrity status.
GET	/	Service name, version, model id, and endpoint list.
OPTIONS	any route	CORS preflight when --cors is enabled.

One model is resident per process. With a Neutrino model, the chat routes are mounted; with a Phonon speech model, the two audio routes are mounted instead. A request to the family that is not mounted returns a 404 naming the route that exists.

Chat-completion fields

Field	Behavior
model	Accepted for client compatibility. One model is resident, and the response uses the configured served id.
messages	Required, non-empty. system, developer, user, assistant, and tool roles. Text content only.
tools / tool_choice	OpenAI function schemas and auto, none, required, or a named function.
functions / function_call	Legacy fields are accepted and converted to the current tools form.
temperature	0 through 2. Explicit values override server profiles.
top_p	0 through 1.
repetition_penalty	Extension from 0.01 through 2. A value of 1 disables it.
max_tokens / max_completion_tokens	Positive integer, clamped to server and model context limits.
stop	String or up to four strings, enforced server-side and streaming-safe.
n	Only 1 is accepted.
stream	Boolean. Uses server-sent events.
stream_options.include_usage	Adds a trailing usage chunk with empty choices.

Unsupported fields

The server warns once per process when it sees unsupported sampling fields.

presence_penalty, frequency_penalty, top_k, min_p
logit_bias, seed, logprobs, top_logprobs
response_format and constrained JSON output are not implemented.

Streaming format

Streaming uses real SSE with Content-Type: text/event-stream and chunked transfer. The sequence is role delta, content deltas, an optional indexed tool-calls delta, finish chunk, optional usage chunk, then [DONE].

text

data: {"object":"chat.completion.chunk","choices":[{"delta":{"role":"assistant"}}]}

data: {"object":"chat.completion.chunk","choices":[{"delta":{"content":"Hello"}}]}

data: [DONE]

If generation fails after the stream starts, the server sends an in-band error frame followed by [DONE].

Audio transcription

POST /v1/audio/transcriptions takes a multipart/form-data body in the OpenAI audio shape, so OpenAI clients work unmodified against base_url=http://127.0.0.1:8000/v1. With --api-key set, every request needs Authorization: Bearer.

Field	Behavior
file	Required. Anything libsndfile decodes: wav, flac, ogg, or aiff at any rate or channel count, resampled to 16 kHz mono. mp3 and m4a are refused with the exact ffmpeg conversion command.
model	Optional but checked. A name this process does not serve returns a 404 model_not_found rather than a transcript from a different model.
response_format	json (default) returns {"text": ...}. text returns the bare transcript as text/plain. verbose_json adds task, language, duration, and the exact decode configuration. Anything else is a 400.

Errors use the same envelope as the chat routes. No file part, malformed multipart, or undecodable audio is a 400; a missing or wrong key is a 401; a body over 32 MB is a 413.

bash

curl -s http://127.0.0.1:8000/v1/audio/transcriptions \
  -H "Authorization: Bearer YOUR_KEY" \
  -F file=@clip.wav -F model=phonon-1 -F response_format=text

Audio streaming

GET /v1/audio/stream upgrades to a standard WebSocket for live transcription. Streaming API covers the full protocol: authentication, the opening frame, audio formats, server frames, and session limits.

Errors and request limits

Errors use {"error":{"message", "type", "param", "code"}}. Unknown routes return an OpenAI-shaped 404. The request-body limit is 32 MB, and chunked request bodies are rejected.

Client token requests are clamped rather than rejected. The server applies --max-new-ceiling, the model’s 40,960-position window, and the remaining positions after prompt tokenization.

Health data

/health reports the model id and kind, container SHA-256 and path, byte size, package version, device and dtype, backend, native runner and session state, sampler defaults, tool profile, stuck-detector configuration and counters, container header, context, and KV reuse statistics.