Fermion ResearchFermion Research

Glossary/facts checked 13 November 2026

Word timestamps in speech recognition

The per-word timing behind subtitles, click-to-seek transcripts and redaction. Three ways systems produce it, and how to request it from the Phonon command line and server.

Speech docs

The short answer

Word timestamps are the start and end times, in seconds from the beginning of the audio, that a speech recognition system attaches to each word of its transcript. They are what subtitle files, click-to-seek transcripts, audio redaction and speaker-labelled transcripts are built from.

Phonon-2.5, Phonon-2.5 English and Phonon-2 return them with fermion transcribe --json, and the server returns them in the OpenAI verbose_json shape.

Three ways to get word timings

Where the times come from depends on the model family.

MethodWhere the times come fromExtra model neededPublic example
From the decoder's own alignmentCTC and transducer models emit tokens at specific audio frames, so each word's frames give its timesNoParakeet TDT 0.6B v3 lists word-level timestamps on its card
Cross-attention plus dynamic time warpingThe decoder's attention weights over audio frames are smoothed and aligned to tokensNo, but timing is indirectWhisper's word_timestamps option (whisper/timing.py)
Forced alignment after transcriptionA separate acoustic model aligns the finished text to the audioYesWhisperX: voice activity detection plus forced phoneme alignment (Bain et al., 2023)

Getting word timestamps from Phonon

On the command line, --json returns a JSON object with the text, a start and end time per segment, and a words list with the start and end of every word in seconds from the start of the file.

On the server started with fermion serve, POST /v1/audio/transcriptions with response_format=verbose_json and timestamp_granularities=word returns a words list (word, start, end) and a segments list in the shape OpenAI clients expect. Uploads go up to 1 GB; the model handles utterances up to 30 seconds and decodes longer audio in chunks.

fermion transcribe phonon-2.5 clip.wav --json
curl -s http://127.0.0.1:8000/v1/audio/transcriptions -F file=@clip.wav -F response_format=verbose_json -F timestamp_granularities=word

What affects timing accuracy

  • Frame resolution: a model's encoder looks at audio in fixed steps, so a word boundary cannot be placed more finely than one step.
  • Fast or overlapping speech: when words run together, boundaries are ambiguous even for human annotators.
  • Pauses and punctuation: some systems attach a pause to the previous word's end, others to the next word's start; check which before cutting audio.
  • Long recordings: audio split into chunks needs each chunk's offset added back, which a well-built tool does for you.

To check timings on your own audio, compare them with a hand-marked or forced-aligned reference and measure the average gap at word starts and ends.

Questions

What are word timestamps?
The start and end time of each word in a transcript, in seconds from the start of the audio. They are used for subtitles, click-to-seek transcripts and redaction.
How do I get word timestamps from Phonon?
Run fermion transcribe phonon-2.5 file.wav --json, which includes a words list with start and end times. On the server, use response_format=verbose_json with timestamp_granularities=word.
Which Phonon models return word timestamps?
Phonon-2.5, Phonon-2.5 English and Phonon-2. Phonon-1 returns null for the words list.
How does Whisper produce word timestamps?
Its word_timestamps option aligns tokens to audio frames from selected cross-attention heads, smoothed with a median filter and aligned with dynamic time warping.
What is forced alignment?
Aligning a known transcript to audio with an acoustic model after transcription. WhisperX, for example, combines voice activity detection with forced phoneme alignment to time each word.
Are word timestamps the same as speaker labels?
No. Timestamps say when each word was spoken; speaker labels (diarization) say who spoke it. Diarization is a separate step that can use word timings to assign words to speakers.

Related

Sources and method

  • Phonon command line and server behaviour: Fermion Research docs (/docs/cli-reference/, /docs/server-reference/), read 2026-11-13.
  • Whisper word timing: github.com/openai/whisper, whisper/timing.py, read 2026-11-13.
  • Bain, Huh, Han, Zisserman, WhisperX: Time-Accurate Speech Transcription of Long-Form Audio, arXiv 2303.00747 (2023-03-01), read 2026-11-13.
  • Parakeet TDT 0.6B v3 model card: huggingface.co/nvidia/parakeet-tdt-0.6b-v3, read 2026-11-13.

Canonical: https://www.fermionresearch.com/glossary/word-timestamps/