The short answer
A Token-and-Duration Transducer (TDT) is a transducer speech recognition model that predicts, at each step, both the next token and its duration: how many audio frames that token covers. During decoding the model jumps ahead by the predicted duration instead of visiting every frame, which makes inference faster than a conventional transducer.
TDT was introduced by Hainan Xu, Fei Jia, Somshubra Majumdar, He Huang, Shinji Watanabe and Boris Ginsburg in "Efficient Sequence Transduction by Jointly Predicting Tokens and Durations" (arXiv 2304.06795, first posted 13 April 2023).
TDT next to CTC and RNN-T
All three are ways to map a long sequence of audio frames to a much shorter sequence of text tokens without a hand-made alignment.
| CTC | RNN-T (transducer) | TDT | |
|---|---|---|---|
| Introduced | Graves et al., ICML 2006 | Graves, arXiv 1211.3711 (2012) | Xu et al., arXiv 2304.06795 (2023) |
| Output per step | One token or a blank per frame | A token, or a blank that moves to the next frame | A token or blank, plus a duration in frames |
| Sees earlier tokens | No: outputs are independent given the audio | Yes, through a prediction network | Yes, through a prediction network |
| Frames visited while decoding | Every frame | Every frame | Can skip frames by the predicted duration |
| Reported effect | Baseline | Baseline for TDT's comparisons | Up to 2.82x faster speech recognition inference than conventional transducers, with better accuracy (paper) |
How it works
A transducer has an encoder that turns audio into frames, a prediction network that reads the tokens emitted so far, and a joint network that combines the two. In a conventional transducer the joint network outputs one distribution over tokens plus a blank; a blank means "move to the next frame".
In TDT the joint network's output is split into two parts that are normalised independently: a distribution over tokens (including blank) and a distribution over a small fixed set of durations. The paper's example set is 0, 1, 2, 3 and 4 frames, and its experiments use sets from 0 to 2 up to 0 to 8. The probability of a token with a duration is the product of the two.
A predicted duration of 2 means the decoder advances two frames after emitting the token. Duration 0 is allowed for real tokens, so several tokens can come out at one frame, but the paper disallows a blank with duration 0, so decoding always moves forward.
What the paper reports, and where TDT shows up
- Speech recognition: up to 2.82 times faster inference than conventional transducers, with better accuracy.
- Speech translation: more than 1 BLEU better on MUST-C, with 2.27 times faster inference.
- Spoken language understanding: intent accuracy up to more than 1 point better, up to 1.28 times faster.
The authors said the implementation would be open-sourced in NVIDIA's NeMo toolkit. Public TDT models include NVIDIA's Parakeet TDT family; the Parakeet TDT 0.6B v3 card lists its architecture as FastConformer-TDT with 600 million parameters and 25 European languages. That model appears as a reference column in our FLEURS results, scored with the same code as the other models.
Speed-ups in a paper are measured against a specific baseline on specific hardware. For a model you plan to deploy, measure real-time factor on your own machine.
Questions
- What does TDT stand for in speech recognition?
- Token-and-Duration Transducer: a transducer that predicts each output token together with the number of audio frames it covers. It was introduced in arXiv 2304.06795 in April 2023.
- Is TDT the same as RNN-T?
- TDT extends RNN-T. Both use an encoder, a prediction network and a joint network; TDT adds a duration output so decoding can skip frames instead of stepping through every one.
- Why is TDT faster than a conventional transducer?
- A conventional transducer visits every audio frame during decoding. TDT predicts how many frames each token covers and jumps ahead by that amount, so it runs the joint network fewer times.
- Does TDT lose accuracy for its speed?
- Not in the paper's results: it reports better accuracy than conventional transducers on speech recognition along with up to 2.82 times faster inference. Results on your own audio can differ.
- Which public models use TDT?
- NVIDIA's Parakeet TDT models, for example Parakeet TDT 0.6B v3, whose card names its architecture FastConformer-TDT.
- Can a TDT model give word timestamps?
- Durations place each token on the audio timeline, and the Parakeet TDT 0.6B v3 card lists word-level and segment-level timestamps as a feature.
Related
Sources and method
- Xu, Jia, Majumdar, Huang, Watanabe, Ginsburg, Efficient Sequence Transduction by Jointly Predicting Tokens and Durations, arXiv 2304.06795 (2023-04-13, revised 2023-05-29), read 2026-11-13.
- Graves, Sequence Transduction with Recurrent Neural Networks, arXiv 1211.3711 (2012-11-14), read 2026-11-13.
- Graves, Fernández, Gomez, Schmidhuber, Connectionist Temporal Classification: Labelling Unsegmented Sequence Data with Recurrent Neural Networks, ICML 2006 (cs.toronto.edu/~graves/icml_2006.pdf), read 2026-11-13.
- Parakeet TDT 0.6B v3 model card: huggingface.co/nvidia/parakeet-tdt-0.6b-v3, read 2026-11-13.
- Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.
Canonical: https://www.fermionresearch.com/glossary/tdt-token-and-duration-transducer/