---
title: "Text normalisation for speech recognition"
description: "Text normalisation rewrites reference and output into one form (case, punctuation, numbers, spelling) before WER is counted. Why scores depend on it."
canonical: "https://www.fermionresearch.com/glossary/text-normalisation-for-asr/"
source: "Fermion Research"
---

Glossary/facts checked 13 November 2026

# Text normalisation for speech recognition

The same transcript can score very differently depending on the normaliser. What the Whisper English normaliser does, and how to read numbers scored with different ones.

[Phonon-2.5 benchmarks and downloads](https://www.fermionresearch.com/models/phonon-2-5/)Open weights on Hugging Face.

## The short answer

Text normalisation for ASR is the step that rewrites both the reference transcript and the model's output into one canonical form, folding case, punctuation, number formats, spelling variants and filler words, before word error rate is counted. It makes the score measure what the model heard rather than how it formatted the text.

Because every normaliser makes different choices, WER depends on it. Two results are only comparable if both were scored with the same normaliser on the same test set.

## What a normaliser changes

Each row is a reference and a model output that mean the same thing. Scored raw, every pair contains errors; the Whisper English normaliser maps both sides to the same text.

| Reference | Model output | Raw comparison | After the Whisper English normaliser |
| --- | --- | --- | --- |
| Hello, World! | hello world | 2 substitutions (case, punctuation) | hello world = hello world |
| Mr. Smith | mister smith | 2 substitutions | mister smith = mister smith |
| It costs 1,000 dollars | it costs $1000 | Several errors | Both become the same digits with a $ sign |
| The colour is grey | the color is gray | 2 substitutions (spelling) | British spellings mapped to American |
| um so we start | so we start | 1 deletion | Filler um removed from both |

## The Whisper English normaliser

OpenAI published an English text normaliser with Whisper (Radford et al., arXiv 2212.04356, 2022), in whisper/normalizers/english.py. The Open ASR Leaderboard applies it to English results at scoring time. In order, it:

- lowercases the text and removes words in square brackets and parentheses;
- removes the fillers hmm, mm, mhm, mmm, uh and um;
- expands contractions and titles (won't to will not, mr to mister, dr to doctor);
- removes commas between digits and full stops not followed by a digit;
- removes symbols and diacritics, keeping signs that carry numeric meaning such as %, $ and €;
- converts spelled-out numbers, ordinals and currency words to digits and symbols;
- maps British spellings to American ones and collapses whitespace.

## Multilingual scoring and diacritics

Removing diacritics is harmless for English but changes the meaning of a score in Czech, French or Greek, where an accent can be the whole difference between two words. Normalisers for other languages make different choices about diacritics, numbers and compound words, so FLEURS numbers from different papers are often scored differently even on the same audio.

Fermion Research's FLEURS scoring keeps diacritics: a missing or wrong accent counts as an error. All models in our FLEURS table were scored with that same code, which is why their numbers compare with each other but not with figures scored elsewhere.

When you read any WER table, look for three things: which normaliser, which test sets, and whether every row was scored with the same code. If a table mixes self-reported numbers from different normalisers, its rows do not line up.

## Questions

## Related

- [Word error rate (WER)](https://www.fermionresearch.com/glossary/word-error-rate/)
- [How Fermion Research measures speech models](https://www.fermionresearch.com/speech/benchmark-methodology/)
- [The Open ASR Leaderboard](https://www.fermionresearch.com/glossary/open-asr-leaderboard/)
- [FLEURS results in 25 languages](https://www.fermionresearch.com/speech/fleurs-results-25-languages/)

## Sources and method

- Whisper English text normaliser: github.com/openai/whisper, whisper/normalizers/english.py, read 2026-11-13.
- Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, arXiv 2212.04356 (2022-12-06), read 2026-11-13.
- Open ASR Leaderboard: github.com/huggingface/open_asr_leaderboard README, read 2026-11-13.
- Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.

Canonical: https://www.fermionresearch.com/glossary/text-normalisation-for-asr/

Source: [https://www.fermionresearch.com/glossary/text-normalisation-for-asr/](https://www.fermionresearch.com/glossary/text-normalisation-for-asr/)
