The short answer
Text normalisation for ASR is the step that rewrites both the reference transcript and the model's output into one canonical form, folding case, punctuation, number formats, spelling variants and filler words, before word error rate is counted. It makes the score measure what the model heard rather than how it formatted the text.
Because every normaliser makes different choices, WER depends on it. Two results are only comparable if both were scored with the same normaliser on the same test set.
What a normaliser changes
Each row is a reference and a model output that mean the same thing. Scored raw, every pair contains errors; the Whisper English normaliser maps both sides to the same text.
| Reference | Model output | Raw comparison | After the Whisper English normaliser |
|---|---|---|---|
| Hello, World! | hello world | 2 substitutions (case, punctuation) | hello world = hello world |
| Mr. Smith | mister smith | 2 substitutions | mister smith = mister smith |
| It costs 1,000 dollars | it costs $1000 | Several errors | Both become the same digits with a $ sign |
| The colour is grey | the color is gray | 2 substitutions (spelling) | British spellings mapped to American |
| um so we start | so we start | 1 deletion | Filler um removed from both |
The Whisper English normaliser
OpenAI published an English text normaliser with Whisper (Radford et al., arXiv 2212.04356, 2022), in whisper/normalizers/english.py. The Open ASR Leaderboard applies it to English results at scoring time. In order, it:
- lowercases the text and removes words in square brackets and parentheses;
- removes the fillers hmm, mm, mhm, mmm, uh and um;
- expands contractions and titles (won't to will not, mr to mister, dr to doctor);
- removes commas between digits and full stops not followed by a digit;
- removes symbols and diacritics, keeping signs that carry numeric meaning such as %, $ and €;
- converts spelled-out numbers, ordinals and currency words to digits and symbols;
- maps British spellings to American ones and collapses whitespace.
Multilingual scoring and diacritics
Removing diacritics is harmless for English but changes the meaning of a score in Czech, French or Greek, where an accent can be the whole difference between two words. Normalisers for other languages make different choices about diacritics, numbers and compound words, so FLEURS numbers from different papers are often scored differently even on the same audio.
Fermion Research's FLEURS scoring keeps diacritics: a missing or wrong accent counts as an error. All models in our FLEURS table were scored with that same code, which is why their numbers compare with each other but not with figures scored elsewhere.
When you read any WER table, look for three things: which normaliser, which test sets, and whether every row was scored with the same code. If a table mixes self-reported numbers from different normalisers, its rows do not line up.
Questions
- What is text normalisation in speech recognition evaluation?
- Rewriting reference and model output into one canonical form (lowercase, no punctuation, standard numbers and spellings, fillers removed) before counting word errors, so formatting differences do not count as recognition errors.
- Why does WER depend on the normaliser?
- Each normaliser folds different things. A transcript that writes $20 where the reference says twenty dollars is two or three errors without a number rule and none with one.
- Which normaliser does the Open ASR Leaderboard use for English?
- The English text normaliser published with OpenAI's Whisper, applied at scoring time to every model's output and the references.
- Does the Whisper English normaliser remove filler words?
- Yes: hmm, mm, mhm, mmm, uh and um are removed before scoring, so a model is not penalised for keeping or dropping them.
- Does Fermion Research keep diacritics when scoring FLEURS?
- Yes. A missing or wrong accent counts as an error, and every model in our FLEURS table was scored with the same code.
- Can I compare WER numbers from two different papers?
- Only if both used the same test sets and the same normaliser. Otherwise rescore both models yourself with one scoring code.
Related
Sources and method
- Whisper English text normaliser: github.com/openai/whisper, whisper/normalizers/english.py, read 2026-11-13.
- Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, arXiv 2212.04356 (2022-12-06), read 2026-11-13.
- Open ASR Leaderboard: github.com/huggingface/open_asr_leaderboard README, read 2026-11-13.
- Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.
Canonical: https://www.fermionresearch.com/glossary/text-normalisation-for-asr/