Fermion ResearchFermion Research

Glossary/facts checked 13 November 2026

The FLEURS benchmark

The standard public test set for multilingual speech recognition: where it comes from, what it measures well, and what it does not.

Phonon-2.5 FLEURS results

The short answer

FLEURS (Few-shot Learning Evaluation of Universal Representations of Speech) is a public speech dataset with read sentences in 102 languages and about 12 hours of speech per language. It was released by Google researchers in 2022 (Conneau et al., arXiv 2205.12446) and is widely used to compare multilingual speech recognition models language by language.

It is built on the FLoRes-101 machine translation benchmark, so the same sentences are read in every language. That makes per-language results line up better than on datasets collected separately in each language.

FLEURS at a glance

Facts from the FLEURS paper and its Hugging Face dataset card, read 2026-11-13.

PropertyFLEURS
Languages102, from more than 10 language families
Speech per languageAbout 12 hours of supervision
Source sentencesFLoRes-101 (n-way parallel: the same sentences in every language)
Speaking styleRead speech
Splitstrain, validation and test for each language
Audio16 kHz
Tasks in the paperSpeech recognition, language identification, translation and retrieval
Where to get ithuggingface.co/datasets/google/fleurs

How to read a FLEURS result

  • Check the language set. An average over 102 languages, over 25 European languages and over 5 languages are different numbers. Phonon-2.5's FLEURS average of 11.71 % is over its 25 languages.
  • Check the split and size. Results should be on the full test split of each language; a sample is noisier.
  • Check the normaliser. Whether diacritics, punctuation and numbers are folded changes the score, especially outside English. Our FLEURS numbers keep diacritics.
  • Expect a wide spread by language. On the same scoring, Phonon-2.5 ranges from 4.80 % in Spanish to 20.83 % in Greek.

What FLEURS does not measure

FLEURS sentences are read aloud from written text, mostly in quiet conditions. Meetings, phone calls, accents, crosstalk and spontaneous speech with hesitations are all harder, and a model's FLEURS rank does not always carry over. For English, the Open ASR Leaderboard's sets (meetings, earnings calls, podcasts and more) cover more conditions. For your own use, test a sample of your own audio.

For a size reference on the same 25 FLEURS sets and the same scoring: Whisper base, which is about the size of Phonon-2.5, averages 53.71 %; Phonon-2.5 averages 11.71 %.

Questions

What is the FLEURS dataset?
A public speech dataset from Google with read sentences in 102 languages, about 12 hours per language, built on the FLoRes-101 translation sentences. It is used to compare multilingual speech recognition models.
How many languages does FLEURS cover?
102 languages, from more than 10 language families, according to the paper and the Hugging Face dataset card.
What does FLEURS stand for?
Few-shot Learning Evaluation of Universal Representations of Speech, the title of the 2022 paper that introduced it (arXiv 2205.12446).
Is FLEURS read or spontaneous speech?
Read speech: speakers read sentences from the FLoRes-101 benchmark. Spontaneous and noisy speech is usually harder than FLEURS suggests.
What is Phonon-2.5's FLEURS word error rate?
11.71 % averaged over its 25 languages, on the full FLEURS test set of each language, scored with one code for all models compared.
Can I compare FLEURS numbers from different papers?
Only if they use the same languages, the full test splits and the same text normaliser; otherwise the numbers can differ for reasons unrelated to the models.

Related

Sources and method

  • Conneau et al., FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech, arXiv 2205.12446 (2022-05-25), read 2026-11-13.
  • FLEURS dataset card: huggingface.co/datasets/google/fleurs, read 2026-11-13.
  • Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.
  • Whisper base on the same 25 FLEURS sets: Phonon-2.5 release post, 2026-10-22.

Canonical: https://www.fermionresearch.com/glossary/fleurs-benchmark/