---
title: "The Open ASR Leaderboard"
description: "The Open ASR Leaderboard runs speech models on the same public test sets with one scoring code. Its English sets, how to read averages and RTFx, and Phonon."
canonical: "https://www.fermionresearch.com/glossary/open-asr-leaderboard/"
source: "Fermion Research"
---

Glossary/facts checked 21 November 2026

# The Open ASR Leaderboard

Hugging Face's public benchmark for speech recognition. A reader's guide: what each test set is, what the average covers, and what RTFx tells you.

[Phonon on the Open ASR Leaderboard](https://www.fermionresearch.com/models/phonon-2-5/open-asr-leaderboard/)

## The short answer

The Open ASR Leaderboard is a public benchmark hosted on Hugging Face that runs speech recognition models on the same test sets with the same scoring code, and reports word error rate per set, an average, and RTFx (seconds of audio transcribed per second of compute). The maintainers run official evaluations themselves, so rows are produced the same way.

It has an English short-form track, an English long-form track and a multilingual track. Its paper (Srivastav et al., arXiv 2510.06961) describes 86 systems and 12 datasets at the time of writing.

## The English test sets

The seven classic sets are the ones most model cards quote. The current English protocol adds URGENT24 and Monsoon en-IN, for nine public sets.

| Set | Audio | Classic seven | Current nine |
| --- | --- | --- | --- |
| LibriSpeech test-clean | Read audiobooks, clean | Yes | Yes |
| LibriSpeech test-other | Read audiobooks, harder speakers | Yes | Yes |
| AMI | Meeting recordings | Yes | Yes |
| Earnings-22 | Company earnings calls | Yes | Yes |
| GigaSpeech | Podcasts, audiobooks and online video | Yes | Yes |
| SPGISpeech | Business and financial speech | Yes | Yes |
| VoxPopuli | European Parliament speeches | Yes | Yes |
| URGENT24 | Speech from the URGENT 2024 challenge, with noise and distortions | No | Yes |
| Monsoon en-IN | Indian English | No | Yes |

## How to read the average and RTFx

- Know which sets an average covers. A seven-set average and a nine-set average are different numbers for the same model, and the leaderboard also evaluates submitted models on private sets. Compare an average only with averages over the same sets.
- Look at the columns, not only the average. Meetings (AMI) and noisy speech (URGENT24) separate models more than clean audiobooks do. If your audio is calls or meetings, those columns matter most.
- Read RTFx as relative speed on the maintainers' hardware. The repository names a single H200 for official runs. RTFx compares models on that machine; it does not tell you speed on a laptop CPU.
- Separate open models from APIs. Closed API rows carry no size or RTFx, and the board snapshot of 2026-09-25 shows the lowest seven-set averages coming from closed APIs.

## Phonon on the leaderboard

Phonon-2.5 English and Phonon-2.5 are listed on the Open ASR Leaderboard since 2026-11-13 (board version 1.16.0), with board averages of 7.05 % and 7.16 % over its default columns (seven public English sets, without LibriSpeech, plus two private sets the maintainers run) and RTFx of 2,110 and 1,898 on the maintainers' GPU. The board removed Phonon-2's row on 2026-11-20. Our own run of the nine public sets with the leaderboard's code, on 2026-10-28, gave 5.70 % for Phonon-2.5 and 5.60 % for Phonon-2.5 English.

On the seven classic sets the averages are 4.82 % for Phonon-2.5 English, 5.06 % for Phonon-2.5 and 5.21 % for Phonon-2. The full per-set tables are on the English benchmark results page.

## Questions

## Related

- [Phonon and the Open ASR Leaderboard](https://www.fermionresearch.com/models/phonon-2-5/open-asr-leaderboard/)
- [English benchmark results](https://www.fermionresearch.com/speech/english-benchmark-results/)
- [Real-time factor and RTFx](https://www.fermionresearch.com/glossary/real-time-factor/)
- [Text normalisation for ASR](https://www.fermionresearch.com/glossary/text-normalisation-for-asr/)
- [Open speech-to-text models under 200 MB](https://www.fermionresearch.com/models/phonon-2-5/small-models/)

## Sources and method

- Open ASR Leaderboard: github.com/huggingface/open_asr_leaderboard README, read 2026-11-13.
- Srivastav et al., Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluation, arXiv 2510.06961 (first version 2025-10-08), read 2026-11-13.
- Test set list: huggingface.co/datasets/hf-audio/open-asr-leaderboard dataset card, read 2026-11-13.
- Phonon nine-set numbers: Fermion Research measurement, 2026-10-28, Open ASR Leaderboard code, full test sets. Board rows: Open ASR Leaderboard version 1.16.2, read 2026-11-21.
- Closed API rows: Open ASR Leaderboard, seven English sets, snapshot 2026-09-25.
- Phonon numbers: Phonon-2.5 release post (/research/phonon-2-5/), 2026-10-22; Fermion Research measurement, full test sets, one scoring code for every model in a table.

Canonical: https://www.fermionresearch.com/glossary/open-asr-leaderboard/

Source: [https://www.fermionresearch.com/glossary/open-asr-leaderboard/](https://www.fermionresearch.com/glossary/open-asr-leaderboard/)
