ASVspoof 5: 53 teams' deepfake speech detectors on 680,774 crowdsourced utterances and 16 attacks

Authors
Wang, Delgado, Evans, Liu, Kinnunen, Tak, Lee, Kukanov, Sahidullah, Todisco, Yamagishi
Venue
IEEE Transactions on Audio, Speech and Language Processing 34, pages 2354 to 2367 (evaluation paper); ASVspoof 5 Workshop, Interspeech 2024 satellite (summary paper)
Published
AIDI-S-0027

Updated ·

ASVspoof 5, the fifth speech deepfake detection challenge, scored submissions from 53 teams on an evaluation set of 138,688 bona fide and 542,086 spoofed utterances and found that the best stand-alone detector in the open condition (team T45) reached a minimum detection cost of 0.0750 and an equal error rate of 2.59%, the best in the closed condition (T32) 0.2436 and 8.61%, and the AASIST baseline 0.7106 and 29.12% (arXiv 2408.08739, Table 4, read 9 November 2026). The results are reported in two primary papers by the organising consortium: the challenge summary (Wang, Delgado, Tak et al., ASVspoof 5 Workshop, Interspeech 2024 satellite, arXiv 16 August 2024) and the evaluation paper (Wang, Delgado, Evans et al., IEEE Transactions on Audio, Speech and Language Processing 34, pages 2354 to 2367, arXiv v1 7 January 2026, v3 11 April 2026) (arXiv 2601.03944, read 9 November 2026). Both were read; the per-team tables are in the 2024 paper and the analyses in the 2026 paper.

ASVspoof 5: 53 teams' deepfake speech detectors on 680,774 crowdsourced utterances and 16 attacks: register plate with the page title and its register ID
The record plate: register ID and title, drawn from the index data.

The record is the first voice entry in the studies index. No product sold under a card in this index is a named participant.

Participants, not products, are the detectors here. The 2026 paper counts 53 teams and the 2024 paper 54 participants; Table 4 of the 2024 paper lists 33 closed-condition and 34 open-condition Track 1 submissions, baselines included. Teams are anonymised by policy: "neither the team name nor the names of team members can be revealed" (arXiv 2601.03944, Section IV footnote 8, read 9 November 2026). Table III of the 2026 paper nonetheless cites the top entries' system-description papers.

What was tested

Detectors tested

The challenge systems named and their index cards
Detector (team ID, rank, condition)System description citedIndex card
T32 (1st, Track 1 closed)none receivedno card
T47 (2nd, closed)ParallelChain Labno card
T24 (3rd, closed)Duroselle et al., data augmentations for the closed conditionno card
T45 (1st, Track 1 open; 1st, Track 2 both conditions)USTC-KXDIGITno card
T36 (2nd, open)Intemano card
T27 (3rd, open)Stourbe et al., WavLM back-endsno card
T23 (4th, open)Whispeakno card
T43 (5th, open)SZU-AFSno card
B01 RawNet2, B02 AASISTorganisers' baselinesno card

Sample

Ranks from Table 4 of arXiv 2408.08739; system descriptions from Table III of arXiv 2601.03944.

None of the voice cards in this index, Pindrop Pulse, Resemble Detect, Reality Defender, Hive, Sensity or AI or Not, appears as a participant. Pindrop appears on the other side of the table: co-author Hemlata Tak's affiliation is Pindrop, and Pindrop co-sponsored the ASVspoof 2024 Workshop (arXiv 2601.03944, author affiliations; arXiv 2408.08739, Acknowledgements, read 9 November 2026). The Pindrop card quotes the vendor's "99% (accuracy)" footnoted to "the ASVspoof5 dataset of 32 engines"; the database carries 32 attack identifiers, A01 to A32, but neither paper prints a Pindrop result.

The data come from the English partition of Multilingual LibriSpeech, crowdsourced audiobook recordings from almost 2,000 speakers, speaker-disjoint across training (18,797 bona fide and 163,560 spoofed utterances, 8 attacks), development (31,334 and 109,616, 8 attacks) and the Track 1 evaluation set (138,688 and 542,086 from 737 speakers, 16 attacks). Spoofed speech was built by data contributors with text-to-speech and voice conversion systems from a legacy MaryTTS unit-selection system to zero-shot YourTTS and XTTS, tuned against surrogate detectors, and seven of the 16 evaluation attacks add Malafide or Malacopula adversarial filters. Evaluation audio is also passed through MP3, Opus, AMR, Speex, M4A, the neural codec Encodec, MP3 plus Encodec, and an 8 kHz telephone condition (arXiv 2408.08739, Section 2 and Tables 1 to 3, read 9 November 2026).

The primary Track 1 metric is the minimum detection cost function with a miss cost of 1, a false alarm cost of 10 and a spoof prior of 0.05, so missing a genuine utterance costs about 1.9 times a false accept; the equal error rate, actual DCF and Cllr are also printed. Track 2 uses the minimum a-DCF. The closed condition allows only challenge data; the open condition allows pre-trained speech foundation models (arXiv 2408.08739, Section 3, read 9 November 2026).

What the study reports

Results as the study reports them, with their base
MetricValueBase
Track 1 closed, top five minDCF and EER (Table 4)T32 0.2436, 8.61%; T47 0.2660, 9.18%; T24 0.2975, 10.43%; T45 0.3948, 14.33%; T13 0.4025, 14.75%680,774 evaluation utterances
Track 1 closed, baselines (Table 4)B02 AASIST 0.7106, 29.12%; B01 RawNet2 0.8266, 36.04%680,774 evaluation utterances
Track 1 open, top five minDCF and EER (Table 4)T45 0.0750, 2.59%; T36 0.0936, 3.41%; T27 0.0937, 3.42%; T23 0.1124, 4.16%; T43 0.1149, 4.04%680,774 evaluation utterances
Teams beating the best baseline, closed (2026 paper, Section IV-A)2733 closed-condition submissions
Track 2 best min a-DCF (Table 5)T45 0.2814 closed, 0.0756 open; organisers' ASV without a detector 0.6869496,632 Track 2 utterances
Encodec compression, closed (2026 paper, Section IV-A-3)top submission minDCF 0.35, second 0.55Encodec-coded evaluation subset
Hardest attack, open (2026 paper, Section IV-A-2)A28 pre-trained YourTTS, minDCF 0.33A28 subset
Cross-dataset EER of four top open systems (2026 paper, Table IV)3.30% to 4.33% on ASVspoof 5; 16.27% to 26.63% on ASVspoof 2019 LA; 6.85% to 14.71% on In-the-Wild3,000 bona fide and 3,000 spoofed utterances per subset

Every figure below is printed in Tables 4 and 5 of the 2024 paper or in the text and Table IV of the 2026 paper.

Values as printed in arXiv 2408.08739, Tables 4 and 5, and arXiv 2601.03944, Sections IV and V-B and Table IV, read 9 November 2026. Per-attack and per-codec minDCFs for the full field appear only as box plots.

Two findings carry beyond the leaderboard. First, calibration: the top systems in both conditions posted an actual DCF of 1.0, meaning that at the pre-set decision threshold they accepted or rejected everything, because their scores were probabilities rather than log-likelihood ratios; T27, which calibrated its scores, posted a Cllr of 0.1927 against T45's 0.7923 (arXiv 2408.08739, Table 4 and Section 6.1, read 9 November 2026). Second, generalisation: when four of the five best open-condition systems were re-run on other databases, every EER but one (T43 on In-the-Wild, 6.85%) rose above 10%, and the authors conclude that "current detection solutions overfit to the acoustic or generative traits of specific datasets" (arXiv 2601.03944, Table IV and Section VI, read 9 November 2026).

Adversarial filters raised error on the attacks they were applied to, and the legacy MaryTTS system (A19) was the hardest attack in the closed condition: "robust performance for relatively advanced attacks is no guarantee of protection against attacks implemented with legacy technology" (arXiv 2601.03944, Section IV-A-1, read 9 November 2026).

Limits the authors state

The organisers list them in Section V-D of the 2026 paper: the task is binary and does not trace the source generator; the data are English-only audiobook-style read speech from one corpus, so "the scenario remains somewhat narrow"; top systems share near-identical architectures, which "may be reaching a bottleneck"; and replayed or re-recorded spoofs were out of scope (arXiv 2601.03944, Section V-D, read 9 November 2026). Evident from the papers: the two give different participant counts (53 and 54); per-attack and per-codec figures for most systems are plotted, not tabulated; and nothing identifies a commercial product.

What this record does not say: a 2.59% equal error rate belongs to an anonymised research ensemble on a 2024 challenge set of read English speech, and the same systems posted 10% to 27% on other databases; no figure here describes Pindrop Pulse or any other product in this index.

Which cards this study feeds

  • No card in the index was tested by this record.

No card takes a result line from this record, because no indexed product is a scored participant. The record is linked from the Pindrop Pulse card as the primary source behind the vendor's "ASVspoof5 dataset of 32 engines" footnote, which the papers neither confirm nor score. It counts toward the voice column of studies by modality as a record with zero indexed detectors, and the content type view lists the eleven cards that state voice. No figure enters the false positive rate ranking or the adversarial robustness ranking, which rank cards. The home page lists which AI detectors exist and who has tested them.

Related records: SAFE 2025 is the other audio challenge in the index, blind and with named teams; RAID 2024 is the shared text benchmark.

Speech is not music: the music detectors question records that none of the five music cards is scored by this or any record, and the Pindrop vs Resemble Detect comparison sets the organiser affiliation noted above beside Resemble Detect's absence from the challenge, both read 10 November 2026.

Questions about this study

What was the best result in ASVspoof 5?

A minimum detection cost of 0.0750 and an equal error rate of 2.59% by team T45 in the open condition, and 0.2436 and 8.61% by team T32 in the closed condition, on 680,774 evaluation utterances (arXiv 2408.08739, Table 4).

Did any commercial voice detector take part in ASVspoof 5?

None is named: teams are anonymised by challenge policy, and Pindrop appears as an organiser affiliation and workshop sponsor rather than a participant (arXiv 2601.03944, footnote 8 and author list; arXiv 2408.08739, Acknowledgements).

How did codecs and adversarial attacks affect ASVspoof 5 detectors?

Encodec neural compression was the hardest condition, with the top closed-condition system at a minDCF of 0.35 and the second at 0.55; adversarial filters raised error on the attacks they were applied to; and the legacy MaryTTS attack A19 was the hardest in the closed condition (arXiv 2601.03944, Section IV-A).

Do ASVspoof 5 results transfer to other audio?

Weakly: four top open-condition systems with 3.30% to 4.33% EER on ASVspoof 5 posted 10.40% to 26.63% on earlier ASVspoof sets and 6.85% to 14.71% on In-the-Wild (arXiv 2601.03944, Table IV).

Source and revisions

Source

Wang, X., Delgado, H., Evans, N., et al. (2026). ASVspoof 5: Evaluation of Spoofing, Deepfake, and Adversarial Attack Detection Using Crowdsourced Speech. IEEE Transactions on Audio, Speech and Language Processing 34, 2354 to 2367, DOI 10.1109/TASLPRO.2026.3682962; read as arXiv:2601.03944 v3, 11 April 2026, https://arxiv.org/abs/2601.03944 and https://arxiv.org/pdf/2601.03944, both read 9 November 2026. Wang, X., Delgado, H., Tak, H., et al. (2024). ASVspoof 5: Crowdsourced Speech Data, Deepfakes, and Adversarial Attacks at Scale. Proceedings of the ASVspoof 5 Workshop, pages 1 to 8; read as arXiv:2408.08739 v1, 16 August 2024, https://arxiv.org/abs/2408.08739 and https://arxiv.org/pdf/2408.08739, both read 9 November 2026. Journal record: Crossref https://api.crossref.org/works/10.1109/TASLPRO.2026.3682962, read 9 November 2026.

Revision log

  1. Record created
  2. Eighth-wave pages linked
  3. Twelfth-wave pages linked

Corrections log

Related reading