ADD 2023: the second Audio Deepfake Detection Challenge, 145 teams across fake-audio game, region location and algorithm recognition
- Authors
- Yi, Tao, Fu, Yan, Wang, Wang, Zhang, Zhang, Zhao, Ren, Xu, Zhou, Gu, Wen, Liang, Lian, Nie, Li (challenge paper); Yi, Zhang, Tao, Wang, Yan, Ren, Gu, Zhou (analysis paper)
- Venue
- IJCAI 2023 Workshop on Deepfake Audio Detection and Analysis (DADA 2023), CEUR Workshop Proceedings 3597, pages 125 to 130; analysis paper arXiv 2408.04967 (v3, 11 December 2024), submitted to the IEEE
- Published
ADD 2023, the second Audio Deepfake Detection Challenge, drew data requests from 145 teams and reports that the best of 49 detection submissions, team B01, reached a weighted equal error rate of 12.45% over two rounds of Mandarin test audio against an average of 49.94%, that the best of 16 manipulation-region-location systems scored 67.13% against a baseline of 42.25%, and that the best of 11 algorithm-recognition systems reached a macro F1 of 89.63% against a baseline of 53.50% (arXiv 2305.13774, Tables 6 to 8 and Section 4.2, read 10 November 2026). The challenge paper, "ADD 2023: the Second Audio Deepfake Detection Challenge" by Jiangyan Yi, Jianhua Tao and sixteen co-authors, appeared in the IJCAI 2023 Workshop on Deepfake Audio Detection and Analysis (DADA 2023, Macao, 19 August 2023), CEUR Workshop Proceedings volume 3597, pages 125 to 130, and as arXiv 2305.13774 on 23 May 2023; the analysis paper, "ADD 2023: Towards Audio Deepfake Detection and Analysis in the Wild" by Yi, Zhang, Tao, Wang, Yan, Ren, Gu and Zhou, is arXiv 2408.04967, version 3 of 11 December 2024, marked as submitted to the IEEE (arXiv 2408.04967, read 10 November 2026). Both were read in full.
The record is the first Mandarin audio entry in the studies index. No product sold under a card in this index is a named participant.
Participants, not products, are the detectors here. Team IDs are assigned by rank within each track (A generation, B detection, C region location, D algorithm recognition) and six organiser baselines carry S numbers; the analysis paper cites the top five system papers in each track (arXiv 2305.13774, Section 4.2; arXiv 2408.04967, Tables 5 to 8, read 10 November 2026).
What was tested
Detectors tested
| Detector (team ID, rank, track) | System as described in the analysis paper | Index card |
|---|---|---|
| B01 (1st, Track 1.2 detection) | Wav2Vec 2.0 front end, AASIST back end, noise, RawBoost and copy-synthesis augmentation | no card |
| B02 (2nd, Track 1.2) | Wav2Vec 2.0; SENet, LCNN and AASIST, weighted average | no card |
| B03 (3rd, Track 1.2) | CQT spectrogram; LCNN and AASIST | no card |
| C01 (1st, Track 2 region location) | Wav2Vec 2.0; ResNet, Transformer and LSTM | no card |
| C02 (2nd, Track 2) | Spectrogram; RCNN-BLSTM | no card |
| D01 (1st, Track 3 algorithm recognition) | STFT and Wav2Vec 2.0; SENet, LCNN-LSTM and TDNN, kNN open-set method | no card |
| D02 (2nd, Track 3) | Eight-model fusion, maximum-similarity open-set method | no card |
| S01 LFCC-GMM, S02 LFCC-LCNN, S03 Wav2Vec 2.0-LCNN, S04 LFCC-LCNN (Track 2), S05 and S06 LFCC-ResNet (Track 3) | organisers' baselines | no card |
Sample
| Text set | Count | Source |
|---|---|---|
| Track 1.2 test, round 1 | 80,000 real, 31,976 fake | Table 3 |
| Track 1.2 test, round 2 | 87,500 real, 30,977 fake | Table 3 |
| Track 2 test | 20,000 real, 30,000 partially fake | Table 3; the partially fake clips come from 148 speakers (analysis paper, Section 4.3) |
| Track 3 test | 10,507 real, 68,983 fake in seven fake classes, 10,469 of them from an unknown algorithm | Tables 2 and 3 |
Ranks from Tables 5 to 8 of arXiv 2305.13774; descriptions from Tables 6 to 8 of arXiv 2408.04967.
None of the voice cards in this index, Pindrop Pulse, Resemble Detect, Reality Defender, Hive, Sensity, ElevenLabs or AI or Not, appears as a participant. The Track 1.2 baseline S03 and most top systems use the wav2vec 2.0 XLSR representation, and the organisers' Track 1.2 round-two target detector is the ASVspoof 2021 RawNet2 baseline, the same architecture Deepfake-O-Meter hosts, which is not a challenge entry (arXiv 2305.13774, Sections 3.1 and 4.1, read 10 November 2026).
The data are Mandarin. Track 1 is a two-round attack-and-defence game: Track 1.1 teams generate fake speech for two AISHELL-3 speakers per round, and Track 1.2 teams detect fake audio whose real side comes from AISHELL-1 and THCHS-30 and whose fake side comes from text-to-speech and voice conversion systems plus the Track 1.1 submissions. Track 2 scores partially fake utterances in which segments of a genuine recording are replaced by synthesised or other real speech, with added noise and format conversion. Track 3 asks which of six known deepfake algorithms, or an unknown seventh, produced a clip (arXiv 2305.13774, Section 2 and Tables 1 to 3, read 10 November 2026).
Counts as printed in arXiv 2305.13774, Tables 1 to 3, read 10 November 2026.
Track 1.2 is scored by weighted EER, 0.4 times the round-one EER plus 0.6 times round two; Track 2 by 0.3 times sentence accuracy plus 0.7 times segment F1; Track 3 by macro-average F1 over the known classes with unknown-class errors counted; and Track 1.1 generators by deception success rate, the share of their clips that detection systems call real (arXiv 2305.13774, Section 3, read 10 November 2026).
What the study reports
| Metric | Value | Base |
|---|---|---|
| Track 1.2, top five weighted EER (Table 6) | B01 12.45%; B02 17.93%; B03 22.13%; B04 22.45%; B05 23.17% | 111,976 round-one and 118,477 round-two test utterances |
| Track 1.2 baselines (Table 6) | S03 wav2vec 2.0-LCNN 30.05%; S01 LFCC-GMM 53.04%; S02 LFCC-LCNN 66.72% | same |
| Track 1.2 field (Section 4.2) | 49 submissions, average weighted EER 49.94%, 11 below the best baseline; 23 submissions at 100% EER in one round (Table 6) | 49 submissions |
| Track 1.1 generation, best and average deception rate (Table 5) | A01 44.97% weighted; average 27.11% | 14 submissions |
| Track 2 region location, top three score (Table 7) | C01 67.13% (sentence accuracy 82.23%, segment F1 60.66%); C02 62.49%; C03 62.02% | 50,000 test utterances |
| Track 2 baseline and field (Table 7, Section 4.2) | S04 42.25%; average 48.82%; 11 teams above baseline | 16 submissions |
| Track 3 algorithm recognition, top three F1 (Table 8) | D01 89.63%; D02 83.12%; D03 75.41% | 79,490 test utterances, 8 classes |
| Track 3 baselines and field (Table 8, Section 4.2) | S06 OpenMax 54.16%; S05 softmax threshold 53.50%; average 62.87%; nine teams above the baselines | 11 submissions |
Every figure below is printed in Tables 5 to 8 or Section 4.2 of the challenge paper, or Tables 5 to 8 of the analysis paper.
Values as printed in arXiv 2305.13774, Tables 5 to 8 and Section 4.2, read 10 November 2026. The analysis paper prints S03 at 30.35% and C03 at 62.42% where the challenge paper prints 30.05% and 62.02% (arXiv 2408.04967, Tables 6 and 7).
Two findings carry beyond the leaderboard. First, the binary task was far from solved on this data: the winning weighted EER of 12.45% sits beside an average of 49.94% because 23 of the 49 submissions posted an EER of 100% in one of the two rounds, and the organisers' own best baseline reached only 30.05% (arXiv 2305.13774, Table 6, read 10 November 2026). The analysis paper attributes the leaders' margin to self-supervised front ends: four of the five best detectors use wav2vec 2.0 or WavLM features, and the winner pairs them with an AASIST back end and RawBoost augmentation (arXiv 2408.04967, Table 6 and Section 5, read 10 November 2026).
Second, locating the fake was harder than detecting it: the best region-location system reached a segment F1 of 60.66% against a sentence-level accuracy of 82.23%, and the organisers conclude that "it is challenging for fake region location", the two new tracks being "the first subchallenge of fake region location and algorithm recognition in the field of deepfake audio detection" (arXiv 2305.13774, Section 4.2 and Table 7, read 10 November 2026).
Limits the authors state
The analysis paper lists them under future directions: current detectors cope poorly with unseen generation methods and adversarial attacks; top models "will degrade significantly when evaluated on the mismatching dataset"; decisions are not interpretable beyond a score; real-time processing was not assessed; and the field's datasets "are mainly focused on single language, most of which in English and Chinese" (arXiv 2408.04967, Section 6, read 10 November 2026).
Evident from the papers: the two give different country counts for the 145 teams, 12 and 15; the Track 1.2 fake side includes clips built by rival participants during the challenge; the papers print no false positive or false negative rate, only EER, score and F1; and nothing identifies a commercial product.
Which cards this study feeds
- No card in the index was tested by this record.
- AIDI-R-07AI image, voice, video and code detectors ranked by independent studies
- AIDI-R-02AI detector false positive rates, ranked
No card takes a result line from this record, because no indexed product is a scored participant. The record counts toward the voice column of studies by modality as a record with zero indexed detectors, and the content type view lists the cards that state voice. No figure enters the false positive rate ranking, which ranks cards. The home page lists which AI detectors exist and who has tested them.
Related records: ASVspoof 2021 is the English challenge whose RawNet2 baseline ADD 2023 reused as a target detector; ASVspoof 5 is its 2024 edition; SAFE 2025 is the index's other audio challenge with named teams.
Questions about this study
What was the best detection result in ADD 2023?
A weighted equal error rate of 12.45% by team B01 (11.56% in round one, 13.05% in round two) over 49 Track 1.2 submissions whose average was 49.94%; the best organiser baseline, a wav2vec 2.0 and LCNN system, scored 30.05% (arXiv 2305.13774, Table 6).
Could ADD 2023 systems locate the manipulated region in a clip?
Partly: the best Track 2 system scored 67.13%, with 82.23% sentence accuracy but 60.66% segment F1, against a baseline of 42.25% and an average of 48.82% over 16 submissions, and the organisers call region location "challenging" (arXiv 2305.13774, Table 7 and Section 4.2).
Could ADD 2023 systems tell which algorithm made a fake?
The best Track 3 system reached a macro F1 of 89.63% over seven known classes and one unknown, against baselines of 53.50% and 54.16% and an average of 62.87% over 11 submissions (arXiv 2305.13774, Table 8).
Did any commercial voice detector take part in ADD 2023?
None is named: teams appear as rank-ordered IDs and no card in this index appears among them (arXiv 2305.13774, Section 4.2; arXiv 2408.04967, Tables 5 to 8).
Source and revisions
Source
Yi, J., Tao, J., Fu, R., Yan, X., Wang, C., Wang, T., Zhang, C. Y., Zhang, X., Zhao, Y., Ren, Y., Xu, L., Zhou, J., Gu, H., Wen, Z., Liang, S., Lian, Z., Nie, S., Li, H. (2023). ADD 2023: the Second Audio Deepfake Detection Challenge. Proceedings of the IJCAI 2023 Workshop on Deepfake Audio Detection and Analysis (DADA 2023), CEUR Workshop Proceedings 3597, 125 to 130; read as arXiv:2305.13774 v1 (23 May 2023), https://arxiv.org/abs/2305.13774 and https://arxiv.org/pdf/2305.13774, both read 10 November 2026; volume contents at https://ceur-ws.org/Vol-3597/, read 10 November 2026. Yi, J., Zhang, C. Y., Tao, J., Wang, C., Yan, X., Ren, Y., Gu, H., Zhou, J. (2024). ADD 2023: Towards Audio Deepfake Detection and Analysis in the Wild. arXiv:2408.04967 v3 (11 December 2024), https://arxiv.org/abs/2408.04967 and https://arxiv.org/pdf/2408.04967, both read 10 November 2026 (no DOI).
). Illuminarty, one of four cards added in the same wave, joined with its one image record from Ha and Passananti 2024, so 18 of 72 cards now qualify.: register plate](/img/independently-tested-480.webp)
