Review of Technological Advancement and Challenge in Speech Spoofing Detection
(1.Key Laboratory of Artificial Intelligence Application Technology State Ethnic Affairs Commission, Qinghai Minzu University, Xining 810007, China; 2.Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin 300072, China)
Abstract
As a natural and convenient interaction mode, speech is widely applied in numerous fields. Speech Spoofing Detection technology has become a key technology for ensuring the security of speech verification and recognition systems. With the rapid development of Text-to-Speech (TTS) synthesis, Voice Conversion (VC) and Deepfake technologies, the generation quality and complexity of spoofed speech are greatly improved, which poses severe challenges to Speech Spoofing Detection. This paper systematically reviews the technological advancement in the field of Speech Spoofing Detection, covering the mainstream methods based on acoustic features and Deep Learning. It first discusses the main types of speech spoofing attacks, TTS-generated speech and VC-spoofed speech. It then analyzes the current mainstream detection models and performance metrics. It finally explores the limitations of existing methods and presents the key problems to be solved urgently in future research. Through a comprehensive analysis of Speech Spoofing Detection techniques, this paper provides new insights for research in this field.
Keywords
automatic speaker verification; spoofing attack; Speech Spoofing Detection; spoofing countermeasure
References
- [1] GOODFELLOW I,POUGET-ABADIE J,MIRZA M,et al. Generative Adversarial Nets [J].Advances in Neural Information Processing Systems,2014,27:2672-2680.
- [2] WU Z,EVANS N,KINNUNEN T,et al. Spoofing and Countermeasures for Speaker Verification: A Survey [J].Speech Communication,2015,66:130-153.
- [3] HAUTAMÄKI R G,KINNUNEN T,HAUTAMÄKI V,et al. I-Vectors Meet Imitators: On Vulnerability of Speaker Verification Systems Against Voice Mimicry [C]//Interspeech 2013. Lyon:ISCA,2013:930-934.
- [4] KINNUNEN T,WU Z Z,LEE K A,et al. Vulnerability of Speaker Verification Systems Against Voice Conversion Spoofing Attacks: The Case of Telephone Speech [C]//2012 IEEE International Conference on Acoustics, Speech and Signal Processing. Kyoto:IEEE,2012:4401-4404.
- [5] KIM E K,LEE S,OH Y H. Hidden Markov Model Based Voice Conversion Using Dynamic Characteristics of Speaker [C]//Eurospeech 1997.Rhodes:ISCA,1997:2519-2522.
- [6] STYLIANOU Y,CAPPÉ O,MOULINES E. Continuous Probabilistic Transform for Voice Conversion [J].IEEE Transactions on Speech and Audio Processing,1998,6(2):131-142.
- [7] ZAREMBA W. Recurrent Neural Network Regularization [J/OL].arXiv:1409.2329 [cs.NE].(2015-02-19).https://arxiv.org/abs/1409.2329.
- [8] VAN DEN OORD A,DIELEMAN S,ZEN H,et al. WaveNet: A Generative Model for Raw Audio [J/OL].arXiv:1609.03499v2 [cs.SD].(2016-09-19).https://arxiv.org/pdf/1609.03499.
- [9] WANG Y,SKERRY-RYAN R J,STANTON D,et al. Tacotron: Towards End-to-End Speech Synthesis [J/OL].arXiv:1703.10135 [cs.CL].(2017-04-06).https://arxiv.org/abs/1703.10135.
- [10] VILLALBA J,LLEIDA E. Speaker Verification Performance Degradation Against Spoofing and Tampering Attacks [C]// Proceedings of Fala 2010.Vigo:Universidade de Vigo,2010:131-134.
- [11] VILLALBA J,LLEIDA E. Detecting Replay Attacks From Far-Field Recordings on Speaker Verification Systems [C]//BioID 2011.Berlin:Springer,2011:274-285.
- [12] KORSHUNOV P,MARCEL S. Vulnerability Assessment and Detection of Deepfake Videos [C]//2019 International Conference on Biometrics.Crete:IEEE,2019:1-6.
- [13] TODISCO M,DELGADO H,EVANS N. Constant Q Cepstral Coefficients: A Spoofing Countermeasure for Automatic Speaker Verification [J].Computer Speech & Language,2017,45:516-535.
- [14] LI X,LI N,ZHONG J,et al. Investigating Robustness of Adversarial Samples Detection for Automatic Speaker Verification [J/OL].arXiv:2006.06186 [eess.AS].(2020-08-07).https://arxiv.org/abs/2006.06186.
- [15] TAK H,TODISCO M,WANG X,et al. Automatic Speaker Verification Spoofing and Deepfake Detection Using Wav2Vec 2.0 and Data Augmentation [J/OL].arXiv:2202.12233 [eess.AS].(2022-02-28).https://arxiv.org/abs/2202.12233.
- [16] GOMEZ-ALANIS A,GONZALEZ-LOPEZ J A,DUBAGUNTA S P,et al. On Joint Optimization of Automatic Speaker Verification and Anti-Spoofing in the Embedding Space [J].IEEE Transactions on Information Forensics and Security,2020,16:1579-1593.
- [17] YOON S H,KOH M S,PARK J H,et al. A New Replay Attack Against Automatic Speaker Verification Systems [J].IEEE Access,2020,8:36080-36088.
- [18] ZENG C,WANG X,COOPER E,et al. Attention Back-End for Automatic Speaker Verification With Multiple Enrollment Utterances [C]//ICASSP 2022. Singapore:IEEE,2022:6717-6721.
- [19] YU H,TAN Z H,MA Z,et al. Spoofing Detection in Automatic Speaker Verification Systems Using DNN Classifiers and Dynamic Acoustic Features [J].IEEE Transactions on Neural Networks and Learning Systems,2017,29(10):4633-4644.
- [20] LAVRENTYEVA G,NOVOSELOV S,TSEREN A,et al. STC Anti-Spoofing Systems for the ASVspoof2019 Challenge [C]//Interspeech 2019.Graz:ISCA,2019:1033-1037.
- [21] TAK H,PATINO J,TODISCO M,et al. End-to-End Anti-Spoofing With RawNet2 [C]//ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).Toronto:IEEE,2021:6369-6373.
- [22] TAK H,JUNG J,PATINO J,et al. End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection [J/OL].arXiv:2107.12710 [eess.AS].(2021-08-23).https://arxiv.org/abs/2107.12710.
- [23] JUNG J W,HEO H,TAK H,et al. AASIST: Audio Anti-Spoofing Using Integrated Spectro-Temporal Graph Attention Networks [C]//ICASSP 2022-2022 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Singapore:IEEE,2022:6367-6371.
- [24] GONG J R,CHEN N Q. Synthetic Voice Spoofing Detection Based on Feature Pyramid Conformer [C]//Interspeech 2023. Dublin:ISCA,2023:2854-2858.
- [25] WANG C,YI J,TAO J,et al. TO-RawNet: Improving RawNet With TCN and Orthogonal Regularization for Fake Audio Detection [J/OL].arXiv:2305.13701 [cs.SD].(2023-05-23).https://arxiv.org/abs/2305.13701.
- [26] NANDWANA M K,KUMAR R,SINGH A,et al. Voice Toxicity Detection Using Multi-Task Learning [C]//2024 IEEE International Conference on Acoustics,Speech and Signal Processing (ICASSP).Seoul:IEEE,2024:331-335.
- [27] WANG C,YI J,TAO J,et al. Detection of Cross-Dataset Fake Audio Based on Prosodic and Pronunciation Features [C]//Interspeech 2023. Dublin:ISCA,2023:3844-3848.
- [28] ZHANG Y,LI Z,LU J,et al. Synthetic Speech Detection Based on the Temporal Consistency of Speaker Features [J].IEEE Signal Processing Letters,2024,31:944-948.
- [29] ZHANG L,WANG X,COOPER E,et al. An Initial Investigation for Detecting Partially Spoofed Audio [J/OL].arXiv:2104.02518 [eess.AS].(2021-06-15).https://arxiv.org/abs/2104.02518.
- [30] ZHANG L,WANG X,COOPER E,et al. Multi-Task Learning in Utterance-Level and Segmental-Level Spoof Detection [J/OL].arXiv:2107.14132 [cs.SD].(2021-08-31).https://arxiv.org/abs/2107.14132.
- [31] ZHU Y P,CHEN Y X,ZHAO Z X,et al. Local Self-Attention-Based Hybrid Multiple Instance Learning for Partial Spoof Speech Detection [J].ACM Transactions on Intelligent Systems and Technology,2023,14(5):1-18.
Funding
National Natural Science Foundation of China (62261045); Key R&D and Transformation Plan of Qinghai Province in 2022 (2022-QY-218); 2021 High level Talents (Doctoral) Program (2021XJG19); Graduate Innovation Project of Qinghai University for Nationalities 2025 (73M2025019)
About the authors
Tang Wenjie (2001-), female, Han ethnicity, from Mianyang, Sichuan, is currently pursuing a master's degree in speech recognition, network and information security; Corresponding author: Geng Taijia (1981-), male, Tibetan, from Xining, Qinghai, professor, master's supervisor, PhD, research direction: speech recognition, speech synthesis, natural language processing; Wei Jianguo (1971-), male, Han ethnicity, Tianjin native, professor, doctoral supervisor, PhD, research interests: speech recognition, voiceprint recognition, medical image analysis.