ASVspoof 2015:

Automatic Speaker Verification

Spoofing and Countermeasures Challenge

Introduction

Do you have a method/algorithm to discriminate between human and synthetic speech (generated from speech synthesis or voice conversion systems)? If so, you are invited to take part in the Automatic Speaker Verification Spoofing and Countermeasures (ASVspoof) Challenge.

Previously, both spoofing attacks and countermeasures have been developed with full knowledge of a particular speaker verification system used for vulnerability assessments. Similarly, countermeasures have been developed with full knowledge of the spoofing attack which they are designed to detect. This is clearly unrepresentative on the real use case scenario in which the specific attack, much less the specific algorithm, can never been known a priori. It is thus likely that the prior work has as much over-exaggerated the threat of spoofing as it has the performance of countermeasures.

The ASVspoof challenge has been designed to help break this mould and to support, for the first time, independent assessments of vulnerabilities to spoofing and of countermeasure performance. While preventing as much as possible the inappropriate use of prior knowledge, the challenge aims to stimulate the development of generalised countermeasures with potential to detect varying and unforeseen spoofing attacks.

The first evaluation, ASVspoof 2015, is being held within the scope of a special session at INTERSPEECH 2015 and with a focus on spoofing detection. Participants are invited to submit spoofing detection results. You will be provide with a spoofing database along with a protocol for experiments. The spoofing database is generated from more than ten voice conversion and speech synthesis spoofing techniques. Your task is to develop a classifier/detector on the development data to deal with some known spoofing attacks, and then apply your classifier or system on a evaluation data with both known attacks and some unknown attacks. Your classifier or system is expected to be reliable to both known and unknown attacks. No prior knowledge of automatic speaker verification technology is required to participate in the challenge!

Evaluation plan: Details of the evaluation plan can be access through this document: http://www.spoofingchallenge.org/asvSpoof.pdf

Results of the Challenge will be disseminated at a Special Session of Interspeech 2015.

The database is now publicly-available: http://dx.doi.org/10.7488/ds/298

Getting the data

The spoofing database is freely-available to all participants, please follow the following steps to get the data:

  1. Please send a request to Zhizheng Wu (zhizheng.wu@ed.ac.uk) to get an username and a password. Please indicate your institute in the email.
  2. Go to this webpage: http://dx.doi.org/10.7488/ds/298
  3. Enter your username and password as requested by above webpage
  4. Download 'ASVspoof2015_development.tar.gz', the size of the file is 5865596645 bytes
  5. Use the md5 checksum to verify your downloads: 0cdc334767cae92c541f220be650ef48

There is a mailing list for discussion for the challenge. Participants can also request a membership of the mailing list when they are requesting the username and password. Anyone who is interested in this topic but does not want to participant in the challenge, you can also request membership of the mailing list by send an email to Zhizheng Wu.

Background information

You could get some background information on spoofing and anti-spoofing from the survey paper:

Zhizheng Wu, Nicholas Evans, Tomi Kinnunen, Junichi Yamagishi, Federico Alegre, Haizhou Li, "Spoofing and countermeasures for speaker verification: a survey", Speech Communication, Volume 66, Pages 130–153, 2015 [PDF]

Submission of Results

Each team is required to submit a brief system description in PDF format, and can submit up to six score files as specified in the evaluation plan.

Each score file should be named according to the following convention <team>_<training-condition>_<submission>.

Each score file is about 1.2Mb after compression (4Mb before compression). Score files can be submitted by different emails, or all the score files are compressed into one archive. The archive name should indicate the team name. Participants need to indicate the primary score file.

The score files should be submitted by email attachment to asvspoof2015@spoofingchallenge.org with the following subject line: ASVspoof submission for <team name>.

Paper Submission

Please submit your challenge paper to the special session in Interspeech 2015: Automatic Speaker Verification Spoofing and Countermeasures (in area 4: Speaker and Language Identification). The organisers also submit an overview paper to the special session: ASVspoof 2015: the First Automatic Speaker Verification Spoofing and Countermeasures Challenge. The paper decribes the ASVspoof database, protocol, and metrics; and also summerises participants results.

Z. Wu, T. Kinnunen, N. Evans, J. Yamagishi, C. Hanilci, M. Sahidullah, A. Sizov, "ASVspoof 2015: the First Automatic Speaker Verification Spoofing and Countermeasures Challenge", Interspeech 2015 (accepted)

Challenge papers and results

Zhizheng Wu, Tomi Kinnunen, Nicolas Evans, Junichi Yamagishi, Cemal Hanilci, Md Sahidullah, Aleksandr Sizov, "ASVspoof 2015: the First Automatic Speaker Verification Spoofing and Countermeasures Challenge", Interspeech 2015 [PDF]

Md Jahangir Alam, Patrick Kenny, Gautam Bhattacharya, Themos Stafylakis, "Development of CRIM System for the Automatic Speaker Verification Spoofing and Countermeasures Challenge 2015", Interspeech 2015 [PDF]

Nanxin Chen, Yanmin Qian, Heinrich Dinkel, Bo Chen, Kai Yu, "Robust Deep Feature for Spoofing Detection - The SJTU System for ASVspoof 2015 Challenge", Interspeech 2015 [PDF]

Artur Janicki, "Spoofing Countermeasure Based on Analysis of Linear Prediction Error", Interspeech 2015 [PDF]

Yi Liu, Yao Tian, Liang He, Jia Liu, Michael T. Johnson, "Simultaneous Utilization of Spectral Magnitude and Phase Information to Extract Supervectors for Speaker Verification Anti-spoofing", Interspeech 2015 [PDF]

Sergey Novoselov, Alexandr Kozlov, Galina Lavrentyeva, Konstantin Simonchik, Vadim Shchemelinin, "STC Anti-spoofing Systems for the ASVspoof 2015 Challenge", arXiv:1507.08074, 2015 [PDF]

Tanvina B. Patel, Hemant A. Patil, "Combining Evidences from Mel Cepstral, Cochlear Filter Cepstral and Instantaneous Frequency Features for Detection of Natural vs. Spoofed Speech", Interspeech 2015 [PDF]

Jon Sanchez, Ibon Saratxaga, Inma Hernaez, Eva Navas, Daniel Erro, "The AHOLAB RPS SSD Spoofing Challenge 2015 submission", Interspeech 2015 [PDF]

Jesus Villalba, Antonio Miguel, Alfonso Ortega, Eduardo Lleida, "Spoofing Detection with DNN and One-class SVM for the ASVspoof 2015 Challenge", Interspeech 2015 [PDF]

Longbiao Wang , Yohei Yoshida, Yuta Kawakami, Seiichi Nakagawa, "Relative phase information for detecting human speech and spoofed speech", Interspeech 2015 [PDF]

Shitao Weng, Shushan Chen, Lei Yu, Xuewei Wu, Weicheng Cai, Zhi Liu, Ming Li, "The SYSU System for the Interspeech 2015 Automatic Speaker Verification Spoofing and Countermeasures Challenge", arXiv:1507.06711, 2015 [PDF]

Xiong Xiao, Xiaohai Tian, Steven Du, Haihua Xu, Eng Siong Chng, Haizhou Li, "Spoofing Speech Detection Using High Dimensional Magnitude and Phase Features: the NTU Approach for ASVspoof 2015 Challenge", Interspeech 2015 [PDF]

Important dates

16th December 2014Release development materials
6th February 2015Release evaluation data
20th February 2015Deadline for receipt of evaluation scores
27th February 2015Anti-spoofing evaluation results returned to participants
20th March 2015Interspeech paper submission deadline
15th June 2015Release meta information (including keys) for evaluation data to participants
September 2015Special session at Interspeech, Dresden, Germany

Organisers

Zhizheng Wu, University of Edinburgh, UK
Tomi Kinnunen, University of Eastern Finland, Finland
Nicholas Evans, EURECOM, France
Junichi Yamagishi, University of Edinburgh, UK