# Voice Deepfake Detection Using the Self-Supervised Pre-Training Model HuBERT
## 4 Experiment
### Datasets and Evaluation Metrics  
This study included experiments on the English ASVspoof 2021 LA dataset, Asvspoof 2019 LA dataset, Chinese FMFCC-A dataset, and Chinese FAD dataset. All four datasets were divided into three parts: training set, development set, and evaluation set, and the speakers in the subsets of the same dataset did not overlap with each other. The ASVspoof 2019 LA dataset is from 107 different speakers and contains real and fake discourse generated using 17 different TTS and VC systems [6]. The training and development sets for the ASVspoof 2021 LA dataset are the same as those released for the ASVspoof 2019 challenge. The evaluation set was recorded by 48 speakers corresponding to the ASVspoof 2019 challenge evaluation set [7]. The FMFCC-A dataset contains a collection of 40,000 synthetic and 10,000 genuine utterances. Moreover, the fake audios was generated based on 11 Mandarin TTS systems and 2 Mandarin VC systems, and the duration is randomly set in the range between 2 and 10 s, with the sampling rate of 16 kHz [8]. The FAD dataset consists of 12 types mainstream voice deepfake techniques such as STRAIGHT, LPCNet, and HifiGAN to generate fake audios, and real audios from six different corpora such as AISHELL1, AISHELL3, and THCHS-30 [26]. The evaluation set of the FAD dataset contains 14,000 utterances generated by four unknown deepfake methods that were not included in the training and validation sets, which can better detect the robustness and generalization of the model in the face of unknown attacks. The specific information of the dataset used in this paper is shown in Table 2.  
To evaluate the performance of the detection system, this study used two evaluation metrics commonly used for audio deepfake detection: equal error rate (EER) and tandem detection cost function (t-DCF) as evaluation metrics. The min t-DCF was proposed by \begin{table}
\begin{tabular}{c c} \hline \hline
**Name** & **Version** \\ \hline CPU & C86 7185 32-core Processor 2.0 GHz \\ Accelerator card & Dcu2 \\ Memory & 16 GB \\ Operating system & CentOS Linux 7.6 64-bit \\ Python & 3.7.2 \\ Deep learning library & PyTorch 1.10.0, fairseq 0.10.0 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Experimental environment. \begin{table}
\begin{tabular}{l c c c c c c c c c} \hline \hline \multicolumn{1}{c}{\multirow{2}{*}{**Database**}} & \multicolumn{3}{c}{**Number of Utterances**} & \multicolumn{3}{c}{**Number**} & \multicolumn{3}{c}{**Evaluation**} & \multirow{2}{*}{**Language**} & \multirow{2}{*}{**Storage Format**} \\ \cline{3-3} \cline{5-10}  & & **Train** & & & **Development** & & & & \\ \cline{3-10}  & **Real** & **Fake** & **Real** & **Fake** & **Real** & **Fake** & & \\ \hline ASVspoof 2021 LA & 231,790 & 2580 & 22,800 & 2548 & 22,296 & 14,816 & 166,750 & English & FLAC \\ ASVspoof 2019 LA & 121,461 & 2580 & 22,800 & 2548 & 22,296 & 7355 & 63,882 & English & FLAC \\ FMFCC-A & 50,000 & 4000 & 6000 & 3000 & 17,000 & 3000 & 17,000 & Chinese & WAV \\ FAD & 115,800 & 12,800 & 25,600 & 4800 & 9600 & 21,000 & 42,000 & Chinese & WAV \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Details of three databases. the ASVspoof 2019 challenge and improved by the ASVspoof 2021 challenge. The specific formulae for \(EER\) and min \(t-DCF\) are as follows:  
\[P_{\text{false}}(\theta)=\frac{\textit{faked voice with score}>\theta}{\textit{ total faked voice}}, \tag{6}\]  
\[P_{\text{miss}}\left(\theta\right)=\frac{\textit{true voice with score}\leq \theta}{\textit{total true voice}} \tag{7}\]  
\[EER=P_{\text{false}}(\theta_{EER})=P_{\text{miss}}(\theta_{EER}) \tag{8}\]  
\[\text{min }t-DCF=\min_{\theta}\Bigl{\{}\text{C}_{0}+\text{C}_{1}P_{miss}( \theta)+\text{C}_{2}P_{false}(\theta)\Bigr{\}} \tag{9}\]  
where the \(EER\) denotes the error rate when the false alarm rate \(P_{false}(\theta)\) and the miss alarm rate \(P_{miss}(\theta)\) are equal, and \(\theta_{EER}\) denotes the threshold value when \(P_{false}(\theta)\) and \(P_{miss}(\theta)\) are equal.