# Voice Deepfake Detection Using the Self-Supervised Pre-Training Model HuBERT
## 3 Methods
### Self-Supervised Pre-Training Speech Models and Fine-Tuning
#### 3.2.1 Self-Supervised Pre-Trained Speech Model  
WenetSpeech is currently the largest open-source Mandarin speech corpus with transcriptions. The data were mainly derived from YouTube videos and Podcast audio, covering various types of recording scenes, background noise, speaking methods, etc. It specifically included 10 scenes such as audiobooks, commentary, documentaries, TV shows, interviews, etc., with more than 10,000 h of data [16]. Among the many self-supervised speech models, this study used the HuBERT pre-training model trained on the WenetSpeech Chinese dataset, and the HuBERT pre-training model trained on the LibriSpeech English dataset, to test the effect on the FMFCC-A Chinese dataset [8] and the ASVspoof 2021 LA English dataset.  
Let \(W\) denotes a speech utterance \(W=[w_{1},w_{2},\cdots,w_{T}]\) of T-frames. As is shown in Figure 2a, the acoustic unit discovery system generates the target label \(f(x)=Z=\{z_{1},z_{2},\cdots,z_{T}\}\) with the k-means clustering algorithm, such as MFCC features. At the same time, \(W\) generates a feature sequence \([x_{1},x_{2},\cdots,x_{T}]\) through a CNN encoder. Let \(M\) be the index of the masked sequence \(X^{\prime}\), and \(X^{\prime}\) represents the masked sequence, using the same strategy as wav2 vec 2.0 to generate the mask, \(X^{\prime}=random(X,M)\). The variable \(p\) represents the proportion of randomly selected starting indices throughout the entire T-frame of speech. The variable \(l\) represents the step size, which is set to 10 and \(M=p\%\times T+l\). If \(t\in M\); then \(x_{t}\) is replaced by an embedded mask, and the mask prediction model \(G\) takes \(X^{\prime}\) as the input and predicts the label distribution of the discrete units.  
The BERT encoder is composed of many layers of transformer encoders. This study used a BERT encoder consisting of 12 layers of transformer encoders. The BERT encoder inputs the mask sequence \(X^{\prime}\) and outputs a feature sequence \(O=[o_{1},o_{2},\cdots,o_{T}]\). The proportion \(p\) of the start index in the entire T-frame of speech is as per that shown in Equation (3):  
\[p_{g}^{(k)}(c\mid\widetilde{X},t)=\frac{\exp\Bigl{(}\text{similarity} \Bigl{(}\mathbf{A}^{(k)}o_{t},\mathbf{e}_{c}\Bigr{)}/\tau\Bigr{)}}{\sum\limits_{c ^{\prime}=1}^{\text{C}}\exp\Bigl{(}\text{similarity}\Bigl{(}\mathbf{A}^{(k)}o_{t}, \mathbf{e}_{c^{\prime}}\Bigr{)}/\tau\Bigr{)}}, \tag{3}\]  
where \(\mathbf{A}\) is the projection matrix, \(e_{c}\) is the embedding for the codebook, similarity calculates the cosine similarity between two vectors, and \(\tau\) is used to scale the logarithmic function, which is set to 0.1.  
By iterative training, the cross-entropy loss functions \(LOSS_{mask}\) and \(LOSS_{unmask}\) are calculated on masked and unmasked units, and then the final loss value LOSS is obtained by weighted summation, as shown in Equation (4):  
\[LOSS=\alpha LOSS_{Mask}+(1-\alpha)LOSS_{unmask}, \tag{4}\]  
The HuBERT model is similar to the classic wav2 vec 2.0 model, but the training methods are different. The latter is to discretize the audio features as a self-supervised target during training, which is characterized by calculating the loss function only in the mask area; the former obtains the training target by carrying out k-means clustering on MFCC features. The results show that the performance of the HuBERT model is better than that of the wav2 vec 2.0 model [9].