Audio Deepfake Detection with Self-Supervised XLS-R and SLS Classifier

Authors: qishan  zhang
shuangbing  wen
tao  hu
Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-07 16:06:50

Abstract: Generative AI technologies, including text-to-speech (TTS) and voice conversion (VC), frequently become indistinguishable from genuine samples, posing challenges for individuals in discerning between real and synthetic content.
This indistinguishability undermines trust in media, and the arbitrary cloning of personal voice signals presents significant challenges to privacy and security.
In the field of deepfake audio detection, the majority of models achieving higher detection accuracy currently employ self-supervised pre-trained models.
However, with the ongoing development of deepfake audio generation algorithms, maintaining high discrimination accuracy against new algorithms grows more challenging.
To enhance the sensitivity of deepfake audio features, we propose a deepfake audio detection model that incorporates an SLS (Sensitive Layer Selection) module.
Specifically, utilizing the pre-trained XLS-R enables our model to extract diverse audio features from its various layers, each providing distinct discriminative information.
Utilizing the SLS classifier, our model captures sensitive contextual information across different layer levels of audio features, effectively employing this information for fake audio detection. Experimental results show that our method achieves state-of-the-art (SOTA) performance on both the ASVspoof 2021 DF and In-the-Wild datasets, with a specific Equal Error Rate (EER) of 1.92\% on the ASVspoof 2021 DF dataset and 7.46\% on the In-the-Wild dataset. Codes and data can be found at https://github.com/QiShanZhang/SLSforADD.

Main paper: 
 introduction: 
 Deepfake audio produced by artificial intelligence algorithms like text-to-speech (TTS) [1] , voice conversion (VC) [2] difficult to distinguish from real samples [3] have the potential to cause significant social and economic damage, have already been used to scam a CEO for 243.000$ [4] . Additionally, the potential for slander, misinformation, and fake news is enormous. There are also many biometric identity authentication applications, such as access control systems, telephone banking, and forensic scenarios [5] . Thus, there is a need for automatic detection and verification of human speech.The audio deepfake detection task emerge as required by the times. The ASVspoof initiative and challenge series [6] was conceived to foster the development of countermeasures to protect against the manipulation of Automatic Speaker Verify (ASV) systems from spoofing attacks. The latest ASVspoof challenge is [7] , which has introduced a new task: Deepfake (DF). This task aims to distinguish genuine utterances from AI-generated fake ones using machine learning techniques. The Audio deepfake detection challenge(ADD) [8, 9] has also held aim to fill the gap between the attack and the defense. Therefore, numerous deepfake audio detection algorithms have been proposed, which can essentially be classified into three categories: one based on handcrafted features such as Linear Frequency Cepstral Coefficients (LFCC) [10] , Mel Frequency Cepstral Coefficients (MFCC) [11] , Constant Q Cepstral Coefficients (CQCC) [12] , etc., another on deep learning features like SincNet [13] , FastAudio [14] , and the third on features from pre-trained self-supervised models such as Wav2vec [15, 16] , XLS-R [17] , HuBERT [18] , WavLM [19] , and Whisper [20] .The flaw of handcrafted features is that they may overlook potentially useful characteristics for identifying deepfake audio, leading to deep learning features increasingly becoming the mainstream solution. Obtaining fake utterances for deep learning models to learn to extract features is costly and technically demanding, further complicated by the continuous emergence of new generative algorithms. Therefore, an efficient method involves using self-supervised pretrained models that can be trained with any bona fide speech data. Using features from pre-trained models has achieved high detection accuracy. However, even with large pre-trained voice models, maintaining high accuracy against previously unseen neural network attack algorithms remains a challenge. Currently, the topperforming model on the ASVspoof 2021 deepfake (DF) eval dataset has achieved an EER of 2.56% [21] , significantly lagging behind the 0.82% [16] achieved on the ASVspoof 2021 logical access (LA) eval dataset. The latter faces 17 unseen attacks, a number that is less than the array faced by the DF dataset.In this paper, we are motivated by the following: (i) We believe that the features in the pre-trained, self-supervised XLS-R hidden layers contain useful discriminatory characteristics of audio, which can be very helpful in identifying deepfake audio, as shown in Figure 1 . Regardless of whether the audio is real or deepfake, the features displayed by different hidden layers of XLS-R exhibit the same pattern. However, the difference map between real audios (row 4 of Figure 1 ) and the difference map between real and deepfake audios (row 5 of Figure 1 ) show different disparities in certain hidden layers. We believe that these differences may serve as effective features for distinguishing deepfake audio. (ii) When the hidden layers of XLS-R indeed offer more effective features for deepfake audio detection, is it still necessary to apply data augmentation and fine-tune pre-trained models?The principal contributions of this work are:• We propose a self-supervised, pre-trained XLS-R modelbased SLS classifier for detecting deepfake audio. Features extracted from various XLS-R Transformer layers are fed into our classifier. The XLS-R features processed by the SLS module have a strong ability to distinguish between real and fake audio and exhibit strong generalization ability. The feature maps of randomly selected real and fake audio processed by SLS have significant differences. At the same time, the module boasts rapid convergence and a simple architecture. • For the first time, the method proposed in this paper has achieved an Equal Error Rate (EER) below 2% on the ASVspoof 2021 DF dataset, demonstrated competitive performance on the ASVspoof 2021 LA dataset, and secured state-of-the-art performance on the In-The-Wild dataset. Provides a new perspective for improving the robustness of deepfake audio detection models. Further analysis of experimental outcomes reveals that fine-tuning continues to play a pivotal role in enhancing the model's performance. Moreover, even with the exploitation of rich hidden layer information from the XLS-R model, the necessity for data augmentation persists. 
related work: 
 In Section 1, we introduce the most effective feature for audio deepfake detection when facing new generative algorithms: features from pre-trained models. This approach significantly enhances the model's recognition accuracy, thereby overcoming the bottleneck of limited training data availability. A number of pre-trained, selfsupervised models are publicly available, including Wav2vec 2.0 [22] , XLS-R [23] (a variant of Wav2vec 2.0), HuBERT [24] , WavLM [19] , and Whisper [20] . In this section, we briefly review prior work that utilizes pre-trained models to enhance model generalizability with the same limited training data as ours.In 2021, Xie et al. [15] for spoofing speech detection. This approach significantly reduced the Equal Error Rate (EER) from the previous state-of-the-art result of 4.07% to 1.15% on the ASVspoof 2019 evaluation set [25] . Martin-Donas [26] designed a system utilizing various transformer layers from XLS-128, a large-scale model for cross-lingual speech representation learning, paired with a simple downstream model to detect deepfake audio, achieving a 4.98% EER on the ASVspoof 2021 DF dataset. Wang et al. [18] explored various pre-trained, selfsupervised speech models as the frontend for spoofing countermeasures. They experimented with Wav2vec2-small, Wav2vec2-large1, Wav2vec2-large2, HuBERT-XL, and XLS-53 (which is similar to Wav2vec2's structure but uses more training data). The experimental results suggest that the backend needs to be deep when the pre-trained frontend is not fine-tuned. In contrast, if the frontend can be fine-tuned, a simple backend with just average temporal pooling and a linear layer is sufficient. Additionally, the XLS-53 frontend with LGF classfier achieved a 4.75% EER on the DF dataset. Tak et al. [16] achieved significantly improved performance in the field of spoofing detection by applying the XLS-R model with a Rawnet2 [27] encoder and AASIST [28] backend, achieving an EER of 2.85% on the ASVspoof 2021 DF eval set. Furthermore, they demonstrated the use of data augmentation, showing its complementary benefits to self-supervised learning. Guo et al. [21] introduced the usage of WavLM as a frontend feature extractor and proposed the Multi-Fusion Attentive (MFA) classifier, based on the attentive statistics pooling layer. The MFA aggregates the output representations of WavLM, focusing on features at various layers and time steps, thus facilitating the extraction of highly discriminative features. These methods pushed the DF eval set's EER to 2.56%.Wang et al. [29] utilized the pre-trained model HuBERT to extract duration features from the waveform, and employed a conformer to extract pronunciation features. These features were then fused with Wav2vec 2.0's output features via an attention mechanism, achieving a 29.53% EER on the In-the-wild dataset. Yang et al. [30] investigated the performance of a broad range of pre-trained models, including XLS-R, HuBERT, WavLM, and Whisper, coupled with a ResNet18 backend. They found that XLS-R performs best on the DF eval set, while HuBERT excels on the In-the-Wild dataset. They proposed two multi-view feature incorporation methods to capture the subtleties of multiple candidate features from XLS-R, WavLM, and HuBERT, thereby enhancing the system's performance and generalizability. This approach achieved a 24.27% EER on the Inthe-Wild dataset. 
proposed method: 
 Our objective is to detect deepfake audio by classifying the contextualized representations derived from various transformer layer outputs of the pre-trained XLS-R model. Previous studies have demonstrated that, for many tasks such as speaker verification or emotion recognition, more discriminative information can be gleaned from the initial or intermediate layers of pre-trained models Since synthetic TTS algorithms cannot accurately mimic the real human speech flow and duration, the voice conversion process introduces vocoder artifacts into the audio signals, and some sound quality may be lost. This may result in the generated sound being unnatural or distorted. Pre-trained models can effectively capture these features for use as input in downstream models. Furthermore, the hidden layers of the transformer may provide representations more suitable for audio deepfake detection tasks. To achieve this, consider 𝑥 to be the waveform of a human voice signal labeled 𝑦 ∈ 0, 1, where 0 signifies a real human voice and 1 denotes a synthetic human voice. Our aim is to construct a classifier ŷ = 𝐹 𝜃 (𝑥) to predict the label of input 𝑥. The binary detection model is constructed as a cascade of neural networks:EQUATIONwhere 𝑊 𝜃 𝑊 represents the front-end XLS-R model for audio representation, equipped with its own set of parameters 𝜃 𝑊 . 𝐶 𝜃 𝑐 denotes a backend binary classifier with a Senstive Layer Select (SLS) module designed to select useful outputs from different XLS-R transformer layers, and 𝜃 𝑐 are its parameters. This classifier can be optimized by solving:EQUATIONwhere L 𝑏 (𝑦, ŷ) represents the cross-entropy loss for binary classification, and 𝑇 denotes the training dataset comprising labeled real and synthetic examples.The model's overall framework is depicted in Fig. 2 .Input raw waveforms are processed by the XLS-R model to obtain contextualized feature representations from various transformer layers.All contextualized representation features from the transformer layers are then input into our classifier.Within the classifier, features from various layers initially undergo weight adjustment, are subsequently summed, and then connected to a simple fully connected layer for binary classification.Detailed explanations of each component are presented in the subsequent subsections.The XLS-R [23] is a large-scale model for cross-lingual speech representation learning, based on wav2vec 2.0 and trained across 128 languages.The raw speech signal 𝑥 is initially processed by a feature encoder comprising several convolutional layers (CNN), extracting vector representations of size 1024 every 20ms, utilizing a receptive field of 25ms. This process yields 𝑧. Subsequently, these encoder features 𝑧 are input into a transformer network comprising 24 layers, which is utilized to derive contextualized representations ℎ. The outputs from the distinct 24 transformer layers are concatenated as: 𝐻 = [ℎ 1 , ℎ 2 , ..., ℎ 𝐿 ]. The model is trained in a self-supervised setting using a contrastive loss. The primary objective is to predict the quantized representations of specific masked encoded features from a set of distractors, utilizing the contextualized representations.Consequently, this model is capable of learning high-level representations of the waveform signal. The features extracted from the pre-trained XLS-R model can be used to train a downstream classifier in a specific task with a relatively few amount of labeled data.Moreover, the XLS-R model and downstream models can be jointly trained in the related task. In this work, the utilization of XLS-R as a pre-trained model is explored. The mathematical representation of processing the audio signal 𝑥 through the XLS-R model is as follows:EQUATIONIn this section, our objective is to employ algorithms capable of selecting useful contextualized representations from different XLS-R transformer layers. Inspired by the SENet, as proposed by Hu Jie et al. [31] , it introduces a mechanism designed to recalibrate channelwise feature responses by explicitly modeling interdependencies between channels. Given 𝐻 = [ℎ 1 , ℎ 2 , ..., ℎ 𝐿 ], our goal is to obtain:EQUATIONWhere ℎ 𝑙 denotes the output of a distinct transformer layer in the XLS-R model, and 𝐿 represents the number of transformer layers.𝛼 𝑙 represents the layer weight, defined as 𝛼 = [𝛼 1 , 𝛼 2 , ..., 𝛼 24 ]. This is derived from 𝐻 ∈ R 𝐿×𝑁 ×1024 , as illustrated in the equation below.EQUATIONIn the above equation, 𝑎𝑣𝑔𝑝𝑜𝑜𝑙 denotes an average pooling operation that is applied along the dimension 𝑁 of 𝐻 ∈ R 𝐿×𝑁 ×1024 . Subsequently, Ĥ ∈ R 𝐿×1×1024 is obtained, and through the full connected layer (FC), Ĥ ∈ R 𝐿×1×1 is produced. Our design rationale is that a single audio frame from avgpooling can represent the entire audio, and representing audio frames through a full connection can ascertain the layer's usefulness. A sigmoid function is then applied to these weights to ensure scaling between 0 and 1, thus facilitating dynamic channel-wise recalibration of the feature maps. Subsequently, the results of the 24 weighted layers are aggregated. This process is illustrated in Fig. 3 . 
experiment: 
 To confirm the motivations outlined in the introduction, we conducted a series of experiments to evaluate the effectiveness of our proposed method. The first set entails jointly fine-tuning the XLS-R We utilized the ASVspoof 2019 LA database [25, 32] training part for training, opting not to use the validation part, because we believe the validation set is traditionally used to prevent overfitting, but the validation part, compared to the train set, only differs in the speaker, we posit that the validation set can only prevent the model from overfitting to the train set's speaker features, it does not possess the ability to prevent overfitting produced by generative algorithms' features, and in the real world, it is necessary to confront features generated by various algorithms, thus, we exclusively use the train set and monitor training loss to determine the occurrence of overfitting. We evaluated our approach on the ASVspoof 2021 LA,DF [33] , and In-The-Wild [34] evaluation sets. The 2021 LA dataset presents a challenge due to its inclusion of codec and transmission variability, elements not present in the training and validation datasets. The DF dataset is more reflective of the performance of audio deepfake detection algorithms compared to the LA dataset; it contains over a hundred generation algorithms not seen in the training set, as well as compression variability during audio transmission. The In-The-Wild dataset consists of 37.9 hours of audio clips that are either fake (17.2 hours) or real (20.7 hours). The fake clips were created by segmenting 219 publicly available videos and audio files that explicitly advertise audio deepfakes. The corresponding genuine instances from the same speakers were collected using publicly available materials, such as podcasts, speeches, etc. The fake and genuine instances are similar in aspects such as background noise, emotions, and duration. Based on the above, the DF and In-the-Wild datasets are the most challenging and capable of reflecting the generality of models. The Equal Error Rate (EER) [35] is used as the evaluation metric. Audio data is cropped or concatenated, yielding segments of approximately 4 seconds in duration (64,600 samples). We employ the pre-trained model XLS-R 300M provided by the literature [23] , which is jointly optimised with the back-end SLS classifier using back-propagation [36] , and utilize the Adam optimizer with the learning rate is 10 -6 and weight_decay 0.0001. The batch size is 5. We set the training duration to 50 epochs, incorporating early stopping technology with a patience of 3. This indicates that when the training loss does not improve for three epochs, the training is stopped, and the model with the lowest loss is chosen for evaluation. All models were trained on a single GeForce RTX 4090 GPU, and all results are reproducible using open source code 1 with the same random seed and GPU environment. RawBoost algorithm 3 [37] is used as the data augmentation method. The RawBoost algorithm adds white noise, which is used to simulate the confrontation caused by electromagnetic interference in audio spoofing detection. The white noise is processed by the FIR filter and added to the audio signal. Table 2 shows a comparison of our results with those of other models on the ASVspoof 2021 LA and DF evaluation datasets. Our method demonstrated the best performance on the DF dataset,. To our knowledge, this is the lowest reported equal error rate (EER) on the DF evaluation dataset, marking the first time EER has dropped below 2%. And achieve the lowest EER on the LA dataset. From Table 2 , it can be seen that manual features such as CQCC and LFCC exhibit significant differences compared to pre trained features when combined with GMM or deep classifier LCNN. [26] and [21] also used hidden layer features, which proves the hypothesis we proposed at the beginning of the experiment. Previous models have limited the performance of hidden layer features, demonstrating the effectiveness of our proposed SLS classifier.As shown in Table 4 , audio generated by the Neural AR vocoder is the most difficult to distinguish under any condition. In the future, efforts can be made to explore methods to reduce this gap. It can be observed that our model has significantly minimized differences among various conditions; compared to the baseline condition C1, there are no significant disparities. Compared to Tak's model, which uses the same data augmentation and front-end, this demonstrates our SLS classifier's ability. The XLS-R features processed through our SLS module produce a very powerful feature map that can distinguish bona-fide or deepfake audio. This is illustrated in Figure 6 .Furthermore, in the LA dataset (Table 5 ), the transmission condition factors significantly influence, resulting in substantial disparities in EER. In Condition C1, which lacks transmission factors, algorithms such as A07, A08, A09, A13, and A14 generate deepfake audio that our model detects with 100% accuracy; moreover, different algorithms do not exhibit particularly large fluctuations in accuracy. The accuracy for Condition 6 dropped by approximately 700%. This is attributed to the use of Data Augmentation (DA) algorithms that adapt to data loss during the compression process. Our primary goal is to address the robustness of new generative algorithms; therefore, we did not implement data augmentation for transmission condition factors.As demonstrated in Table 3 , our model exhibited unprecedented generalization capabilities on the In-The-Wild dataset, achieving state-of-the-art performance. This result further substantiates the Table 4 : Results in terms of Equal Error Rates (EERs, %) for each codec condition (DFC1-DFC9) and different generative algorithm vocoders on the ASVspoof 2021 DF evaluation partition. In the leftmost column, 'V' represents Vocoder, 'T' traditional vocoder, 'C' Wav concatenation, 'N' autoregressive neural vocoder, 'Nn' non-autoregressive use,'U' unkonw vocoder, and 'P' weighted pooling. In each condition, the left column represents our results, and the right column shows the results from Tak [16] . The underlying reason is likely attributable to our utilization of rich hidden layer features. Compared to prior work utilizing hidden layer features [21, 26] , the weight assignment function is identified as playing a crucial role. These studies employ softmax for weight assignment, thereby limiting the classifier to the utilization of features from no more than one layer. The ablation study Section 4.4 and Figure 4 support this viewpoint.V C1 C2 C3 C4 C5 C6 C7 C8 C9 Pooled T 1.Compared to prior work [16, 30] , our model architecture achieves high accuracy despite its simplicity. This phenomenon prompts a reevaluation of audio deepfake detection model design principles. The comparison of model convergence speeds, as illustrated in Figure 5 , shows our model converges faster than Tak's model [16] , provided all other training hyperparameters remain constant.This further substantiates the capability of XLS-R's hidden layers to provide a rich and effective feature set for discriminating against audio deepfake.Based on the above experimental results, we introduce a new perspective on utilizing the XLS-R hidden layers, which contain rich features capable of enhancing deepfake audio detection performance. We propose an SLS classifier that uses a simple architecture and achieves fast convergence, resulting in state-of-the-art (SOTA) performance across three challenging datasets.Demonstrated that every component of our proposed model is essential in this section. Table 6 presents the results of our ablation experiments on each component of the modified architecture. Clearly, the sensitive layer selection module plays a critical role in the model. Clearly, we utilized the rich hidden layer features of XLS-R and the SLS effectively processed its features, our model still achieved cutting-edge performance without fine-tuning XLS-R. However, compared to fine-tuning, accuracy on the DF and In-the-Wild datasets declined. Conversely, on the LA dataset, there was an improvement, likely due to fine-tuning accelerating the model's adaptation to certain DF dataset features.Without data augmentation, the model's performance significantly decreased across all three datasets, possibly due to conditions not encountered during XLS-R's pre-training, like encoding loss factors affecting audio data. In the absence of data augmentation, the model's accuracy was higher on the LA dataset than on the DF dataset, aligning with our intuition given the DF dataset's challenging nature. Accuracy also declined on the In-The-Wild dataset. This indicates that our data augmentation methods not only counteract encoding loss effectively but also accommodate the distribution of other real-world factors.To test whether each hidden layer provides valuable discriminative features, we removed the first five layers of features with low weights, as shown in Figure 4 . The experimental results showed that although the features in the first few layers were assigned lower weights, they still provided useful discriminative features. After removing these first five layers, the error rates of the three datasets increased.The final ablation experiment confirmed our earlier hypothesis: replacing the weighted classification function in the SLS module with softmax limited the features provided by XLS-R. Simply aggregating all features from XLS-R yielded better results in the DF and LA datasets compared to softmax, whereas in the In-The-Wild dataset, softmax outperformed simple aggregation. We conclude that the In-The-Wild dataset encompasses the most diverse range of real-world factors affecting audio. Therefore, when XLS-R provides all feature representations, simply aggregating them results in chaotic feature maps from the SLS module, underscoring the need to differentiate between the features in real-world scenarios. The experiments suggest that our proposed SLS classifier is a promising option.This contrasts with previous studies [26] that used softmax, a difference we attribute to our unique approach in feature processing. We treat weighted features as a single-channel image and apply direct pooling to reduce data volume. Unlike the fully connected layers employed in these studies, our pooling method retains more original features.Upon further analysis of the SLS module's weight distribution map, we observed that the SLS module effectively discerns the disparity between the feature maps of genuine and fake audio. This selective aggregation of feature maps facilitates classification by downstream models. As illustrated in Figure 6 , randomly selected examples from the DF dataset underscore this remarkable capability. Moreover, it exhibited consistency across various datasets. However, in rows eight and nine of the In-the-Wild dataset, the SLS module showed some blurring compared to its performance in the first two datasets, with less distinct attention to the differences between genuine and fake audio. By analyzing the weight distribution diagram of the SLS module in conjunction with the feature difference diagram of different audio samples from Figure 1 , it can be observed that the primary differences between the gap diagram of real audio samples and the gap diagram between real and deepfake audio samples occur in layers 19, 20, and 23. This explains the phenomenon of weight distribution decreases for layers 19, 20, and 23 shown in Figure 6 . However, this is just one possible preliminary explanation, and future work could involve more statistical efforts in the area of interpretability. 
conclusion: 
 In this study, we validated the hypothesis that hidden layers of pretrained models harbor more abundant features for detecting audio deepfake. We introduced a classifier featuring a sensitive layer selection module, achieving state-of-the-art (SOTA) performance on two challenging datasets. One dataset, DF, features spoofed utterances generated by over 100 different attack algorithms, while In-The-Wild showcases real-world data distribution, thereby proving the approach's correctness and practical feasibility. Furthermore, our findings confirm that data augmentation is essential, even with the use of richer hidden layer features, and that fine-tuning enhances performance. While our model maintains state-of-the-art (SOTA) performance, it also boasts faster convergence and fewer parameters, prompting a reevaluation of future model design.Future work could focus on designing strategies that more effectively utilize hidden layer features, as outlined in this paper, along with developing more precise and complex classification models. Our classifier structure is very simple, indicating significant potential for improvement. Utilizing more effective deep learning modules may offer further enhancements to model performance. The currently used pre-trained models are based on bonafide audio. A feasible solution might be to retrain these models with deepfake audio from more varied scenarios. 
 
 Appendix:  


Decision: The authors propose a deepfake speech detection method based on the combination of XLS-R and SLS techniques. The method is trained on the ASVspoof19 dataset and evaluated on the ASVspoof21 dataset and In-the-wild dataset. The paper reports good performance on both datasets.

Although the paper receives 'borderline accept' as the final decision from the reviewers, the rebuttal introduces new experiments and results. To me, for equitable evaluation against other conference submissions, I lean towards primarily considering the paper's existing results and experiments. The reviewer 'bq28' requests the inclusion of new content to the manuscript presented in the rebuttal. One reviewer ('NDq5' ) thinks that without a subjective experiment, the claims in the paper cannot be substantiated. However, in the rebuttal, it is stated that the subjective experiment was not conducted. It is recommended that the authors adequately address reviewer concerns in the camera ready version.

The reviewers highlight the following strengths and limitations:

Strengths:
1. The proposed method reports better performance on the popular ASVspoof21 and In-the-wild datasets

Limitations:
1. The subjective experiment was not conducted.
2. The suitability of Unimedia/unimodal in nature but of sufficient interest to the MM community
3. The innovation contributed by this manuscript is limited when compared to [21]
4. The grammar and typos.

Reviews: review: paper_summary: paper_summary: Thank you to all the authors for their efforts and for the opportunity to review the manuscript. In this paper, the authors propose a method for deepfake audio detection using pre-trained XLS-R transformer layer outputs and a sensitive layer selection module. Experiments show an improvement in deepfake audio detection performance on the ASVspoof 2021 DF Dataset and significant improvement on In-the-Wild Dataset. The code will be publicly released after paper’s acceptance.
strength_weakness: strength_weakness: 1. Besides several typos, the introduction is well-written. The motivation and contributions of the paper are clearly highlighted, and help to get an overview of the entire paper.
2. The method’s novelty lies in the implementation of the combination of XLS-R transformer layer outputs and Layer Select Network.
3. It is good to see that XLS-R model provides good performance in feature selection as described in the paper. However, is there any reasoning behind choosing the cross-lingual XLS-R model and not any other pre-trained transformer model for speech recognition? Is there an ablation which supports this reasoning? If the answer is already present in the manuscript, please highlight this point carefully.
4. It is interesting to note the reasons behind not using a validation set to prevent overfitting because in this specific case, the validation set cannot fully prevent overfitting on the type of deepfake speech generation methods.
5. Exhaustive experimental analysis has been provided to prove the effectiveness of the proposed approach.
limitations: limitations: 1. Line 94 - “Using features from pre-trained models has achieved high detection accuracy”. Is there a basis for this statement in the introduction? Perhaps, a citation or two can be added here to strengthen the argument. This is because any result comparing pre-trained models and hand-crafted features has not been provided in the paper or supplementary material. So, a citation will be helpful.
2. Figure 1, even though very illustrative, is not complete in analysis. This is because feature maps of only a single genuine audio and only a single deepfake audio are shown. What about the “Difference” between two genuine audios or two deepfake audios? If the difference is significant and perceptually identifiable in those cases too, then it invalidates the visual hypothesis related to feature maps. Please clarify.
3. Novelty: How is the proposed approach significantly different than a prior work “Martin-Donas[26]” in line 170, which also used transformer layer outputs and a downstream model, as described in the paper. Is the difference between the two approaches just the use of XLS-128 in one and XLS-R in the other? An elaboration about this in the relative work section would be helpful for the readers. Also, line 171 is confusing because it leaves the impression that XLS-128 is an entirely different model than XLS-R. Later in the paper, in line 332, it is mentioned that XLS-R is trained across 128 languages, which implies that both terms indeed refer to the same model. Similarly in Table 2, what is the difference between XLSR and XLS-R? So: 1) What separates the proposed method from prior work? 2) Is there a reason behind using different terms for “probably” the same model?
4. What features are “z” in Figure 2?
5. Table 2 and Table 3 look lighter w.r.t the number of comparison methods. For example, in Table 2, what is the performance of non XLS-R (e.g. Wav2vec) based models or spec based models? Since the methods in Table 2 and Table 3 do not overlap, it is difficult to conclude the correspondence in performance on both datasets.

Besides, there are some minor grammatical errors and typos which could be removed by proof-reading:

1. In Line 20, there should be “a” instead of “an” because SLS does not start with a vowel.
2. In Line 45, it should be “produced” instead of “produce”.
3. In Line 46, there is missing space between “text-to-speech” and “(TTS)”, similarly in “voice conversion” and “(VC)”. This has happened at several spaces in the paper. It needs to be made consistent.
4. Line 65-66, “There also …” has a missing “are”.
5. Line 68, “systems,telephone banking” has a missing space before ‘,’.
6. Line 82 - It is always proper to list full forms of abbreviations, the first time abbreviations are used in text, for example, LFCC, MFCC, EER and CQCC.
7. Figure 1 caption - Not sure what “dicramenate” is. The caption needs to be fixed grammatically.
8. Lines 196 and 197 -  XLSR has been listed twice within the same sentence, and is written differently than “XLS-R” in Line 151.
9. Line 267 -  End of sentence has a missing full stop.
10. Line 276 - it should be “{0,1}” as per set notation.

I am sure there are more which I could not list here. A proof-read and grammar check is strongly recommended. With the high amount of grammar errors and typos in the paper, some sections can become very hard to read.

Overall, the paper presents an interesting approach for deepfake audio detection. Some merits of the method include not requiring fine-tuning or augmentation, and direct extraction of distinguishing features from pre-trained transformer layers. There is a lot of experimental analysis provided in the paper, however comparison methods are not consistent. Moreover, there are numerous typos and grammar errors throughout the paper indicating a lack of proof-reading. More work is needed in the writing of the manuscript.
paper_summary: paper_summary: The authors proposed a deepfake speech detection method based on the combination of XLS-R and SLS techniques.

The proposed method was trained on ASVspoof19 dataset and evaluated on ASVspoof21 dataset and In-the-wild dataset.

The proposed method achieved reportedly the best performance on both datasets.
strength_weakness: strength_weakness: - The proposed method achieved reportedly the best performance on the popular ASVspoof21 and In-the-wild datasets
- The EER of the proposed method is significantly better than other methods on In-the-wild dataset
limitations: limitations: - The paper has a lot of formatting problems.
  - Please make sure there is a space between parenthesis and English text
  - Please make sure there is a space behind periods
  - Please make sure there is a space between citation bracket and English text

- In the abstract the authors refers to XLS-R without any further description, which is very confusing
- What is the difference between Layer Select Network (LSN) and Sensitive Layer Selection (SLS)?
- The description of XLS-R model is too vague. It is an important part of the proposed method, which requires more details.
- What are the pretrain data and method for XLS-R? Is there any potential overlap problem between the pretrain data of XLS-R and the test data used in this paper?
- What is the metric used in Table 4?
- Please briefly describe rawboost
- I am not convinced with claim the "human eye can directly identify the authenticity by observing the SLS feature maps" unless subjective tests are conducted 
- The proposed method achieved significantly better performance compared to all other methods designed for in-the-wild dataset. The authors' explanation for this phenomenon is not very strong.
  - It seems like the authors proposed the SLS technique that boosted the performance significantly. The design of SLS seems to be applicable to other neural network architecture. Have the authors implemented the "rich feature" strategy for other neural network architectures? Does this strategy ubiquitously improve the performance of all neural network architectures? If so, the SLS technique itself seems to have more profound influence. If not, why does SLS work better with XLS-R?
paper_summary: paper_summary: This paper makes use of the hidden layer features from XLS-R for audio deepfake detection. Experiments show that the proposed deepfake detection model achieves SOTA performance and the authors validate the effectiveness of the designed SLS module.
strength_weakness: strength_weakness: The paper is well-organized and the analysis of the method is detailed. The authors conduct numerous comparative experiments which make the proposed model to be persuasive.
limitations: limitations: -- about novelty
The main contribution of this paper is to use the hidden layer features of XLS-R for audio deepfake detection, in order to capture some useful information for detection. However, deepfake detection is essentially a binary classification task, and utilizing mid layer features for classification tasks is not a novel solution. What's more, the designed SLS module achieves the fusion of features taken from each layer, and such adaptive weight allocation is common in feature fusion. 
I believe that the proposed model is going to achieve satisfactory results in ADD or ASVspoof challenges, but it lacks striking novelty as frontier research. 

-- about experiment
How to prove that each hidden layer feature is effective and indispensable? The shallow hidden layer features which are fed to SLS for the fusion process have not been proven to be valuable in the experiment. As shown in Fig. 4, the weights of these layers are quite small compared to the deeper layers, and the connections between the Transformer layers result in a lot of redundancy in the underlying information contained in these mid layer features. For example, will excluding the hidden layer features of the first five Transformer layers cause a significant decrease in the model's deepfake detection performance? This is related to the core idea of the paper, but the paper failed to validate that.

-- others
In Fig. 3, the operations between data are missing and this figure seems to be unnecessary, as Fig. 2 also contains the detailed structure of the SLS module and I think Fig. 3 can be merged with Fig. 2.
The ablation study shows that fine-tuning and data augmentation significantly improve the model's performance. Has the fine-tuning and DA conducted for the baseline methods that utilize XLSR?
paper_summary: paper_summary: The paper proposes a new deepfake audio detection model that leverages a pre-trained model called XLS-R and incorporates a Sensitive Layer Selection (SLS) module to enhance the detection of fake audio. The experiment shows that this method performs well on in-the-wild datasets.
strength_weakness: strength_weakness: 1: This manuscript presents intriguing visual analyses, exemplified by Figures 1 and 4. The author's utilization of sigmod and softmax for result analysis is thought-provoking.

2: The proposed method demonstrates competitive performance in the wild dataset.

3: The proposed method exhibits simplicity and rapid convergence.
limitations: limitations: From a technical standpoint, the innovation contributed by this manuscript is limited when compared to [21].