I Can Hear You: Selective Robust Training for Deepfake Audio Detection
Authors: Zirui Zhang
Wei Hao
Aroon Sankoh
William Lin
Emanuel Mendiola-Ortiz
Junfeng Yang
Chengzhi Mao
Venue: ICLR 2025 Poster (2025)
Keywords: Deepfake audio detection
Audio augmentations
Frequency-Selective Adversarial Training
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Recent advances in AI-generated voices have intensified the challenge of detecting deepfake audio, posing risks for scams and the spread of disinformation. To tackle this issue, we establish the largest public voice dataset to date, named DeepFakeVox-HQ, comprising 1.3 million samples, including 270,000 high-quality deepfake samples from 14 diverse sources. Despite previously reported high accuracy, existing deepfake voice detectors struggle with our diversely collected dataset, and their detection success rates drop even further under realistic corruptions and adversarial attacks. We conduct a holistic investigation into factors that enhance model robustness and show that incorporating a diversified set of voice augmentations is beneficial. Moreover, we find that the best detection models often rely on high-frequency features, which are imperceptible to humans and can be easily manipulated by an attacker. To address this, we propose the F-SAT: Frequency-Selective Adversarial Training method focusing on high-frequency components. Empirical results demonstrate that using our training dataset boosts baseline model performance (without robust training) by 33%, and our robust training further improves accuracy by 7.7% on clean samples and by 29.3% on corrupted and attacked samples, over the state-of-the-art RawNet3 model.

de Hoyos| (2024). However, the suc- cess of Al-synthesized human voices Published as a conference paper at ICLR 2025
000
I CAN HEAR YOU: SELECTIVE ROBUST TRAINING
001
002
FOR DEEPFAKE AUDIO DETECTION
003
004
Zirui Zhang', Wei Hao', Aroon Sankoh', William Lin', Emanuel Mendiola Ortiz',
005
Junfeng Yang', Chengzhi Mao"
006
007
Columbia University, WWasington University in St. Louis, 3 New York University, "Pennsylvania State University, 'Rutgers University
008
009
010
{zz3093, wh2473) @ @columbia.edu edu, a.j.sankoh @ wusttleedu w12612 @ @nyu.edu, emo5318@ psu.edu, junfeng @ cs. cs.columbia. edu., cm1838@ rutgers.edu
011
012
013
014
ABSTRACT
015
016
Recent advances in Al-generated voices have intensified the challenge of detecting deepfake audio, posing further risks for the spread of scams and disinformation. To tackle this issue, we establish a large scale public voice dataset DeepFake Vox- HQ. comprising 1.3 million samples, including 270,000 high-quality deepfake samples from 14 diverse sources. Despite previously reported high accuracy, ex- isting deepfake voice detectors struggle with our diversely collected dataset, and their detection success rates drop even further under realistic corruptions and ad- versarial attacks. We conduct a holistic investigation into factors that enhance model robustness and show that incorporating diversified set of voice augmen- tations is beneficial. Moreover, we find that the best detection models often rely on high-frequency features, which are imperceptible humans and can be easily manipulated by an attacker. To address this, we propose the F-SAT: Frequency- Selective Adversarial Training method focusing on high-frequency components. Empirical results demonstrate that our training dataset boosts baseline model per- formance (without robust training) by 33%, and our robust training further im- proves accuracy by 7.7% on clean samples and by 29.3% 3% corrupted and at- tacked samples, over the state-of-the-arr RawNet3 model.
017
018
019
020
021
022
023
024
025
026
027
028
029
030
031
032
033
034
INTRODUCTION
035
036
Al-generated voices have become
Number of Samples
80
Thketing Depphen (Thideg
increasingly realistic due to larger datasets and enhanced model pacities (Ju et al. 2024; [Nckhhar al. 2024). and they have been used in many important applica- tions (CalahorraCandao & Martín-
037
60
Cartge Deplie Tlstad
Decision Bourlay
038
039
40
040
20
041
042
%
0.02 0.10 0.21 0.35 0.50 0.65 0.79 0.90 0.98 1.00 Prediction Score
043
Figure The distribution of deepfake samples over pre- dicted scores using the state-of-the-art detector (Jung et al. 2022) trained on the In-the-Wiie dataset (Müller al. 2022). with decision boundary at 0.5. Tests on original, corrupted, attacked, and real-world deepfake audio reveal significant shifts in prediction scores, highlighting that train- ing solely on current public datasets without robust training methods leads to poor performance.
poses significant security risks, in- cluding deepfake voice fraud and scams (Tak et al.) 2021, Sun et al.] Yang et al. 2024 A recent CNN report reveals a fraud in Hong Kong where a finance worker sent 049
044
045
046
047
2023
048
$25 million to scammers after a video
050
call with a deepfake 'chief financial officer'. The voice was created by Al model, highlighting the risk of such technology.
051
052
Due to the importance of this problem, a number of work has investigated detecting Al-generated audio. Despite previously reported high detection accuracy on public datasets (Todisco al. 2019
053

---

Published as a conference paper at ICLR 2025
1.0
Fitzting Irdtunn 400
054
10344
055
0.8
4800
056
Accuracy
057
0.6
058
0.4
Hlucesy Saitaro Isdow 2000
059
1030
060
Real Accuracy
061
Fake Accuracy
062
1000 2000 3000 4000 5000 6000 7000 8000 Center Frequency (Hz)
063
064
Figure 2: We apply a high-pass filter to audio samples to remove low-frequency components. The x-axis represents the center frequency of the filter applied. Notably, there is a marked decline for
065
066
RawNet3 Model in detection performance for real audio starting at 4000 Hz and for fake audio at
067
6000 Hz, suggesting that existing audio models often rely on high frequency signals for prediction.
068
069
070
Frank& & Schönherr 2021), ) existing deepfake voice detectors perform poorly under real-world con- ditions (Xu et al.) 2020; Müller et al. 2022 Radford et al. 2023). This is because the established benchmarks are often trivial, small, outdated, and homogeneous. Consequently, models trained and validated solely on these datasets fail to generalize to more diverse and challenging real-world deep- fake samples. Moreover, deep learning models for audio particularly vulnerable to adversarial attacks (Szegedy et al.] 2013) (Zhang et al. 2019), where an attacker can subtly alter audio inputs in ways that are imperceptible to humans but mislead models into incorrect classifications. Fig- ure [ illustrates a dramatic shift in the models' prediction scores when exposed to these factors, underscoring the need for more robust training methodologies.
071
072
073
074
075
076
077
078
To address the above limitations, we first created the largest deepfake audio dataset to date, DeepFakeVox-HIQ. including 270,000 high-quality deepfake samples from 14 diverse and distinct origins. We show that simply training on our collected dataset can produce new statc-of-the-ar models.
079
080
081
082
Moreover, we find that even the state-of-the-art AI-voice detection model (RawNet3) often depend on high-frequency features to make decisions (see Figure 2). which are imperceptible to humans. On the other hand, the low frequency signals can be heard by humans but are not relied on by the model make predictions. As result, natural corruptions in high frequency or attackers can easily manipulate the model by changing the high frequency signals, reducing the detection's 's robustness.
083
084
085
086
087
088 In an initial study, we observed that standard adversarial training on raw waveforms not only fails to bolster robustness but also diminishes performance on unattacked data. To address these short- Srrquency-Sclectiie
089
090
comings, we propose frequency components. Since our adversarial training is targeted, we can mitigate specific vulnera- bilities without touching the true features at lower frequencies, thus enhancing the model's s resilience to corruptions and attacks while maintaining high accuracy on clean data.
Adversarial Training (F-SAT), which focuses on high-
091
092
093
094
Visualizations and empirical experiments demonstrate that using only our training dataset, we can produce othee-of-the-art models, achieving 33% improvement on the out-o--distribution portion of our test set, which includes 1,000 deepfake samples from the top five AI voice synthesis companies and 600 samples from social media. Additionally, by incorporating random audio augmentations, our model achieves the highest accuracy across 24 different types of corruptions. Furthermore, after applying F-SAT, our model further achieves 30.4% improvement against adversarial attacks in the frequency domain and an 18.3% improvement against unseen attacks targeting raw waveform data
095
096
097
098
099
100
in the time domain.
101
102
103
2 RELATED WORK
104
105
Al-synthesized human voice: Al voice synthesis generally falls into two categories: text-to-speede
106
(TTS) and voice conversion (VC). TTS systems convert written text into spoken audio using de- sired voice. This process typically involves three main components: a text analysis module that transforms text into linguistic features, an acoustic model that converts these features into mel-
107
2

---

Published as a conference paper at ICLR 2025
108
Dataset
#Real Utt
#Fake Utt
Language
Conditions
Year
#Fake Source
Fake Type
109
ASVSpoof19
12k
109k
English
Clean
2019
TTS.
110
ASVSpoof21
22k
589k
English
Clean, Noisy
2021
UNK
TIS, VC
111
WavFake
14k
90k
English. Japanese
Clean
2021
TIS
ADD2022
61k
251k
Chinese
Clean
2022
UNK
TTS, VC
112
In-The Wild
20k
12k
English
Clean, Noisy
2022
19
TIS
113
Vib
13k
79k
English
Clean
2023
6
TTS
114
Our Train
690k
640k
English
Clean, Noisy
2024
40
TTS. VC
Our Test
3k
3k
English
Clean, Noisy
2024
15
TTS, VC
115
116
Table 1: Comparison of Deepfake Audio Datasets
117
118
119
spectrogram, and vocoder. Some of the leading TTS models include StyleTTS (Li et al. 2024).
120
VoiceCraft (Peng et al., 2024), XTTS (Casanova et al.) 2024), and Tortoise-TTS (Betker (023)) ) and are known for their ability to produce high-quality audio that closely mimics real human speech.
121
122
VC models, in contrast, take an audio sample from one person and transform it to sound like an- other person for the same speech content. Recent VC approaches primarily operate within the mel- spectrum domain (Ju et al.) 2024 [Shen al.) 2023: Popov et al. (02I). using deep neural networks to shift the mel-spectrograms from the source to the target voice.
123
124
125
126
Detection Method: AI deepfake detection models based deep learning be grouped into two main categories: those processing raw audio end-to-end and those analyzing spectrum. The first category includes models like RawNet2, which employs Sinc-Layers (Ravanelli & Bengio 2018) to extract features directly from waveforms, and RawGAT-ST, which utilizes spectral and temporal sub- graphs (Tak et a 2021). RawNet3 (Jung et al. 2022) which begins by using parameterized fil- terbanks Zeghidour et al. (2018) to extract a time-frequenc" representation from the raw waveform and then is followed by three backbone blocks with residual connections, structural approach that sets it apart from SCAPA-TDNI (Desplanques et al.) 2020). These models process the audio data in its raw form to capture nuanced details directly impacting model performance.
127
128
129
130
131
132
133
134
135
The second category of AI deepfake detection models involves transforming raw audio into spectro- grams for analysis. This process utilizes extracted features such as Mel Frequency Cepstral Coef- ficients (MFCCs) (Sahidullah & Sahal (2012), Constant Q Cepstral Coefficients (CQCCs) (Todisco) al. 2017). bit-rate (Borzi et al. (022). The analysis of these features is conducted using tradi- tional machine learning methods like Gaussian Mixture Models (GMMs) (Todisco et al. (019) or advanced neural networks such as Bidirectional Long Short-Term Memory (Bi-LSTM) (Akdeniz &
136
137
138
139
140
141
Becerikli 2021), ResNet (Alzantot et al., (019)) and Transformers (Zhang et al. 1021 These meth-
142
ods enable deeper and more intricate pattern recognition, enhancing the model 's ability to identify and classify deepfake audio accurately.
143
144 Ad Adversarial Attack: Neural networks are highly vulnerable to nearly imperceptible perturbations, known as adversarial attacks (Szegedy al. 2013). Although initial studies focused image mod- els, speech tasks are similarly susceptible. Notably, adversarial attacks generated on spectrograms deceive 2D CNN models and, when converted back waveforms, can also effectively fool 1D CNN models (Koerich et al. 2020).
145
146
147
148
149
Adversarial training, initially developed for image processing systems (Madry al. 2017 Mao ct al. 2019), has been increasingly adapted to the audio domain (Chen et al.) (023), particularly to enhance the robustness of applications such as Automatic Speech Recognition (ASR) and speaker verification systems. Another defense method, smoothing, based on additive noise masking, has demonstrated great improvements in model robustness for these tasks (Olivier al.) 2021 Cohen et al., 2019). Additionally, a defensive strategy that employs diffusion models to counter adversarial audio attacks (Wu al., (023) effectively smooths out perturbations and prevents attackers from altering audio signals without significantly compromising signal quality.
150
151
152
153
154
155
156
157
While these techniques enhance robustness, they hurt the detection on unattacked audio, highlight- ing trade-off between robustness and accuracy (Zhang et al. 2019 Tsipras et al.) 2018). This compromise is particularly critical in scenarios that demand high accuracy and user satisfaction. Furthermore, the generalizability of adversarially trained models new and unseen attacks remains limited (Rajaratnam et al. 2018), raising questions about their effectiveness in rapidly evolving threat environments.
158
159
160
161
3

---

Published as a conference paper at ICLR 2025
162
3.
DEEPFAKEVOX-HQ
163
164
165
3.1
TRAINING DATASET
166
167
In this section, we introduce new training dataset and a rigorous test set. In contrast to prior dataset, our dataset large, diversified, realistic, and up-to-date, as shown in Table [ Prior detectors show poor generalization capabilities in realistic settings, as shown in Figure 8 Both our training and test- ing datasets integrate the latest advancements in Al voice synthesis technologies. Additionally, the testing dataset includes several new models not covered in the training dataset, specifically designed to test the generalization ability of our detection systems.
168
169
170
171
172
173
High quality deepfake samples: The limitations of existing public datasets, which often lack high- quality deepfake samples, can potentially impair model performance. To address this, we have in- vestigated more than 30 recent advancements in Text-to-Speect (TTS) and Voice Conversion (VC) models developed in the past few years. Our training dataset now includes deepfake audio sam- ples generated using the top seven TTS models: MetaVoice-IBB IB (Liu et al. (201]2022.) StyleTTS-v2 (Li et al. 2024), VoiceCraft (Peng et al. 2024), WhisperSpeech (Radford et al. 2023). Vokan TTS, XTTS- v2 (Caaanvaa et al. 2024). and Elevenlabs. We use four datasets VCTK (Yaagiishl) 2012),
174
175
176
177
178
179 LibriS periSpeechhPo (Panayotov et al. 2015), In-The-Wiile (Müller et al. (2022). and AudioSet (Gemmeke
180
al. 2017 generate deepfake audio. These datasets include both clean, high-quality and noisy, low ww-quality real audio, ensuring that the deepfake audio produced is highly diverse and accurately
181
182
reflects real-world conditions.
183
For both fake and real audio, having only high-quality samples is insufficient. A broader diversity of samples is essential for the training dataset. Thus, for real audio, we utilize portions from six public audio datasets: VCTK, LibriSpeech, AudioSet, ASRspoof2019 (Toiicco ct al., 2019), Voxcelebl (Nagrani et al., 2017), and ASRspoof2021 (Liu et al., (023), with half consisting of clean audio and the other half of noisy audio. For fake audio, include two low-
Reference data:
184
185
186
187
188
quality fake audio datasets: WaveFake (Frank & Schönherr (021) and ASRspoof2019 to further enhance the diversity of the training material.
189
190
191
192
3.2
TESTING DATASET
193
194
Our test dataset comprises approximately 6,000 samples, with equal balance between real and fake audio. All data used is legally permissible, as detailed in Section A.I where we discuss the sources and usage policies in further detail.
195
196
197
Real audio samples are sourced from recent celebrity speeches and conversational videos. For the fake audio, we not only utilize samples created using the seven latest TTS models but have also expanded our dataset to include contributions from eight of the most advanced AI voice synthesis models or commercial software currently available, namely Voice (Du et al. (024)) Resemble, Speechify, LOVO AI, Artlist, and Lipsynthesis. Additionally, this set includes fake audio directly collected from social media platforms like YouTube and X, further enriching the dataset with diverse range of real-world scenarios. This comprehensive composition is strategically designed to rigorously test the generalization capabilities of our model.
198
199
200
201
202
203
204
205
Figure 8 shows that training with previous public datasets yields lower accuracy on ours. Additionally, removing high-quality deepfake samples from our training set significantly also re- duces accuracy, highlighting their importance."
Insight:
206
207
208
209
210
METHOD
211
212
213
In this section, we present our selective adversarial training approach, F-SAT. We also present a taxonomy of the most common corruptions and attacks in audio processing, which can be used for robust evaluation in realistic settings. Additionally, we discuss the implementation of Rand- Augmentation for audio further enhance the robustness of our detection system.
214
215
4

---

Published as a conference paper at ICLR 2025
Step1: Decompose
Step2: Iteratively find worst-case perturbation
Step3: Train Model
uedate
216
Lass and gradient
Cross Entropy Loss (H(P, y))
217
Selective Perturbation
Input sudio (x)
218
add to
Detection model
219
Prediction e
Label (y]
220
STFT
Magnitude (Xp)
221
Detection model
222
IST
223
Perturbed audio (x)
224
Phase (Kg)
225
Figure 3: F-SAT Pipeline
226
227
4.1
F-SAT: FREQUENCY- CREQUEENYY-SHLECTVVE SELECTIVE ADVERSARIAL TRAINING
228
229
Let x be a waveform input audio, and y be its ground-truth category label, To perform classification, neural networks commonly learn predict the category Fx by optimizing the cross-entropy Hly.y' y) between the predictions and the ground truth. We use RawNet3 (Jung et al. 2022) that can process waveform audio input. The network parameters are estimated by minimizing the expected
230
231
232
233
value the objective:
yx(x, H (x))))
234
235
Time domain Attack: Adversarial attacks on audio can be directly added to the waveform. Let the additive perturbations be ō. For attacks in the time domain, we directly add the waveform x. Due to the huge amount of freedom of the attack vector 5, the added attack vectors are often high frequency. We formulate this attack as: x' = + 6.
236
237
238
239
Frequency Domain Attack: Attacks can also be applied in the frequency domain, accessed through a reversible Fourier transformation of the waveform. We transform the waveform x to the frequency domain X using the Short-Time Fourier Transform (STFT). The attack modifies X by adding 8, yielding X'= + ô. The Inverse Short-Time Fourier Transform (ISTFT) then reverts X' back to the time domain, creating the manipulated audio waveform x'.
240
241
242
243
244
Frequency-Selese SSlecttve Attack: Compared to time-domain attacks, frequency-domain attacks provide an inductive bias that enables the crafting of more controlled perturbations, such as those within a restricted bandwidth. Building on this, we introduce a Frequency- Attack that targets specific frequency ranges within the magnitude component of waveform. We define this attack
245
246
247
248
249
x' = s(x, 6, Si,
250
where and fa represent the lower and upper frequency boundaries of the range where the attack is to be applied. The procedure can be described as follows:
251
252
Starting with STFT result X, we first compute its magnitude component X, and phase component X To map the frequency range to the index range of the STFT spectrum, we calculate the lower and upper boundary indices ri and ra of the FFT spectrum using formulas:
253
254
255
256
ft x Am
fu 7.
Ta
257
Sr
258
where nm is the number of FFT points, and sr is the sampling rate of the original signal x. We then define mask function M using diagonal matrix D such that
259
260
261
DER {1
if k otherwise
262
263
This matrix D selectively targets the desired frequency components within the specified range (ri, ru] for perturbation. The selective perturbation then defined as:
264
265
266
5_ M(5, rt, 5.
267
By adding this perturbation ò, to the magnitude component of the spectrogram, the perturbed spec- trogram can be represented as:
268
269
X' (X, + 5₂) dX (X, +D e
5

---

Published as a conference paper at ICLR 2025
270
Finally, employ the ISTFT to convert the attacked spectrogram X' back into the tine-domann signal x', thus generating the attacked audio waveform x'.
271
272
Moreover, attackers design these worst-case perturbations to disrupt trained network Fa. aiming to maximize misclassifications by optimizing the objective:
273
274
275
x arg max Lo(x',)) y), s.t.
||x' - x|1g <
276
where the perturbation x' x is constrained by the norm to be less than c, ensuring minimal deviation.
277
278
279
F-SAT: Based on the Brcquency-Selection Attack, we propose F-SAT, an adversarial training method which optimizes perturbations in the frequency domain by targeting the magnitude component within specific frequency ranges.
280
281
282
x = 11x+s d a M (sw(VV/CCltss.
283
284
285
where xxx+5 represents the projection onto the allowable perturbation set S, defined by the condition
286 ||x' - xl|p S e. The parameter a denotes the step size of the update, and Vx is the gradient of the loss function with respect the perturbed input x'.
287
288
after K iterations (where n 2 K), we feed the most perturbed sample back into the detection model and calculate the cross-entropy loss between it and the ground truth
As illustrated in Figure 3
289
290
291 label:
Crobust (x', .y) (Fo
292
Additionally, to maintain a balance between accuracy original and attacked samples, we train the model using both the original and perturbed samples. We introduce a parameter y to control the trade-off between clean loss and robust loss. The clean loss is defined as:
293
294
295
Celean (x. y) (Fx())).
296
297
The total loss Frotal is then computed as a weighted sum of clean and robust losses.
298
Liotal = Letean + T Lrobust
299
Since time domain attacks are often high frequency, by focusing the adversarial training on the high frequency, our approach not only enhances model robustness against Trequency-tareetee attacks, but also improves defenses against time-domain attacks.
300
301
302
303
4.2
RANDAUGMENT FOR AUDIO
304
305
Realistic Corruptions for Deepfake Audio Generation: shown Figure 9 we present a tax- onomy for analyzing the robustness of deepfake audio detection systems that incorporate the most common corruptions and attacks. Corruptions in audio result from unintentional modifications dur- ing recording, processing, transmission, impacting noise levels and frequency responses. Ad- versarial attacks, in contrast, are intentional manipulations aimed at deceiving detection systems through subtle changes across various attack methodologies.
306
307
308
309
310
311
Experiments show that the best detection model, RawNet3, experiences a great drop in accuracy when faced with audio corruptions. Inspired by the image-based RandAug- ment (Cubuk et al. 2020), which improves model robustness, we adapted this method for audio. We selectively apply N transformations from the available options, each assigned uniform prob- ability. An additional probability p determines whether each transformation is applied at given instance, to balance the accuracy between original and corrupted samples. The magnitude of each transformation is controlled within predefined boundaries, with the intensity randomly sampled from this range.
RandAugment:
312
313
314
315
316
317
318
319
320
EXPERIMENTS
321
322
We first compare approach to existing state-of-the-art methods across three benchmarks and demonstrate improved accuracy. We then assess its robustness against corruption and adversarial attacks. We finally conduct ablation study enhancements the detection system's robustness.
323

---

Published as a conference paper at ICLR 2025
1.0
RawNet2
Accuracy
0.8
RawNet3
324
0.6
Ours
325
0.4
326
0.2
327
0.0
328
Roal Audio
PlayHT2.
Resentle
LOVo
10 Sounnddbbarrdd
Youtube
CosyValee
Lipsy
Eleveriable
ShyeTTB v2
XTTS stl
Malavoioe
Whlsper
Vola Crff 2.0
Vokan 719
329
330
Fake sources covered training
Fake sources covered training
331
Figure 4: Detailed results across different AI-voice synthesis models our test set. Fake audio is categorized into sources included and not included in training. Models evaluated include RawNet2, RawNet3, and Ours. Our method outperforms all others across all different sources.
332
333
334
335
5.1
DATASET
336
DeepFake Vox-HQ VO (test) Our custom test dataset incorporates 14 of the latest and highest quality TTS and VC models to generate fake audio. It also includes fake audio samples directly collected from social media platforms such as YouTube and X, providing diverse set of real-world scenarios. For the experiments described in this paper, we specifically utilize seven fake sources not present in the training set. However, we also include samples from seven other fake sources used during training to facilitate future research.
337
338
339
340
341
342
343
ASV spoot2019 (Todisco et al. 2019) derived from the VCTK base corpus, which includes speech data captured from 107 speakers. It contains three major forms of spoofing attacks, namely synthetic, converted, and replayed speech.
344
345
346
347
5.2
BASELINE
348
RawNet2 (Tak al. 2021 employs Sinc-Layers (Ravanelli & Bengio 2018) to directly extract features from audio waveforms. These layers function as band-pass filters that enhance the detection of spoofed audio content.
349
350
351
352
RawGAT-ST Tak et al. 2021) utilizes spectral and temporal sub-graphs integrated with a graph pooling strategy, effectively processing complex auditory environments.
353
354
Te-ResNet (Zhang et al. 2021) processes synthetic speech detection by first extracting MFCC from input speech. These coefficients are used as features for a CNN that extracts spatial features, fol- lowed by a Transformer that analyzes these to detect characteristics of synthetic speech effectively.
355
356
357
RawNet3 (Jung et 2022) begins by using parameterized filterbanks extract a time-frequency representation from the raw waveform. This is followed by three backbone blocks with residual connections, a structural approach that deviates from ECAPA-TDNN (Desplanques et al. 2020).
358
359
360
361
5.3
MAIN RESULT
362
363
Results on Original Data: We trained all models on DeepFake and tested them across two
364
benchmarks: our test set and ASospoff2019, as shown in Table 2 Our method achieved state-of- the-art results across both benchmarks. Compared with RawNet3, our method shows improvements of 7.8% F1 score on DeepFakeVox
365
366
(test), 1.1% 11% FI score on ASVspoof2019.
367
Among all baseline models, Rawnet3 performs the best. Therefore, for subsequent evaluations involving corruption and attacks, we will apply robust training methods to Rawnet3 and conduct comparisons.
368
369
370
Figure 4 outlines results across various Al-voice synthesis models on our test set. For the fake sources are included in the training, all models demonstrated high accuracy However, for un- foreseen source, our method significantly outperformed others, particularly those from real-world social media platforms like YouTube and Lipsyne, by up to 50% points, highlighting its superior generalization capabilities on aat-of-distribution data.
371
372
373
374
375
376
Result on Corrupted data: are depicted in the green region with F-SAT, and in the red region without F-SAT. They outperform all other baseline methods across 24 types of corruptions, with average absolute increase of 15.3%
Figure 5a shows detailed results for various corruptions. Our methods
377
7

---

Published as a conference paper at ICLR 2025
Approach
DeepFake Vox-HQ
ASVspoof2019
378
379
Real Acc
Fake Acc
F1
Real Acc
Fake Acc
F1
380
TE-ResNet
0.859
0,433
0.629
0.894
0,973
0.933
381
RawGAT-ST
0.926
0.698
0.810
0.943
0,951
0.947
382
RawNet2
0.946
0.578
0.754
0.930
0.985
0,957
383
RawNet3
0.961
0.759
0.861
0.970
0,908
0.939
384
Ours
0.975
0.964
0.974
0.950
0.982
0.966
385
386
Table 2: Comparative performance of our method and baseline models on DeepFakeVox-HQ and
387
ASVspoof2019 test sets, our method outperforms prior approaches. (We ensured our deepfake test set does not overlap with the training set.)
388
389
390
orgin
391
streeph
Trackgrund
FreceS- 5k) sttack (lokg)
392
tonh disto
COlL nolse
393
seven band
noile
(reat)
Time sttack 1400
394
simulard
ehidyt nolee
395
pobertt
for
396
Frespt-Ikc ottcck (fukch
397
postingfine
bandsoss
398
lowshetf
bondssop fiber
399
Lovposo Mlar
trae(t-ak omoos (reali
Niwegn (-t0))))) brtack (poo)
400
tumber
401
sighther
high poso filer
8ih
Giffcing dtstortion
Freq2-aicy (fanc)
Fregto AN attock (fain))
402
tointranition
50)
Serto 2012 rescoiing
403
(a) Performance across various corruptions
(b) Performance across different types of attacks
404
405
Figure 5: Comparative results showcasing the accuracy of our method against various types of corruptions and attacks. Our method outperforms the baseline across nearly all tested conditions.
406
407
408
points.
However, as we can observe, aliasing has the most negative impact on accuracy. During aliasing. audio is intentionally downsampled and then resampled back, which not only removes figg-fryy features but also introduces distortions characteristic of aliasing. In addition, low- pass filters, time stretching, and noise also significantly impact performance more than others.
409
410
411
412
413
414
5.4
RESULTS ON ATTACKED DATA
415
416
To assess our model's robustness against adversarial attacks, we employ the Projected Gradient Descent (PGD) attack, well-established method in adversarial training. We compare F-SAT with conventional adversarial training and smoothing defense methods.
417
418
419
Attack Settings for Evaluation and Perceptibility Assessment: In the main paper, we evaluate the -PG attack on the time domain (e = x 10 a = 4 x 10-5, iter 2) and on the magnitude 10 a 4 10-4 iter 2). For the frequency
420
421
component of the frequency domain (e domain, we progressively expand the attacked frequency range to include 6-8 kHz, 4-8 kHz, 2-8 kHz, and 0-8 kHz. The average attack result is calculated across these ranges. More comprehensive attack analyses are provided in Appendix A2. To ensure the imperceptibiliity of the attacks in both domains, we evaluate the Signal-to-Noise Ratio (SNR) of the attacks compared to the original audio.
422
423
424
425
The SNR values are 58.4 dB for the time domain attack and 68.7 dB for the frequency domain attack, indicating that the attacks remain imperceptible.
426
427
428
Compared with other defense method: We compare our Method F-SAT with other defense meth-
429
ods MAD smoothing (Olivier et al. 2021). Gaussian smoothing (Cohen et al. 2019), and standard
430
adversarial training (Mkadry et al. 2017) the RawNet3 model against various adversarial at-
431
tacks in both time and frequency domains. As shown in Figure [b] F-SAT outperforms all other methods.
8

---

Published as a conference paper at ICLR 2025
1,0
Real Aeouaiyy Paka Audio Accuney
1.0
0.8
0.8
Audi Acnunny
432
0.6
0.6
433
0.4
0.4
434
0.2
0.2
435
436
0.0
No allack
Time Domain Specemagntude Specoplose AMtck Type
0.0
44-8k
Froquancy
437
(a) Impact of Different Attack Types
(b) Frequency Range for Attack (e = 0.01)
438
439
Figure 6: Exploring Model Vulnerabilities: (a) shows accuracy under under various attack types; (b) details how attacks targeting magnitude in various frequency domains affect accuracy. We show attacks based on magnitude in the frequency domain highly effective at degrading model perfor- mance, with those targeting higher frequency ranges having a greater impact.
440
441
442
443
5.5
ABLATION STUDY AND ANALYSIS
444
445
Choice of attack type for F-SAT: As discussed in Section 4.2, there are three types of adversarial attacks on audio: time domain, frequency domain on magnitude, and frequency domain on phase. All of these can be employed for adversarial training. However, our focus on targeting the magnitude component in attacks is explained in Figure 6a] Through our evaluation, we found that attacks based on magnitude in the frequency domain are particularly effective at degrading model performance. This is due to the fact that changes in magnitude directly affect the amplitude audio signals, crucial for maintaining
446
447
448
449
450
acoustic features. Such changes alter the sound intensity across frequencies, complicating the model's task of distinguishing key characteristics between real and fake audio. Conversely, phase attacks have minimal impact on the model's predictions, phase alterations primarily influence spatial audio perception and do not significantly affect feature detection.
451
452
453
454
455
We further validate our conclusions experimentally, shown in Table Our findings indicate that F-SAT excels in adversarial training against attacks on both time and frequency domains. Surpris- ingly, incorporating adversarial training focused on the time domain appears diminish overall accuracy on original data and does not effectively enhance robustness against attacks in either do- main. Direct attacks in the time domain may disrupt most features critical for differentiating between real and fake audio, potentially overburdening the model during training.
456
457
458
459
460
461
Additionally, the introduction of RandAug to the RawNet3 baseline model significantly enhances accuracy corrupted data, suggesting that the model benefits from exposure to varied conditions during training.
462
463
464
Approach
Origin
Corruption
Attack(Time)
Attack(Frequency)
465
Real Acc
Fake Ace
Fl
Real Ace
Fake Ace
F1
Real Ace
Fake Ace
Real Ace
Fake Acc
Fl
466
Rew3Net3
0.961
0,759
0.861
0.947
0.556
0.720
0.979
0.035
0.134
0.804
0.083
0.202
467
RandAug
0.976
0,947
0.968
0.899
0.826
0.860
0.747
0.661
0.693
0.630
0.573
0.613
468
RandAug++AT(Tino
0.947
0.825
0.891
0.881
0.772
0.818
0.871
0.124
0.211
0.665
0.152
0.207
RandAAgg+ F-SAT
0.975
0.964
0.974
0.948
0.830
0,885
0.902
0,880
0.910
0.933
0.915
0.924
469
470
Table 3: Ablation study evaluating the impact of our RandAug and F-SAT, comparing their effects against time-domain adversarial training (AT) and phase-based adversarial training. RandAug en- hances robustness to corruptions, while F-SAT improves robustness to adversarial attacks.
471
472
473
Impact of frequency range for F-SAT: The importance of high-frequenee components is under- scored in Figure 6b] which shows the model's performance under gradient- based adversarial attacks across different frequency ranges. Attacks targeting higher frequencies notably degrade deepfake detection more than lower frequencies, underscoring the vulnerability of high-frequency features. By focusing attacks on high-frequency regions, we preserve low-frequency integrity, simplifying the model's task of distinguishing deepfake audio features. This approach maintains high accuracy on unattacked data and improves adversarial robustness.
474
475
476
477
478
479
480
Further analysis on selecting the optimal frequency range for F-SAT is presented in Table 4 Ad- versarial training within the 4k to 8k frequency range yields the best performance. We observe that increasing the attack frequency range decreases accuracy on original data due to the distortion of more critical features, adding complexity the model's learning process.
481
482
483
484
485
Moreover, we observe that narrowing the frequency range F-SAT increases the accuracy for at- tacked fake data while decreasing it for attacked real data. This supports the findings shown in

---

Published as a conference paper at ICLR 2025
Approach
Original
Attack (Tīme)
Attack (Frequency)
486
Real Ace
Fake Acc
F1
Real Acc
Fake Ace
F1
Real Acc
Fake Acc
F1
487
F-SAT (0-8kHz)
0.977
0.818
0.898
0.978
0.795
0.877
0.970
0.751
0.862
488
F-SAT (2-8kHz)
0,983
0.896
0.940
0.976
0.736
0.831
0.979
0.754
0.836
489
F-SAT (4-8kHz)
0.975
0.964
0,974
0.902
0.880
0.910
0.933
0.915
0.924
490
F-SAT (6-8kHz)
0.971
0.974
0.952
0.912
0.850
0.879
0.923
0.896
0.907
491
492
Table 4: The results of frequency range selections for F-SAT. 4-8k is the most effective range.
493
494
Figure 2 which highlight a significant gap between the frequency domains where real and fake au- dio features predominantly exist. Fake audio features are concentrated in higher frequency ranges compared to real audio. In most real-world applications, criminals aim to make fake audio sound real enough to deceive detectors for committing fraud, Therefore, focusing adversarial training on high frequencies effectively enhances the robustness of fake audio.
495
496
497
498
499
0.96
500
0.925
0.850
0.94
501
00.00
0.877
roustack
20.00
0.92
no sttcch atteck (3rm] attedk (raguency)
502
2 0.825
06800
abock (hequarry)
stack Dins]
0.88
0.86
503
0.800
0.84
504
0.778
0.82
505
0.750
50
1000-01
Attck Magnitude
1 OOF
02
000-03
0.80
Ralio Robust Clean
1.00E-01
SAP
1.00-03
506
(a) Impact of Attack Magnitude on Detection (b) Impact of ratio y between robust and clean Accuracy
507
loss (Brebust/ Leteon) on Detection Accuracy
508
509
Figure 7: Exploration of hyperparameters control attacks, balancing accuracy and robustness. (We train Rawnet3 + RandAug + F-SAT on 10% of DeepFakeVoc-HQ))
510
511
Balance Between Clean and Attacked Data: Figure 7a demonstrates that F-SAT achieves optimal performance with an attack magnitude of = 0.01, maintaining high accuracy on both attacked and clean data. Figure 7b reveals that ratio of 0.1 between robust and clean loss, (Troobust/ / Teetaa)), results in the highest overall accuracy. The results show that excessive focus on attacked data does not consistently boost robustness and may decrease clean data accuracy. Similarly, an undue em- phasis on clean data does not reliably enhance accuracy and can weaken robustness. Thus, striking a balance between clean and attacked samples critical for optimal model performance.
512
513
514
515
516
517
518
The fundamental rationale for our algorithm lies in the dual nature of nigh-frequency features: while they are crucial for identifying deepfake audio, they are also susceptible to adversarial attacks. In contrast, low-frequency features, though more robust, do not provide sufficient information on their own to train an effective detector. Therefore, simply using a lowpass filter to eliminate all high-frequency features is not a feasible strategy; instead, powerful and resilient detector must retain and protect these high-frequency features.
Additional Analysis of the F-SAT:
519
520
521
522
523
524
Additionally, our F-SAT approach specifically targets the magnitude component of audio signals while preserving the phase. This focused strategy significantly bolsters the effectiveness of our method. To substantiate this approach, conducted rreueency-eelectvv adversarial training on the high-frequency phase component, which led to a substantial decline in accuracy on unattacked data. This outcome suggests that the model may overfit to artifacts introduced by phase perturbations, thereby diminishing its generalization capacity.
525
526
527
528
529
530
531
CONCLUSION
532
533
In our study, we introduce a dataset PeecFakeVo--HQ that addresses diversity and quality issues in prior datasets, and provide a taxonomy to explore common audio corruptions and attacks. We find that leading AI voice detection models depend on vulnerable high-frequency features. This discovery leads us to develop F-SAT, a targeted adversarial training method that focuses on high- frequency components while while preserving the integrity of low-frequency features. Our approach effectively maintains accuracy on unattacked data and enhances robustness against various attacks. These results pioneer robust training for detecting fake audio for the first time, opening up a new direction for identifying such threats.
534
535
536
537
538
539
10

---

Published as a conference paper at ICLR 2025
540
ACKNOWLEDGMENTS
541
542
This work was supported in part by multiple Google Cyber NYC awards, Columbia SEAS/EVPR Stimulus award, and Columbia SEAS- KFAI Generative Al and Public Discourse Research award.
543
544
545
REFERENCES
546
547
Fulya Akdeniz and Yacsar Becerikli. Detection of copy-move forgery in audio signal with mel fre- quency and delta-mel frequency kepstrum coefficients. In 2021 Innovations in Intelligent Systems and Applications Conference (ASYU), pp. 1-6. IEEE, 2021.
548
549
550
Moustafa Alzantot, Ziqi Wang, and Mani B Srivastava. Deep residual neural networks for audio spoofing detection. arXiv preprint 100.00090700000,,200
551
552
553
James Betker. Better speech synthesis through scaling, arXiv preprint arXiv:2305.07243, 2023.
554
Stefano Borzi, Oliver Giudice, Filippo Stanco, and Dario Allegra. Is synthetic voice detection research going into the right direction? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 71-80, 2022.
555
556
557
558
Guillermo Galahorra-Candao and María José Martin-de Hoyos. The effect of anthropomorphism of virtual voice assistants on perceived safety as an antecedent to voice shopping. Computers in Human Behavior, 153:108124, 2024.
559
560
561
Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea, Logan Hart, Aya Al- jafari, Joshua Meyer, Reuben Morais, Samuel Olayemi, et al. Xtts: massively multilingual zero-shot text-to-speech model. arXiv preprini arXiv:2406.04990 2024.
562
563
564
565
Guangke Chen, Zhe Zhao, Fu Song, Sen Chen, Lingling Fan, Feng Wang, and Jiashui Wang. To- wards understanding and mitigating audio adversarial examples for speaker recognition. JEEE Transactions on Dependable and Secure Computing, 20(5):3970-3987, 2023. doi: 10.1
566
567
TDSCC.202.3320667
568
569
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. international conference on machine learning, pp. 1310-1320. PMLR, 2019.
570
571
572
Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 702-703, 2020.
573
574
575
Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. Ecapa-tdnn: Emphasized chan- nel attention, propagation and aggregation in tdnn based speaker verification. arXiv preprint arXiv:2005.07143. 2020.
576
577
578
579
Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Sigi Zheng, Yue Gu, Ziyang Ma, et al. Cosyvoice: scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. arXiv preprint vrXiv:2407.05407 2024.
580
581
582
Joel Frank and Lea Schönherr. Wavefake: A data set to facilitate audio deepfake detection. arXiv preprint arXiv:2111.02813 2021.
583
584
585
Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 776-780, IEEE. 2017.
586
587
588
589
Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Donghaa Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. Naturatspeech 3: Zero-shot speech synthesis with factor- ized codec and diffusion models. arXiv preprint arXiv:2403.03100 2024.
590
591
592
593
Jee-weon Jung, You Jin Kim, Hee-Soo Heo, Bong-Jin Lee, Youngki Kwon, and Joon Son Chung. Pushing the limits of raw waveform speaker recognition. arXiy preprint 280848,20, 2022.

---

Published as a conference paper at ICLR 2025
594
Karl Michel Koerich, Mohammad Esmailpour, Sajjad Abdoli, Alceu de S Britto, and Alessandro L Koerich. Cross- rrosstrpprssnntation transferability of adversarial attacks: From spectrograms to audio waveforms. In 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1-7. IEEE, 2020.
595
596
597
598
Yinghao Aaron Li, Cong Han, Vinay Raghavan, Gavin Mischler, and Nima Mesgarani. Styletts 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information Processing Systems, 36, 2024.
599
600
601
602
Songxiang Liu, Dan Su, and Dong Yu. Meta-voice: Fast few-shot style transfer for expressive voice cloning using meta learning. arXiv preprint arXiv:21 2021.
603
604
605
Xuechen Liu, Xin Wang, Md Sahidullah, Jose Patino, Héctor Delgado, Tomi Kinnunen, Massi- miliano Todisco, Junichi Yamagishi, Nicholas Evans, Andreas Nautsch, al. Asvspoof 2021: Towards spoofed and deepfake speech detection in the wild. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2507-2522, 2023.
606
607
608
609
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, arXiv preprint arXiv:1706.06600 2017.
610
611
612
613
Aleksander Mkadry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. stat, 1050(9), 2017.
614
615
Chengzhi Mao, Ziyuan Zhong, Junfeng Yang, Carl Vondrick, and Baishakhi Ray. Metric learning for adversarial robustness. Advances in neural information processing systems, 32, 2019.
616
617
618
Nicolas M Müller, Pavel Czempin, Franziska Dieckmann, Adam Froghyar. and Konstantin Böttinger. Does audio deepfake detection generalize? arXiv preprint arXiv:2203.1626: 2022.
619
620
Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. Voxceleb: a large-scale speaker identifi- cation dataset. arXiv preprint arXiv: 1006. 02.0, 2017.
621
622
623
Paarth Neekhara, Shehzeen Hussain, Subhankar Ghosh, Jason Li, Rafael Valle, Rohan Badlani, and Boris Ginsburg. Improving robustness of llm-based speech synthesis by learning monotonic alignment. arXiv preprint arXiv:2406. 17957, 2024.
624
625
626
Raphael Olivier, Bhiksha Raj. and Muhammad Shah. and audio. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2995-2999. IEEE, 2021.
adversarial defense for speech
627
628
629
630
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206-5210. IEEE, 2015.
631
632
633
Puyuan Peng, Po-Yao Yao Huang, Daniel Li, Abdelrahman Mohamed, and David Harwath. Voicecraft: Zero-shot speech editing and text-to-speech in the wild. arXiv preprint arXiv:2403. 16973, 2024.
634
635
636
Vadim Popov, Ivan Vovk, Vladimir Gogoryan, Tasnima Sadekova, Mikhail Kudinov, and Jiansheng Wei. Diffusion-based voice conversion with fast maximum likelihood sampling scheme. arXiv preprint arXhv:2109.138211 2021.
637
638
639
Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. Robust speech recognition via scage weak supervision. In International conference on ma- chine learning, pp. 18492-28511 18. PMLR, 2023.
640
641
642
643
Krishan Rajaratnam, Kunal Shah, and Jugal Kalita. Isolated and ensemble audio preprocessing methods for detecting adversarial examples against automatic speech recognition. arXiv preprint arXiv: 043977 2018
644
645
646
647
Mirco Ravanelli and Yoshua Bengio. Speaker recognition from raw waveform with sincnet. In 2018 IEEE spoken language technology workshop (SI.7), pp. 1021-1028. IEFE, 2018.
12

---

Published as a conference paper at ICLR 2025
648
Chandan KA Reddy, Vishak Gopal, and Ross Cutler. Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6493-6497. IEEE, 2021.
649
650
651
Md Sahidullah and Goutam Saha. Design, analysis and experimental evaluation of block based transformation in mfcc computation for speaker recognition. Speech communication, 54(4):543- 565, 2012.
652
653
654
655 Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. arXiv preprini arXiv:2304.099 16, 2023.
656
657
658
Chengzhe Sun, Shan Jia, Shuwei Hou, and Siwei Lyu. Ai-synthesized voice detection using neural vocoder artifacts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 904-912, 2023.
659
660
661
662
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1312. 2013.
663
664
Hemlata Tak, Jose Patino, Massimiliano Todisco, Andreas Nautsch, Nicholas Evans, and Anthony Larcher. End-to-end anti-spoofing with rawnet2. In ICASSP 2021-2022 IEEE International Con- ference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6369-6373 IEEE, 2021.
665
666
667
668
Massimiliano Todisco, Héctor Delgado, and Nicholas Evans. Constant q cepstral coefficients: A
669
spoofing countermeasure for automatic speaker verification. Computer Speech & Language, 45: 516-535, 2017.
670
671
Massimiliano Todisco, Xin Wang, Ville Vestman, Md Sahidullah, Héctor Delgado, Andreas Nautsch, Junichi Yamagishi, Nicholas Evans, Tomi Kinnunen, and Kong Aik Lee. Asvspoof 2019 Future horizons in spoofed and fake audio detection. arXiv preprint arXiv: arXiv:19040055: 1904.05411, 2019.
672
673
674
675
Dimitris Tsipras, Shibani Santurkar, Logan Engstrom, Alexander Turner, and Aleksander Madry. Robustness may be odds with accuracy. arXiv preprint orXiv: /805. /2/22, 2018.
676
677
678
Shutong Wu, Jiongxiao Wang, Wei Ping, Weili Nie, and Chaowei Xiao. Defending against adver- sarial audio via diffusion model. arXiv preprint arXivv2303.01507 2023.
679
680
Ruilin Xu, Rundi Wu, Yuko Ishiwaka, Carl Vondrick, and Changxi Zheng. Listening to sounds of silence for speech denoising. Advances in Neural Information Processing Systems, 33:9633- 9648, 2020.
681
682
683
684
Junichi Yamagishi.
English multi-speaker corpus for cstr voice cloning toolkit. http://homepages. inf. ed. ac. whiyanaass/pge//pgeSS/ppe html, 2012.
URL
685
686
Yujie Yang, Haochen Qin, Hang Zhou, Chengeheng Wang, Tianyu Guo, Kai Han, and Yunhe Wang. A robust audio deepfake detection system via multi-view feature. In ICASSP 2024-2024 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 13131-13135. IEEE, 2024.
687
688
689
690
691 Neil Zeghidour, Nicolas Usunier, Iasonas Kokkinos, Thomas Schaiz, Gabriel Synnaeve, and Em- manuel Dupoux. Learning filterbanks from raw speech for phone recognition. In 2018 IEEE international conference on acoustics, speech and signal Processing (ICASSP), pp. 5509-5513.
692
693
IEEE, 2018
694
695
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning. pp. 7472-7482. PMLR, 2019.
696
697
698
Zhenyu Zhang, Xiaowei Yi, and Xianfeng Zhao. Fake speech detection using residual network with transformer encoder. In Proceedings of the 2021 ACM workshop on information hiding and multimedia security, pp. 13-22, 2021.
699
700
701
13

---

Published as a conference paper at ICLR 2025
702
A
APPENDIX
703
704
A.1
LEGAL PERMISSIONS FOR BENCHMARK DATASET
705
706
Our benchmark dataset sourced from three main platforms: YouTube, commercial software, and open-source projects from GitHub. For both commercial software and open-source projects, have reviewed their respective policies and confirmed that they permit the sharing of generated data for non-commercial use.
707
708
709
710 To address concerns regarding YouTube's 's terms service and ethical considerations, we will not directly distribute any content sourced from YouTube. Instead, will provide only metadata (video ID, start time, and end time), ensuring that researchers can access the content through YouTube's official interface compliance with the platform's 's policies.
711
712
713
714
715
A.2
COMPARISON OF DATASETS
716
717
ASVapooi
718
0.50
0.96
0.50
2019
719
In- Wild
720
0.50
0.94
0.45
0.57
721
Waveeikke
722
Train
1.00
723
724
weo teres
0.50
0.99
725
726
Ctrrs
(mmll)
0.64
0.99
0. 0.90
727
ASVigoe/2019
In Wild WondFaln
728
Tist Set
729
Figure 8: Performance of the RawNet3 baseline model on various datasets. 'Ours (train, w/o new)' represents training dataset after removing all high-quality deepfake samples.
730
731
732
A.3
EXPLANATION OF SOME TYPES OF CORRUPTIONS
734
735
Air absorption refers to the phenomenon where high-frequency sound waves more strongly attenuated by air molecules. As sound travels through the air, it loses energy, partic- ularly at higher frequencies, due to air viscosity and thermal conduction.
Air Absorption:
736
737
738
A room simulator is a digital tool that emulates the acoustics of different rooms or spaces. It includes parameters such as room size, wall materials, and shape, which affect how sound reflects and diffuses.
Room Simulator:
739
740
741
Peaking:
Peaking refers adjusting a specific range of frequencies around central frequency. is commonly used in parametric equalizers.
742
743
744
components are sampled below the Nyquist rate, causing them to appear as lower frequencies. This can be represented using the Nyquist theorem:
Aliasing: Aliasing occurs when
745
746
sample > 2 fmax=
747
748
where sample is the sampling frequency, and fmax is the maximum frequency of the signal. When this condition violated, the signal folded back into the lower frequencies, creating aliasing artifacts.
749
750
Bit-Crush: Bit-crushing reduces the bit depth of an audio signal, causing quantization errors.
751
Tanh Distortion: Tanh distortion form of soft clipping achieved using the hyperbolic tangent function. The output is given by:
752
753
= tanh(kx),
754
755
where x is the input signal, and & controls the amount distortion. As k increases, the function approximates hard clipping effect.
14

---

Published as a conference paper at ICLR 2025
756
background noise, oolor nolse, short noise, grisssann noise
Notse
757
758
Acousic
band posd, band-stog, high-poss, pass, lone- high- low-shelf, peaking
759
Filler
760
Compption
Distortion
aliasing luk dig, tanh distoction
761
Dynamic Range Contzul
gain transstion
762
Equalizarion
seven- oeven-land parmetric EQ
763
Frequency Stilling
julc shill
764
Robustress
Temporil Modifications
time mask, time stretch
765
Compression
AAC,
766
767
A ttck Objective
unbaryettd, lamelaal (Fake unly)
768
W hito box
gradient-hos based (POSM, PCD)
Atakk Method
769
Blark box
transfr lasaad, query based
770
A ttack
Perturiation Domain
time domain, fraqumay dumain
771
Perturbation Strmngth
impenooptible, perceptiale
772
Global
whole andho
773
Soope
time segment, frequency
774
775
Figure 9: Overview of various corruption types and adversarial attack strategies affecting audio robustness. The diagram categorizes different forms of corruptions (e.g., noise, filtering, distortion) and adversarial attacks (e.g., white-box, black-box) based their methods, objectives, and scope of perturbation. This framework outlines the challenges in ensuring the robustness of audio systems against both environmental corruption and intentional adversarial manipulation.
776
777
778
779
780
781
Gain Transition: Gain transition refers smoothly adjusting the amplitude over time.
782
783
Seven-Band Parametric EQ: seven-band parametric EQ provides independent control over seven frequency bands. Each band can be adjusted using three parameters: center frequency (fo), gain (G), and bandwidth (Q). The overall transfer function is a combination of individual band filters:
784
785
786
787
H( = II H&U), 2=1
788
789
where Hi(f) represents the frequency response of the i-th band.
790
791
Time Mask: Time masking involves temporarily silencing or removing a segment of audio.
792
793
A.4
RANDAUGMENT FOR AUDIO
794
795
796
audio, transforessorss
"backaround_nodse"" 'eotor_neian "short matort_nodse', 'guussian_noise', jadr_absorption', 'roon_ 'band_pass', "high_shetr', shetr'', "peaktin',, 'atsasing', 'bst_orush', "clip",
797
band_stopp 'high_pass', 'tanh, "tanh,distortal distortion', ; "gain, transition', www.bbbnndppaaeerrc..oo piithhsshftt', 'tins, "tine. streetn"
798
799
def rand_augmant_ wtw._aagnnntnuddsssraaaaaoo (sanpee
800
nex Apply randos augnentations to an oudio souple.
Angs:
801
sueple
augnentatiors apply
Probebility applying augnentation
802
Returns:
An sugnented oudio sample
803
804
operations randon, choice (audio_ transforrs,
805
raturn cperations(caapaa) &F rando() p slse sampla
806
Figure 10: Python code for RandAugment for audio
807
808
Details of RandAugment for Audio are shown in Figure 10 In our experiments, we set = 1 and
p 0.9.
15

---

Published as a conference paper at ICLR 2025
810
A.5
ROBUSTNESS TO COMPRESSION
811
812
To evaluate the robustness of our detection model to compression, we tested two lossy formats: MP3 and AAC. The evaluation utilized RawNet3 combined with RandAug and F-SAT. As shown in the 3 both MP3 and AAC compression had minimal impact on detection accuracy.
813
814
815
816
Format
Real
Fake
Avg
817
Origin (90% wav 10% mp3)
97.5%
96.4%
97.0%
818
MP3
97.5%
95.6%
96.6%
ACC
96.9%
96.6%
96.8%
820
821
Table 5: Detection Results for Compressed Audio Formats
822
823
824
A.6
COMPREHENSIVE ANALYSIS OF ALL ATTACK TYPES
825
Table presents the detailed adversarial attack results for RawNet3 and our method under vari- ous conditions. includes both white-box and black-box approaches and examines the impacts of attacks in both the time and frequency domains, detailing the different attack hyperparameters used.
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
849
850
851
852
853
854
856
857
858
859
860
861
862
16

---

Published as a conference paper at ICLR 2025
864
Domain
Iters
Restart
Source
RawNet3
Maw.Net3+RandAue
RawNet3+RandAuug SAT
865
Real Acc
Fake Ace
Real Ace
Fake Acc
Real Acc
Fake Ace
No Attack
96.1
75.9%
97.6%
94.7%
97.5%
96.4%
867
868
2
A
97.9% %
3.50%
74.7%
66.1%
90.2%
88.0%
5
1
A
90.6 %
2.10%
34.4%
42.4%
66.7%
80,9%
869
5
2
A
90.2 %
1.70%
33.2%
39.4%
65.4%
80.3%
870
2
1
A
98.9 %
8.10%
79.8%
69.0%
92.9%
89.2%
871
Time
5
1
92.1 %
2.90%
43.7%
52.4%
74.4%
84.0%
5
2
91.6 %
3.20%
42.1%
50.3%
73.2%
83.1
872
B
100.0%
17.3%
%
91.7%
98.7%
96.4%
100.0%
16.0%
98.9%
92.1%
98.7%
96.3%
874
5
2
100.0%
16.2%
98.9% %
92.1%
98.7%
96.3%
875
2
1
A
65.0%
5.7%
38.5%
42.9%
87.0%
86.3%
5
23.8%
2.9%
7.5%
14.9%
63.8%
69.4%
876
5
2
23.0%
2.7%
7.3%
15.2%
63.7%
69.2%
877
2
1
77.3%
14.9%
50.6%
55.1%
90.2%
88.1 %
Frequency (0-8k Hz)
878
31.6%
4.92%
14.9%
27.6%
72.7%
74.6%
879
2
23.8%
2.4%
7.8%
16.0%
63.8%
66.4%
99.8%
34.60%
85.98%
98.7%
96.8%
99.8%
29.5%
98.9%
85.3%
98.7%
96.8%
881
2
100.0%
31.9%
98.9%
84.9%
98.7%
96.7%
882
2
1
RANALAE
74.3%
6.8%
50.4%
49.3%
93.1%
89.5 %
883
5
34.0%
4.29%
12.9%
22.9%
83.5%
82.6%
5
2
33.0%
4.44%
12.7%
21.4%
83.0%
82.1 %
884
84.0%
16.5%
59.7%
63.5%
94.0%
89.8%
885
Frequency (2-8k Hz)
43.8%
7.00%
23.5%
35.2%
87.5%
85.1%
886
I
2
33.5%
12.9%
21.3%
82.9%
80.3%
887
99.8%
84.2%
98.6%
97.0%
100.0%
28.4%
98.4%
84.7%
98.7%
95.9%
2
99.8%
31.0%
99.1%
84.2%
98.6%
96.0%
889
2
A
87.2%
8.5%
72.9%
61.3%
96.7%
94.6%
890
5
1
54.9%
7.78%
35.4%
36.3%
96.2%
93.4%
891
2
54.4%
6.83%
35.1%
36.3%
95.7%
93.2%
2
91.4%
18.3%
80.0%
71.9%
97.1%
95.6%
892
Frequency (4-8k Hz)
1
64.9%
10.5%
47.3%
50.6%
96.5%
94.5%
893
2
54.6%
7.1%
36.0%
36.3%
96.0%
93.4%
894
99.7%
34.4%
99.0%
88.0%
98.0%
96.3%
895
99.7%
29.2%
99.0%
87.5%
98.0%
96.3%
5
2
99.7%
32.5%
99.2%
87.8%
98.0%
96.3%
896
2
1
A
95.2%
12.3%
90.1%
76.7%
96.5%
95.6%
897
5
1
83.8%
16.0%
70.6%
65.6%
95.7%
95.3%
898
5
2
83.7%
15.7%
70.5%
65.0%
95.9%
94.9%
899
Frequency (6-8k Hz)
95.7%
21.9%
92.7%
80.4%
96.7%
95.9%
1
86.8%
16.8%
78.4%
70.4%
96.0%
95.9%
900
5
2
83.2%
15.9%
70.5%
63.4%
95.9%
95.8%
901
2
99.7%
38.3%
98.9%
89.1%
97.0%
96.4%
902
99.7%
32.2%
98.9%
88.5%
97.1
96.4%
2
B
99.7%
36.8%
99.0%
88.5%
97.3%
96.4
903
904
Table 6: Adversarial Attack Results RawNet3 and Its Variants under Various Conditions. For the attack scenarios, we include both white-box and black-box approaches: A represents tests with the same model and identical weights, while A' indicates the same model but with different weights, and B denotes tests on a completely different model. For attacks in the time domain, we use e =
905
906
907
908
10 and a 10 For attacks the frequency domain, the parameters are 10-3 and
909
a = 4- 14-10-
910
911
912
913
914
915
916
917
17

---

Published as a conference paper at ICLR 2025
918
A.7
DATASET DETAILS
919
920
Fake Source
Total Duration (Hours)
Audio Count
Mean Duration (Seconds)
921
922
VCTK
14.1
12.0k
4.2
923
Librispeech (Train)
961.1
281.2k
12.3
924
In-The-Wilds Wilds
14.6
9.3k
5,7
925
ASRspoof2019 (LA)
11.9
12.5k
3,4
Voxceleb1
340.4
[48.6k
8.2
926
Audioset (Narration)
50.1
12.2k
14.8
927
928
Table Summary of fake audio sources data
929
930
931
932
Fake Source
Total Duration (Hours)
Audio Count
Mean Duration (Seconds)
933
Metavoice
189.1
61.7k
11.0
934
StyleTTSSv2
186.6
61.6k
10.9
935
XTTS-v2
175.5
61.8k
10.2
936
VoiceCraft
119.9
59.4k
7.3
937
Whisperspeech
155.2
61.9k
9,0
938
Vokan-TTS
161.7
61.6k
9.4
Elevenlabs
3.3
3.2k
3,7
939
ASRspoof2019 (LA)
97.8
109.0k
3.2
940
Wavefake (English)
198.7
117.9k
6.1
941
Table 8: Summary of fake audio sources data
943
944
945
VCTK Speaker id: p244
In-the-Wild Speaker: Alan Watts
946
Model
Ovrl MOS
Sig MOS
Bak MOS
P808 MOS
Ovrl MOS
Sig MOS
Bak MOS
PS08 MOS
947
Real refer
3.26
3.56
4.04
3.61
3.02
3.40
3.74
3.57
948
metavoice
3,29
3,58
4.05
3.63
3.15
3.52
3.88
3,55
StyleTTS v2
3.28
3.56
4.08
3.87
3,28
3.57
4.06
3,83
950
XTTS
3.13
4.00
3.78
3.41
3.98
3.70
VoiceCraft
3.16
3.51
3,94
3.61
3.01
3.34
3.80
3.43
951
Whisperspocch
3.28
3.56
4.07
3.82
3.15
3.44
3.99
3.59
952
Vokan- TTS
3.23
3.55
4.01
3.71
2.94
3.39
3.66
3.60
953
Table MOS Scores for Various TTS Models
954
955
956
Table 7 presents the real audio data we utilized from previously published datasets. For generating deepfake audio for training set, we employ models such as XTTS v2, StyleTTS v2, Metavoice, Whisperspeech, Vokan-TTS, VoiceCraft, and Elevenlabs. Additionally, for the test set, Cosyvoice, PlayHT 2.0, Resemble, LOVO AI, and Lipsynthesis to create deepfake voices. We introduce post-processing augmentations to generate noisy deepfakes. Four real datasets- (28.5k), Audioset (narration) ( (12.2k), and In-The- Wild (real parts: 9.3k) utilized generate deepfake voices for the training set. Details of our generated deepfake audio and references to previous public deepfake audio datasets are presented in Table 8 Addition- ally, we employed DNSMOS (Reddy et al.) 2021 to quantitatively measure the synthetic speech quality across these models, using scale from to 5, where higher values indicate better quality. As demonstrated in Table Metavoice and StyleTTS outperform other AI voice synthesis models.
957
958
959
960
(12.0k), Librispeech--lean-
961
962
964
965
966
967
968
969
970
971
18

---

Published as a conference paper at ICLR 2025
972
A.8
F-SAT's TRAINING EFFICIENCY
973
974
F-SAT's training efficiency is influenced by hyperparameters such as attack iterations and restart counts, which identify the worst-case perturbation. Drawing insights from "Fast Is Better Than Free: Revisiting Adversarial Training," we optimized these parameters by setting restarts to one and attack iterations one or two, while employing larger attack magnitude to enhance robustness. Training time is shown in the Table [O] Although F-SAT requires longer training times, it improves aceuracy by an average of 9% on original data and 43% on attacked data compared Standard Adversarial Training. We should not compromise accuracy merely to accelerate training
975
976
977
978
979
980
981
Description
w/o Adversarial Training
Standard Adversarial Training
F-SAT
982
983
Training Duration (Days) Number of Epochs
2
4.5
984
15
15
15
985
Hardware Used
A100 GPU
A100 GPU
A100 GPU
986
987
Table 10: Training time comparasion
968
989
A.8.1
TRAINING HYPERPARAMETER
990
991
Here the Training hyperparameter of F-SAT for Table
992
Training Hyperparameters
993
994
Learning Rate (Ir): 1 10-5
995
Epochs: 15
Batch Size (bs): 16
997
998
Optimizer: adam
999
Augmentation Number
1 or 2
1000
Augmentation Probability (aug prob):: 0.9
1001
1002
LR Scheduler (Warmup Cosine)
1003
1004
Warm-up Epochs: 1
1005
Warm-up LR: x 10-6
1006
Minimum LR: 10-7
1007
1008
Attack Hyperparameters
1009
1010
Attack Type: loc
1011
Epsilon: 0.005, Alpha: 0.002
1012
Gamma (control ratio of clean loss and robust loss): 0.1
1013
Attack Iterations:
1014
1015
Restarts:
1016
Frequency Range: 4-8k Hz
1017
1018
Mixup Hyperparameters
1019
Mixup Alpha: 0.5
1020
1021
1022
1023
1024
1025
19


Reviews:
[{"review_id": "DoeAWiGmLw", "reviewer": "Reviewer_EGHT", "summary": "The paper attempts to improve deepfake detection by (1) proposing a large training and evaluation dataset called DeepFakeVox-HQ containing diverse synthetic and real speech recordings, (2) proposing a data augmentation method similar to randaugment for deepfake detectors and (3) proposing a frequency-selective adversarial training (F-SAT) method to make deepfake detectors more robust to adversarial attacks. \n\nDeepFakeVox-HQ is a large dataset containing real and synthetic speech from existing datasets, speech generated by SOTA speech synthesis models as well as deepfakes found in-the-wild (on social media, etc.). Results show that models trained on DeepFakeVox-HQ generally perform better on existing deepfake while the models trained on the existing datasets have weak performance on DeepFakeVox-HQ, which indicates that DeepFakeVox-HQ includes information that prior works do not provide. DeepFakeVox-HQ will likely be a useful resource in deepfake research. \n\nThe proposed RandAugment scheme for deepfake detection utilizes a large bank of audio augmentations during training and yields significant improvements in deepfake detection accuracy.\n\nThe key contribution of F-SAT is to add adversarial perturbations to only certain frequency bands, which apparently results in lesser degradation of accuracy on un-perturbed data while providing greater robustness than standard adversarial training.", "questions": "1. What frequency range is the spec-magnitude attack being applied over in Figure 8a?\n1. Do you have any _formal_ explanation for why the time-domain attack is less successful than the spec-magnitude attack? To me this seems counter-intuitive because the STFT is a linear and (mostly) invertible function so, from the perspective of the optimization, it should not matter if the attack was computed in the frequency domain or the time domain. I would be very interested in seeing more explanation for why the time-domain attack is unable to reach the solution acheived by the frequency-domain attack. Please also provide the detailed settings for all the adversarial attacks (time, frequency and phase domain) used in AT, F-SAT and during evaluation.\n1.In principle, a frequency selective adversarial attack could be constructed entirely in the time domain by applying a band-pass filter to the adversarial perturbation after each optimization step (i.e. include the BP filter as part of the projection operation). This might be less computationally intensive than the proposed approach. Can you provide some discussion on why the proposed approach was favored?\n1. Why is the performance of the model trained on DeepFakeVox-HQ so low on the In-the-wild dataset (see Figure 3)", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. Paper is generally well-written and easy to read but some important details are missing\n1. DeepFakeVox-HQ is a novel dataset containing data from prior datasets as well as novel deepfakes generated from SOTA speech synthesis models. I appreciate that the authors have curated a test set containing deepfake generation methods not covered in the training set _as well as deepfakes gathered from the internet_.  I encourage the authors to consider uploading the dataset to a platform like Huggingface Hub.\n1. The proposed randaugment data augmentation method is effective at improving deepfake detection for RawNet3 models and is likely to be widely adopted if the source code is easy to use (I looked at the README in the attached supplemental material but did not find any instructions for the augmentation).\n1. The proposed adversarial training method improves deepfake detection accuracy on clean and adversarially perturbed recordings (though I have some reservations regarding the experimental setup).", "weaknesses": "1. Some important details about the proposed approaches are not mentioned in the paper. \n   \n   1. The value of $\\epsilon$ and $p$ (or $q$) used in adversarial training methods should be mentioned in the main body of the paper. Currently, it is mentioned in the caption of a table in the appendix\n   1. The settings used for adversarial attacks during AT, F-SAT and evaluation need to be mentioned.\n   1. The parameters of the augmentations used in randaugment need to be mentioned at least in the appendix.\n   1. Detailed composition of DeepFakeVox-HQ needs to be mentioned including \n\n      1. the method used for generating deepfakes (particularly noisy deepfakes), \n      1. the number of audios from each deepfake generation system, \n      1. the demographic distribution of the real and fake speakers,\n      1. the number of utterances used from each of the datasets from prior works, \n      1. quantitative measurement of synthetic speech quality using metrics like DNSMOS, or NORESQA.\n\n1. The choice of accuracy as a metric seems to be inappropriate for a binary classification task. I would suggest using F1-score and equal error rate as the metrics. Moreover, reading tables and plots with two accuracy metrics for accuracy is a little confusing.\n1. Conducting adversarial attacks in the frequency domain and reverting to the temporal domain is not novel and has been done before [2].\n1. There is no comparison with other adversarial defenses for audio models. Many of the defenses created for speech and speaker recognition will also apply to the deepfake detection scenario. One method that is quite simple is [3]\n1. The common practice is to use signal-to-noise ratio (SNR) as the bound for adversarial attacks in the audio domain [1] instead of $\\ell_p$ bounds. I would highly recommend the authors use SNR as well. It is fairly straightforward to convert SNR to $\\ell_2$ bounds and vice-versa. The main advantage of using SNR is that one has an idea how _perceptible_ the adversarial attack is.\n\n1. Clarity issues:\n   1. The caption Figure 9 needs to state that the results are of F-SAT\n   1. Add results for 0-8K in Figure 8b \n\n\n[1] Carlini, Nicholas, and David Wagner. \"Audio adversarial examples: Targeted attacks on speech-to-text.\" 2018 IEEE security and privacy workshops (SPW). IEEE, 2018. \n\n[2] Koerich, Karl Michel, et al. \"Cross-representation transferability of adversarial attacks: From spectrograms to audio waveforms.\" 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 2020.\n\n[3] Olivier, Raphael, Bhiksha Raj, and Muhammad Shah. \"High-frequency adversarial defense for speech and audio.\" ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eNUIJtj79i", "reviewer": "Reviewer_ijUY", "summary": "This paper addresses the challenge of deepfake audio detection, presenting two major contributions: (1) the creation of DeepFakeVox-HQ, the largest and most diverse public dataset for deepfake audio detection, which enables realistic testing conditions and exposes limitations in existing models, and (2) the introduction of Frequency-Selective Adversarial Training (F-SAT), a novel approach that improves detection robustness by focusing on high-frequency audio components. The work is well-written and logically structured, making complex concepts accessible, and holds significant potential for advancing the robustness and reliability of deepfake audio detection models.", "questions": "1.Given that F-SAT focuses on high-frequency perturbations, have the authors considered whether these perturbations might be perceptible to human listeners? \n\n\n\n\n2. Were all baseline models subjected to similar adversarial training procedures as the proposed F-SAT model? Consistency in adversarial training across baseline models is essential to ensure a fair comparison of robustness improvements. If not, would the authors consider including adversarially trained baselines in future comparisons?\n\n3 . How sensitive is F-SAT to the choice of hyperparameters, particularly the frequency ranges and perturbation magnitudes used for adversarial training?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.DeepFakeVox-HQ stands out as a substantial addition to the field, with over 1.3 million samples, including 270,000 high-quality deepfake samples from 14 sources. This dataset addresses the limitations of existing datasets in diversity and scale, making it a valuable resource for benchmarking future detection models. Releasing this dataset would have a broad impact on the community.\n\n2.The F-SAT method is an important innovation, targeting high-frequency features that are critical for detection but vulnerable to adversarial manipulation. This frequency-focused adversarial training enhances model robustness without compromising accuracy on clean data, addressing a key gap in existing deepfake detection methods.\n\n3.Comprehensive Experimental Evaluation:  \n   The experimental design is extensive, evaluating performance across standard benchmarks (ASVspoof2019 and WaveFake) as well as the authors' own test dataset. F-SAT demonstrates clear improvements in robustness across multiple corruption and adversarial attack scenarios. The addition of an ablation study further supports the effectiveness of the proposed method.\n\n4. Extending RandAugment from image processing to audio is an inventive adaptation that helps improve model robustness on both clean and corrupted audio. This demonstrates the authors' resourcefulness in leveraging existing techniques and could be beneficial for future work in audio data augmentation.", "weaknesses": "1. The paper does not specify whether baseline models were subjected to adversarial training. If only the F-SAT model received this enhancement, it could bias the results. Including adversarially-trained versions of baseline models using contemporary adversarial methods would provide a fairer comparison and highlight F-SAT’s unique advantages.\n\n2. While F-SAT’s focus on high-frequency components is intriguing, the rationale behind the reliance on high frequencies for detecting deepfake audio could be further elaborated. \n\n3.Adversarial training, especially in the frequency domain with iterative updates, can be computationally demanding. Assessing F-SAT's efficiency, particularly compared to baseline models, would improve the paper's practicality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RxGgwxuqqk", "reviewer": "Reviewer_erKc", "summary": "Glad to review the paper.\nThis paper proposes a novel method, F-SAT for deepfake audio detection.\nThe topic of this work is promising, and the paper is easy to follow.\nI believe this work has reference values to domain-related researchers.", "questions": "My main concern is about the generalization and robustness issues listed above.\nI will consider changing my score based on the author's responses and other reviewers.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "Three main contributions involved in this work include (1) a carefully organized dataset, (2) a deepfake detection method, and (3) the ability against adversarial attacks (with the setting focusing on high-frequency signals).\nIn general, the contributions of this work are multi-fold.", "weaknesses": "My major concern is whether the contributions (or advantages) of this work are over-claimed.\nRegarding the dataset, although it is well organized and processed, the samples are generated using existing approaches, thus, \"the largest\" is not a significant contribution.\nRegarding generalization, as in Table 2, the significantly superior results of the proposed method are achieved on the self-organized dataset, DeepFake Vox-HQ. However, as the author introduced in Section 3, there are overlapped synthesis methods between training and testing data in this group of results (as in Figure 6). Thus the results in DeepFake Vox-HQ can not indicate out-of-distribution generalization.\nRegarding enhancing robustness, in the last paragraph of the related section, the referenced solutions were published in 2019,2018 and 2018, I am not sure whether any recent works focus on the adversarial issue, that should be discussed or compared.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FwD3v5NWNb", "reviewer": "Reviewer_NjHP", "summary": "The paper \"I CAN HEAR YOU: SELECTIVE ROBUST TRAINING FOR DEEPFAKE AUDIO DETECTION\"\nintroduces the DeepFakeVox-HQ data set, which contains audio from 14 sources.\nIn addition to the dataset, the authors introduce Frequency-Selective Adversarial Training (F-SAT), a training method that focuses on the high-frequency part of the spectrum. In addition to FSAT, this paper evaluates robustness concerning various input perturbations.", "questions": "- Which model was used to create the plot on the left side in Figure 2?\n- How many hours of speech does the dataset encompass exactly? How long are the samples?\n- Are the samples aligned? Do all models synthesize speech using the same input sentences?\n- Is it possible to add a data sheet that outlines the exact sources and utterance lengths per source?\n- Are the WaveFake test samples also part of the DeefFakeVox-HQ test set?\n- WaveFake contains Japanese language JSUT samples.\n    - Are these part of the dataset?\n    - Should the Caption of Table 1 make this explicit? Since WaveFake is listed as \n     English-language data set, I assume JSUT is not considered a part of WaveFake in this paper.\n    - Do the Utterance numbers in table one exclude JSUT?\n    - If yes, should this be mentioned somewhere else?\n\n- Is it possible to include leading works from the audio classification world, like the Audio Spectrogram Transformer (AST)[1], in the evaluations? Related work [2] found it to perform well on the WaveFake-dataset. It would be interesting if it also outperforms other methods on DeepFakeVox-HQ.\n\n- The Wavefake paper [3] trains with binary settings with fake audio from a single source and measures generalization. Training on which source network led to the numbers in Table 2? Are the numbers comparable to the related work?\n\n- Which software libraries have been used to implement this project?\n\n- Which hyperparameters underpin the network training?\n\nRelated work:\n[1] AST: Audio Spectrogram Transformer, https://arxiv.org/pdf/2104.01778,\n[2] Towards generalizing deep-audio fake detection networks, https://arxiv.org/pdf/2305.13033,\n[3] WaveFake: A Data Set to Facilitate Audio Deepfake Detection, https://arxiv.org/abs/2111.02813", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles a significant problem.\n- The related work is well-researched and described.\n- The adversarial attack perspective is interesting.\n- Authors ensure their results are up to date, combining existing datasets with samples from commercial models.", "weaknesses": "- Traditional compression algorithms like MP3 remove high-frequency content; according to line 84, FSAT focuses on this part of the spectrum.\n- If I understand correctly, compression is not part of the corruption set, as shown in Figure 7. Including compression would have been important for real-world applicability.\n- Data-set details like the length in hours or training hyperparameters like the learning rate are missing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)