P HASE F OOL : P HASE - ORIENTED A UDIO A DVERSARIAL E XAMPLES VIA E NERGY D ISSIPATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Audio adversarial attacks design perturbations onto inputs that lead an automatic speech recognition (ASR) model to predict incorrect outputs.
Current audio adversarial attacks optimize perturbations with different constraints (e.g. l p -norm for waveform or the principle of auditory masking for magnitude spectrogram) to achieve their imperceptibility.
However, the existing audio adversarial attacks neglect the inﬂuence of phase spectrogram.
In this work, we propose a novel phaseoriented algorithm named PhaseFool that can efﬁciently construct imperceptible audio adversarial examples with energy dissipation.
Speciﬁcally, we leverage the spectrogram consistency of short-time Fourier transform (STFT) to adversarially transfer phase perturbations to the adjacent frames of magnitude spectrogram and dissipate the energy patterns in the spectrogram.
Moreover, we propose a weighted loss function to improve the imperceptibility of PhaseFool.
Experimental results demonstrate that PhaseFool can generate full-sentence imperceptible audio adversarial examples and achieve a 6.64x generation speed-up over current state-of-the-art imperceptible counterparts (the imperceptibility is veriﬁed through a human study 1 ). Further robustness evaluations demonstrate that the adversarial examples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses.
Most importantly, our PhaseFool is the ﬁrst to exploit the phase-oriented energy dissipation in the audio adversarial example rather than add perturbations on the audio waveform like most of the previous works.
I NTRODUCTION
Adversarial examples are speciﬁcally designed instances that cause a machine-learning algorithm to produce a misclassiﬁcation (Szegedy et al., 2013; Biggio et al., 2013).
Since researchers have successfully created adversarial examples to make automatic speech recognition (ASR) system transcribe the audio as any different sentence (Carlini & Wagner, 2018; Vaidya et al., 2015; Carlini et al., 2016), audio adversarial attacks have become more and more popular.
Those attack algorithms usually optimize perturbations added on the benign audio waveform with different constraints based on l p -norm (Carlini & Wagner, 2018; Zhang et al., 2020).
Although the perturbations introduced is small in magnitude, noises in the audio adversarial examples are usually perceptible.
Later, algorithms based on psychoacoustic principle (Qin et al., 2019; Sch¨onherr et al., 2019) successfully generate effectively imperceptible adversarial examples by hiding the adversarial perturbations to regions of magnitude spectrogram where it will not be heard by a human.
In current imperceptible targeted audio adversarial attacks, only waveform and magnitude spectrogram are taken into account leaving phase and its nature unexplored.
Since the structure of natural speech is highly complicated, these two types of algorithms face several challenges:
• Waveform-oriented attacks are usually not imperceptible enough.
Although constraining perturbations with l 2 -norm or l ∞ -norm (Carlini & Wagner, 2018; Liu et al., 2020) these attacks introduce audible noises to the adversarial examples.
Later works focused on the temporal dependency (Zhang et al., ) successfully reduce human-perceptible noises
[Figure]
but their perturbations are still audible, due to the uncorrelated relationship between psychoacoustic perception and l p -norm metrics.
• The optimization of attacks based on the psychoacoustic principles is highly timeconsuming and hard to converge.
Due to the determining inﬂuence of magnitude spectrogram on human perception, the attack that leverages the psychoacoustic principle of auditory masking (Qin et al., 2019) successfully generates imperceptible audio adversarial examples.
However, it needs two-stage optimization where the ﬁrst stage of optimization focuses on ﬁnding a relatively small perturbation to fool the network (as is done in prior work (Carlini & Wagner, 2018)) and the second stage makes the adversarial examples imperceptible.
The search process of the optimal balance between hiding the adversarial perturbations in the inaudible regions and fooling the ASR systems is empirically difﬁcult.
Generating a single full-sentence adversarial example requires approximatelyiterations until it ﬁnally converges to an imperceptible solution.
This research is motivated by a psychoacoustic principle that phase manipulation in one-channel audios imposes tiny effects on human perception (Vary & Eurasip, 1985).
Besides, the inherent spectrogram consistency of short-time Fourier transform (STFT) reveals the fact that the adversarial perturbations in a frame of phase spectrogram will transfer to adjacent frames of magnitude spectrogram (Grifﬁn & Lim, 1984) and dissipate the energy patterns in the spectrogram.
These observation motivates us to exploit phase spectrogram for generating audio adversarial examples.
Considering the relationship between phase spectrogram and the spectrogram consistency of STFT, in this work, we propose a novel phase-oriented algorithm called PhaseFool that efﬁciently generates imperceptible audio adversarial examples via energy dissipation.
We prove that the phase-oriented perturbations always reduce the magnitude of the spectrogram in the corresponding areas via spectrogram consistency, which is called energy dissipation.
Leveraging such a phenomenon, we bring new directions for audio adversarial attacks and are able to construct imperceptible audio adversarial examples efﬁciently.
To the best of our knowledge, we are the ﬁrst to consider phase spectrogram in the audio adversarial attacks.
Our contributions can be summarized as follows:
• We leverage the spectrogram consistency of STFT to construct phase-oriented audio adversarial examples, which brings new directions for imperceptible audio adversarial attacks.
• We investigate the relationship between phase perturbations and magnitude spectrogram, which is called energy dissipation.
Instead of the common l p distance metrics or magnitude-oriented metrics based on the psychoacoustic principle, we propose a weighted loss function based on the phenomenon of energy dissipation to improve the imperceptibility of phase-oriented audio adversarial examples.
• We further evaluate the robustness of PhaseFool by constructing adversarial examples that can remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses.
We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015) and implement successful attacks against a pretrained DeepSpeech 2 model (Amodei et al., 2016) and an end-to-end Transformer-based ASR system (Winata et al., 2020).
The results show that in terms of imperceptibility, PhaseFool greatly surpasses current state-of-the-art algorithms based on l p -norm metrics and shows comparable imperceptibility with the attacks based on the psychoacoustic principle of auditory masking (veriﬁed by a human study).
Besides, PhaseFool achieves a 6.64x generation speed-up over current state-of-the-art imperceptible counterparts. Furthermore, in the robustness evaluations, the adversarial examples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses.
We attach the adversarial examples in the experiments and the implementation for reproducibility in the supplementary material.
B ACKGROUNDS
In this section, we brieﬂy overview the background of this work, including audio adversarial examples and phase-aware audio signal processing.
Audio Adversarial Examples.
After the early works (Gong & Poellabauer, 2017; Ciss ´ e et al., 2017) towards neural networks based ASR systems successfully generate untargeted adversarial examples that produce incorrect but arbitrary transcriptions.
Recently, more and more efforts have been made in the audio domain (Alzantot et al., 2018; Carlini & Wagner, 2018).
Among them, one of the most outstanding studies is the iterative optimization-based attack (Carlini & Wagner, 2018), which introduces a two-stage attack and directly generates targeted adversarial examples using a gradient-descent minimization method.
Nevertheless, the generated adversarial examples tend to include noticeable noise.
Thus, two recent approaches (Qin et al., 2019; Sch¨onherr et al., 2019) introduce hearing thresholds for designing inaudible perturbations by curbing the signal variation below the threshold of human perception.
However, these imperceptible attacks require two stages of optimization and suffer from slow convergence rates.
Phase-aware Audio Signal Processing.
The interest in utilizing phase information in the ﬁeld of audio signal processing is constantly increasing (Mowlaee et al., 2016).
The STFT phase has been widely used in signal enhancement (Miyahara & Sugiyama, 2014; Choi et al., 2019; Hu et al., 2020) and signal reconstruction (Mowlaee & Kulmer, 2015).
The phase in the harmonic domain is used in the speech synthesis (Degottex & Erro, 2014; Ai & Ling, 2020) and the phase in perceptual bands is used in speech watermarking (Wang et al., 2021).
The important information in the phase brings positive contributions to the above ﬁelds.
Nevertheless, the existing audio adversarial attacks directly neglect the inﬂuence of phase spectrogram (Sch¨onherr et al., 2019).
In this work, instead of introducing perturbations to the audio waveform like most of the previous works, we propose to leverage the spectrogram consistency and generate phase-oriented perturbations for constructing imperceptible audio adversarial examples.
P HASE - ORIENTED A UDIO A DVERSARIAL E XAMPLES VIA E NERGY D ISSIPATION
In this section, we ﬁrst deﬁne the preliminaries of the phase-oriented audio adversarial attack.
Then we investigate the relation between phase spectrogram and magnitude spectrogram under spectrogram consistency and prove that phase perturbations always reduce the magnitude of the spectrogram in the corresponding areas under spectrogram consistency, which is called energy dissipation (as is illustrated in Figure 1).
Finally, we design a novel phase-oriented loss function based on energy dissipation for constructing imperceptible audio adversarial examples. [Figure] [Figure] [Figure]
W Invertible
[Figure]
3. P RELIMINARIES OF P HASE - ORIENTED A UDIO A DVERSARIAL A TTACK 3.1. P ROBLEM D EFINITION
Given an input audio waveform x and a transcription y corresponding to x . Denote the automatic speech recognition (ASR) system as f ( · ) that outputs a transcription. The objective of the targeted audio adversarial attack is to ﬁnd a small perturbation δ that constructs an imperceptible and targeted example x (cid:48)
=
x + δ to fool the ASR system f ( · ) so that f ( x (cid:48) ) = y t , where y t is the targeted adversarial transcription. Denote L net ( f ( x (cid:48) ) , y t ) as the loss function for the ASR model that is minimized when f ( x (cid:48) )
= y t .
Then the general formulation of targeted adversarial attack for ASR system can be deﬁned as:
[Equation]
(1) where λ > 0 is the trade-off parameter between the imperceptibility and loss function L net ( f ( x + δ ) , y t ) (in some formulations λ = 0), (cid:15) control the maximum perturbation value, and p could be speciﬁed as different values, according to the attacker’s requirement (e.g. p
= 2 in C&W’s attack (Carlini & Wagner, 2018)).
Then we will describe how to generate phase-oriented and targeted audio adversarial examples in the next subsection.
Denote φ ( · ) as the function of the STFT and φ − 1 ( · ) as the function of the inverse STFT and
X =
φ ( x ) = | X | · e iθ x , where | X | and θ x = ∠ X representing the spectral magnitude and the instantaneous phase spectrogram, respectively.
Denote T ( X ) =
φ ( φ − 1 ( X )) .
We assume that window function w , window length N and window shift R is ﬁxed and a signal can be perfectly reconstructed from its STFT spectrogram through the inverse STFT in this setting.
We propose to generate perturbations on the instantaneous phase spectrogram and reconstruct the audio adversarial examples with the inverse STFT as the following formulation:
[Equation]
where E ( · ) is the feature extraction function for the ASR model and the φ − 1 ( | X | · e iθ x · e jδ ) is the generated audio adversarial example.
As shown in Figure 2, the spectrogram X (cid:48)
=
| X | · e iθ x · e jδ perturbed by P ( X ) is usually located outside the STFT spectrogram domain.
Considering the phenomenon of spectrogram consistency (Le Roux et al., 2010), it is easy to prove the following two conclusions: 1) | X (cid:48) | (cid:54) = |T
( X (cid:48) )
| mathematically; 2) φ − 1 ( X (cid:48) ) and φ − 1 ( T ( X (cid:48) )) represent the same time-domain audio signals.
Since the STFT and iSTFT functions are differentiable, these conclusions enable us to adversarially manipulate the reconstructed magnitude spectrogram by introducing perturbations to the original phase spectrogram, which is called phase-oriented audio adversarial attack.
Then we detail how the perturbations on a frame of phase spectrogram inﬂuence the adjacent frames of corresponding magnitude spectrogram and reveal the energy dissipation nature of the phase-oriented audio adversarial attack.
3. R ELATION A NALYSIS BETWEEN P HASE AND M AGNITUDE
This subsection mainly discusses the essence of the relationship between phase perturbations and magnitude spectrogram frames.
Reconstructing the time-domain signals after introducing perturbations to the phase spectrogram includes an overlap-add process.
For simplicity, we assume that the window shift length R is equal to half of the window length N , which means the adjacent frames have 50% overlapped signals.
As shown in Figure 2, the signals x m − 1 ( n (cid:48) ) and x m ( n ) in adjacent frames m − 1 and m overlap, where n (cid:48) and n represent the time indices and n (cid:48)
= n + N 2 .
x m ( n ) are obtained by an inverse Fourier transform as the following:
[Equation]
where j is the imaginary unit and X m [ k ] are spectrogram samples in frames m at k -th frequency bin. x m − 1 ( n (cid:48) ) can also be obtained with Equation 3.
Then we introduce perturbations δ m to the frame m and the corresponding signal ˆ x m ( n ) becomes:
[Equation]
Since the human auditory system is relatively less sensitive to the tiny time delay caused by phase ro- tation, the phase perturbations are inherently more imperceptible than the perturbations on the audio waveform.
Then we further investigate how phase perturbations affect the magnitude spectrogram.
The reconstructed signal ˜ x m ( n ) with the overlap-add process is obtained as:
[Equation]
We substitute X m − 1 [ k ] and X m [ k ] with the Fourier transform of x m − 1 ( n (cid:48) ) and x m ( n ) , then the overlapping area of ˜ x m ( n ) in Equation 5 will become:
[Equation]
The overlapping area of the x m − 1 ( n (cid:48) ) and x m ( n ) in Figure 2 (b) satisﬁes the following relationship based on spectrogram consistency:
[Equation]
With Equation
[Equation]
we can deduce the ﬁnal formulation for the reconstructed signal x m ( n ) as following:
[Equation]
where the clean audio can be represented by setting δ m − 1 [ k ] = 0 and δ m
[ k ] = 0 .
Therefore, the ratio γ m
[ k ] of the reconstructed signal’s spectrogram energy to the clean audio’s satisﬁes:
[Equation]
where γ m [ k ] is ranged within [0 , 1] .
It means that the phase perturbations δ m − 1 [ k ] , δ m [ k ] always cause loss of the reconstructed magnitude spectrogram, which is called ‘energy dissipation‘.
Till now, we reveal the essence of phase-oriented audio adversarial attack: it slightly dissipates the energy patterns in the spectrogram and constructs the adversarial examples with the loss function in Equation 2 in the meantime.
3. O PTIMIZATION WITH E NERGY D ISSIPATION
Simply minimizing the l p distortion of (1 − γ ) is a possible solution.
However, the l p based metrics yield more audible perturbations in the practical attacks due to the differences between l p distance and human perceptibility.
In this paper, we propose a weighted loss function to constrain phase perturbations in an imperceptible range.
Since the perturbations of our attack always cause energy loss to the spectrogram, this function should dissipate the energy that is crucial for the classiﬁcation model and meanwhile preserve the energy that is important for human perception.
According to the characteristics of human auditory systems, we divide the spectrogram into three parts: 1) the signals that are important for human perception; 2) the signals that are relatively unimportant for human auditory systems; 3) the inaudible signals that are below the threshold in quiet; For 1) and 2), we manage to constrain their γ in different levels β 1 and β 2 , respectively.
For 3), we do not constrain the corresponding γ since the energy dissipation in these areas can not be perceived.
We divide quiet signals 3) following Lin & Abdulla (2015).
The signals 1) are the local maxima that have the largest magnitude within 3 bins in the spectrum frames (in order to preserve the signals in high frequency we do not adopt Bark scale) and the signals 2) are the rest of the signals in the spectrogram.
Weighted Loss function. We formulate the construction of imperceptible phase-oriented adver- sarial examples as minimizing the following weighted loss function:
[Equation]
N − 1 (cid:88) k = I 2 ( max { β 2 − γ m [ k ] , 0 } ) , (11) where I 1 and I 2 are the indicator functions that are equal to 1 when the γ m [ k ] is in the corresponding part in the spectrogram and α 1 = 1 , α 2 = 0 .
05 are trade-off parameters.
We initialize the parameter λ
as 1 × 10 − 4 and update λ every 10 iterations according to the performance of the attack.
Since the hinge loss is utilized to constrains the dissipation ratio γ of the part 1) and 2) above the levels β 1 = 0 .
98 and β 2 = 0 .
90 , the tiny energy losses caused by the phase perturbations will be perceived as almost inaudible volume differences instead of the echoes or noises;
E XPERIMENTAL S ETUP
Datasets. We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015), which contains 1000 hours of 16KHz English speech derived from audiobooks.
We randomly selectaudio examples from the test-clean dataset as source examples, and 100 separate transcriptions from the same dataset to be the targeted transcriptions following the experimental setup with (Carlini & Wagner, 2018; Qin et al., 2019).
We ensure that each target transcription is around the same length as the original transcription because it is unrealistic to perturb a short audio sample (e.g.,words) to have a much longer transcription (e.g., 30 words).
Examples of the original and targeted transcriptions are available in the supplementary material.
Implementation Details.
We successfully implement our PhaseFool and several state-of-the-art attack baselines against a pretrained DeepSpeech 2 model (Amodei et al., 2016) and an end-to-end speech recognition model based on the low-rank Transformer architecture following Winata et al. The DeepSpeech 2 model is an open-source ASR system with Connectionist Temporal
Classiﬁcation (CTC) method (Graves et al., 2006) and LSTM as its main components, which takes the log spectrograms as the input features.
And the low-rank Transformer-based ASR system that takes log-mel spectrograms as its input features is composed of lightweight transformer architecture and is trained with the cross-entropy loss.
Since the feature extraction is differentiable, our phase attack can be easily extended to other ASR systems.
All experiments are carried out on 1 Nvidia GTX 3080 GPU with 10 GB memory.
We use Adam (Kingma & Ba, 2015) as the optimizer with β 1 = 0 .
9 , β 2 = 0 .
98 , (cid:15) =
10 − 9 and the learning rate is set as 5 × 10 − 3 .
Threat Model.
As is done in most previous works, we assume a white-box threat model, i.e. the adversary is allowed to compute gradients through the model in order to generate adversarial examples.
Considering that current white-box audio adversarial attacks (Carlini & Wagner, 2018;
Yakura & Sakuma, 2019; Qin et al., 2019) have been widely applied into the black-box attacks (Yuan et al., 2018; Chen et al., 2020) as the base generation methods and shown great improvements in terms of robustness or imperceptibility, our work only focus on investigating the white-box model, as is done in most previous works.
Evaluation Metrics.
Word error rate (WER). We calculate the WER metric which is deﬁned as WER =
S + D
+ I N × 100% , where S , D , and I are the numbers of substitutions, deletions, and insertions, respectively, and N is the total number of words.
Particularly, we choose the targeted transcription as the label for calculating the WER in the experiments 3 .; 2) Success Rate (SR).
SR is the ratio of examples that can be successfully recognized as the malicious target texts by ASR systems.
Here, “successfully recognized” means that the adversarial example is recognized as the malicious target transcription;
R ESULTS
In this section, we evaluate the performance of PhaseFool in terms of imperceptibility, generation speedup, robustness against simulated environmental distortions and state-of-the-art defenses.
The extra analysis and ablation studies can be found in Appendix A.
5. I MPERCEPTIBILITY A NALYSIS
Experimental Design.
We construct 100 full-sentence targeted adversarial examples sampled from the LibriSpeech test-clean dataset for our PhaseFool attack.
For comparison, we then generate adversarial examples using the prior state-of-the-art works as baselines, including: 1) C&W’s attack (Carlini & Wagner, 2018) , a two-stage adversarial attack for audio based on CTC loss and l pnorm; 2) IPC attack (Zhang et al., 2020) , a new Iterative Proportional Clipping (IPC) algorithm that generates robust audio adversarial examples with temporal dependency; 3) Qin’s attack (Qin et al., 2019) , the imperceptible attack that leverages the psychoacoustic principle of auditory masking.
We ensure that all attacks have a 100% success rate and the target transcription is consistent among the experiments to exclude other interference factors.
The user study is conducted to verify the imperceptibility of these attacks.
We recruit 24 users online and ask them to tell us if the examples have any background noise (e.g., static, echoing, people talking in the background) following (Qin et al., 2019), which is the most representative experiment of how an attack would work in practice.
The users are asked to listen to each sample with headphones on, and answer questions about a set of audio samples.
In all experiments, users can replay the audio samples multiple times. [Table] [Table]
Result Analysis.
As shown in Table 1, 31% of the users believe that the clean audio samples contain noises.
Only 43% and 46% of the users believe that the adversarial examples generated by the PhaseFool against the DeepSpeech 2 system and low-rank Transformer system contain some noises, which is similar to Qin’s attack. Meanwhile, PhaseFool and Qin’s attack greatly surpass the l p -norm based C&W’s attack and IPC attack.
It can be seen that our phase-oriented attack shows comparable imperceptibility with the attacks that leverage the psychoacoustic principle of auditory masking.
However, our attack needs fewer steps and only one optimization stage to generate imperceptible audio adversarial examples rather than complex two optimizations stages in Qin et al.
We will further explore the generation speed of these attacks in the next subsection.
[Table] [Table]
5. G ENERATION S PEEDUP
We evaluate the generation speed of PhaseFool compared with several state-of-the-art audio adversarial attacks in the previous experiment.
We generate 100 adversarial examples for each algorithm with the 100% success rate and ensure that the original audio samples and the target transcriptions are consistent among the experiments to exclude other interference factors.
The lengths of the selected audio range from 3 seconds to 10 seconds (5.18 seconds on average) and the detailed analysis of the selected samples can be found in the Appendix A.4.
As shown in Table 2, it can be seen that PhaseFool can generate a successful audio adversarial example for the low-rank Transformer system in around 30 seconds on average, which is a 9.24x speedup compared with the imperceptible counterpart Qin’s attack (Qin et al., 2019). Moreover, for the DeepSpeech 2 system, Qin’s attack usually requires over 20 minutes to generate an adversarial example while our PhaseFool only needs around 3 minutes.
Although the constraints used in the baseline attacks can improve their imperceptibility, these constraints signiﬁcantly slow down the convergence rates of the optimization process.
Besides, the two-stage optimizations make C&W’s attack and Qin’s attack requires even more steps to generate adversarial examples.
Empirically, the computational complexity of audio adversarial attack mainly depends on iteration steps and the attacked model f ( · ) . Since our PhaseFool is a one-stage attack without complex computations and only requires 700 steps to obtain a successful adversarial example, the generation speed of PhaseFool is much faster than the baseline attacks.
5. R OBUSTNESS A NALYSIS
We conduct the robustness analysis to explore whether the adversarial examples constructed by
PhaseFool can remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses.
We generate 100 adversarial examples only for the DeepSpeechsystem in this experiment.
We adopt the Expectation over Transformation (EOT) algorithm
(Athalye et al., 2018) to simulate transformations caused by environmental distortions and defenses and incorporate the transformations into the generation process following the previous works (Yakura &
Sakuma, 2019; Qin et al., 2019).
The details of the experimental setup can be found in Appendix
A.1.
The results are shown in Table 3 and Table 4.
Robustness Against Defenses.
To compare the robustness of attacks against adversarial audio defenses, PhaseFool and the baseline attacks are applied with the following defenses: 1) Temporal Dependency (TD) defense (Yang et al., 2019) , the defense that uses the temporal dependency in audio data to gain discriminate power against audio adversarial examples; 2) Dropout
[Table]
defense (Jayashankar et al., 2020) , the defense based on the uncertainty introduced by dropout in neural networks; 3) Downsample defense (Yang et al., 2019) , a common defense that down-samples a band-limited audio ﬁle without sacriﬁcing the quality of the recovered signal while mitigating the adversarial perturbations in the reconstruction phase.
4) Noise defense , a common defense that introduces white Gaussian noise to disable the adversarial perturbations.
We follow the same experimental procedures as the corresponding defenses and adopt their evaluation metric: the area under curve (AUC) score.
As shown in Table 3, the AUC scores for PhaseFool’s adversarial examples against the four defenses are mostly distributed between 57% ∼ 68%, which means that the classiﬁers of these defenses have poor performance on our adversarial examples.
However, although
IPC attack shows great performance against the TD defense, its adversarial examples are easily detected by the dropout defense and the noise defense.
Besides, Qin’s attack is also not robust enough with the noise defense.
These results demonstrate the robustness of our phase-oriented adversarial examples against several sophisticated defenses.
Robustness Against Environmental Distortions.
To demonstrate the robustness of PhaseFool against environmental distortions, we evaluate the attack success rate (SR) and the word error rate (WER) of adversarial examples generated by PhaseFool and baseline attacks in the simulated environments that contain reverberations and white Gaussian noise following the previous works (Qin et al., 2019; Yakura & Sakuma, 2019).
Besides, to investigate whether the adversarial examples of
PhaseFool can still remain imperceptible to the human ear in these scenarios, we also conduct the user study with the same experimental settings in Subsection 5.1.
As shown in Table 4, the adversarial examples generated by PhaseFool achieve approximately 40% attack success rate in the reverberant and noisy environments.
However, the imperceptibility of Qin’s attack signiﬁcantly drops in the reverberation environment, while the imperceptibility of our PhaseFool does not degrade so much relatively.
Since the phase-oriented adversarial examples sound more like echoes or blurred pronunciation, our PhaseFool shows better imperceptibility in the attacks for the environments that contain distortions.
C ONCLUSIONS
In this work, we propose PhaseFool, a phase-oriented audio adversarial attack, which leverages energy dissipation to efﬁciently construct imperceptible audio adversarial examples.
Leveraging the phenomenon of energy dissipation, we bring new directions for audio adversarial attacks.
Moreover, we propose a weighted loss function to improve the imperceptibility of phase-oriented audio adversarial examples.
Experiments on the LibriSpeech dataset (Panayotov et al., 2015) demonstrate that our PhaseFool signiﬁcantly surpasses current state-of-the-art algorithms based on l p -norm metrics and shows comparable imperceptibility with the attacks based on the psychoacoustic principle of auditory masking.
Besides, PhaseFool achieves a 6.64x generation speed-up over current state-ofthe-art imperceptible counterparts.
Moreover, in the robustness evaluations, the adversarial examples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses.
For future work, we will continue to improve the imperceptibility of the phase-oriented audio adversarial examples, and apply PhaseFool to black-box settings for practical attacks at commercial ASR systems.
We will also investigate PhaseFool’s capacity in online learning scenarios.
A A PPENDIX A. T HE D ETAILED S ETUP OF R OBUSTNESS E VALUATION
In the robustness evaluation, we adopt the Expectation over Transformation (EOT) algorithm
(Athalye et al., 2018) to simulate transformations caused by environmental distortions and defenses and incorporate the transformations into the generation process.
Then we transcribe the newly obtained audio adversarial examples and ﬁnally calculate SR and WER to evaluate their robustness.
A.1. T HE S ETUP OF E VALUATION AGAINST D EFENSES
For TD defense, we randomly select the ﬁrst k portion of the adversarial examples ( k is chosen from { 1 3 , 1 2 , 2 3 , 1 1 } ) and adopt the EOT algorithm to make the generated adversarial examples adaptive to the TD defense.
In the defense stage, we choose k = 12 and calculate the AUC scores of WER following (Yang et al., 2019).
For Dropout defense, we randomly choose the dropout rate p of ASR systems from { 0 . 00 , 0 . 05 , 0 . 10 , 0 . 15 } and adopt the EOT algorithm to make the generated adversarial examples adaptive to the Dropout defense.
In the defense stage, we set the defense dropout rate as p = 0 .and
calculate the AUC scores of the SVM-F classiﬁer following (Jayashankar et al., 2020).
For Downsample defense, we conduct the EOT algorithm by performing the attack on the downsampled elements of the original audio sequence.
In the attack stage, we randomly choose whether to downsample the adversarial examples from 16000 Hz to 8000 Hz.
In the defense stage, we downsample the audio from 16000 Hz to 8000 Hz and calculate the AUC scores of WER following (Yang et al., 2019).
For Noise defense, we randomly add White Gaussian noise with the boundary of ±(cid:52) to the generation process of the adversarial examples.
Since the input audio is normalized, here we randomly choose the (cid:52) from { 0 . 01 , 0 . 02 , 0 . 03 } in the attack stage.
In the defense stage, we choose (cid:52) = 0 .and calculate the AUC scores of WER.
A.1. T HE S ETUP OF E VALUATION AGAINST E NVIRONMENTAL D ISTORTIONS
For the reverberant scenarios, we use an acoustic room simulator that applies the classic Image
Source Method introduced in (Scheibler et al., 2018) to create the room impulse response r .
We set the room conﬁgurations (room dimension, source audio and target microphone’s locations, and reverberation time) following Qin et al.
Then, the generated room impulse response r is convolved with the clean audio to create the speech with reverberation.
In the attack stage, we generate 1000 random room conﬁgurations sampled from the distribution as the training room set.
And the test room set includes another 100 random room conﬁgurations sampled from the same distribution for the 100 audio adversarial examples generated by different attacks.
For the noisy scenarios, we randomly add White Gaussian noise with the boundary of ±(cid:52) to the generation process of the adversarial examples.
Since the input audio is normalized, here we randomly choose the (cid:52) from { 0 . 01 , 0 . 02 , 0 . 03 } .
In the test stage, we choose (cid:52) = 0 . 02 .
Note that this experiment aims at evaluating the attack success rate and imperceptibility of attacks in the noisy scenarios, which is different from the evaluation against noise defense.
A. A BLATION S TUDIES
We evaluate the generation speed and imperceptibility in the ablation studies to explore the inﬂuences of the weighted loss function proposed in Equation 11.
As shown in Table 5 and Table 6, when we do not constrain the phase perturbations and optimize the example until it is successful, PhaseFool only needs 305 and 212 steps on average to construct an audio adversarial example for the DeepSpeech 2 system and low-rank Transformer system, correspondingly.
It can be seen that the examples without constraints are even more imperceptible than the l p -norm based attacks that need thousands of steps. But these examples introduce more noise than PhaseFool with the weighted hinge loss in Equation 11, which demonstrates the effectiveness of our proposed loss function.
However, when we constrain the phase perturbations with l 2 loss, the noises in adversarial examples are even louder than the examples generated without constraint.
It demonstrates the importance of curbing the energy dissipation ratio γ above the weighted level for different types of regions with the loss function proposed in Subsection 3.3. [Table] [Table]
A. I NTERPRETATION OF A TTACKS
For further investigating the interpretation of different attacks, we present the pixel-wise difference of the adversarial examples in the mel-spectrogram domain to visualize the learned adversarial perturbations in Figure 3.
The perturbations of C&W’s attack are disorderly distributed in the melspectrogram while Qin s attack and IPC attack tend to hide perturbations in the utterance regions.
Our PhaseFool attack provides additional insights about the vulnerability of ASR models, i.e. the inﬂuences of different regions in the spectrogram for the audio adversarial attack.
For example, as shown in Figure 3, the perturbations generated by our PhaseFool attack mainly occur on the positions around the harmonics in the mel-spectrogram.
The perturbation positions reveal the fragile part of the spectrogram for the ASR system’s prediction.
With the l 2 loss function, PhaseFool signiﬁcantly perturbs the mel-spectrogram regions that are important for human perception and cause audible echoes in the audio adversarial examples.
When we remove the constraints of PhaseFool, the magnitude of perturbations in the mel-spectrogram grows obviously, which demonstrates the effectiveness of the proposed weighted loss. spectrogram.
A. L ENGTH ANALYSIS
In all experiments, we randomly select 100 audio examples from the test-clean set of the LibriSpeech dataset (Panayotov et al., 2015) as source examples, and 100 separate transcriptions from the same dataset to be the target transcriptions following the experimental setup with (Carlini & Wagner, 2018; Qin et al., 2019). We ensure that each target transcription is around the same length as the original transcription because it is unrealistic to perturb a short audio sample (e.g., 10 words) to have a much longer transcription (e.g., 30 words). Figures 4 illustrates the comparison of the text length between original transcriptions and target transcriptions.
And Figures 5 shows the lengths ofselected audio samples in the experiments. A. A NALYSIS ON THE C ONVERGENCE OF L OSS AND L EVENSHTEIN D ISTANCE
Here we analyze the CTC loss and Levenshtein distance convergence processes when we attack the DeepSpeech 2 model with PhaseFool.
We randomly select 20 samples in the experiments. As shown in Figure 6, the CTC loss of most cases quickly converges to a value close to 0. The Levenshtein distance in Figure 6 converges to 0 on all cases before 700 iteration steps.
Note that when the value of Levenshtein distance becomes 0, the adversarial example is successfully generated.