P HASE F OOL : P HASE - ORIENTED A UDIO A DVERSARIAL E XAMPLES VIA E NERGY D ISSIPATION
Anonymous authors
Paper under double-blind review
A BSTRACT
Audio adversarial attacks design perturbations onto inputs that lead an automatic speech recognition (ASR) model to predict incorrect outputs.
Current audio adversarial attacks optimize perturbations with different constraints (e.g. l p -norm for waveform or the principle of auditory masking for magnitude spectrogram) to achieve their imperceptibility.
Since phase is not relevant for speech recognition, the existing audio adversarial attacks neglect the inﬂuence of phase spectrogram.
In this work, we propose a novel phase-oriented algorithm named PhaseFool that can efﬁciently construct imperceptible audio adversarial examples with energy dissipation.
Speciﬁcally, we leverage the spectrogram consistency of short-time Fourier transform (STFT) to adversarially transfer phase perturbations to the adjacent frames of magnitude spectrogram and dissipate the energy that is crucial for ASR systems.
Moreover, we propose a weighted loss function to improve the imperceptibility of PhaseFool.
Experimental results demonstrate that PhaseFool can inherently generate full-sentence imperceptible audio adversarial examples with the 100% targeted success rate within 500 steps on average (9.24x speed-up over current state-of-the-art imperceptible counterparts), which is veriﬁed through a human study 1 .
Most importantly, our PhaseFool is the ﬁrst to exploit the phaseoriented energy dissipation in the audio adversarial examples rather than add perturbations on the audio waveform like most previous works.
I NTRODUCTION
Adversarial examples are speciﬁcally designed instances that cause a machine-learning algorithm to produce a misclassiﬁcation (Szegedy et al., 2013; Biggio et al., 2013).
Since researchers have successfully created adversarial examples to make automatic speech recognition (ASR) system transcribe the audio as any different sentence (Carlini & Wagner, 2018; Vaidya et al., 2015; Carlini et al., 2016), audio adversarial attacks have become more and more popular.
Those attack algorithms usually optimize perturbations added on the benign audio waveform with different constraints based on l p -norm (Carlini & Wagner, 2018; Zhang et al., 2020).
Although the perturbations introduced is small in magnitude, noises in the audio adversarial examples are usually perceptible.
Later, algorithms based on psychoacoustic principle (Qin et al., 2019; Sch¨onherr et al., 2019) successfully generate effectively imperceptible adversarial examples by hiding the adversarial perturbations to regions of magnitude spectrogram where it will not be heard by a human.
In current imperceptible targeted audio adversarial attacks, only waveform and magnitude spectrogram are taken into account leaving phase and its nature unexplored.
Since the structure of natural speech is highly complicated, these two types of algorithms face several challenges:
• Waveform-oriented attacks are usually not imperceptible enough.
Although constraining perturbations with l 2 -norm or l ∞ -norm (Carlini & Wagner, 2018; Liu et al., 2020), these attacks introduce audible noises to the adversarial examples.
Later works focused on the temporal dependency (Zhang et al., 2020) successfully reduce human-perceptible noises but their perturbations are still audible, due to the uncorrelated relationship between psychoacoustic perception and l p -norm metrics.
[Figure] • The optimization of attacks based on the psychoacoustic principles is highly timeconsuming and hard to converge.
Due to the determining inﬂuence of magnitude spectrogram on human perception, the attack that leverages the psychoacoustic principle of auditory masking (Qin et al., 2019) successfully generates imperceptible audio adversarial examples.
However, it needs two-stage optimization where the ﬁrst stage of optimization focuses on ﬁnding a relatively small perturbation to fool the network (as is done in prior work (Carlini & Wagner, 2018)) and the second stage makes the adversarial examples imperceptible.
The search process of the optimal balance between hiding the adversarial perturbations in the inaudible regions and fooling the ASR systems is empirically difﬁcult.
Generating a single full-sentence adversarial example requires approximatelyiterations until it ﬁnally converges to an imperceptible solution. This research
is motivated by a psychoacoustic principle that phase manipulation in one channel audios imposes tiny effects on human perception
(Vary & Eurasip, 1985)
Besides, although phase is not relevant for speech recognition the inherent spectrogram consistency
of
short-
time Fourier transform (STFT
) reveals
that
the
adversarial
perturbations
in a frame of phase spectrogram will transfer to adjacent frames of magnitude spectrogram (Grifﬁn & Lim, 1984).
And these observation motivates us to exploit phase spectrogram for generating audio adversarial examples.
Considering the relationship between phase spectrogram and the spectrogram consistency of STFT, in this work, we propose a novel phase-oriented algorithm called PhaseFool that efﬁciently generates imperceptible audio adversarial examples via energy dissipation.
We prove that the phaseoriented perturbations always reduce the magnitude of the spectrogram in the corresponding areas via spectrogram consistency, which is called energy dissipation.
Leveraging such a phenomenon, we bring new directions for audio adversarial attacks and are able to perform a bi-directional attack that dissipates the crucial energy for ASR models and meanwhile, construct the perturbations for the adversarial attack.
To the best of our knowledge, we are the ﬁrst to consider phase spectrogram in the audio adversarial attacks.
Our contributions can be summarized as follows:
• We leverage the spectrogram consistency of STFT to construct phase-oriented audio adversarial examples, which make further steps towards imperceptibility and efﬁciency.
• We further investigate the relationship between phase perturbations and magnitude spectrogram, which is called energy dissipation.
Instead of the common l p distance metrics or magnitude-oriented metrics based on the psychoacoustic principle, we propose a weighted loss function based on the phenomenon of energy dissipation to improve the imperceptibility of phase-oriented audio adversarial examples.
• Further investigating properties of PhaseFool, we observe additional insights about the vulnerability of ASR systems.
It brings a better interpretation of audio adversarial attacks,
i
e. the perturbation positions reveal that which part of one magnitude spectrogram is important but also fragile for the corresponding ASR system.
We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015) and implement a successful attack against a Transformer-based ASR system (Winata et al., 2020b).
The results show that in terms of imperceptibility, PhaseFool greatly surpasses current state-of-the-art algorithms based on l p -norm metrics and shows comparable imperceptibility with the attacks based on the psychoacoustic principle of auditory masking (veriﬁed by a human study).
Furthermore, it can generate full-sentence audio adversarial examples with the 100% targeted success rate in around 30 seconds and achieves 9.24x speedup over the current state-of-the-art imperceptible counterpart.
We attach the adversarial examples in the experiments and the implementation for reproducibility in the supplementary material.
B ACKGROUNDS
In this section, we brieﬂy overview the background of this work, including audio adversarial examples and phase-aware audio signal processing.
Audio Adversarial Examples.
After the early works (Gong & Poellabauer, 2017; Ciss ´ e et al., 2017) towards neural networks based ASR systems successfully generate untargeted adversarial examples that produce incorrect but arbitrary transcriptions.
Recently, more and more efforts have been made in the audio domain (Alzantot et al., 2018; Carlini & Wagner, 2018).
Among them, one of the most outstanding studies is the iterative optimization-based attack (Carlini & Wagner, 2018), which introduces a two-stage attack and directly generates targeted adversarial examples using a gradient-descent minimization method.
Nevertheless, the generated adversarial examples tend to include noticeable noise.
Thus, two recent approaches (Qin et al., 2019; Sch ¨ onherr et al., 2019) introduce hearing thresholds for designing inaudible perturbations by curbing the signal variation below the threshold of human perception.
However, these imperceptible attacks require two stages of optimization and suffer from slow convergence rates.
Phase-aware Audio Signal Processing.
The interest of utilizing phase information in the ﬁeld of audio signal processing is constantly increasing (Mowlaee et al., 2016).
The STFT phase has been widely used in signal enhancement (Miyahara & Sugiyama, 2014; Choi et al., 2019; Hu et al., 2020) and signal reconstruction (Mowlaee & Kulmer, 2015).
The phase in the harmonic domain is used in the speech synthesis (Degottex & Erro, 2014; Ai & Ling, 2020) and the phase in perceptual bands is used in speech watermarking (Wang et al., 2021).
The important information in the phase brings positive contributions to the above ﬁelds.
Nevertheless, the existing audio adversarial attacks directly neglect the phase spectrogram, due to the fact that phase is not relevant for speech recognition.
In this work, instead of introducing perturbations to the audio waveform like most of previous works, we propose to leverage the spectrogram consistency and generate phase-oriented perturbations for constructing imperceptible audio adversarial examples.
P HASE - ORIENTED A UDIO A DVERSARIAL E XAMPLES VIA E NERGY D ISSIPATION
In this section, we ﬁrst deﬁne the preliminaries of the phase-oriented audio adversarial attack.
Then we investigate the relation between phase spectrogram and magnitude spectrogram under spectrogram consistency and prove that phase perturbations always reduce the magnitude of the spectrogram in the corresponding areas under spectrogram consistency, which is called energy dissipation (as is illustrated in Figure 1).
Finally, we design a novel phase-oriented loss function based on energy dissipation for constructing imperceptible audio adversarial examples.
[Figure]
3. P RELIMINARIES OF P HASE - ORIENTED A UDIO A DVERSARIAL A TTACK
Denote the ASR classiﬁcation model as f : { X } → Y , with X ∈ [0 , 1] t × c being the audio space.
Y ∈ { 0 , 1 , ..., k } n × 1 is the sequence of the k-class output space corresponding to the transcription.
Speciﬁcally, denote x ∈ X as the input audio, y ∈ Y as the ground-truth label, and y t ∈ Y as the target transcription, and L net ( f ( x ) , y t ) be a loss function for the threat model that is minimized when f ( x ) =
y t .
Then the general formulation of adversarial attack for ASR system is:
[Equation]
(1) where L net is the loss function used in the ASR model, λ > 0 is the trade-off parameter between the imperceptibility and adversarial loss (in some formulations λ = 0), (cid:15) control the maximum perturbation value, and p could be speciﬁed as different values, according to the attacker’s requirement (e.g. p
= 2 in C&W’s attack (Carlini & Wagner, 2018)).
Denote φ ( · ) as the function of the STFT and φ − 1 ( · ) as the function of the inverse STFT and
X = φ ( x ) = | X | · e iθ x , where | X | and θ x = ∠ X representing the spectral magnitude and the instantaneous phase spectrogram, respectively. Denote T ( X ) =
φ ( φ − 1 ( X )) . We assume that window function w , window length N and window shift R is ﬁxed and a signal can be perfectly reconstructed from its STFT spectrogram through the inverse STFT in this setting.
We propose to generate perturbations on the instantaneous phase spectrogram and reconstruct the audio adversarial examples with the inverse STFT as the following formulation:
[Equation]
where E ( · ) is the feature extraction function for the ASR model and the φ − 1 ( | X | · e iθ x · e jδ ) is the generated audio adversarial example.
As shown in Figure 2, the spectrogram X (cid:48)
=
| X | · e iθ x · e jδ perturbed by P ( X ) is usually located outside the STFT spectrogram domain.
Considering the phenomenon of spectrogram consistency (Le Roux et al., 2010), it is easy to prove the following two conclusions: 1) | X (cid:48) | (cid:54) = |T
( X (cid:48) )
| mathematically; 2) φ − 1 ( X (cid:48) ) and φ − 1 ( T ( X (cid:48) )) represent the same time-domain audio signals.
Since the STFT and iSTFT functions are differentiable, these conclusions enable us to adversarially manipulate the reconstructed magnitude spectrogram by introducing perturbations to the original phase spectrogram, which is called phase-oriented audio adversarial attack.
Then we detail how the perturbations on a frame of phase spectrogram inﬂuence the adjacent frames of corresponding magnitude spectrogram and reveal the energy dissipation nature of the phase-oriented audio adversarial attack.
3. R ELATION A NALYSIS BETWEEN PHASE AND MAGNITUDE
This subsection mainly discusses the essence of the relationship between phase perturbations and magnitude spectrogram frames.
Reconstructing the time-domain signals after introducing perturbations to the phase spectrogram includes an overlap-add process.
For simplicity, we assume that the window shift length R is equal to half of the window length N , which means the adjacent frames have 50% overlapped signals.
As shown in Figure 2, the signals x m − 1 ( n (cid:48) ) and x m ( n ) in adjacent frames m − 1 and m overlap, where n (cid:48) and n represent the time indices and n (cid:48)
= n + N 2 .
x m − 1 ( n (cid:48) ) and x m ( n ) are obtained by an inverse Fourier transform as the following:
[Equation]
where j is the imaginary unit, X m − 1 [ k ] and X m [ k ] are spectrogram samples in frames m − 1 and m at k -th frequency bin.
Then we introduce perturbations δ m − 1 , δ m to the frame m − 1 , m and the corresponding signals become:
[Equation]
Since the human auditory system is relatively less sensitive to the tiny time delay caused by phase rotation, the phase perturbations are inherently more imperceptible than the perturbations on the audio waveform.
Then we further investigate how phase perturbations affect the magnitude spectrogram.
The reconstructed signal ˜ x m ( n ) in the frame m is obtained as:
[Equation]
We substitute X m − 1 [ k ] and X m [ k ] with the Fourier transform of x m − 1 ( n (cid:48) ) and x m ( n ) , then the overlapping area of ˜ x m ( n ) in equation 7 will become:
[Equation]
The overlapping area of the x m − 1 ( n (cid:48) ) and x m ( n ) in Figure 2 satisfy the following relationship based on spectrogram consistency:
[Equation]
With equation
[Equation]
we can deduce the ﬁnal formulation for the reconstructed signal x m ( n ) as following:
[Equation]
where the clean audio can be represented by setting δ m − 1 [ k ] = 0 and δ m
[ k ] = 0 .
Therefore, the ratio γ m
[ k ] of the reconstructed signal’s spectrogram energy to the clean audio’s satisﬁes:
[Equation]
where γ m [ k ] is ranged within [0 , 1] . It means that the phase perturbations δ m − 1 [ k ] , δ m [ k ] always cause loss of the reconstructed magnitude spectrogram, which is called ‘energy dissipation‘.
Till now, we reveal the bi-directional essence of phase-oriented audio adversarial attack: it dissipates the critical energy for the classiﬁcation model and constructs the adversarial perturbations in the meantime.
3. O PTIMIZATION WITH E NERGY D ISSIPATION
Simply minimizing the l p distortion of (1 − γ ) is a possible solution.
However, the l p based metrics yield more audible perturbations in the practical attacks due to the differences between l p distance and human perceptibility.
In this paper, we propose a weighted loss function to constrain phase perturbations in an imperceptible range.
Since the perturbations of our attack always cause energy loss to the spectrogram, this function should dissipate the energy that is crucial for the classiﬁcation model and meanwhile preserve the energy that is important for human perception.
According to the characteristics of human auditory systems, we divide the spectrogram into three parts: 1) the signals that are important for human perception; 2) the signals that are relatively unimportant for human auditory systems; 3) the inaudible signals that are below the threshold in quiet; For 1) and 2), we manage to constrain their γ in different levels β 1 and β 2 , respectively.
For 3), we do not constrain the corresponding γ since the energy dissipation in these areas can not be perceived.
We divide quiet signals 3) following Lin & Abdulla (2015).
The signals 1) are the local maxima that have the largest magnitude within 3 bins in the spectrum frames (in order to preserve the signals in high frequency we do not adopt Bark scale) and the signals 2) are the rest of the signals in the spectrogram.
Weighted Loss function We formulate the construction of imperceptible phase-oriented adversarial examples as minimizing the following weighted loss function:
[Equation]
where I 1 and I 2 are the indicator functions that are equal to 1 when the γ m [ k ] is in the corresponding part in the spectrogram and α 1 = 1 , α
2 = 0 . 05 are trade-off parameters. We initialize the parameter λ as 1 × 10 − 4 and update λ every 10 iterations according to the performance of the attack.
Since the hinge loss is utilized to constrains the dissipation ratio γ of the part 1) and 2) above the levels β 1 = 0 .
98 and β 2 = 0 .
90 , the tiny energy losses caused by the phase perturbations will be perceived as almost inaudible volume differences instead of the echoes or noises;
We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015), which containshours of 16KHz English speech derived from audiobooks.
The train-clean-100 and train-cleandatasets are used to train the threat ASR model.
Then we randomly select 100 audio examples from the test-clean dataset as source examples, and 100 separate transcriptions from the same dataset to be the targeted transcriptions.
We ensure that each target transcription is around the same length as the original transcription because it is unrealistic to perturb a short audio sample (e.g., 10 words) to have a much longer transcription (e.g., 30 words).
Examples of the original and targeted transcriptions are available in the supplementary material.
[Table]
4. S ETTINGS AND M ETRICS
Implementation Details. We implement an end-to-end speech recognition model based on the low-rank transformer architecture following Winata et al.
(2020a).
The transformer model consists of four encoder layers and four decoder layers with the dim inner of 2048, dim model of 512, and dim emb of 512.
The number of attention heads is set to 8.
The encoder module adopts a VGG net (Simonyan & Zisserman, 2015) with a 6-layer CNN architecture as the frontend feature extractor.
The VGG net consists of convolutional layers that learn a universal representation of the spectrogram and generate input embedding.
Since the feature extraction is differentiable, our phase attack can be easily extended to the ASR models that take other acoustic features as input.
All experiments are carried out on 1 Nvidia GTX 3080 GPU with 10 GB memory.
We use Adam (Kingma & Ba, 2015) as the optimizer with β 1 = 0 .
9 , β 2 = 0 .
98 , (cid:15) =
10 − 9 and the learning rate is set as 5 × 10 − 3 .
Threat Model. As is done in most previous works, we assume a white-box threat model, i.e. the adversary is allowed to compute gradients through the model in order to generate adversarial examples.
Considering that there are many ways to convert a white-box model to a black-box model Chen et al.
(2020) and to improve the robustness against distortions Yakura & Sakuma (2019);
Liu et al.
(2020) and defencesZhang et al.
(2020), our work focus on the white-box model, as is done in most previous works.
Evaluation Metrics.
We calculate the Word error rate (WER) metric which is deﬁned as WER =
S + D
+ I N × 100% , where S , D , and I are the numbers of substitutions, deletions, and insertions, respectively, and N is the total number of words.
Particularly, we choose the target transcription as the label for calculating the WER in the experiments.
R ESULTS
In this section, we evaluate the performance of PhaseFool in terms of imperceptibility, generation speedup, interpretation of attacks and ablation studies.
5. I MPERCEPTIBILITY A NALYSIS
Experimental Design. We construct 100 full-sentence targeted adversarial examples sampled from the LibriSpeech test-clean dataset for our PhaseFool attack.
For comparison, we then generate adversarial examples using the prior state-of-the-art works as baselines, including: 1) C&W’s attack (Carlini & Wagner, 2018) , a two-stage adversarial attack for audio based on CTC loss and l pnorm; 2) IPC attack (Zhang et al., 2020) , a new Iterative Proportional Clipping (IPC) algorithm that generates robust audio adversarial examples with temporal dependency; 3) Qin’s attack (Qin et al., 2019) , the imperceptible attack that leverages the psychoacoustic principle of auditory masking.
We ensure that all attacks have a 100% success rate and the target transcription is consistent among the experiments to exclude other interference factors.
The user study is conducted to verify the imperceptibility of these attacks.
We recruit 24 users online and give them one of the two types of questions following (Qin et al., 2019), including: 1) clean or noisy, we give users a set of audio samples and ask them to tell us if they have any background noise (e.g., static, echoing, people talking in the background), which can represent the imperceptibility of attacks in practice; 2) identical
or not, we present users with two audio samples and ask them if the audio samples are identical, or if there are any differences, which is the most challenging experiment.
The users are asked to listen to each sample with headphones on, and answer questions about a set of audio samples (one user can only answer one type of the questions).
In all experiments, users can replay the audio samples multiple times when they are unsure of the answer. Result Analysis. As shown in Figure 1, 33% ∼ 40% of users believe that the clean audio samples contain noises.
Only 47% and 46% of the users believe that the adversarial examples generated by the Qin’s attack and PhaseFool attack contain some noises, which greatly surpass the l p -norm based C&W’s attack and IPC attack.
In the more challenging experiment 2), there are only 8% and 9% users who can distinguish the adversarial examples of the Qin’s attack and PhaseFool attack from the clean audio, respectively.
It can be seen that our phase-oriented attack shows comparable imperceptibility with the attacks that leverage the psychoacoustic principle of auditory masking. However, our attack only needs 500 steps and one optimization stage to generate a full-sentence audio adversarial example rather than the 5000 step and two optimizations stage in Qin et al. We will further explore the generation speed of these attacks in the next subsection. [Table]
[Table]
5. G ENERATION S PEEDUP
We evaluate the generation speech of PhaseFool compared with several state-of-the-art audio adversarial attacks in the previous experiment.
We generate 100 adversarial examples for each algorithm with the 100% success rate and ensure that the original audio samples and the target transcriptions are consistent among the experiments to exclude other interference factors.
As shown in Table 2, it can be seen that PhaseFool can generate a successful full-sentence audio adversarial example in around 30 seconds on average, which is a 9.24x speedup compared with the imperceptible counterpart Qin’s attack (Qin et al., 2019).
PhaseFool is a one-stage attack without complex computations and only requires 500 steps to obtain a successful and imperceptible adversarial example, which demonstrates the superiority of the phase-oriented audio adversarial attack. 5. A BLATION S TUDY
We evaluate the efﬁciency and imperceptibility in the ablation studies to explore the inﬂuences of the weighted loss function proposed in equation 13. As shown in Table 3, when we do not constrain the phase perturbations and optimize the example until it is successful, PhaseFool only needssteps on average to construct an audio adversarial example.
It can be seen that such examples are more imperceptible than the l p -norm based attacks that need thousands of steps but introduce more noise than the psychoacoustic methods and PhaseFool with the weighted hinge loss in equation 13, which demonstrates the effectiveness of our proposed loss function. However, when we constrain the phase perturbations with l 2 loss, the noises in adversarial examples are even louder than the examples generated without constraint. It demonstrates the importance of curbing the energy dissipation ratio γ above the weighted level for different types of regions with the loss function proposed in subsection 3.3.
5. I NTERPRETATION OF A TTACKS
For further investigating the interpretation of different attacks, we present the pixel-wise difference of the adversarial examples in the mel-spectrogram domain to visualize the learned adversarial perturbations in Figure 3.
The perturbations of C&W’s attack are disorderly distributed in the melspectrogram while Qin s attack and IPC attack tend to hide perturbations in the utterance regions. Our PhaseFool attack provides additional insights about the vulnerability of ASR models, i.e. the inﬂuences of different regions in the spectrogram for the audio adversarial attack. For example, as shown in Figure 3, the perturbations generated by our PhaseFool attack mainly occur on the positions around the harmonics in the mel-spectrogram. The perturbation positions reveal that which part of the spectrogram is important but also fragile for its prediction by the DNN model.
With the l 2 loss function, PhaseFool signiﬁcantly perturbs the mel-spectrogram regions that are important for human perception and cause audible echoes in the audio adversarial examples. When we removing the constraints of PhaseFool, the magnitude of perturbations in the mel-spectrogram grows obviously, which demonstrates the effectiveness of the proposed weighted loss.
C ONCLUSIONS
In this work, we propose PhaseFool, a phase-oriented audio adversarial attack, which leverages energy dissipation to efﬁciently construct imperceptible audio adversarial examples. Leveraging such a phenomenon, we bring new directions for audio adversarial attacks and are able to perform a bi-directional attack that dissipates the crucial energy for ASR models and meanwhile, construct the perturbations for the adversarial attack. Moreover, we propose a weighted loss function to improve the imperceptibility of phase-oriented audio adversarial examples.
Experiments on the LibriSpeech dataset (Panayotov et al., 2015) demonstrate that our PhaseFool signiﬁcantly surpasses current stateof-the-art algorithms based on l p -norm metrics and shows comparable imperceptibility with the attacks based on the psychoacoustic principle of auditory masking.
Moreover, it can generate fullsentence adversarial examples with the 100% targeted success rate within 30 seconds approximately and achieves 9.24x speedup over the current state-of-the-art imperceptible counterparts.
For future work, we will continue to improve the imperceptibility of the phase-oriented audio adversarial examples, and apply PhaseFool to over-the-air and black-box settings for practical attacks.
We will also investigate PhaseFool’s capacity in online learning scenarios.