PhaseFool: Phase-oriented Audio Adversarial Examples via Energy Dissipation
Authors: Ziyue Jiang
Yi Ren
Zhou Zhao
Venue: ICLR 2022 Submitted (2022)
Keywords: Audio adversarial examples
audio adversarial attacks
automatic speech recognition

Abstract: Audio adversarial attacks design perturbations onto inputs that lead an automatic speech recognition (ASR) model to predict incorrect outputs. Current audio adversarial attacks optimize perturbations with different constraints (e.g. lp-norm for waveform or the principle of auditory masking for magnitude spectrogram) to achieve their imperceptibility. Since phase is not relevant for speech recognition, the existing audio adversarial attacks neglect the influence of phase spectrogram. In this work, we propose a novel phase-oriented algorithm named PhaseFool that can efficiently construct imperceptible audio adversarial examples with energy dissipation. Specifically, we leverage the spectrogram consistency of short-time Fourier transform (STFT) to adversarially transfer phase perturbations to the adjacent frames of magnitude spectrogram and dissipate the energy that is crucial for ASR systems. Moreover, we propose a weighted loss function to improve the imperceptibility of PhaseFool. Experimental results demonstrate that PhaseFool can inherently generate full-sentence imperceptible audio adversarial examples with the 100% targeted success rate within 500 steps on average (9.24x speed-up over current state-of-the-art imperceptible counterparts), which is verified through a human study. Most importantly, our PhaseFool is the first to exploit the phase-oriented energy dissipation in the audio adversarial examples rather than add perturbations on the audio waveform like most previous works.

Under review as a conference paper at ICLR 2022
PHASEFOOL: PHASE-ORIENTED AUDIO ADVERSAR- IAL EXAMPLES VIA ENERGY DISSIPATION
Anonymous authors Paper under double-blind review
ABSTRACT
Audio adversarial attacks design perturbations onto inputs that lead an automatic speech recognition (ASR) model to predict incorrect outputs. Current audio ad- versarial attacks optimize perturbations with different constraints (e.g. ny-norm for waveform or the principle of auditory masking for magnitude spectrogram) to achieve their imperceptibility. However, the existing audio adversarial attacks ne- glect the influence of phase spectrogram. In this work, we propose a novel phase- oriented algorithm named PhaseFool that can efficiently construct imperceptible audio adversarial examples with energy dissipation. Specifically, we leverage the spectrogram consistency of short-time Fourier transform (STFT) to adversari- ally transfer phase perturbations to the adjacent frames magnitude spectrogram and dissipate the energy patterns in the spectrogram. Moreover, we propose a weighted loss function to improve the imperceptibility of PhaseFool. Experimen- tal results demonstrate that PhaseFool can generate full-sentence imperceptible audio ad odversarial examples and achieve a 6.64x generation speed-up over current sate-of-the-aad art imperceptible counterparts (the imperceptibility verified through a human st Further robustness evaluations demonstrate that the adversarial examples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses. Most importantly, our Phasefoool is the first to exploit the phase-oriented energy dissi- pation in the audio adversarial example rather than add perturbations the audio waveform like most of the previous works.
INTRODUCTION
Adversarial examples specifically designed instances that cause machine-learning algorithm to produce a misclassification (Szegedy et al.) 2013 Biggio et al., 2013). Since researchers have successfully created adversarial examples to make automatic speech recognition (ASR) system tran- scribe the audio as any different sentence (Carlini & Wagner, 2018 Vaidya et al., 2015 Carlini et al.) 2016). audio adversarial attacks have become more and more popular. Those attack algorithms usu- ally optimize perturbations added on the benign audio waveform with different constraints based on 1y-norm (Carlini & Wagner, 20188 Zhang et al., 2020). Although the perturbations introduced is small in magnitude, noises in the audio adversarial examples are usually perceptible. Later, al- gorithms based on psychoacoustic principle (Qin et al. 2019; Schönnerr et al. (019)) successfully generate effectively imperceptible adversarial examples by hiding the adversarial perturbations to regions of magnitude spectrogram where it will not be heard by a human.
In current imperceptible targeted audio adversarial attacks, only waveform and magnitude spectro- gram are taken into account leaving phase and its nature unexplored. Since the structure of natural speech is highly complicated, these two types of algorithms face several challenges:
Waveform-oriented attacks are usually not imperceptible enough. Although constraining perturbations with 42-norm or -nom (Carlini & Wagner, 2018 Liu et al.) 2020), these attacks introduce audible noises to the adversarial examples. Later works focused on the
temporal dependency (Zhang et al. 2020) successfully reduce human-percoptible noises
'Generated audio adversarial examples can he found our demo website: htt:s::/phasefoor Jamn-ggithubb.oo//hasefooo

---

Under review as a conference paper at ICLR 2022
Clean
W&s's Attack
IPC Attack
Qin's 's Attack
PhaseFool
PhascFool w/o weighted loss
Figure 1: Visualizations of the ground-truth mel-spectrograams and the mel-spectrograms generated by several state-of-the-ar attacks and our PhaseFool. Attacks based on the ly-norm (e.g. C&W's W's at- tack (Carlini & Wagner 2018) and IPC attack (Zhang al. 2020) introduce obvious perturbations to the mel-spectrogram and cause perceptible noises. Qin s attack leverages the psychoacoustic prin- ciple of auditory masking and hides noises in the inaudible regions while our PhaseFool dissipates the energy patterns that are important for classification while less important for human perception.
but their perturbations are still audible, due to the uncorrelated relationship between psy- choacoustic perception and 1y-norm metrics.
The optimization of attacks based on the psychoacoustic principles is highly time- consuming and hard to converge. Due to the determining influence of magnitude spec- trogram on human perception, the attack that leverages the psychoacoustic principle of auditory masking (Qin et al., 2019) successfully generates imperceptible audio adversarial examples. However, it needs two-stage optimization where the first stage of optimiza- tion focuses on finding a relatively small perturbation to fool the network (as is done in prior work (Carlini & Wagner 2018)) and the second stage makes the adversarial examples imperceptible. The search process of the optimal balance between hiding the adversarial perturbations in the inaudible regions and fooling the ASR systems is empirically diffi- cult. Generating single full-sentence adversarial example requires approximately 5000 iterations until it finally converges to an imperceptible solution.
This research is motivated by psychoacoustic principle that phase manipulation in one-channel audios imposes tiny effects on human perception (Vary & Eurasip 1985). Besides, the inherent spectrogram consistency of short-time Fourier transform (STFT) reveals the fact that the adversarial perturbations in frame of phase spectrogram will transfer adjacent frames of magnitude spectro- gram (Griffin & Lim 1984) and dissipate the energy patterns in the spectrogram. These observation motivates us exploit phase spectrogram for generating audio adversarial examples.
Considering the relationship between phase spectrogram and the spectrogram consistency of STFT, in this work, we propose a novel phase-oriented algorithm called PhaseFool that efficiently generates imperceptible audio adversarial examples via energy dissipation. We prove that the phase-oriented perturbations always reduce the magnitude of the spectrogram in the corresponding areas via spec- trogram consistency, which is called energy dissipation. Leveraging such a phenomenon, new directions for audio adversarial attacks and are able to construct imperceptible audio adversarial examples efficiently. To the best of our knowledge, are the first consider phase spectrogram in the audio adversarial attacks. Our contributions can be summarized as follows:
bring
We leverage the spectrogram consistency of STFT to construct -oiented audio adver- sarial examples, which brings new directions for imperceptible audio adversarial attacks.
We investigate the relationship between phase perturbations and magnitude spectro- gram, which is called energy dissipation. Instead of the common I, distance metrics or magnitde-oriented metrics based on the psychoacoustic principle, propose weighted
2

---

Under review as a conference paper at ICLR 2022
loss function based on the phenomenon of energy dissipation to improve the imperceptibil- ity of phase-oriented audio adversarial examples.
We further evaluate the robustness of PhaseFool by constructing adversarial examples that can remain effective even after applying realistic simulated environmental distortions and several state-of- ahe- defenses.
We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015) and implement suc- cessful attacks against a pretrained DeepSpeech 2 model (Amodei et al. (2016) and an end-to-end Transformer- -aasedd ASR system (Winata et al. (2220). The results show that in terms of impercepti- bility, PhaseFool greatly surpasses current state-of-the-art algorithms based on ly-norm metries and shows comparable imperceptibilityy with the attacks based on the psychoacoustic principle of audi- tory masking (verified by a human study). Besides, PhaseFool achieves a 6.64x generation speed-up over current -fate-offlee tfte--ffthhhart the-art imperceptible counterparts. Furthermore, in the robustness evaluations, the adversarial examples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-art defenses. We attach the adversar- ial examples in the experiments and the implementation for reproducibility in the supplementary material.
2 BACKGROUNDS
In this section, we briefly overview the background of this work, including audio adversarial exam- ples and phase-aware audio signal processing.
Audio Adversarial Examples. After the early works (Gong & Poellabauer 2017 Cissé et al, 2017) towards neural networks based ASR systems successfully generate untargeted adversarial examples that produce incorrect but arbitrary transcriptions. Recently, more and more efforts have been made in the audio domain (Alzantot et al. 2018 Carlini & Wagner 2018). Among them, one attack Carlini & Wagner 2018).
of the most outstanding studies is the iterative optimization-base which introduces two-stage attack and directly generates targeted adversarial examples using gradient-descent minimization method. Nevertheless, the generated adversarial examples tend to include noticeable noise. Thus, two recent approaches (Qin et al., 2019 Schönherr al. 2019)
introduce hearing thresholds for designing inaudible perturbations by curbing the signal variation below the threshold of human perception. However, these imperceptible attacks require two stages of optimization and suffer from slow convergence rates.
Phase-aware Audio Signal Processing. The interest in utilizing phase information in the field of audio signal processing constantly increasing (Mowlaee et al. 2016). The STFT phase has been widely used in signal enhancement (Miyahara & Sugiyama 2014| Choi et al. [011 Hu al.) 2020) and signal reconstruction (Mowlaee & Kulmer| 2015). The phase in the harmonic domain is used in the speech synthesis (Degottex & | 2014 Ai & Ling 2020) and the phase in perceptual bands is used in speech watermarking (Wang et al. 2021). 1). The important information in the phase brings positive contributions to the above fields. Nevertheless. the existing audio adversarial attacks directly neglect the influence of phase spectrogram (Schönherr et al.] 2019). In this work, instead of introducing perturbations the audio waveform like most of the previous works, propose to leverage the spectrogram consistency and generate phase-oriented perturbations for constructing imperceptible audio adversarial examples.
3 PHASE-ORIENTED AUDIO ADVERSARIAL EXAMPLES VIA ENERGY DISSIPATION
In this section, we first define the preliminaries of the phase-oriented audio adversarial attack. Then we investigate the relation between phase spectrogram and magnitude spectrogram under spectro- gram consistency and prove that phase perturbations always reduce the magnitude of the spectro- gram in the corresponding areas under spectrogram consistency, which is called energy dissipation (as is illustrated in Figure D. Finally, we design a novel phase-oriented loss function based on energy dissipation for constructing imperceptible audio adversarial examples.
3

---

Under review as a conference paper at ICLR 2022
Complex domain
(n²)
JSTFT
I.m
Time n'
0
N/2
N
Time-domain audio signals
averttble
POO
STFT
3m (12)
Anvrrtibee
domain
Time
N/2
N
(b)
Figure 2: The Illustration of spectrogram consistency. (a) demonstrates that if the spectrogram X perturbed by P(X), it usually locates outside of the STFT spectrogram domain making non- invertible. (b) is the weerlap-add process with 50% overlapped signals in the inverse STFT.
3.1 PRELIMINARIES OF PHASE-ORIENTED AUDIO ADVERSARIAL ATTACK
3.1.1 PROBLEM DEFINITION
Given an input audio waveform r and a transcription corresponding to x. Denote the automatic speech recognition (ASR) system as f(-) that outputs transcription. The objective of the targeted audio adversarial attack is to find a small perturbation ö that constructs an imperceptible and targeted example x' z + 8 to fool the ASR system f(-) so that f(x') yt. where y: the targeted adversarial transcription. Denote (nn( Mt) as the loss function for the ASR model that is minimized when fa=) = ye- Then the general formulation of targeted adversarial attack for ASR system can be defined as:
argmin (x 8). + M| s.t.l1ll <E,
where À 0 is the trade-off parameter between the imperceptibility and loss function (uel (f(r + 6), yt) (in some formulations À = 0), control the maximum perturbation value, and p could be specified different values, according tack (Carlini & Wagner, 2018)). Then we will describe how to generate phase-oriented and targeted audio adversarial examples in the next subsection.
the attacker's requirement (e.g. p 2 in C&W's at-
3.1.2 PIIASE-ORIENTED ADVERSARIAL EXAMPLE GENERATION
Denote <b-) as the function of the STFT and a-1() as the function of the inverse STFT and X d()) = XX| e where X and 02 -X representing the spectral magnitude and the instantaneous phase spectrogram, respectively. Denote T(X) (()=1(X)). We assume that window function w, window length N and window shift is fixed and a signal can be perfectly reconstructed from its STFT spectrogram through the inverse STFT in this setting. We propose to generate perturbations on the instantaneous phase spectrogram and reconstruct the audio adversarial examples with the inverse STFT as the following formulation:
argmin ( e0 edd))) s.t. - 555m
(2)
where E(-) is the feature extraction function for the ASR model and the Φ x - edd) is the generated audio adversarial example.
As shown in Figure cated outside the STFT
dd perturbed by P(X) is usually lo- spectrogram domain. Considering the phenomenon of spectrogram consis- 2010), it is easy to prove the following two conclusions: 1) |X'| * [T(X') (X*) and o (T('') represent the same time-domain audio signals. Since the STFT and iSTFT functions are differentiable, these conclusions enable us to adversarially ma- nipulate the reconstructed magnitude spectrogram by introducing perturbations to the original phase spectrogram, which is called crisetoriented audio adversarial attack. Then detail how the pertur- bations on a frame phase spectrogram influence the adjacent frames corresponding magnitude spectrogram and reveal the energy dissipation nature of the phase-oriented audio adversarial attack.
the spectrogram X' = |X| e th
tency (Le Roux et al.
mathematically; 2) Ø-
4

---

Under review as a conference paper at ICLR 2022
3.2 RELATION ANALYSIS BETWEEN PHASE AND MAGNITUDE
This subsection mainly discusses the essence of the relationship between phase perturbations and magnitude spectrogram frames. Reconstructing the time-domain signals after introducing perturba- tions to the phase spectrogram includes overlap-add process. For simplicity, window shift length R is equal to half of the window length N, which means the adjacent frames have 50% overlapped signals. As shown in Figure 2 the signals 2m-1 (n') and Fm (n) in adjacent frames 1 and overlap, where n' and represent the time indices and n' + N. Zm (n) are obtained by an inverse Fourier transform as the following:
assume that the
N-1
I'm Im (n) = = (Xm) = N
Σ km
Xm
Ikle x
where j is the imaginary unit and Xm [k] are spectrogram samples in frames m at k-th frequency
bin. 2m-1(n') (n²) can also be obtained with Equation 3 Then we introduce perturbations 6n to the
frame m and the corresponding signal 8m fm(n) (n²) becomes:
N-1
(n) =
1
Σ ==
Xm Km]k 20 (6m([k]
(4)
N
Since the human auditory system is relatively less sensitive the tiny time delay caused by phase ro- tation, the phase perturbations are inherently more imperceptible than the perturbations on the audio waveform. Then we further investigate how phase perturbations affect the magnitude spectrogram. The reconstructed signal Z. (n) with the overlap-add process is obtained as:
N-1
(m) fmm-1(m)) Rm (n)
1
x
[k]e
Regnn 25mm
Σ Xm Meee 2 (6mlk)
N
k=0
(5)
We substitute Xm-1(k) and Xm [k] with the Fourier transform of (10) and xm then the overlapping area of I' (n) in Equation will become:
N-1
N/2-1
(m)
1 N
Im- 1 (n' 194]
Σ
Im (
(6)
N/=
The overlapping area of the Im-1(n'' based on spectrogram consistency:
(n') and 2m (m) in Figure 2 (b) satisfies the following relationship
N Tn (1 + ====m(0)) x1), s.l. 8...0 N 2
With Equation] 7 we can deduce the final formulation for the reconstructed signal 7m(n) as follow- ing:
2/2-1
Im (n)
Σ
jmm [K] + (36m
(8)
N
==0
k==
where the clean audio can be represented by setting 8m-1 [k] 0 and 8m [k]
0. Therefore, the
ratio 7m [k] of the reconstructed signal's 's spectrogram energy to the clean audio 's satisfies:
|e56m. 144) + [6 /2 2 2 cos (0m-1mmm 6m [k])
Yom [k]
e, + 10
where 7mm [k] is ranged within [0, 1J. It means that the phase perturbations 8m- [A], 6. always cause loss of the reconstructed magnitude spectrogram, which called "energy dissipation Till now, reveal the essence of phase-oriented audio adversarial attack: it slightly dissipates the energy patterns in the spectrogram and constructs the adversarial examples with the loss function in Equation 2 in the meantime.
5

---

Under review as a conference paper at ICLR 2022
3.3 OPTIMIZATION WITH ENERGY DISSIPATION
Simply minimizing the lp distortion of (1 7) is possible solution. However, the lp based metrics yield more audible perturbations in the practical attacks due to the differences between lp distance and human perceptibility. In this paper, we propose weighted loss function to constrain phase perturbations in an imperceptible range. Since the perturbations of our attack always cause energy loss the spectrogram, this function should dissipate the energy that is crucial for the classification model and meanwhile preserve the energy that is important for human perception. According to the characteristics of human auditory systems, we divide the spectrogram into three parts: 1) the signals that are important for human perception: 2) the signals that are relatively unimportant for human auditory systems: 3) the inaudible signals that are below the threshold in quiet; For 1) and 2), we manage to constrain their in different levels 3n and 32, respectively. For 3), we do not constrain the corresponding y since the energy dissipation in these areas can not be perceived. We divide quiet signals 3) following Lin & Abdulla (2015). The signals 1) are the local maxima that have the largest magnitude within 3 bins in the spectrum frames (in order to preserve the signals in high frequency we do not adopt Bark scale) and the signals 2) are the rest of the signals in the spectrogram.
Weighted Loss function. We formulate the construction of imperceptible phase-oriented adver- sarial examples as minimizing the following weighted loss function:
L Lma( - . e)))), ve) A Limp
(10)
Limp(7) = Σ
In (marr (31 .m (m]{)].0)) + 02
Σ Σ
In(max - 7m [A] 03),
(11)
W.LDD
where I, and In are the indicator functions that are equal to when the Ym [k] is in the corresponding part in the spectrogram and 01 1, 02 0.05 are trade-off parameters. We initialize the parameter A as 1 x 10-4 and update À every 10 iterations according to the performance of the attack. Since
the hinge loss utilized constrains the dissipation ratio y of the part 1) and 2) above the levels B1 = 0.98 and 32 = 0.90, the tiny energy losses caused by the phase perturbations will be perceived as almost inaudible volume differences instead of the echoes or noises;
EXPERIMENTAL SETUP
Datasets. We conduct experiments on the LibriSpeech dataset (Panayotov et al., 2015), which contains 1000 hours of 16KHz English speech derived from audiobooks. We randomly select 100 audio examples from the test-clean dataset as source examples, and 100 separate transcriptions from the same dataset to be the targeted transcriptions following the experimental setup with (Carlini & Wagner 2018 Qin et al., 2019). We ensure that each target transcription is around the same length as the original transcription because it unrealistic to perturb a short audio sample (e.g.. 10 words) to have much longer transcription (e.g., 30 words). Examples of the original and targeted transcriptions are available in the supplementary material.
Implementation Details. We successfully implement our PhaseFool and several state-of-the-art attack baselines against a pretrained DeepSpeech 2 model (Amodei et al. 2016) and an end-to-end speech recognition model based on the low-rank Transformer architecture following Winata et al. 2020 The DeepSpeech 2 model is an open-source ASR system with Connectionist Temporal Classification (CTC) method (Graves et al.] (006) and LSTM as its main components, which takes the log spectrograms as the input features. And the low-rank Transformer-based ASR system that takes log-mel spectrograms its input features is composed of lightweight transformer architecture and is trained with the cross-entropy loss. Since the feature extraction is differentiable, our phase attack can be easily extended to other ASR systems. All experiments are carried out on I Nvidia GTX 3080 GPU with 10 GB memory. We use Adam (Kingma & Ba 2013) as the optimizer with 31 0.9, 32 0.98, 10 9 and the learning rate is set X 10
TThe DeepSpeech 2 model follows the Pytorch implementation in the adversarial robustness toolbox Nico- lae al. 2018 and the low-rank Transformer model implemented following Winata et al. (2020)

---

Under review as a conference paper at ICLR 2022
Threat Model. As is done in most previous works, we assumee white-box threat model, i.c. the adversary is allowed to compute gradients through the model order to generate adversarial examples. Considering that current white-box audio adversarial attacks (Carlini & Wagner 2018
Yakura & Sakuma 2019 QQn et al. 2019) have been widely applied into the black-box attacks Yuan al. 2018] Chen et al. 2020) the base generation methods and shown great improvements in terms of robustness or imperceptibility, our work only focus on investigating the white-box model, as is done in most previous works.
Evaluation Metrics. 1) Word enror rate (WER). We calculate the WER metric which is defined as WER = S+D+I x 100%. where S. D. and I are the numbers of substitutions, deletions, and insertions, respectively, and N is the total number of words. Particularly, we choose the targeted transcription as the label for calculating the WER in the experiment 2) Success Rate (SR). SR the ratio of examples that can be successfully recognized as the malicious target texts by ASR systems. Here, "successfully recognized" means that the adversarial example is recognized as the malicious target transcription;
N
RESULTS
In this section, we evaluate the performance of PhaseFool terms of imperceptibility, generation speedup, robustness against simulated environmental distortions and state-of-the-ar defenses. The extra analysis and ablation studies can be found in Appendix
5.1 IMPERCEPTIRILTT ANALYSIS
Experimental Design. We construct 100 full-sentence targeted adversarial examples sampled from the LibriSpeech test-clean dataset for our PhaseFool attack. For comparison, we then gen- crate adversarial examples using the prior state-of-the-art works as baselines, including: 1) C&W's attack (Carlini & Wagner 2018), two-stage adversarial attack for audio based CTC loss and 4p- norm; IPC attack (Zhang et al. 2020), a new Iterative Proportional Clipping (IPC) algorithm that generates robust audio adversarial examples with temporal dependency; 3) Qin 's attack (Qin et al. 2019), the imperceptible attack that leverages the psychoacoustic principle of auditory masking. We ensure that all attacks have 100% success and the target transcription is consistent among the experiments to exclude other interference factors. The user study is conducted to verify the imper- ceptibility of these attacks. We recruit 24 users online and ask them to tell if the examples have any background noise (e.g., static, echoing, people talking in the background) following (Qin et al., 2019), which is the most representative experiment of how an attack would work in practice. The users are asked to listen to each sample with headphones on, and answer questions about set of audio samples. In all experiments, users can replay the audio samples multiple times.
Result Analysis. As shown in Table [ 31% of the users believe that the clean audio samples contain noises. Only 43% and 46% of the users believe that the ad- versarial examples generated by the Phase- Fool against the DeepSpeech 2 system and low-rank Transformer system contain some noises, which is similar to Qin's attack. Meanwhile, PhaseFool and Qin's attack greatly surpass the 4--norm based C&W's attack and IPC attack. It can be seen that phase-oriented attack shows compara- ble imperceptibility with the attacks that leverage the psychoacoustic principle of au- ditory masking. However, our attack needs fewer steps and only one optimization stage to generate imperceptible audio adversarial examples rather than complex two optimizations stages in Qin al. (2019. We will further explore the generation speed of these attacks in the next subsection.
Classified as Noisy +
Method
DeepSpeech 2
Transformer
Clean
31%
31%
C& W's Attack
75%
77%
IPC Attack Qin's Attack
59%
68%
49%
45 %
PhaseFool
43%
46%
Table 1: Results of the user study for imperceptibility. *4" means lower is more imperceptible.
This calculation method of WER follows Carlini & Wagner 2018 Qin al. 2019 Liu al. 2020

---

Under review as a conference paper at ICLR 2022
DeepSpeech
Transformer
Method
Steps
Convergence Times (8) +
Steps 4
Convergence Times (s) +
C&W's Attack
4000
1031.16 1 369.03
4000
211.87 1: 12.31
IPC Attack
3000
871.72 = 328.35
3000
158.75 + 10.99
Qin's Attack
5000
1310.86 583.28
5000
305,91 8.02
PhaseFool
700
197.85 = 56.09
500
33.10 + 3.27
Table 2: Comparison of the generation speed with 95% confidence intervals. Convergence Times (s) means that the algorithm can generate a successful audio adversarial examples within the listed seconds on average.
Defenses
Method
TD
Dropout
Downsample
Noise
C&W's Attack
57.78%
63.75%
67.05%
63.70%
IPC Attack
52.67%
71.62%
68.95%
74.42%
Qin's Attack
58.22%
61.20%
66.74%
82.64%
PhaseFool 57.02%
64.09%
68.60%
62.81%
Table 3: Area under curve (AUC) scores for different defenses against PhaseFool and various base- line attacks.
5.2 GENERATION SPEEDUP
We evaluate the generation speed of PhaseFool compared with several state-of-the-ar audio adver- sarial attacks in the previous experiment. We generate 100 adversarial examples for each algorithm with the 100% success rate and ensure that the original audio samples and the target transcriptions are consistent among the experiments to exclude other interference factors. The lengths lected audio range from 3 seconds to I0 seconds (5.18 seconds on average) and the detailed analysis of the selected samples can be found in the Appendix AA.4] As shown in Table 2 it can be seen that PhaseFool can generate successful audio adversarial example for the low-rank Transformer system in around 30 seconds on average, which is 9.24x speedup compared with the imperceptible counterpart Qin's in's attack (Qin et al.) 2019). Moreover, for the DeepSpeech 2 system, Qin's 's attack usually requires over 20 minutes to generate an adversarial example while our PhaseFool only needs around 3 minutes. Although the constraints used in the baseline attacks improve their impercep- tibility, these constraints significantly slow down the convergence rates of the optimization process. Besides, the two-stage optimizations make C&W's attack and Qin's attack requires even more steps to generate ad adversarial examples. Empirically, the computational complexity of audio ad adversarial attack mainly depends on iteration steps and the attacked model f(-). Since our PhaseFool is a one-stage attack without complex computations and only requires 700 steps to obtain successful
the se-
adversarial example, the generation speed PhaseFool is much faster than the baseline attacks.
5.3 ROBUSTNESS ANALYSIS
We conduct the robustness analysis to explore whether the adversarial examples constructed by PhaseFool can remain effective even after applying realistic simulated environmental distortions and several state-of-the-at defenses. We generate 100 adversarial examples only for the DeepSpeech system in this experiment, We adopt the Expectation over Transformation (EOT) algorithm (Atha- [ye al. 2018) to simulate transformations caused by environmental distortions and defenses and incorporate the transformations into the generation process following the previous works (Yakura & Sakuma 2019 Qin et al. 2019). The details of the experimental setup can be found Appendix
A.1 The results are shown in Table 3 and Table
Robustness Against Defenses. To compare the robustness of attacks against adversarial audio defenses, PhaseFool and the baseline attacks are applied with the following defenses: 1) Tem- poral Dependency (TD) defense (Yang et al, 2019), the defense that uses the temporal depen- dency in audio data gain discriminate power against audio adversarial examples; 2) Dropout
8

---

Under review as a conference paper at ICLR 2022
Reverberation
Noise
Method
SR
WER +
Classified as Noisy /
SR
WER
Classified as Noisy +
C&W's Attack
47%
22.69%
100%
55%
15.38%
88%
IPC Attack
21%
37.07%
97%
37%
27.94%
73%
Qin's Attack
43%
26.44%
85%
19%
43.45%
51%
PhaseFool 38%
29.62%
61%
44%
16.34%
48%
Table 4: The results of robustness evaluation against reverberant and noisy environments.
defense (Jayashankar et al. 2020). the defense based on the uncertainty introduced by dropout in neural networks; 3) Downsample defense (Yang al. 2019), a common defense that down-samples a band-limited audio file without sacrificing the quality of the recovered signal while mitigating the adversarial perturbations in the reconstruction phase. 4) Noise defense, a common defense that introduces white Gaussian noise to disable the adversarial perturbations. We follow the same ex- perimental procedures as the corresponding defenses and adopt their evaluation metric: the area under curve (AUC) score, As shown in Table [] the AUC scores for PhaseFool's adversarial ex- amples against the four defenses are mostly distributed between 57%-668, which means that the classifiers of these defenses have poor performance IPC attack shows great performance against the TD defense, its adversarial examples are easily de- tected by the dropout defense and the noise defense. Besides, Qin's 's attack is also not robust enough with the noise defense. These results demonstrate the robustness of our phase-oriented adversarial examples against several sophisticated defenses.
adversarial examples. However, although
Robustness Against Environmental Distortions. To demonstrate the robustness of PhaseFool against environmental distortions, we evaluate the attack success rate (SR) and the word error rate (WER) of adversarial examples generated by PhaseFool and baseline attacks in the simulated envi- ronments that contain reverberations and white Gaussian noise following the previous works (Qin) et al. 2019 Yakura & Sakuma 2019). Besides, to investigate whether the adversarial examples of Phase Fool can still remain imperceptible to the human ear in these scenarios, we also conduct the user study with the same experimental settings in Subsection 5. As shown in Table 4 the adver- sarial examples generated by PhaseFool achieve approximately 40% attack success rate in the re- verberant and noisy environments. However, the imperceptibility of Qin's attack significantly drops in the reverberation environment, while the imperceptibility of our PhaseFool does not degrade so much relatively. Since the phase-oriented adversarial examples sound more like echoes or blurred pronunciation, our PhaseFool shows better imperceptibility in the attacks for the environments that contain distortions.
CONCLUSIONS
In this work, we propose PhaseFool, a phase-oriented audio adversarial attack, which leverages en- ergy dissipation to efficiently construct imperceptible audio adversarial examples. Leveraging the phenomenon of energy dissipation, we bring new directions for audio adversarial attacks. Moreover, we propose a weighted loss function to improve the imperceptibility of phase-oriented audio adver- sarial examples. Experiments on the LibriSpeech dataset (Panayotov et al.) (015) demonstrate that our PhaseFool significantly surpasses current state-of-the-an algorithms based on 4y-norm metrics and shows comparable imperceptibility with the attacks based on the psychoacoustic principle of auditory masking. Besides, PhaseFool achieves a 6.64x generation speed-up over current state-of- the-art imperceptible counterparts. Moreover, in the robustness evaluations, the adversarial exam- ples constructed by PhaseFool remain effective even after applying realistic simulated environmental distortions and several state-of-the-arr defenses.
For future work, we will continue improve the imperceptibility of the phase-oriented audio adver- sarial examples, and apply PhaseFool to black-box settings for practical attacks at commercial ASR systems. We will also investigate PhaseFool's capacity in online learning scenarios.

---

Under review as a conference paper at ICLR 2022
REFERENCES
Yang Ai and Zhen-Hua Ling. A neural vocoder with hierarchical generation of amplitude and phase spectra for statistical parametric speech synthesis. IEEE ACM Trans. Audio Speech Lang. Process., 28:839-851, 2020. doi: 1009/TAALP.2020.99977 10.1009/TASLP.220.2229
URL
Moustafa Alzantot, Bharathan Balaji, and Mani Srivastava. Did you hear that? adversarial examples against automatic speech recognition. arXiv preprint arXiv: 1a00:0005444 2018.
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. Deep speech 2: End-to- end speech recognition in english and mandarin. In International conference on machine learning.
pp. 187. PMLR, 2016.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing robust adversarial examples. In International conference on machine learning, pp. 284-293. PMLR, 2018.
Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Gior- gio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Joint European conference on machine learning and knowledge discovery databases, pp. 387-402. Springer, 2013.
Nicholas Carlini and David A. Wagner. Audio adversarial examples: Targeted attacks on speech- to-text. In 2018 JEEE Security and Privacy Workshops. SP Workshops 2018, San Francisco. CA, USA, May 24, 2018, pp. 1-7. IEEE Computer Society, 2018. doi: 10. 1009/PW/ 2018.00009
URL httpt:::///.m. http:/ttts////oooo /ttp:::/ddo..rgggoo 0.1 09/ SPW.201 8.00009
Nicholas Carlini, Pratyush Mishra, Tavish Vaidya, Yuankai Zhang, Micah Sherr, Clay Shields, David Wagner, and Wenchao Zhou. Hidden voice commands. In 25th (USENIX) Security Symposium ((USENIX) Security 16), pp. 513-530, 2016.
Yuxuan Chen, Xuejing Yuan, Jiangshan Zhang, Yue Zhao, Shengzhi Zhang, Kai Chen, and Xi- aoFeng Wang. Devil's whisper: A general approach for physical adversarial attacks against commercial black-box speech recognition devices. In Srdjan Capkun and Franziska Roesner (eds.), 29th USENIX Security Symposium, USENIX Security 2020, August 12-14, 2020, pp. 2667-2684. USENIX Association, 2020. URL http://wwwweeeiioo///oeeoeeeeeee usenixeeccritty///reeettt..ooo cettitatiioooceeeyyuueuu yuxuan
Hyeong-Seok Choi, Jang-Hyun Kim, Jaesung Huh, Adrian Kim, Jung-Woo Ha, and Kyogu Lee. Phase-aware speech enhancement with deep complex u-net. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA. May 6-9, 2019. OpenReview.net, 2019.
forum i SKeRTsAcRR
Moustapha Cissé, Yossi Adi, Natalia Neverova, and Joseph Keshet. Houdini: Fooling deep struc- tured prediction models. CoRR, abs/1707.05375. 2017. URL
/bbs/
17C7.05373
Gilles Degottex and Daniel Erro. A measure of phase randomness for the harmonic model in speech synthesis. In Fifteenth Annual Conference of the International Speech Communication Associa- tion, 2014.
Yuan Gong and Christian Poellabauer. Crafting adversarial examples for speech paralinguistics applications. CoRR, abs/1711.03288 2017. UKLL:tttp//aarrr.ooo//////......
Alex Graves, Santiago Fernández, Faustino Gomez, and Jürgen Schmidhuber. Connectionist tem- poral classification: labelling unsegmented sequence data with recurrent neural networks. In Proceedings of the 23rd international conference on Machine learning, pp. 369-376, 2006.
Daniel Griffin and Jae Lim. Signal estimation from modified short-time fourier transform. JEEE Transactions acoustics, speech, and signal processing, 32(2))236-243, 1984.
10

---

Under review as a conference paper at ICLR 2022
Yanxin Hu, Yun Liu, Shubo Lv, Mengtao Xing, Shimin Zhang, Yihui Fu, Jian Wu, Bihong Zhang, and Lei Xie. DCCRN: deep complex convolution recurrent network for phase-aware speech en- hancement. In Helen Meng. Bo Xu, and Thomas Fang Zheng (eds.), Interspeech 2020, 21st An- nual Conference of the International Speech Communication Association, Virtual Event, Shang- hai, China, 25-29 October 2020, pp. 2472-2476. ISCA, 2020. doi: 00.2143//nterspecch..00 2537. URL https: http://////..00r0/0002222//0//10
Tejas Jayashankar, Jonathan Le Roux, and Pierre Moulin. Detecting audio attacks on ASR systems with dropout uncertainty. In Helen Meng. Bo Xu, and Thomas Fang Zheng (eds.), Interspeech 2020, 21st Anmual Conference of the International Speech Communi- cation Association, Virtual Event, Shanghai, China, 25-29 October 2020, pp. 4671-4675. ISCA, 2020. doi: 10.214477lmterspeechh. 2020 1846. URL http::///6l.ogg/00..21
Interspeech.2c20-1 1846
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.). 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: 7/arxiv.ozg abs 11b.b///11.6000
Jonathan Le Roux, Hirokazu Kameoka, Nobutaka Ono, and Shigeki Sagayama. Fast signal recon- struction from magniuue stft spectrogram based on spectrogram consistency. Proc. DAFx, volume 10, pp. 397-403, 2010.
Yiqing Lin and Waleed H Abdulla. Principles of psychoacoustics. In Audio Watermark, pp. 15-49. Springer, 2015.
Xiaolei Liu, Kun Wan, Yufei Ding. Xiaosong Zhang, and Qingxin Zhu. Weighted-samphe audio adversarial example attack. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 4908 4915, 2020.
Ryoji Miyahara and Akihiko Sugiyama. An auto-focusing noise suppressor for cellphone movies based on phase randomization and power compensation. In 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2199 2203. IEEE, 2014.
Pejman Mowlaee and Josef Kulmer. Phase estimation in single-channel speech enhancement: Limits-potential. JEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(8): 1283-1294, 2015.
Pejman Mowlace, Rahim Sacidi, and Yannis Stylianou. Advances in phase-aware signal processing in speech communication. Speech communication, 81:1-29, 2016
Maria- Irina Nicolae, Mathicu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wis- tuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial robustness toolbox v1.2.0. CoRR, 1807 2018. URL
Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5206-5210 IEEE, 2015.
Yao Qin, Nicholas Carlini, Garrison W. Cottrell, Ian J. Goodfellow, and Colin Raffel. Impercep- tible, robust, and targeted adversarial examples for automatic speech recognition. In Kama- lika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Confer- ence on Machine Learning. ICML 2019, 9-15 June 2019, Long Beach, California, USA, vol- 97 of Proceedings of Machine Learning Research, pp. 5231-5240. PMLR, 2019. URL http:://rrtt http::////ooeeengg..... r.prrsss/vil w91////nnnnn http://nnnaahtmll.
Robin Scheibler, Eric Bezzam, and Ivan Dokmanic. Pyroomacousties: A python package for audio room simulation and array processing algorithms. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada, April 15-20, 2018, pp. 351-355. IEEE, 2018 doi: 10. 10.1109/CASSP 2018.8461330 URL https: /
10.1109/ TCASSP 2018.8461318
11

---

Under review as a conference paper at ICLR 2022
Lea Schönherr, Katharina Kohls, Steffen Zeiler, Thorsten Holz, and Dorothea Kolossa. Ad- versarial attacks against automatic speech recognition systems via psychoacoustic hiding. In 26th Annual Network and Distributed System Security Symposium, NDSS 2019. San Diego California, USA, February 24-27, 2019. The Internet Society, 2019. URL https:
//www.ndss sympos womppssium.orge. httpg//ssssspp///errrrrraaaaaaaa
inst-
aut automatic speech recocnition seeertttttttccccceettttttttttoo wwwptttiiooosssstsneeeee systems via pycpppsccoocouttiiiciiiiii
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 13/2v61312.61999 2013.
Tavish Vaidya, Yuankai Zhang, Micah Sherr, and Clay Shields. Cocaine noodles: exploiting the gap between human and machine speech recognition. In 9th {USENIX) Workshop on Offensive Technologies ( ([WOOT} 15), 2015.
Peter Vary and Member Eurasip. Noise suppression by spectral magnitude estimation- and theoretical limits Signal processing, 8(4):387-400, 1985.
Shengbei Wang, Weitao Yuan, Zhen Zhang, Jianming Wang, and Masashi Unoki. Synchronous multi-bit audio watermarking based on phase shifting. In JEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2021, Toronto, ON, Canada, June 6-11, 2021, pp. 2700-2704. IEEE, 2021. doi: 10. 00.100//CAASP3992222000001 2011199922..2211144339 URL Stthttp::
Genta Indra Winata, Samuel Cahyawijaya, Zhaojiang Lin, Zihan Liu, and Pascale Fung. Lightweight and efficient end-to-end speech recognition using low-rank transformer. In 2020 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing, ICASSP 2020, Barcelona, Spain, May 4-8, 2020, pp. 6144-6148. IEEE, 2020. doi: 10.1 D0.109/CCASSP44776.2000 200.7766.22209033873 URL 2.000 021.1.....22..00372 9053878
Hiromu Yakura and Jun Sakuma. Robust audio adversarial example for physical attack. In Sarit Kraus (ed.), Proceedings of the Twenty-Eightt International Joint Conference on Artificial Intel- ligence, IJCAI 2019, Macao, China, August 10-16, 2019, pp. 5334-5341. ijcai.org, 2019. doi: 10.24963/jjai..201 wataaa..111//70 91791
Zhuolin Yang. Bo Li, Pin-Yu Chen, and Dawn Song. Characterizing audio adversarial exam- ples using temporal dependency. In 7th International Conference on Learning Representa- tions, ICLR 2019. New Orleans, LA, USA, May 6-9, 2019. OpenReview.net, 2019. URL
Xuejing Yuan, Yuxuan Chen, Yue Zhao, Yunhui Long, Xiaokang Liu, Kai Chen, Shengzhi Zhang, Heqing Huang, Xiaofeng Wang, and Carl Gunter. Commandersong: A systematic approach for practical adversarial voice recognition. In 27th {USENIX} Security Symposium (UUSENIX) Security 18), pp. 49-64, 2018.
Hongting Zhang, P Zhou, Q Yan, and Xiao Yang Liu. Generating robust audio adversarial examples with temporal dependency. In Proc. Int. Joint Conf. Artif. Intell., pp. 1-5, 2020.
A APPENDIX
A.1 THE DETAILED SETUP OF ROBUSTNESS EVALUATION
In the robustness evaluation, adopt the Expectation over Transformation (EOT) algorithm (Atha- Iye et al. 2018) to simulate transformations caused by environmental distortions and defenses and incorporate the transformations into the generation process. Then transcribe the newly obtained audio adversarial examples and finally calculate SR and WER to evaluate their robustness.
A.1.1 THE SETUP OF EVALUATION AGAINST DEFENSES
For TD defense, we randomly select the first & portion of the adversarial examples (k is chosen from 33 2: 3: })) and adopt the EOT algorithm to make the generated adversarial examples adaptive to
12

---

Under review as a conference paper at ICLR 2022
the TD defense. In the defense stage, we choose k 1/2 and calculate the AUC scores of WER following Yang et al. 2019).
For Dropout defense, {0.00, 0.05, 0.10, 0.15] and adopt the EOT algorithm to make the generated adversarial examples adaptive the Dropout defense. In the defense stage, set the defense dropout rate as p 0.05 and calculate the AUC scores of the SVM-F classifier following Jayashankar et al.) 2020
randomly choose the dropout rate p of ASR systems from
For Downsample defense, conduct the EOT algorithm by performing the attack the down- sampled elements of the original audio sequence. In the attack stage, we randomly choose whether to downsample the adversarial examples from 16000 Hz 8000 Hz. In the defense stage, we down- sample the audio from 16000 Hz to 8000 Hz and calculate the AUC scores of WER following Yang al. 2019).
For Noise defense, we randomly add White Gaussian noise with the boundary of = to the gener- ation process of the adversarial examples. Since the input audio normalized, here we randomly choose the A from f0.01, 0.02, 0.03) in the attack stage. In the defense stage, we choose A 0.02 and calculate the AUC scores of WER.
A.1.2 THE SETUP OF EVALUATION AGAINST ENVIRONMENTAL DISTORTIONS
For the reverberant scenarios, we use an acoustic room simulator that applies the classic Image Source Method introduced in (Scheibler al., 2018) create the room impulse response r. We set the room configurations (room dimension, source audio and target microphone's locations, and reverberation time) following Qin et al. (2019). Then, the generated room impulse response is convolved with the clean audio to create the speech with reverberation. In the attack stage, we generate 1000 random room configurations sampled from the distribution as the training room set. And the test room set includes another 100 random room configurations sampled from the same distribution for the 100 audio adversarial examples generated by different attacks.
For the noisy scenarios, we randomly add White Gaussian noise with the boundary of =ı to the generation process of the adversarial examples. Since the input audio normalized, here we ran- domly choose the A from {0.01, 0.02, 0.03). In the test stage, we choose A 0.02. Note that this experiment aims at evaluating the attack success rate and imperceptibility of attacks in the noisy scenarios, which is different from the evaluation against noise defense.
A.2 ABLATION STUDIES
We evaluate the generation speed and imperceptibility in the ablation studies explore the influ- ences of the weighted loss function proposed in Equation II] As shown in Table 5 and Table 6 when do not constrain the phase perturbations and optimize the example until is successful, PhaseFool only needs 305 and 212 steps on average to construct an audio adversarial example for the DeepSpeech system and low-rank Transformer system, correspondingly. It can be seen that the examples without constraints are even more imperceptible than the 1n-norm based attacks that need thousands of steps. But these examples introduce more noise than PhaseFool with the weighted hinge loss in Equation []] which demonstrates the effectiveness of our proposed loss function. How- ever, when we constrain the phase perturbations with l2 loss, the noises adversarial examples are even louder than the examples generated without constraint. It demonstrates the importance of curb- ing the energy dissipation ratio y above the weighted level for different types of regions with the loss function proposed in Subsection 3.3
DeepSpeech 2
Method
Steps 4
Convergence Times (s) 4
Classified as Noisy 1
PhaseFool
700
197.85++56.09
43%
PhaseFool with 12 loss
700
195.48-1.5.229
75%
PhaseFool w/o constraint
305
88362
64%
Table 5: The results of ablation studies against the DeepSpeech 2 system.
13

---

Under review as a conference paper at ICLR 2022
Transformer
Method
Steps
Convergence Times (s) +
Classified as Noisy +
PhaseFool
500
33.10.1..27
44%
PhascFool with l2 loss
500
32.54=3.50
72%
PhascFool w/o constraint
17.80
67%
Table 6: The results of ablation studies against the low-rank Transformer system.
A.3 INTERPRETATION OF ATTACKS
For further investigating the interpretation of different attacks, present the pixel-wise differ- ence the adversarial examples in the mel-spectrogram domain to visualize the learned adversarial perturbations in Figure 3 The perturbations of C& attack are disorderly distributed in the mel- spectrogram while Qin attack and IPC attack tend to hide perturbations in the utterance regions. Our PhaseFool attack provides additional insights about the vulnerability of ASR models, i.e. the influences of different regions in the spectrogram for the audio adversarial attack. For example, as shown in Figure 3] the perturbations generated by our PhaseFool attack mainly oceur on the posi- tions around the harmonics in the mel-spectrogram. The perturbation positions reveal the fragile part of the spectrogram for the ASR system's prediction. With the /2 loss function, PhaseFool sig- nificantly perturbs the mel-spectrogram regions that are important for human perception and cause audible echoes in the audio adversarial examples. When we remove the constraints of PhaseFool, the magnitude of perturbations in the melestcctn grows obviously, which demonstrates the effectiveness of the proposed weighted loss.
PhaseFool
PhaseFooo with lg loss
PhuseFool w/o constraint
C&W's Attack
IPC Attack
Qin's Attack
Figure 3: Pixel-wise difference the generated audio adversarial examples in the mel-spectrogram domain. The perturbations of C&W's attack are disorderly distributed attack and IPC attack tend to hide perturbations in the utterance regions while our PhaseFool attack generates perturbations around the harmonics in the spectrogram.
in the mel-spectrogram. Qin
A.4 LENGTH ANALYSIS
In all experiments, we randomly select 100 audio examples from the test-clean of the LibriSpeech dataset (Panayotov et al., 2015) source examples, and 100 separate transcriptions from the same dataset to be the target transcriptions following the experimental setup with (Carlini & Wagner) 2018 Qin et al. 2019). We ensure that each target transcription is around the same length as the original transcription because it unrealistic to perturb a short audio sample (e.g.. 10 words) to have much longer transcription (e.g., 30 words). Figures illustrates the comparison of the text length between original transcriptions and target transcriptions. And Figures 5 shows the lengths of 100 selected audio samples in the experiments.
14

---

Under review as a conference paper at ICLR 2022
Original transcnppior
Targec transer pluo
Count
20
40
100
120
1/0
150
Length
Figure 4: The length distribution of 100 text examples randomly sampled from Librispeech in our experiments.
Chuic
Longth
Figure 5: The length distribution of 100 audio examples randomly sampled from Librispeech in our experiments.
A.S ANALYSIS ON THE CONVERGENCE OF LOss LEVENSHTEIN DISTANCE
Here we analyze the CTC loss and Levenshtein distance convergence processes when we attack the DeepSpeech 2 model with PhaseFool. We randomly select 20 samples the experiments. As shown in Figure 6 the CTC loss of most cases quickly converges to value close to O. The Levenshtein distance in Figure converges to on all cases before 700 iteration steps. Note that when the value of Levenshtein distance becomes 0, the adversarial example is successfully generated.
15

---

Under review as a conference paper at ICLR 2022
Lawerann dictance
40
20
50
20
100
200
300
400
500
600
700
:00
200
300
400
500
600
700
Beration steps
Iteratton steps
04
12
25
Lawanchain
100
200
300
400
500
500
700
100
200
300
403
50c
500
700
Iteration steps
Iteratton
steps
Figure 6: The Illustraiion of the CTC loss and Levenshtein distance convergence processes of Phase- Fool. We randomly picked 20 samples in the experiments.
16


Reviews:
[{"review_id": "_Ga7idD4Bmx", "reviewer": "Reviewer_rkdE", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall, I believe that the paper has great potential since the idea of energy dissipation is quite interesting but there are several concerns that need to be addressed. If the following concerns are addressed, I will consider to increase my score. Please see below my concerns and questions to the authors in descending order of importance (0 is the most important):\n\n0. The main problem of this paper stems from the limited experimental setup that the authors have chosen to show the significance of their contributions. Specifically, the authors have only tested their algorithm for generating sequences with 100% WER and with a white-box ASR which was trained by the authors. It would be interesting to see if those adversarial examples are valid for pre-trained black box models even with not 100% WER. Moreoever, the 100% WER might be misleading since it would make much more sense to have a small change in a few words that would change the meaning than completely altering the whole target sentence. Furthermore, it might be the case that the other algorithms could alter a couple of words fairly easily but not all of them.\n\n1. The authors provide only a very limited amount of adversarial examples in their supplementary material (which the authors also do not provide details if they are random or cherry-picked). Even from those limited examples I would admit that there are evident artifacts in PhaseFool’s adversarial examples. To the best of my hearing ability I would also say that the adversarial examples coming from the “Imperceptable” sound less artifact-y. I do not consider that the difference is so significant but I would like to hear more random examples to conclude.\n\n2. There are several premises and claims in this paper which are at least ill-posed and sometimes wrong. For example, “Since phase is not relevant for speech recognition, the existing audio adversarial attacks neglect the influence of phase spectrogram.” is not true in cases where there are several speakers and a model (or even a human) uses the phase (aka inter-aural time-difference) between the two sources to better isolate and understand the words spoken by a speaker from a specific location. Another example, “Till now, we reveal the bi-directional essence of phase-oriented audio adversarial attack: it dissipates the critical energy for the classification model and constructs the adversarial perturbations in the meantime.” How is the latter statement shown? To the best of my knowledge, there is no “known” frequency region critical energy for classification for all ASR models. \n\n3. I think that calling the inference/convergence times “efficiency (s)” is rather misleading. Moreover, the authors should report the number of trainable parameters, as well as the actual memory requirements in order to conclude that their algorithm is indeed more efficient and also discuss ways if their algorithm could be somehow more efficiently parallelized.\n\n4. I am not fully convinced that an ASR model which does not use an STFT or a log-mel representation would also be vulnerable to an attack such as the one proposed in this paper. Specifically, with a small and simple adaptive front-end, one could possibly learn some real over-complete basis to be more robust against attacks like  PhaseFool. Real-basis have been successfully applied towards separation and enhancement problems but also show that can effectively represent speech with even a single layer [1, 2, 3, 4]. Could the authors provide ablation studies by replacing the fixed STFT layers with adaptive convolutions and still show that their findings still hold?\n\n5. Based on the aforementioned point, the vulnerability of ASR models to phase perturbations or echoes is known for many years now [5] which partly explains the efficiency of PhaseFool that can easily drive the ASR model towards a mis-prediction while also not altering the structure of the sound significantly. Can PhaseFool be applied in reverberant scenarios where there are also other phase artifacts and still cannot be perceived by the human ear? Optionally, it would be wise to assume that there is at least some naive Wiener or minimum variance distortionless response (MVDR) filter and see how and if PhaseFool can trick those front-ends as well. There are other more sophisticated defences that the authors could also try, such as [6], to show that the proposed adversarial examples are not easily detectable by automated systems.\n\n6. Table 1. should be WER 100% and the arrow next to WER should be facing up since  all of them are adversarial algorithms.\n\n7. Could the authors specify what are the exact time-lengths of the input audio that the ~30seconds inference time corresponds to?\n\n[1] Venkataramani S, Casebeer J, Smaragdis P. End-to-end source separation with adaptive front-ends. In2018 52nd Asilomar Conference on Signals, Systems, and Computers 2018 Oct 28 (pp. 684-688). IEEE.\n\n[2] Luo, Y. and Mesgarani, N., 2019. Conv-tasnet: Surpassing ideal time–frequency magnitude masking for speech separation. IEEE/ACM transactions on audio, speech, and language processing, 27(8), pp.1256-1266.\n\n[3] Tzinis, E., Venkataramani, S., Wang, Z., Subakan, C. and Smaragdis, P., 2020, May. Two-step sound source separation: Training on learned latent targets. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 31-35). IEEE.\n\n[4] Mimilakis, S.I., Drossos, K. and Schuller, G., 2021, January. Unsupervised interpretable representation learning for singing voice separation. In 2020 28th European Signal Processing Conference (EUSIPCO) (pp. 1412-1416). IEEE.\n\n[5] Xiong, F., Meyer, B.T., Moritz, N., Rehr, R., Anemüller, J., Gerkmann, T., Doclo, S. and Goetze, S., 2015. Front-end technologies for robust ASR in reverberant environments—spectral enhancement-based dereverberation and auditory modulation filterbank features. EURASIP Journal on Advances in Signal Processing, 2015(1), pp.1-18.\n\n[6] Jayashankar, T., Roux, J.L., Moulin, P. (2020) Detecting Audio Attacks on ASR Systems with Dropout Uncertainty. Proc. Interspeech 2020, 4671-4675, doi: 10.21437/Interspeech.2020-1846.\n\n===== POST - REBUTTAL =========\nThe authors have addressed some of my concerns, thus, I increase my score from 3 -> 5.", "summary_of_the_paper": "This paper introduces a new way of constructing adversarial examples for automatic speech recognition using the phase of the short-time Fourier transformation. The authors conduct experiments and subjective evaluations to show how their adversarial examples compare to previous state-of-the-art algorithms. The experimental results show that the proposed phase-oriented audio adversarial samples can be produced in reduced time as well as they are less perceivable by humans.", "summary_of_the_review": "Although the idea proposed in the paper is interesting and could have a great potential in the way that adversarial examples are generated for ASR systems. However, the current version of the paper displays only a limited experimental setup of the proposed algorithm that does not help to show the significance of the algorithm. I am willing to increase my score if my above concerns are addressed. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "W2KF4a7k42b", "reviewer": "Reviewer_vUjU", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n1. The idea of phase-based adversarial attack makes sense in that the phase information is usually known as less perceptible than magnitude information.\n2. The authors theoretically show that the phase-based perturbation leads to energy dissipation.\n3. The experiment results are good, and is even better than the magnitude-based method.\n\nWeakness:\n1. It could have been nicer if the authors had experimented with the raw-waveform-based ASR networks.\n2. It could have been nicer if the authors had experimented the method on more diverse audio NNs.", "summary_of_the_paper": "This paper proposes a novel adversarial attack method that attacks ASR network.\nThe idea is that the attack is likely to be imperceptible when phase information is perturbed because phase information is known to give releatively small perceptible difference than magnitude information.\nThe perturbation on phase information influences the magnitude information which is known as energy dissipation.\nExperiment results show that it reaches almost state-of-the-art results in terms of imperceptibility. The advantage of the proposed method is that it can generate the imperceptible perturbation using fewer steps than the previous state-of-the-art method.\nThey also qualitatively show that it is usually the harmonic parts of speech where the energy dissipates.\n", "summary_of_the_review": "The idea is solid.\nExperiment results are good.\nWriting is straightforward and easy to follow.\nAll in all, I recommend to accept the paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "QYQJtvtaATI", "reviewer": "Reviewer_BtbA", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main strength of the paper is the novelty of the energy dissipation.\n\nMain weaknesses include:\n1. As pointed out by reviewer rkdE, the claim that serves as the motivation \"Since phase is not relevant for speech recognition, the existing audio adversarial attacks neglect the influence of phase spectrogram.\" is not true. It is not shown in the paper what this claim is based on. I guess perhaps the authors thought traditional filterbank acoustic features for monaural ASR only keep magnitudes, which is certainly true. However, phase information is certainly important for ASR not only with binaural or microphone array data but also for monaural -- this is possibly why ASR systems with raw-waveform features can outperform those with filterbank features in many recent studies. \n2. The type of ASR input features used is missing in the paper (perhaps log-mel spectrograms from Fig. 1?). I traced into the paper that provides the ASR model architecture (Winata et al., 2020a), but did not find any clear evidence for the feature type. This is critical here since filterbank-related features (including MFCCs) do not have explicit phase information. Perturbing phase information can reduce the magnitude of the spectrogram is certainly good evidence, but doesn't this contradict with the claim \"Since phase is not relevant for speech recognition, the existing audio adversarial attacks neglect the influence of phase spectrogram.\"?\n3. The ASR method used in this paper is not common. The effectiveness of such adversarial attacks is certainly related to the detailed model architecture and how it was trained. I would expect the method may not work for NN-HMMs, CTC, and stateless RNN-T. \n4. It is also difficult for me to understand how to use such white-box adversarial attack techniques for ASR in practice. Industrial ASR systems are often trained on an enormously large dataset with many data augmentation techniques applied, which should be a lot more robust against energy dissipation. I understand the authors rely on (Chen et al., 2020) to convert a white-box model to a black-box model. However, the methods proposed in (Chen et al., 2020) are not convincible enough to me, since most of the industrial ASR systems studied in that paper are possibly NN-HMMs and CTC, which should be robust against adversarial attacks. \n5. Although the adversarial attack is mostly a machine learning research topic, the novelty and the required knowledge are mostly signal processing. Perhaps ICASSP would be a more suitable place to publish this work than ICLR. \n6. Contains many mistakes and typos, for instance:\n\n(a) \"Denote the ASR classification model as f : {X} → Y , with X ∈ [0, 1]t×c being the audio space.\"\n \nI guess c is the number of bits related to the floating number precision, but it is not defined at all.\n\n(b) The reference (Winata et al., 2020a) and (Winata et al., 2020b) are the same paper.\n\n(c) In the equation, L_{net} means n, e, t are three variables. Should be L_{\\text{net}} instead.\n\n...\n ", "summary_of_the_paper": "This paper proposed a novel idea that generates imperceptible adversarial attacks for ASR by perturbing the phase information. Proof that phase perturbations reduce the magnitude of the spectrogram is provided. White-box-threat-model-based experimental results showed the proposed method can generate effective adversarial examples for an academic Transformer-based ASR system with better efficiency than the state-of-the-art counterpart methods. ", "summary_of_the_review": "In conclusion, the paper presents a novel idea to perform adversarial attacks for audio-based systems, but the experiment setup and results need improvement. The amount of work required is perhaps more than what could be finished based on the time constraint. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "1_1RxUlTcp", "reviewer": "Reviewer_sPgr", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper provides an interesting perspective to generate stealthy adversarial audios. The paper is easy to follow and the proposed method is well-motivated.\nHowever, the evaluation of the proposed attack in the paper is a bit weak. For instance, the paper didn't evaluate the proposed attack against different adversarial audio defenses to evaluate the resiliency of the attack. \nIt would be helpful if the authors can test the generated adversarial audios against different ASR systems.\nFurther, it would be important and interesting to test the adversarial transferability and other properties of the generated adversarial audios. ", "summary_of_the_paper": "This paper proposes a phase-oriented algorithm PhaseFool to efficiently construct imperceptible audio adversarial attacks with energy dissipation.  The authors leverage the spectrogram consistency of STFT to adversarially transfer phase perturbations to the adjacent frames and dissipate the energy that is crucial for ASR systems. Empirical evaluations show that the attack effectiveness of the proposed attack is high.", "summary_of_the_review": "Overall the paper is well-organized and the proposed method is interesting.\nIf the authors can add necessary evaluation, such as the attack effectiveness comparison with other attacks against the state of the art defenses; the adversarial transferability analysis, and the adversarial evaluation against different ASR systems, I would like the increase my score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposed a novel phase-oriented algorithm to efficiently construct imperceptible audio adversarial attacks. It leverages the spectrogram consistency of STFT to adversarially transfer phase perturbations to the adjacent frames and dissipate the energy that is crucial for ASR systems. Empirical evaluations show that the attack effectiveness of the proposed attack is high.

As agreed by the reviewers the method is very interesting, but the experimental justification is limited, lacking strong SOTA baseline ASR systems, different ASR model architectures, the adversarial transferability analysis etc. The author did add DeepSpeech 2 results to the initial version's Low-rank Transformer only results, which is still not convincing enough. The author also commented that they will "add more comprehensive experiments with different systems and …architectures in our next version of the paper" which is not in the current paper yet. Hence, resubmission with more experimental evaluations is recommended. 

The decision is mainly due to the weak experimental justification.
Decision: Reject