Defending against Adversarial Audio  via Diffusion Model
Authors: Shutong Wu
Jiongxiao Wang
Wei Ping
Weili Nie
Chaowei Xiao
Venue: ICLR 2023 poster (2023)
Keywords: Adversarial attack and defense
AI security
speech recognition
diffusion models

Abstract: Deep learning models have been widely used in commercial acoustic systems in recent years. However, adversarial audio examples can cause abnormal behaviors for those acoustic systems, while being hard for humans to perceive. Various methods, such as transformation-based defenses and adversarial training, have been proposed to protect acoustic systems from adversarial attacks, but they are less effective against adaptive attacks. Furthermore, directly applying the methods from the image domain can lead to suboptimal results because of the unique properties of audio data. In this paper, we propose an adversarial purification-based defense pipeline, AudioPure, for acoustic systems via off-the-shelf diffusion models. Taking advantage of the strong generation ability of diffusion models, AudioPure first adds a small amount of noise to the adversarial audio and then runs the reverse sampling step to purify the noisy audio and recover clean audio. AudioPure is a plug-and-play method that can be directly applied to any pretrained classifier without any fine-tuning or re-training. We conduct extensive experiments on the speech command recognition task to evaluate the robustness of AudioPure. Our method is effective against diverse adversarial attacks (e.g. L2 or L∞-norm). It outperforms the existing methods under both strong adaptive white-box and black-box attacks bounded by L2 or L∞-norm (up to +20% in robust accuracy). Besides, we also evaluate the certified robustness for perturbations bounded by L2-norm via randomized smoothing. Our pipeline achieves a higher certified accuracy than baselines.

Published as a conference paper at ICLR 2023
DEFENDING AGAINST ADVERSARIAL AUDIO VIA DIFFUSION MODEL
Shutong Wu Jiongxiao Wang'
Wei Ping
Weili Nie
Chaowei Xiao
Arizona State University "Shanghai Jiao Tong University NNIDIA
ABSTRACT
Deep learning models have been widely used in commercial acoustic systems in recent years. However, adversarial audio examples can cause abnormal be- haviors for those acoustic systems, while being hard for humans to perceive. Various methods, such as transformation-bart based defenses and adversarial train- ing, have been proposed to protect acoustic systems from adversarial attacks, but they are less effective against adaptive attacks. Furthermore, directly apply- ing the methods from the image domain lead suboptimal results because the unique properties of audio data. In this paper, we propose an adversar- ial purification-based defense pipeline, AudioPure, for acoustic systems via off- the-shelf diffusion models. Taking advantage of the strong generation ability of diffusion models. AudioPure first adds a small amount of noise the adversar- ial audio and then runs the reverse sampling step to purify the noisy audio and recover clean audio. AudioPure applied to any pretrained classifier without any fine-tuning or re-training. We conduct extensive experiments on speech command recognition task to evaluate the robustness of AudioPure. Our method is effective against diverse adversar- ial attacks (e.g. C2 or -norm). outperforms the existing methods under both strong adaptive white-box and black-box attacks bounded by L2 Loo- norm (up to +20% in robust accuracy). Besides, we also evaluate the certified ro- bustness for perturbations bounded by L2-norm via randomized smoothing. Our pipeline achieves a higher certified accuracy than baselines. Code is available at http://www...oo//yyy//atat////aa
plug-and-play method that can be directly
INTRODUCTION
Deep neural networks (DNNs) have demonstrated great successes in different tasks in the audio do- main, such as speech command recognition, keyword spotting, speaker identification, and automatic speech recognition. Acoustic systems built by DNNs (Amodei et al., [0166] [hhen et al.] 2019) are applied in
applications ranging from making phone calls controlling household se- curity systems. Although DNN- models have exhibited significant performance improvement,
extensive studies have shown that they are vulnerable to adversarial examples (Szegedy et al. 2014]
Carlini & Wagner 2018 Qin et al. 2019 Du et al. 2020 Abdullah et al. 2021 Chen et al. 202 where attackers add imperceptible and carefully crafted perturbations to the original audio to mis- lead the system with incorrect predictions. Thus, it becomes crucial to design robust DNN-based acoustic systems against adversarial examples.
To address it, existing works (e.g., Rajaratnam & Alshemali, 2018 Yang et al., 2019) have tried to leverage the temporal dependency property of audio to defend against adversarial examples. They apply the time-domain and frequency-domain transformations prove the robustness. Although they can alleviate this problem to some extent, they are still vulner- able against strong adaptive attacks where the attacker obtains full knowledge the whole acoustic system (Tramer et al., (2020). Another way to enhance the robustness against adversarial examples adversarial training (Goodfellow et al. 2015)] Madry et al. 2018) that adversarial perturbations have been added to the training stage. Although it has been acknowledged as the most effective defense, the training process will require expensive computational resources and the model still
the adversarial examples im-
work done during the internship ASU.

---

Published as a conference paper at ICLR 2023
vulnerable to other types of adversarial examples that are not similar to those used in the training process (Tramer& & Bonch 2019).
Adversarial purification (Yoon et al., 2021 Shi et al.) 2021 Nie et al. 2022) is another family of defense methods that utilizes generative models to purify the adversarial perturbations of the input examples before they are fed into neural networks. The key of such methods is to design an effective generative model for purification. Recently, diffusion models have been shown to be the state-of-the-art models for images (Song & Ermon 2019| Ho al. 2020 Nichol & Dhariwal 2021] Dhariwal & Nichol, 202T) and audio synthesis (Kong et al. 2022 Chen et al. (021b). It motivates the community to use it for purification. In particular, in the image domain, DiffPure (Nie et al.
applies diffusion models as purifiers and obtains good performance in terms of both clean and
robust accuracy on various image classification tasks. Since such methods do not require training the model with pre-defined adversarial examples, they can generalize to diverse threats. Given the significant progress of diffusion models made in the image domain, it motivates us to ask; possible to obtain similar success in the audio domain??
Unlike the image domain, audio signals have some unique properties. There are different choices of audio representations, including raw waveforms and various types of time-frequency representa- tions (e.g.. MMll spectrogram, MFCC). When designing an acoustic system, some particular audio representations may be selected the target features, and defenses that work well on some features may perform poorly on other features. In addition, one may think of treating the 2-D timefrequency representations (i.e., spectrogram) images, where the frequency-axis is set height and the time- axis is set as width, then directly apply the successful DiffPure (Nie et al. (022)) from the image domain for spectrogram. Despite the simplicity, there are two major issues: i) the acoustic system can take audio with variable time duration as the input, while the underlying diffusion model within DiffPure can only handle inputs with fixed width and height. ii) Even if we apply it in fixed-length segment-wise manner for the time being, it still achieves the suboptimal results as we will demon- strate in this work. These unique issues pose a new challenge of designing and evaluating defense systems in the audio domain.
In this work, we aim to defend against diverse unseen adversarial examples without adversarial training. We propose a play-and-plug purification pipeline named AudioPure based on pre-trained diffusion model by leveraging the unique properties of audio. In specific, our model consists of two main components: (1) waveform-bast diffusion model and (2) classifier. It takes the audio waveform as input and leverages the diffusion model to purify the adversarial audio perturbations. Given adversarial input formatted with waveform, AudioPure first adds a small amount of noise via the diffusion process to override the adversarial perturbations, and then uses the truncated reverse process to recover the clean sample. The recovered sample is fed into the classifier.
We conduct extensive experiments to evaluate the robustness of our method on the task of speech command recognition. We carefully design the adaptive attacks so that the attacker can accurately compute the full gradients to evaluate the effectiveness of our method. In addition, we also com- prehensively evaluate the robustness of our method against different black-box attacks and the Ex- pectation Over Transformation (EOT) attack. Our method shows a better performance under both white-box and black-box attacks against diverse adversarial examples. Moreover, we also evaluate the certified robustness of AudioPure via randomized smoothing, which offers provable guaran- tee of model robustness against Ca-based perturbation. We show that our method achieves better certified robustness than baselines. Specifically, our method obtains a significant improvement (up to +20% at most in robust accuracy) compared to adversarial training, and over 5% higher certified robust accuracy than baselines. To the best of our knowledge, we the first to use diffusion mod- els to enhance the security of acoustic systems and investigate how different working domains of defenses affect adversarial robustness.
2 RELATED WORK
Adversarial attacks and defenses. (Zegedy et al. (2014) introduce adversarial examples, which look similar to normal examples but will fool the neural networks to give incorrect predictions. Usually, adversarial examples are constrained by L, norm to ensure the imperceptibility. Recently,
stronger attack methods are emerging (Madry et al. 2018 Carlini & Wagner 2017 Andriishchennoo
et al. 2020; Croce& & Hein| 2020 Xiao et al. 20T8a 2019 2022bla
Cao al. 2019h a 2022a)
2

---

Published as a conference paper at ICLR 2023
Diffusion Model
Diffusion
Reverse
Adversarial
Purified
Audio
AudioPure
Audio
STFT Rescale
Classifler
Update adversarial audio using gradients backpropagated through
Figure 1: The architecture of the whole acoustic system protected AudioPure (black line in the figure) and the adaptive attack (orange line runs the reverse process to recover purified audio. Next, the purified audio is transformed into the spectrogram, and the spectrogram is fed into the classifier to get predictions. The attacker updates the adversarial audio based on the gradients backpropagated through SDE. Without AudioPure, the adversarial audio transfers to the spectrogram and feeds into the classifier directly.
figure). AudioPure first adds noise to adversarial audio and then
In the audio domain, Carlini & Wagner (2018) introduce audio adversarial examples, and Qin et al. 2019) manage make them more imperceptible. Black-box attacks (Du al. 2020: Chen al., 2021a a are also developed, aiming to mislead the end-to-end acoustic systems.
In order protect neural networks from adversarial attacks, different defense methods are proposed. The most widely used one is adversarial training (Madry et al. (2088), which deliberately uses adver- sarial examples as the training data of neural networks. The main problems of adversarial training are the accuracy drop of benign examples and the expensive computational cost. Many improved
versions of adversarial training aim to alleviate these problems (Wong et al.) 2020 Shafahi et al.,
2019 Zhang et al.) 2019b a Sun et al. [2021] Cao et al. 2022b, Zhhng et al. (019c).. Another line of work is adversarial purification Yoonn al. 2021 Shi et al.) 2021
[ Nie al., 2022), which uses gen- erative models to remove the adversarial perturbations before classification. Both of these two types of defenses are mainly developed for computer vision tasks and cannot be directly applied to the audio domain. In this paper, explicitly design a defense pipeline according to the characteristics of audio data.
Speech processing. Many speech processing applications vulnerable to adversarial attacks, including speech command recognition (Warden 2018). keyword spotting (Chen et al. 2014] Li
al. 2019), speaker identification (Reynolds et al. 2000 Ravanelli & Bengio [018]] [Snyder al.
2018), and speech recognition (Amodei et al.) 2016 Shen et al. 2019; Ravanelli et al. [019). In
particular, speech command recognition closely related to keyword spotting, and can viewed as
speech recognition with limited vocabulary. In this work, we choose speech command recognition as the testbed for the proposed AudioPure pipeline. The proposed pipeline is applicable for keyword spotting and speech recognition.
speech command recognition system consists of a feature extractor and classifier. The feature extractor processes the raw audio waveforms and outputs acoustic features, e.g. Mel spectrograms or Mel-frequency cepstral coefficients (MFCC). Then these features are fed into the classifier, and the classifier gives predictions. Given the 2-D spectrogram features, convolutional neural networks
for images are readily applicable (Simonyan & Zisserman 2015: He al., 2016 Zagoruyko &
Komodakis
2016 Xie al. 2017: Huang et 2017).
3 METHOD
3.1 BACKGROUND OF DIFFUSION MODELS
diffusion model normally consists forward diffusion process and a reverse sampling process. The forward diffusion process gradually adds gaussian noise to the input data until the distribution
3

---

Published as a conference paper at ICLR 2023
of the noisy data converges standard Gaussian distribution. The reverse sampling process takes the standard gaussian noise as input and gradually denoises the noisy data to recover clean data. At present, diffusion models can be divided into two different types: discrete-time diffusion models based on sequential sampling, such as SMLD Song & Ermon (2019), DDPM (Ho et al. 2020), and DDIM (Song al. 2021ap, and continuous-time diffusion models based on SDEs (Song al., 2021c). Song et al. (202lc) also build the connection between these two types of diffusion models.
Denoising Diffusion Probabilistic Models (DDPM) (Ho et al. 2020 is of the most widely used diffusion models. Many of the subsequently proposed diffusion models, including DiffWave for audio (Kong et al., 202T), are based on the DDPM formulation. In DDPM, both the diffusion and reverse processes are defined by Markov chains. For input data Xo E R we denote X ~ g(Xo) as the original data distribution, and X1, x1....; XN are intermediate latent variables from the distributions (x1 .x.).....00)) g(xx xx)-)), where N is the total number of steps. Generally, with a pre-defined or learned variance schedule 31, .I.....,9 .N (usually linearly increasing small constants), the forward transition probability g(x,. (nns) can be formulated as:
xx,, xn-1) (xxx v 8,,, B,1),
Based on the variance schedule {3n). set of constants is defined as:
an Sn, an II
an. Bn
1- an
Bay
(2)
B1,
1
n=1
and using the reparameterization trick, we have:
q(x,,|xo) Nxxi vān (1 I,
When n gradually gets larger to infinity, (xx,/xo) will converge to standard Gaussian distribution. Mcanwhile, for the reverse process, we have:
Xn-1 pa (xx= 1i n),, ou Xn,, n n)1),
(4)
where the mean term [4g] (X,, n) and the variance term of (x,, n) is instantiated by parameter 8. Ho et al. (2020); Kong et al. (2021)
neural network eg define na, and is fixed to a constant:
Bn
so(x..
o0(x., = VBn..
Ho (x n)
Xn
(5)
Van
on
We denote x, (x0, E) = Xanxo +
an)E, N(0, I), and the optimization objective is:
N
0r = arg max
X Ex() 6 co-oooaaxo + v(1 - -n),,,,
(6)
where An the weighting coefficient (Ho et al. 2020).
According to Song et al. (2021c), as N + DDPM becomes VP-SDE, a continuous-tims formu- lation of diffusion models. Particularly, the forward SDE formulated as:
dx -
-=====3()xx 3t)xxdt + v 3(ttdw..
where / E [0, 1], dt is an infinitesimal positive time step, w is standard Wiener process, B(t) is the continuous-time noise schedule. Similarly, the reverse SDE can be defined as:
dx = - 8()[(x [ + 2V lo (x)] + /B /B(t)dw,
(8)
where d! is an infinitesimal negative time step, and w a reverse-time standard Wiener process.
3.2 AUDIOPURE: A PLUG-AND-PLAY DEFENSE FOR ACOUSTIC SYSTEMS
To standardize the formulation of the defense, suggested by Nie al. 2022 continuous-tima formulation defined by Eq. 7 and Eq. 8 Note that since the existing pretrained DiffWave models (Kong et al. 202T) are based DDPM, will use their equivalent VP-SDE.
the

---

Published as a conference paper at ICLR 2023
If we use the Geller-aruyamaa method to solve the VP-SDE and the step size At x. the sampling of the reverse- reverse-time SDE will be equivalent to the reverse sampling of DDPM (detailed proofs can be
found in Song et al. (222cc)) Under this prerequisite, we have & where n E {1....,.A. N). We
define B( ( = Bn. a x = Onn 3(x) == Bn.. and x == X,, Given an adversarial example n' Fado the input at t 0, i.e. Xo Xadvs we first run the forward SDE from 0 to t N by solving Eq. 7 (it is equivalent to running n²" DPPM steps), which yields:
x((*) = V (ā(t*) Xadu + V1 - or(())) )2
2 ~ (0,
Next, we run the truncated reverse SDE from t= t* to 2022), we define an SDE solver sdeint that uses the Euler-Maruyama method, and sequentially takes in six inputs: initial value, drift coefficient, diffusion coefficient, Wiener process, initial time, and end time. The reverse output x0) at t = 0 can be formulated as:
by solving Eq. Similar to Nie al.
(0)= sdeint(x(t"), Srev: grev: W, 1 tt,t).
(10)
where the drift and diffusion coefficients are:
Sreu (x, 1) == : 3(1[xx
- + (x, ().
greu (1) = VBL).
(11)
Note that we use a diffusion coefficient different from Nie et al. (2022) for the purpose of cleaner output (see the detailed explanation in Section 3.3) Next, we use the discrete-time noise estima-
tor Eø (Xn, 12) to compute the continuous-time score estimator sp(x, t). By defining (x(x(t), t) ==
Eo (x( N). n)) (x,,,, n) with : N. the score function in the reverse VP-SDE can be estimated as:
E0 (x. sx(x V1 a(l)
Vx lr
(12)
Accordingly, x0),, the purified output of the adversarial input x(0) Xadus is fed into the later stages the acoustic system to make predictions. The whole purification operation can be denoted function Purifier R IR Rd
(Va("> )xedo + V1 (("²" N )z frevs Grev. w, N N:0)
Purifier(Xadee n²) sdeint
(13)
The acoustic systems are usually built on the features extracted from the raw audio. For example, the system can extract Mel spectrogram as the features: 1) it first applies short-time Fourier transfor- mation (STFT) on the time-domain waveform to get linear- spectrogram, and 2) it then rescales the frequency band to the Mel-scale. We denote this process as Wave2Mel IRd - ! x IR". which is a differentiable function. Then the classifier F 1T x R + R (usually convolutional network) takes the Mel spectrogram as the input and gives predictions.
Since both the time domain waveform and time frequeery domain spectrogram go through the pipeline, the purifier can be applied in either the time domain or time-frequency domain. If the purifier is applied in the time domain, the whole defended acoustic system AS Rdd R R can be formulated as:
AS(Xadv: n²)
(aatieeMM((Pufffrrrrx........ n))))
(14)
where the waveform Purifier is based on DiffWave
Meanwhile, if we want to purify the input adversarial examples the time-frequency domain, we can choose diffusion model used for image synthesis, and apply it to the output spectrogram of Wave2Mel. We denote this purifier as Purifiersspec 1 R" Rm x R", In this scenario, the whole defended acoustic system will be:
AS AS(xadv n²" F (PPrrffeersped (anveMMe((Xddx))) n²*)
(15)
The architecture of the whole pipeline is illustrated in Figure. [ For the purification in the time- frequency domain spectrogram, we use an Improved DDPM Nichol & Dhariwal 202T) trained on the Mel spectrograms of audio data and denote it DiffSpec. We compare these two purifiers and discover that the purification in the time domain waveform is more effective to defend against adversarial audio. Detailed experimental results can be found in Sec. 4.2
5

---

Published as a conference paper at ICLR 2023
3.3 TOWARDS EVALUATING AUDIOPURE
Adaptive attack For the forward diffusion process formulated Eq. 9 the gradients of the output x(t") w.t. the input x() a constant. For the reverse process formulated as Eq. [0] the adjoint method (Li et al. (2020) is applied to compute the full gradients of the objective function L w.E.t a(t") without any out-of-memory issues, by solving another augmented SDE:
x(1))
x(0)
free Öless z 8x
(Grer:1 )
(
w(1 w(1 11) -1 .0,0
BC
sdeint
ac Ox(()
(16)
(x((*))
where 1 and O represent the vectors of all ones and all zeros, respectively.
SDE modifications for clean output We observe that directly applying the framework of et al. (2022) to the audio domain will cause the performance degradation. That is, when converting the discrete-time reverse process of DiffWave (Kong et al. (021) to its corresponding reverse VP-SDE
in Eq. 8 the output audio still contains much noise, resulting lower classification accuracy. We identify two influencing factors and solve this problem by modifying the SDE formulation.
The first factor is the diffusion error due the mismatch of the reverse variance between the discrete and continuous cases. Ho et al. (2020) observed that both of = Sr and oî = B₁ get similar results experimentally in the image domain. However, we find that it is not the case in the audio modeling with diffusion models. For audio synthesis using DiffWave trained with o Be, if we switch the reverse variance schedule to o 3r the output audio becomes noisy. Thus, in Sec. 3.2 we define
((x) B, and use the diffusion coefficient grev match the variance 3, DiffWave.
/ B(t) Eq. 11 instead of grev / (t) to
The second factor is the inaccuracy from the continuous- time noise schedule B(t) = Bo ( Bo)t and a(t) e Ig (())ds used by Nie et al. (0022). The impact of the difference between 3(t) 9o + (3x- -0 and 3ive cannot be negligible, especially when N is not large enough (e.g. N 200 for the pretrained DiffWave model we use). Besides, when t is close to 0, @(t) e S6 3(s)da is not a good approximation of de any more. Thus, we define the continuous-time noise schedule directly based on the discrete schedule, namely, )(x)= B, and ā( (t) = an, for the purpose of better denoised output and more accurate gradient computation.
EXPERIMENTS
In this section, we first introduce the detailed experimental settings. Then we compare the perfor- mance of our method and other defenses under strong white-box adaptive attack where the attacker has full knowledge about the defense and black-box attacks. To further show the robustness of our method, we also evaluate the certified accuracy via randomize smoothing Cohen et al. (2019), which provides a provable guarantee of model robustness against L2 norm bounded adversarial perturba- tions.
4.1 EXPERIMENTAL SETTINGS
Dataset. Our method is evaluated on the task of speech command recognition. We use the Speech Commands dataset Warden 2018), which consists of 85,511 training utterances, 10, 102 validation utterances, and 4,890 tests utterances. Following the setting Kong et al. (202T). we choose the utterances which stand for digits 0 ~ 9 and denote this subset as SCO9.
Models. We use DiffWave (Kogg et al. 2021) and DiffSpec (based on Improved DDPM (Nichol & Dhariwal 2021)) as our defensive purifier,, which are representative diffusion models on the wave- form domain and the spectral domain respectively. We use the unconditional version of Diffwave with the officially provided pretrained checkpoints. Since the Improved DDPM model does not pro- vide the pretrained checkpoint for audio, we train it from scratch on the Mel spectrograms of audio from SC09. The training details and hyperparameters are in the appendix A For the classifier, use ResNeXt-29. 29-8-44(88i et al.) 2017) for spectrogram representation and M5 Net et al., 2017) for waveform except the experiments for ablation studies.

---

Published as a conference paper at ICLR 2023
Table 1: Performance against adaptive attacks among different methods.
f... whitc-bxx
Co white-box
C... black-hox
PGDDc PGDDan
PGD
PGila
PGDbo
PGD
PGDDo
PGDDo
PGDsa
PGD5a PGD..co
FAKEBOB
None Yang et al.
100
2
21
0
MS
Yans
100
6
4
1
1
0
21
16
LPP
Rajertram & Althetial
2018
100
2
0
0
0
20
BPF
Kapratman & Alshomall
2018
99
1
0
18
Madry of 2018
73
AUdiOPur6
97
89 89
89
85
84
84
89
86
83
85
84
86
Attacks. For white-box attacks, we use PGD (Madry al. 2018 with different iteration steps from 10 to 100 among T. and L2 norms. The attack budget is set to 0.002 for -norm constraint except the ablation study and 0.253 for L2 norm constraint. For black-box attacks, we apply query-based attack, FAKEBOB (Chen et al. (222Ia), and set the iteration steps to 200, NES samples to 200, and the confidence coefficient A 0.5.
Baselines. We compare our method with two types of baselines including: (I) transformation-barl
defense (Yang et al. 2019; Rajaratnam & Alshemali, 2018) including average smoothing (AS), median smoothing (MS), downsampling (DS), low-pass filter (LPF), and band-pass aadd-pass filter (BPF), and (2) adversarial training based defense (AdvTr) (Madry et al., 2018). For adversarial training, we follow the setting of Chen et al. (2022), using L. PGD1o with 0.002 and ratio 0.5.
4.2
MAIN RESULTS
We evaluate AudioPure (n*=3 by default) under adaptive attacks, assuming the attacker obtains full knowledge of our defense. We use the adaptive attack algorithm described in the previous section so that the attacker is able to accurately compute the full gradients for attacking. The results are shown in Table We find that the baseline munsformation-basee defenses (Yang al., 2019 Rajaratnam & Alshemali] 2018), including average smoothing (AS), median smoothing (MS), downsampling (DS), low-pass filter (LPF), and band-pass filter (BPF), virtually broken through by white-box attacks with up to 4% robust accuracy. For the adversarial vraining-based method (AdvTr) trained on -norm adversarial examples, although it achieves 71% robust accuracy against -based adversarial examples, such the method does not work so well on other types of adversarial examples (i.e.. Ca-based method), achieving 65% robust accuracy under Ca-based PDG100 attack. Compared with all baselines, AudioPure can obtain much higher robust accuracy, about 10% improvements on average, on -based adversarial examples, and is equally effective against Ca-based white-box attacks, achieving 84% robust accuracy.
We also evaluate AudioPure on black-box attacks including: FAKEBOB (Chnn al. 2021a) and transferability-baar attacks. The results of FAKEBOB are shown in Table [ indicating that our method can keep effective under the query-based black-box attack. The results the transferability- based attack are in the appendix B They draw the same conclusion. These results further verify the effectiveness of our method. All results indicate that AudioPure can work under diverse attacks with different types of constraints, while adversarial training has to apply different training strategies and re-train the model, making it less effective among unseen attacks than our method. We report the actual inference time in Appendix ! and compare out method with more existing methods in Appendix F G and H Additionally, we conduct experiments on the Qualcomm Keyword Speech Dataset (Kim al., 2019), and the results and details in Appendix B In this dataset, our method is still effective against adversarial examples.
4.3 ABLATION STUDY
PGD steps To ensure the effectiveness of PGD attacks, test different iteration steps from 10 to 150. As Figure. 2a illustrates, the robust accuracy converges after iteration steps n 2 70.
Effectiveness against Expectation sion and reverse process of AudioPure consist of many randomized sampling operations, we apply the expectation transformation (EOT) attack evaluate the effectiveness of AudioPure with
transformation (EOT) attack. Besides, since the diffu-

---

Published as a conference paper at ICLR 2023
100
200
Robust Accuracy
Robust: Accuracy
40
43
None
None
AdvTr
Adv
AudioPure
Audio Pure
steps
EOT
(a) robust accuracy different PGD steps
(b) robust accuracy with different EOT size
Figure 2: The performance baseline defense, denoted as None), adversarial training (denoted AdvTr), and AudioPure under attacks with different iteration steps EOT size. (a) indicates step convergence, and the attack is almost optimal when iterating over 70 steps. (b) shows that increasing EOT size can barely affect the robustness of our method.
The robust accuracy under PGD1o with different attack budget e when using different reverse steps n².. Larger requires larger n" to ensure better robustness.
Table 2:
Diffusion Steps
Attack Budget
n'
n²
=
0.004
76
86
83
74
66
0.008
27
85
84
68
Table 3: Ablation studies among different model architectures. The robust accuracy is evaluated under Coo- PGDzo. Our method is effective on various models with different architectures.
ResNeXt-29-8-66
VGG-19-BB
WedeRes
DenseNet-BCG BC-NNee---C-00-22
M5
Defease
Cleun
Robust
Clean
Robust
Clean
Robust
Clean
Clean
Robust
None
100
1
100
2
100
100
5
94
12
AudioPure
97
84
99
81
99
94
different BOT sample sizes. Figure 2b demonstrates the result. We find that AudioPure is effective among different EOT sizes.
Attack budget We evaluate the effectiveness of our method among different including e = (0.002, 0.004, 0.008, 0.0163. Since the diffusion steps n" are the hyperparameters for Au- dioPure, we conduct experiments among different n². As shown in Table 2 if n² is larger than 2, AudioPure will show strong effectiveness among different e. When increases, it requires a larger n" to achieve the optimal robustness since larger adversarial perturbation requires large noise from the forward process of the diffusion model to override the adversarial perturbations and the corresponding larger step recover purified audio. However, if the n* is too large, it will override the original audio information as well so that the recovered audio from the diffusion model will lose the original audio information, contributing to the performance drop. Furthermore, we explore the extent of the diffusion model for purification in Appendix
based classifier: Moreover, we apply AudioPure different classifiers, including spectrogram- 2at-:eseNee22--00/agguu
Architectures.
& Zisserman, 2015), ResNeXt-22-8-6Al et al. 2017]
Komodakis 2016) DenseNet-BCC-OO-- 102--aang et al. 2017 and
wave-form based classifier: M5 (Dai et al. 2017). Table 3 shows that method effective for various neural network classifiers.
Audio representations Audio has different types of representations including raw waveforms or time-freque frequeey representations (e.g., Mel spectrogram). We conduct an ablation study to show the effectiveness of diffusion models by using different representations, including DiffWave, diffusion model for waveforms (Kong et al., (02T)) and DiffSpec, a diffusion model for spectrogram based on the original image model (Nichol & Dhariwal 2021). The results are shown in Table 4 We find
8

---

Published as a conference paper at ICLR 2023
Table Ablation studies among different audio representations. We implement AudioPure using two differ- ent diffusion models as purifiers, Dive and DiftSpec, that respectively process the representations in the time domain and time-frequene frequency domain.
Defense
Clean
Ls white box
La white box
PGDD,o
PGDDg
PGDDs
PGDDg
PGDrg
PGDD
PGD
PGDDg0
PGD
PGD2
PGDD.0
DirWare 97
89
89
85
84
84
89
86
83
85
84
DifSSpec
99
92
78
73
72
71
62
54
49
Table 5: Certified accuracy for different methods. For each noise level a, add the same level of noise train the classifier and apply to RS Gaussian.
Noise
Certified radios (C))
0.25
0.50
075
1:25
1.50
RS Vanilla
a = 0.5
1.0
30
8
21
8
12
8
6
7
3
3
3
3
Gaussian
0.5
49
39
33
23
6
3
18
AucioPure
8 - 0.5
1.0
45
27
22
40
33
16
27
15
12
17
11
13
8
that the DiffWave consistently outperforms DiffSpec against L2 and Lic adversarial examples. Moreover, compared with DifjWave, despite DiffSpec achieve higher clean accuracy, only achieves 49% robust accuracy, a significant 35% performance drop against Ly-based adversarial examples. We think the potential reason is that the tbort-time Fourier transform (STFT) an operation of infor- mation compression. The spectrogram contains much less information than the raw audio waveform. This experiment shows that the domain difference contributes to significantly different results, and directly applying the method from the image domain can lead to suboptimal performance for audio. also verifies the crucial design of AudioPure for adversarial robustness.
4.4 CERTIFIED ROBUSTNESS
In this section, we evaluate the certified robustness of AudioPure via randomized @moothing(oohe et al. 2019). Here we draw N = 100, 000 noise samples and select noise levels E {0.5, 1.0 for certification. Note that we follow the same setting from Carlini et al. (2022) and choose to use the one-shot denoising method. The detailed implementation of our method could be found in Appendix
C We compare our results with randomized smoothing using the vanilla classifier and Gaussian augmented classifier, denoted RS- Vanilla and RS-Gausst Gaussian respectively. The results shown in Table We also provide the certified robustness under different L2 perturbation budget with different Gaussian noise a {0.5, 1.0] in Figure A Appendix C By observing we find that our method outperforms baselines for a better certified accuracy except a = 0.5 at radius. We also notice that the performance of our method will be even better when the input noise gets larger. This may be due to AudioPure can still recover the clean audio with a large Ca-based perturbation while Gaussian augmented model could even be converged when training with such large noise.
results,
5 CONCLUSION
In this paper, we propose an adversarial purification-based defense pipeline for acoustic systems. To evaluate the effectiveness of AudioPure, we design the adaptive attack method and evaluate our method among adaptive attacks, FOT attacks, and black-box attacks. Comprehensive experiments indicate that our defense more effective than existing methods (including adversarial training) among the diverse type of adversarial examples. We show AudioPure achieves better certifiable ro- bustness via Randomized Smoothing than other baselines. Morcover, our defense can be a universal plug-and-pen method for classifiers with different architectures.
Limitations. AudioPure introduces the diffusion model, which increases the time and computa- tional cost. Thus, how to improve time and computational efficiency is important future work. For example, it is interesting to investigate the distillation technique (Salimans & Ho 2022) and fast sampling method (Kong & Ping. 2021) reduce the computation complexity introduced by diffusion models.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENT
We thank Prof. Xiaolin Huang from Shanghai Jiao Tong University for the valuable discussions. Shutong Wu is partially supported by the National Natural Science Foundation of China (61977046), Shanghai Science and Technology Program (2251 105600), and Shanghai Municipal Science and Technology Major Project (2011SHDDZX0102)
ETHICS STATEMENT
Our work proposes defense pipeline for protecting acoustic systems from adversarial audio ex- amples. In particular, study focuses speech command recognition, which is closely related to keyword spotting systems. Such systems are well known to vulnerable to adversarial attacks. Our pipeline will enhance the security aspect of such real-world acoustic systems and benefit the social beings. The Speech Commands dataset used in our study are released by others and has been publicly available for years. The dataset contains various voices from anonymous speakers. To the best of our knowledge, it does not contain any privacy-related information for these speakers.
REFERENCES
Hadi Abdullah, Muhammad Sajidur Rahman, Washington Garcia, Kevin Warren, Anurag Swarnim Yadav, Tom Shrimpton, and Patrick Traynor. Hear" no evil", see" kenansville"* Efficient and transferable black-box attacks speech recognition and voice identification systems. In 2021 JEEE Symposium on Security and Privacy (SP), pp. 712-729. IEEE, 2021.
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. Deep speech 2: End-to- end speech recognition in english and mandarin. In International conference machine learning, pp. 173-182. PMLR, 2016.
Maksym Andriushchenko, Francesco Croce, Nicolas Flammarion, and Matthias Hein. Square at- tack: a query-efficient black-box adversarial attack via random search. In European Conference Computer Vision, pp. 484-501. Springer, 2020.
Yulong Cao, Chaowei Xiao, Benjamin Cyr. Yimeng Zhou, Won Park. Sara Rampazzi, Qi Alfred Chen, Kevin Fu, and Z. Morley Mao. Adversarial sensor attack on lidar-based perception in autonomous driving. CCS, 2019a.
Yulong Cao, Chaowei Xiao, Dawei Yang, Jing Fang, Ruigang Yang, Mingyan Liu, and Bo Li. Adver- sarial objects against lidar-based autonomous driving systems. arXiv preprint 2019b.
Yulong Cao, Chaowei Xiao, Anima Anandkumar, Danfei Xu, and Marco Pavone. Advdo: Realistic adversarial attacks for trajectory prediction. In European Conference on Computer Vision, pp. 36-52. Springer, 2022a.
Yulong Cao, Danfei Xu, Xinshuo Weng, Zhuoqing Mao, Anima Anandkumar, Chaowei Xiao, and Marco Pavone. Robust trajectory prediction against adversarial attacks. CORL (Oral), 2022b.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 20/7 ieee symposium security and privacy (sp), pp. 39-57. Iece, 2017.
Nicholas Carlini and David Wagner. Audio adversarial examples: Targeted attacks on speech-to- text. In 2018 IEEE security and privacy workshops (SPW), pp. 1-7. IEEE, 2018.
Nicholas Carlini, Florian Tramer, Zico Kolter, et al. (certified!!) adversarial robustness for free! arXiv preprint arXiv:2206.1000 2022.
Guangke Chen, Sen Chenb, Lingling Fan, Xiaoning Du, Zhe Zhao. Fu Song, and Yang Liu. Who is real bob? adversarial attacks on speaker recognition systems. 2021 IEEE Symposium on Security and Privacy (SP), pp. 694-711. IEEF, 2021a.
10

---

Published as a conference paper at ICLR 2023
Guangke Chen, Zhe Zhao, Fu Song, Sen Chen, Lingling Fan, Feng Wang, and Jiashui Wang. To- wards understanding and mitigating audio adversarial examples for speaker recognition. IEEE Transactions Dependable and Secure Computing, 2022.
Guoguo Chen, Carolina Parada, and Georg Heigold. Small- footprint keyword spotting using deep neural networks. In 2014 IEEE International Conference on Acoustics, Speech and Signal Pro- cessing (ICASSP), pp. 4087-4091. IEEE, 2014.
Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wave- grad: Estimating gradients for waveform generation. In International Conference on Learning Representations, 2021b.
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. International Conference on Machine Learning, pp. 1310-1320. PMLR, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustmess with an ensemble of diverse parameter-free attacks. In International conference on machine learning, pp. 2206- 2216. PMLR, 2020.
Wei Dai, Chia Dai, Shuhui Qu, Juncheng Li, and Samarjit Das. Very deep convolutional neural networks for raw waveforms. In 2017 IEEE international conference acoustics, speech and signal processing (ICASSP), pp. 421 421- IEEE, 2017.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis, Advances in Neural Information Processing Systems, 34:8780-8794, 2021.
Chris Donahue, Julian McAuley, and Miller Puckette. Adversarial audio synthesis. In International Conference on Learning Representations, 2018.
Tianyu Du, Shouling Ji, Jinfeng Li, Qinchen Gu, Ting Wang, and Raheem Beyah. Sirenattack: Generating adversarial audio for end-to-end acoustic systems. In Proceedings of the 15th ACM Asia Conference Computer and Communications Security, pp. 357-369, 2020.
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In International Conference on Learning Representations, 2015.
Shixiang Gu and Luca Rigazio. Towards deep neural network architectures robust to adversarial examples. arXiv preprint arXiv:1412.5068, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in 2020.
Neural Information Processing Systems,
Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv preprini arXiv:1908.00 (9t.:::000077. 02729, 2019.
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference computer vision and patlern recognition, pp. 4700-4708, 2017.
Sonal Joshi, Saurabh Kataria, Yiwen Shao, Piotr Zelasko, Jesus Villalba, Sanjeev Khudanpur, and Najim Dehak. Defense against adversarial attacks on hybrid speech recognition using joint ad- versarial fine-tuning with denoiser. arXiv preprint arXiv:2204.0385// 2022.
Byeonggeun Kim, Mingu Lee, Jinkyu Lee, Yeonseok Kim, and Kyuwoong Hwang. Query-by- example on-device keyword spotting, 2019.
Zhifeng Kong and Wei Ping. On fast sampling diffusion probabilistic models. arXiv preprint arXiv:2106.000 003.00 2021.
11

---

Published as a conference paper at ICLR 2023
Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, 2021.
Juncheng Li, Shuhui Qu, Xinjian Li, Joseph Szurley. Zico Kolter, and Florian Metze. Adversarial music: Real world audio adversary against wake-word detection system. Advances in Neural Information Processing Systems, 32, 2019.
Xuechen Li, Ting-Kam Leonard Wong, Ricky TQ Chen, and David Duvenaud. Scalable gradients for stochastic differential equations. In International Conference on Artificial Intelligence and Statistics, pp. 3870-3882. PMLR, 2020.
Yi I Luo and Nima Mesgarani. Conv-tasnet:
Surpassing ideal time-frequency magnitude masking for speech separation. IEEE/ACM transactions audio, speech, and language processing, 27(8): 1256-1266, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018
Muzammal Naseer, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Fatih Porikli. A self- supervised approach for adversarial robustness. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 262 222,,71,25 2020.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising difffusion probabilistic models. In International Conference on Machine Learning, pp. 8162-8171. PMLR, 2021.
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. In International conference on machine learning. PMLR, 2022.
Yao Qin, Nicholas Carlini, Garrison Cottrell, Ian Goodfellow, and Colin Raffel. Imperceptible, robust, and targeted adversarial examples for automatic speech recognition. In International con- ference machine learning, pp. 5231-5240. PMLR, 2019.
Krishan Rajaratnam and Basemah Alshemali. Speech coding and audio preprocessing for mitigating and detecting audio adversarial examples on automatic speech recognition. http::htt///cc /tct:://cs.ttt uocs. edu/ wtwpk//itt//oo/////////aaaaa///e
pdd.. 2018.
Mirco Ravanelli and Yoshua Bengio. Speaker recognition from raw waveform with sincnet. In 2018
IEEE Spoken Language Technology Workshop (SLT), pp. 1021-1028. IEEE, 2018.
Mirco Ravanelli, Titouan Parcollet, and Yoshua Bengio. The pytorch-kaldi speech recognition toolkit. In ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 6465-6469, IEEE, 2019.
Douglas Reynolds, Thomas Quatieri, and Robert B Dunn. Speaker verification using adapted gaussian mixture models. Digital signal processing, 10(1-3):19 2000.
Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models, arXiv preprint 0tt::2222.0052222 2022.
Hadi Salman, Mingjie Sun, Greg Yang, Ashish Kapoor, and J Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. Advances in Neural Information Processing Systems, 33:21945 229977 2020.
Pouya Samangouei, Maya Kabkab, and Rama Chellappa. Defense-gan gan: Protecting classifiers against adversarial attacks using generative models. In International Conference on Learning Represen- fations, 2018.
Simo Särkkä and Amo Solin. Applied Stochastic Differential Equations, volume 10. Cambridge University Press, 2019.
12

---

Published as a conference paper at ICLR 2023
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Ad- vances in Neural Information Processing Systems, 32, 2019.
Changhao Shan, Junbo Zhang, Yujun Wang, and Lei Xie. Attention-based end-to-end models for small-footprint keyword spotting. Proc. Interspeech 2018, pp. 2037-2041, 2018.
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, et al. Lingvo: a modular and scalable framework for sequence-to-sequenee modeling. arXiv preprint arXiv: 2019.
Changhao Shi, Chester Holtz, and Gal Mishne. Online adversarial purification based on self-
supervised learning. In International Conference on Learning Representations, 2021.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
David Snyder, Daniel Garcia- Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur. X- vectors: Robust dnn embeddings for speaker recognition. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp. 5329-532235 5333. IEEE, 2018.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna-
fional Conference on Learning Representations, 2021a.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score- based diffusion models. Advances in Neural Information Processing Systems, 14:1415-1428, 2021b.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Representations, 2021c.
Jiachen Sun, Yulong Cao, Christopher Choy, Zhiding Yu, Anima Anandkumar, Zhuoqing Morley Mao, and Chaowei Xiao. Adversarially robust 3d point cloud recognition using self-superrisions. NewrlPS, 34:1 14915598-5,,1,, 2021.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfel- low, and Rob Fergus. Intriguing properties of neural networks. In International Conference on Learning Representations, 2014.
Florian Tramer and Dan Boneh. Adversarial training and robustness for multiple perturbations. Advances in Neural Information Processing Systems, 32, 2019.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Aleksander Madry. On adaptive attacks to adversarial example defenses. Advances in Neural Information Processing Systems, 33:1633- 1645, 2020.
Pete Warden. Speech commands: dataset for limited-vocahulary speech recognition. arXiv 2018.
preprint
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020.
Chaowei Xiao, Bo Li, Jun- Yan Zhu, Warren He, Mingyan Liu, and Dawn Song. Generating adver- sarial examples with adversarial networks. In IJCAI, 2018a.
Chaowei Xiao, Jun-Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially trans- formed adversarial examples. In International Conference on Learning Representations, 2018b.
Chaowei Xiao, Dawei Yang. Bo Li. Jia Deng, and Mingyan Liu. Meshady: Adversarial meshes for visual recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 0898-6907,655
13

---

Published as a conference paper at ICLR 2023
Chaowei Xiao, Zhongzhu Chen, Kun Jin, Jiongxiao Wang, Weili Nie, Mingyan Liu, Anima Anand- kumar, Bo Li, and Dawn Song. Densepure: Understanding diffusion models towards adversarial robustness. arXiv preprint 2tth::2211.00322.. 2022a.
Chaowei Xiao, Xinlei Pan, Warren He, Jian Peng, Mingjie Sun, Jinfeng Yi. Mingyan Liu, Bo Li, and Dawn Song. Characterizing attacks on deep reinforcement learning. AAMAS, 2022b.
Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual trans- formations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1492-1500, 2017.
Zhuolin Yang. Bo Li, Pin-Yu Yu Chen, and Dawn Song. Characterizing audio adversarial examples using temporal dependency. In International Conference on Learning Representations, 2019.
Jongmin Yoon, Sung Ju Hwang, and Juho Lee. Adversarial purification with score- based generative models. In International Conference on Machine Learning, pp. 12062-12072. PMLR, 2021.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In British Machine Vision Conference 2016. British Machine Vision Association, 2016.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. Advances in Neural Information Processing Systems, 32, 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing, Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. In International conference on machine learning. pp. 7472-7482, PMLR, 2019b.
Huan Zhang, Hongge Chen, Chaowei Xiao, Bo Li, Duane Boning, and Cho-Jui Hsich. Towards stable and efficient training of verifiably robust neural networks. ICLR, 2019c.
14

---

Published as a conference paper at ICLR 2023
APPENDIX
A DETAILS ON TRAINING THE IMPROVE DDPM.
We train an Improved DDPM using the official repository diffusion). For the UNet model, set tmage_size num res blocks = 128. For diffusion flags, we set N = 200, 31 = 0.0001, 3x = 0.02 and the linear variance schedule. For the model training. set the learning rate to Ie 4 and the batch size to 230. The training loss has converged after 80,000 training steps, and we use this checkpoint build our purifier.
32, num_channels
3, and
B ADDITIONAL EXPERIMENTS OF TRANSFER-BAL -BASED ATTACK
We additionally evaluate our method under transfer-based attack, where we assume the attacker can only get the output logits of the acoustic system but have no knowledge about the used defense.
We use model functional stealing train surrogate model. Specifically, we first feed input ex- amples into the acoustic system consisting of DiffWave and a ResNeXt classifier and get the output logits. Then we use these output logits of the acoustic system as labels and train new surrogate ResNeXt model, which has the same architecture as the classifier in the acoustic system. The results are shown in Table A The Stealing Acc. denotes the accuracy of the surrogate classifier using the predictions of the defended acoustic system as ground truth. The Transfer to Vanilla and Trans- fer to Defended represent the undefended vanilla classifier and the defended acoustic system. The surrogate classifier is attacked to generate adversarial examples, and these adversarial examples are transferred to evaluate the robustness the undefended vanilla classifier and the defended acoustic system.
Table A: Transfer-based attack via model functional stealing. We train surrogate model, using the outputs of the defended acoustic system as labels. Then adversarial examples are generated by attacking the surrogate model and transferred the undefended vanilla classifier and the defended acoustic system.
Transfer to Vanilla
Transfer to Defended
Stealing Target
Stealing Acc.
Clean
Robust
Clean
Robust
AudioPure (n* 1) 100
100
22
100
AudioPure (n* 5)
98
100
58
96
94
C DETAILS ABOUT CERTIFIED ROBUSTNESS
Randomized smoothing (Cohen et al. 2019 provides a provable robustness guarantee in Ca-norm by evaluating models under noise. Usually. the performance of the vanilla classifier will degrade when feeding the Gaussian perturbed inputs. To alleviate this problem, we can re-train new net- work or fine-tune pretrained network on Gaussian augmented data. However, both of them could take a lot of time on training. Another way is to apply a denoiser before the vanilla classifier, named denoised smoothing (Salman et al.) 2020). Since the reverse process of the diffusion model can be seen as a good denoiser, we can use a pretrained diffusion model as plug-and-ple play method to make compute the corresponding diffusson
any model certifiably robust. For given noise level o, step t* which adds the same level of noise to the input examples. The diffusion process can be reformulated
I-ān ān
=
Va VI āiz = Vān +
z):
z N~ N(0,1)
(S17)
while the noisy input & of randomized smoothing is
x Xy + Voz,
N(O,
(S18)
15

---

Published as a conference paper at ICLR 2023
So obtain n² 1-ûn. after multiplying rescale coefficient vā, the input x.
On
According to Carlini et al. (2022). single reverse step is able to recover an image with high accuracy for the classifier and can largely save computational time by directly recovering the data through Xo -/2 (Xn, v1 Üi Eø (v, n)). So can just apply one-shot denoising instead of running full steps our reverse process.
Figure [A] shows the certified accuracy of AudioPure compared with RS-Gaussian and RS-V Vanilla. The results show that the certified robustness of our method is consistently better than baselines except at small certified radii when d 0.50.
sigma 0.5, N=100000
sigma 1.0, N N=100000
AudioPure
25
AudioPure
RS-Gaussian
RS-Gaussian
Certitied Accuracy
40
RS Da-Vanilla
Certified Accuracy
20
RS-Vanilla
30
15
20
10
10
0
3
4
Certified Radius
Certified Radius
Figure A: Certified robustness (C2) with different input noise level a. Larger a ensures better ro- bustness under larger perturbations, but the performance for benign inputs will be degraded.
D THEORETICAL ANALYSIS ON THE PURIFICATION ABILITY
Theorem D. 1. Assume that p(x) and g(x) are respectively the data distribution clean examples and the data distribution of adversarial examples. We Pt and qe to represent the respective distribution of (l) when xt)) ~ p(x) and x(1) when sxl) q(x). Then we have
a ODxl(pillp Øt
<0
(S19)
where the equality is established only Pt g. This inequality indicates that as increases from 0 to 1, the KL divergence of Pt and 9e monotonically decreases. In other words, when the diffusion steps n² increases, more of the adversarial perturbations will be removed. Considering that the original semantic information will also be removed if n* is too large, which affects the clean accuracy, there should be a trade-off when we set n for the diffusion model purifier.
Following Nie et al. (2022); ; Song al. (2021b). firstly formulate the Fokker-Planck equation (Säkkkä Solin 2019 of the forward SDE in Eq. (where we define f(x, 4) = (())
Proof:
and g(t) / B()) as:
@p.(x)
V f(z,t)p((z) 2 (t(()aa)(())
Øt
= - (fle,t)iio)) - V
(S20)
V. (1p(a, t)p((z))
where (a, = 2 (t) log fxx t). Assuming Pt and q are smooth and fast decaying, i.e. for any i 1,, 1,..., d, we have
a log 0,
lim (x(x) 2 2,000
lim (x Oz,
log (x())) 0
(S21)
Oz,
16

---

Published as a conference paper at ICLR 2023
for x.,, the i-th dimension of x R.. Then we reformulate the KL divergence Ö k (pellat)
Öt / (z(2)) log
pt(x) (x(x)
de
1 = =v (flx,t)pi()) -0:00)02999000 ( logp((z))
Pr(x) de +
pr(x)
v (1p(2, ())( log
V. (hp(1, ())
(1(
(1(x)
(S22)
/ (, h, t)] V (z) V log de (x))dd
((t) / p((z)|l V log q (x)1
(1)Dr())l)))
where (p(pellg:) is the Fisher divergence. Considering that g²( = B(t) > 0, and the Fisher divergence Dy (Pellllee (le)) > 0 and the equality is established only if Pt qc. result, we have Eq where the equality is established only if p: = qt-
S19
EXPERIMENTS ON THE QUAL.OMM KEYWORD SPEECH DATASET
In addition to the commonly used SCO9, for more comprehensive consideration, also conduct experiments on the Qualcomm Keyword Speech Dataset (Kim et al., 2019), denoted as QKW in the following. QKW consists of 4270 utterances belonging four classes, with variable durations from 0.48s to 1.92s. We split them into training set (3770 utterances), a validation set (400 utterances), and test (100 utterances). To handle the variable-sized input, we train an Attention Recurrent Convolutional Network (Shan et al. (2018) and save the checkpoint with the highest accuracy on the validation set. Then we finetuned the DiffWave model on QKW for 50,000 steps, with Ir 2e and
2 for 3 GPU. The results under L.o PGD with 0.002 are shown Table B We can observe that AudioPure can still achieve non-trivial robustness and handle the audio with variable time duration well.
Table B: We apply AudioPure to the Qualcomm Keyword Speech Dataset. The diffusion steps n²* is
set to 2.
Defense
Clean
Robust
None
100
0
AudioPure
91
61
F FINE-TUNING ON ADVERSARIAL EXAMPLES
AudioPure takes advantage of pretrained diffusion models. We wonder whether the purification performance will be improved if fine-tuned on adversarial examples. And we further fine-tune the
DiffWave model by augmenting self-supervised perturbation (SSP) (Naseer al. 2020). Specifi- cally, we use STFT (rescaling to the Mel-scale) as our feature extractor and maximize the following objective generate perturbed examples:
arg max A (z, z') STFT((
s.t. x' zo1
(S23)
x²'
where is the clean example and x' is the perturbed example. We then use gradient descent to optimize the perturbed example by:
2'41 clip(a'
sign (vA((, 6, +
(S24)
for /. = 1...., T. Here we use T = 100, 0.002 and = 0.0004. Next, we fine-tune the pretrained DiftWave model on the SSP examples, minimizing the following loss:
Lruning Landio XL
(S25)
17

---

Published as a conference paper at ICLR 2023
where
Luudio = MSE(2, Puriiferr(,n)))
(S26)
fent Fent MTS
(S27)
We choose À = 0.1 and use SGD to optimize Lruning. setting the learning rate to le - 5. The results are shown Table As result, does not improve the performance of AudioPure (with 3) under Lo PGD1o 10 and PGD with 0.002. These results further verify the effectiveness of using pretrained models.
Table C: We fine-tune the pretrained Wave model adversarial examples generated by SSP. After fine-tuning, the performance not improved.
Defense
Clean
PGD10
PGD7o
None
100
3
AudioPure
97
89
84
SSP-Tuned AudioPure
97
89
82
G COMPARISON WITH OTHER DENOISER-BAN BBSED DEFENSE
We compare AudioPure with DefenseGAN (Samangouei et 2018 and Joint Adversarial Fine- tuning (Joshi et al.] [2022). For DefenseGAN, which is originally designed to defend against adver- sarial images by finding the optimal noise that generates the most similar image to adversarial counterpart, we adopt it to the audio domain, choosing WaveGAN (Donahue et al., 2018 as the GAN model this pipeline. We train a WaveGAN on the SC09 dataset for 100 epochs, using the Adam optimizer with (r le 3. 8 0.5. and 32 0.9. For Joint Adversarial Fine-tuning, we follow the setting of Joshi et al. (2022). using a Conv-TasNet (Luo Mesgarani 2019) as the denoiser. And like Joshi al. (2022). craft offline adversarial SC09 dataset against the pre- Then train Conv-TasNet model OffAdv-SC09 for 30 epochs to get the pretrained denoiser. We denote the defense using the pretrained Conv-TasNe as CTN Baseline. Based on the adversarial ex- amples generated by attacking the whole acoustic system, we only update the Conv-TasNet denoiser Dur- ing the adversarial tuning, we use L. PGD1o attack with 0.002. After tuning for 1000 steps with batch_size size = 20, we calculate the clean and robust accuracy (under L.o PGD10 and PGDzo with 0.002) on the same test used in our paper.
trained classifier by using L- PGD- 100 attacks with c = 0.002 (denoted as OffAdv-SCoI
while keeping the classifier frozen, and denote this method as CTN Ady-Finetune-Join
We report the results in Table D We find that DefenseGAN based on WaveGAN cannot work well the audio domain. It shows the impact domain differences with respect to the final results and verifies the importance of our pipeline design. Besides, the Conv-TasNet denoiser is less effective than diffusion models against adaptive attacks, even after fine-tuning.
Table D; We compare AudioPure with different denoiser-basee based defenses, DiffWave is proven to be a more effective purifier.
Defense
Clean
PGD
PGD
None
100
AudioPure
97
89
84
DefenseGAN
8
0
0
CTN Baseline
98
13
1
CTN Adv-Finetune-Joimt-fuue
90
52
41
18

---

Published as a conference paper at ICLR 2023
H COMPARISON WITH THE REGULARIZATION-BASS DEFENSE
Gu Rigazio 2014); Hoffman et al. 2019) introduce the input-output Jacobian matrix of the net-
work as a regularization term in the optimization objective, formulated
Lreg Σ ((or, Ui) ^ll af(x))
||
(S28)
where 2; E R the input data, Yr E IR" the label, C Rd X + R is the original loss function, and : - R" is the neural network. By minimizing the Frobenius norm of the Jacobian matrix, the adversarial robustness of the network will be improved. For a more comprehensive study, we also compare AudioPure with this regularization-based method, using different A. The results shown in Table where we denote the regularization- -based defense as Jacobian-Reg.
Table E: We compare AudioPure with the regularization-baan defense, using different X.
Defense
Clean
PGD1o
PGD70
None
100
AudioPure
97
89
84
Jacobian-Reg (A=le-8)
45
5
Jacobian-Reg (A=1e-9)
84
27
15
Jacobian-Ree -eg (A=1e-10)
91
31
18
Jacobian-Reg (A=le-11)
96
19
4
I EXPERIMENTS ON LARGER ATTACK BUDGETS.
conduct additional experiments explore the po- tential of the diffusion model for purification. We select c = {0.01, 0.02, 0.03, 0.04, 0.05), and set the diffusion steps n The results are shown in Table We find that our method still achieves 42% accuracy at 0.03, which brings significant distortions to audio. Our method keeps the ability to purify adversarial perturbations until 0.05. We also visualize the audio waveforms under attacks with different e, illustrated in Figure B It is easy to observe significant noise in them.
Besides the results of different in Table
Table F: We explore the potential of DiffWave under larger attack budgets. The diffusion steps n²* is set to
Attack Budget
= 0.01
== 0.02
= 0.03
6 = 0.04
0.05
Robust Acc.
82
67
42
14
ADDITIONAL INFERENCE TIME COST.
Due to the introduction of diffusion models, AudioPure will bring additional time cost during infer- ence. As shown in Table G we compute the time cost per audio, averaged on 100 examples and the time duration for each example around one second. We evaluate it on an NVIDIA RTX 3090 GPU with Intel TM 19- CPU @ 3.50GHz and 64 GB RAM.
Table G: The inference time cost when using different diffusion steps n²*
Diffusion Steps
n*
n²*
n²
n²
7t"
n1
n=
10
Time Cost (s)
0.0967
0.5522
0.7876
1.0162
1.4795
2.0125
2.6839
19

---

Published as a conference paper at ICLR 2023
clean audio
adversarial audio with eps = 0.01
(b)
adversarial audio with eps 0.02
(c)
Figure B: Visualizations of the clean audio and adversarial audio with different attack budgets.
20

---

Published as a conference paper at ICLR 2023
adversarial audio with eps = 0.03
090
adversarial audio with eps = 0.04
(c)
adversarial audio with eps 0.05
(I)
Figure B: Visualizations of the clean audio and adversarial audio with different attack budgets.
21


Reviews:
[{"review_id": "vqITkah_ui", "reviewer": "Reviewer_NJPC", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper introduces a DiffPure inspired method (Diffusion models for adversarial robustness) in the context of speech-reco acoustic models. The adaptation of DiffPure, called AudioPure, operates in time domain with a diffusion process before the stft/mel feature extraction. The paper focuses on measuring the performance of different attack strategies, including white-box attacks where the diffusion approach really shines. ", "summary_of_the_review": "This is a good research, in term of adapting a very promising approach of adversarial robustness (DiffPure) from image to speech.\nBecause of that this work is not completely new, but more like a solid port of an existing strong baseline from a different modality to the speech domain. That said, the details and the strength of the results may still warrant relatively broad interest for the work, especially after few tweaks and improvements.", "strength_and_weaknesses": "The paper is very extensive, both in term of motivation, background, details of the experiments. \nThe performance especially for white-box / adaptive attacks is rather impressive. This seems to be a trend with the latest diffusion-based adversarial robustness methods. \nActual quantification of inference time increase / cpu requirements would be good to have to complete the picture in term of practical applicability of the method proposed.\nOne area about adversarial robustness is missing, the regularization based defenses, with seminal paper: https://arxiv.org/pdf/1412.5068.pdf\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "v9_uolF7tA6", "reviewer": "Reviewer_phaA", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes an adversarial purification-based defense strategy to defend against adversarial audio examples. This method is called AudioPure which is achieved by the off-the-shelf diffusion model's generation. AudioPure adds noise to the adversarial audio and then utilizes the reverse sampling step to purify the noisy audio. AudioPure is employed as a plug-and-play module for any pretrained classifier. The experiments are conducted on representative datasets to show the robustness achieved by AudioPure under white-box and black-box attacks. ", "summary_of_the_review": "This paper proposes a diffusion-based strategy for the adversarial defense of the acoustic systems. And my main concerns are the novelty and the experimental details.", "strength_and_weaknesses": "Strength:\nThis is the first work using the diffusion model to achieve the robustness of the acoustic systems, and the results demonstrate the effectiveness of the proposed method.\n\nWeakness:\n1. The experiments are only conducted on one dataset, and more datasets should be involved to show the generalization of the proposed strategy.\n\n2. As shown in Table 2, there are different optimized diffusion steps faced with different attack budgets. And how to decide the optimal value for an input audio with unknown budgets?\n\n3. If the diffusion model is finetuned according to the input adversarial audio samples, like the strategy of [A], will be performance be improved?\n\n[A] A self-supervised approach for adversarial robustness, CVPR2020", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "VzhP5Ys5rx", "reviewer": "Reviewer_fScq", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors address the problem of defending against adversarial audio attacks using audio purification based on recently popular diffusion models for recovering clean audio. Overall the idea is quite interesting and the experimental evaluation shows strong performance of the proposed idea. ", "summary_of_the_review": "Overall I really liked the idea of using the diffusion model to re-generate clean audio from adversarial audio in order to defend against adversarial attacks. The paper is well written with insightful experiments. ", "strength_and_weaknesses": "Strengths:\n1. Excellent idea which is well described\n2. The empirical evaluations are also well described and insightful. \n\nWeakness:\n1. the technical sections in Section 3 are still a difficult read and the presentation can be improved. \n2. It is not clear how the diffusion process to a certain extent removes the adversarial perturbation. I thought this would be the most interesting question to be addressed by this paper but I was not able to find a crisp answer to this question. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SxBLXHtqM68", "reviewer": "Reviewer_6825", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work proposes a technique called AudioPure where pre-trained diffusion-based speech denoisers are used for adversarial defense. The diffusion model is intended to be used in forward and backward steps to arrive from adversarial signal to purified audio. The paper proposes to use DiffWave and DiffSpec to be able to operate in time and time-frequency domains. The theory behind SDE is thoroughly discussed. SDE modifications to get cleaner audio is very interesting.\nThe choice of end application is Speech Command Recognition. There are many experiment sub-sections. Mainly the proposed technique is compared with Adversatial training method. Tuning experiments include varying PGD steps for attack, EOT size, n*, choice of classifier, \naudio features, and RS variance. In all experiments, proposed technique shows promise.", "summary_of_the_review": "The choice of problem is good. The choice of model (diffusion) is also interesting. The breath of experiments is impressive.\nAlthough as explained earlier, novelty is low. Investigating diffusion models specifically trained for cleaning adversarial audio should be prefered in revision of paper.\nAlso, stronger evaluation (higher eps) is needed while also comparing with DefenseGAN and denoiser defenses.", "strength_and_weaknesses": "Strengths:\n1. Providing code is encouraging.\n2. Breath of experiments is great.\n3. Paper is good step in promoting diffusion models for adversarial defense.\n\nLimitations:\n1. Using existing pre-trained models as defense may not be enough to satisfy novelty requirements. The training of DiffWave and DiffSpec is not modified to account for adversarial noise.\n2. Comparison with four-year old Adversarial training is not enough. DefenseGAN and denoiser defense are close to state-of-the-art defenses. Examples of such works:\n\nSamangouei, Pouya, Maya Kabkab, and Rama Chellappa. \"Defense-gan: Protecting classifiers against adversarial attacks using generative models.\" arXiv preprint arXiv:1805.06605 (2018).\n\nJoshi, Sonal, et al. \"Defense against Adversarial Attacks on Hybrid Speech Recognition using Joint Adversarial Fine-tuning with Denoiser.\" arXiv preprint arXiv:2204.03851 (2022).\n\n3. \"Limitation\" section is hastily written. It may be removed.\n4. I could not understand the intuition behind the proposal. Why should diffusion models work for cleaning adversarial signals?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster