How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective
Authors: Yimeng Zhang
Yuguang Yao
Jinghan Jia
Jinfeng Yi
Mingyi Hong
Shiyu Chang
Sijia Liu
Venue: ICLR 2022 Spotlight (2022)
Keywords: Zeroth-Order Optimization
Black-Box Defense
Gradient-Free
Adversarial Robustness
Certified Defense

Abstract: The lack of adversarial robustness has been recognized as an important issue for state-of-the-art machine learning (ML) models, e.g., deep neural networks (DNNs). Thereby, robustifying ML models against adversarial attacks is now a major focus of research. However, nearly all existing defense methods, particularly for robust training, made the white-box assumption that the defender has the access to the details of an ML model (or its surrogate alternatives if available), e.g., its architectures and parameters. Beyond existing works, in this paper we aim to address the problem of black-box defense: How to robustify a black-box model using just input queries and output feedback? Such a problem arises in practical scenarios, where the owner of the predictive model is reluctant to share model information in order to preserve privacy. To this end, we propose a general notion of defensive operation that can be applied to black-box models, and design it through the lens of denoised smoothing (DS), a ﬁrst-order (FO) certiﬁed defense technique. To allow the design of merely using model queries, we further integrate DS with the zeroth-order (gradient-free) optimization. However, a direct implementation of zeroth-order (ZO) optimization suffers a high variance of gradient estimates, and thus leads to ineffective defense. To tackle this problem, we next propose to prepend an autoencoder (AE) to a given (black-box) model so that DS can be trained using variance-reduced ZO optimization. We term the eventual defense as ZO-AE-DS. In practice, we empirically show that ZO-AE-DS can achieve improved accuracy, certiﬁed robustness, and query complexity over existing baselines. And the effectiveness of our approach is justiﬁed under both image classiﬁcation and image reconstruction tasks.

Published as a conference paper at ICLR 2022
HOW TO ROBUSTIFY BLACK-BOX ML MODELS? A GEROTH-ORDER OPTIMIZATION PERSPECTIVE
Yimeng Zhang Michigan State University
Yuguang Yao Michigan State University
Jinghan Jia
Michigan State University
Jinfeng Yi JD AI Research
Mingyi Hong University of Minnesota
Shiyu Chang UC Santa Barbara
Sijia Liu
Michigan State University
ABSTRACT
The lack of adversarial robustness has been recognized as an important issue for state-of-the-art machine learning (ML. models, e-g., deep neural networks (DNNs). Thereby, robustifying ML models against adversarial attacks is now major fo- cus of research. However, nearly all existing defense methods, particularly for robust training, made the white-box assumption that the defender has the access the details of an ML model (or its surrogate alternatives if available), e.g.. its architectures and parameters. Beyond existing works. in this paper we aim to address the problem of black-box defense: How to robustify a black-box model using just input queries and output feedback? Such problem arises in practical scenarios, where the owner of the predictive model reluctant to share model information in order to preserve privacy. To this end, propose general no- tion of defensive operation that can be applied to black-box models, and design it through the lens of denoised smoothing (DS), a first-order (FO) certified de- fense technique. To allow the design of merely using model queries, we further integrate DS with the zeroth-order (gradient-free) optimization. However, a di- rect implementation of zeroth-order (ZO) optimization suffers a high variance of gradient estimates, and thus leads to ineffective defense. To tackle this problem, we next propose to prepend an autoencoder (AE) to given (black-bon model so that DS can trained using variance-reduced ZO optimization. We term the eventual defense ZO-AE-DS. In practice, we empirically show that Z0-AE- DS can achieve improved accuracy, certified robustness, and query complexity over existing baselines. And the effectiveness of our approach is justified under both image classification and image reconstruction tasks. Codes are available at htttps://qitee wtt.tu..ccooaaaaaaaaaeeeeaeeeeae htt.://www......eaoooooeooeeoaaa fense
INTRODUCTION
ML models, DNNs in particular, have achieved remarkable success owing to their superior predictive performance. However, they often lack robustness. For example, imperceptible but carefully-crafted input perturbations can fool the decision of well-trained ML model. These input perturbations
refer to adversarial perturbations, and the adversarially perturbed (test-time) examples are known as adversarial examples or adversarial attacks (Goodfellow et al.) [2015] Carlini & Wagner 2017]
Papernot et al.) 2016). Existing studies have shown that it is not difficult to generate adversarial
attacks. Numerous attack generation methods have been designed and successfully applied to (i)
different use cases from the digital world to the physical world, e.g., image classification (Brown
ct al.
2017
Li et al. 2019; Xu et al., 2019 Yuan et al. 202I). object detection/trackin (Eykholl
et al.
2017
Xu et al. 2020 Sun et al. 2020). and image reconstruction (Antun et al., 2020 Raj
et al.
2020
Vasiljević et al. 2021), ). and (ii) different types of victim models, e.g., white-bol models
whose details can be accessed by adversaries (Madry et al.] 2018: Carlini & Wagner 2017 Tramer
ct al. 2020) Croce & Hemn 2020 Wang et al. 2021), and black box models whose information is not
disclosed to adversaries (Papernot et al. 2017 Tu et al. 2019 [lyas al. 2018a| Liang et al. 2021

---

Published as a conference paper at ICLR 2022
Given the prevalence of adversarial attacks, methods to robustify ML models are now major focus in research. For example, adversarial training (AT) (Madry et al. 2018), which has been poised one of the most effective defense methods (Athalye et al. 2018)) employed min-max optimization to minimize the worst-case (maximum) training loss induced by adversarial attacks. Extended from AT, various empirical defense methods were proposed, ranging from supervised learning. semi-supervised
learning, to unsupervised learning (Madry et al.)
2018 Zhang et al. 2019b Shafahi et al. 20119] Zhang al. 2019a; Carmon et al. 2019; Chen et detense, certified defense is another research focus, which aims to train provably robust ML models
2020; Zhang et al.) 202T). p. In addition to empirical
and provide certificates of robustness (Wong Kolter 2017; Raghunathan et al.) 2018 Katz et al.,
2017 Salman et al. 2019 2020, (021). Although exciting progress has been made in adversarial defense, nearly all existing works ask a defender to perform over while-box ML models (assuming non-confidential model architectures and parameters). However, the white-box assumption may restrict the defense application in practice. For example, a model owner may refuse to share the model details, since disclosing model information could hamper the owner's privacy, e.g., model inversion attacks lead to training data leakage (Fredrikson et al., 2015). Besides the privacy consideration, the white-box box defense built upon the (end-to-end) robust training (e.g., AT) is computationally intensive, and thus is difficult to scale when robustifying multiple models. For example, in the medical domain, there exist massive pre-trained ML models for different diseases using hundreds of neuroimaging datasets (Sisodiya et al. 2020). Thus, robustly retraining all models becomes impractical. Taking the model privacy and the defense efficiency into consideration, we ask:
Is it possible to design an adversarial defense over black-box models using only model queries?
Extending adversarial defense to the black-box regime (that we call 'black- box defense") is highly non-trivial due to the challenge of black-box op- timization (i.e., learning over black- box models). To tackle this prob- lem, the prior work (Salman et al.] 2020) leveraged surrogate models as approximations of the black-box models, over which defense can be conducted following the white- box setup. Yet, this still requires to have access to the information on the victim model type and its function. In practice, those conditions could be difficult to achieve. For example, if the domain knowledge related to medicine or healthcare is lacking (Qayyum et al.] 2020 Finlayson et al.) 2019). then it will be difficult to determine a proper surrogate model of medical ML system. Even if black-box model estimate can be obtained using the model inversion technique (Kumar & Levine 2019), a significantly large number of model queries are needed even just for tackling a MNIST-level prediction task (Oh et al.) 2219)). Different from (Salman et al. (2020), we study an authentic black- box scenario, in which the interaction between defender and model is only based on input-output function queries (see Fig.[] D. To our best knowledge, this is the first work to tackle the problem of query-based black-box detense.
Output foodback
Adverssriial attack
Defensive operation
Black-box model
Inference
R
Ostrich
Imput
query
Figure 1: Illustration of defense against adversarial attacks for entirely black-box models.
Contributions. We summarize our contributions below.
② (Formulation- wise) We formulate the problem of black-box defense and investigate it through the lens of zeroth-order (ZO) optimization. Different from existing works, our paper aims to design the restriction-leatt black-box defense and our formulation is built upon a query-based black-box setting, which avoids the use of surrogate models.
(Methodology-wii) wise)) We propose a novel black-box defense approach, ZO AutoEncoder Denoised Smoothing (ZO-AE-DS), which is able to tackle the challenge of 70 optimization in high dimensions and convert a pre-trained non-robust MI. model into a certifiably robust model using only function queries.
@ (Experiment-wise) We verify the efficacy of our method through an extensive experimental study. In the task of image classification, the proposed ZO-AE-DS significantly outperforms the ZO baseline built upon (Salman et al.) 2020). For instance, we can improve the certified robust accuracy of ResNet- 110 on CIFAR- 10 from 19.16% (using baseline) to 54.87% (using ZO-AE-DS) under adversarial perturbations with E, norm less than 64/255. We also empirically show that our proposal stays effective even in the task of image reconstruction.
2

---

Published as a conference paper at ICLR 2022
2 RELATED WORK
Empirical defense. An immense number of defense methods have been proposed, aiming to improve model robustness against adversarial attacks. Examples include detecting adversarial attacks (Guo et al. 2017: Meng & Chen 2017|Gong et al.
2017 Grosse al.
2017 Metzen et al.
2017) and training robust ML models (Madry et al. 2018] Zhang et al., 2019b Shafahi et al. 2019
Wong et al, 2020; Zhang et al., 2019a AAthayy et al., 2018 Cheng et al. 2017; Wong Kolter
2017. Salman et al., 2019 Raghunathan et al. [018 Katz et al., 2017). In this paper, we focus
on advancing the algorithm foundation of robust training over black-box models. Robust training can be broadly divided into two categories: empirical defense and certified defense. In the former category, the most representative method is AT (adversarial training) that formulates adversarial defense as a two-player game (between attacker and defender) (Madry et al. 2018). Spurred by AT, empirical defense has developed rapidly. For example, in (Zhang et al.) (019bb) TRADES was proposed to seek the optimal trade-off between accuracy and robustness. In (Stanforth al. (0199]
Carmon et al. 2019), unlabeled data and self-training were shown effective to improve adversarial defense in both robustness and generalization. In (Shafahi al. 2019] Wong al.) 2020) Zhhng iet al. 2019a| Andriushchenko & Flammarion 2020). to improve the scalability of adversarial defense, alternatives of AT were developed. Despite the effectiveness of empirical defense against adversarial attacks (Athalye et al., 2018), it lacks theoretical guarantee (known as 'certificate") > for the achieved robustness. Thus, the problem of certified defense arises.
computationally
Certified defense. Certified defense seeks to provide a provably guarantee of ML models. One line of research focuses on post-hoe formal verification of pre-trained ML model. The certified robustness is then given by a 'safe' input perturbation region, within which any perturbed inputs will not fool the given model (Katz et al. 2017 Fillrs 2017: Bunel et al. 2018 Dutta et al.! 2017).
Since the exact verification is computationally intensive, series of work (Raghunathan et al. 2018
Dvijotham et al.) 2018 Wong & Kolter 2017 Weng et al. 2018a by Wong et al.) (018)) proposed 'incomplete" verification, which utilizes convex relaxation to aver-approximate the output space of a predictive model when facing input perturbations. Such a relaxation leads to fast computation in the verification process but only proves lower bound of the exact robustness guarantee. Besides the post-hoc model verification with respect to each input example, another line of research focuses on in-processing certification-awal training and prediction. For example, randomized smoothing (RS) transforms an empirical classifier into provably robust one by convolving the former with an isotropic Gaussian distribution. It was shown in (Cohen et al., 2019) that RS can provide formal guarantees for ad adversarial robustness, Different types of RS -rioenete provable defenses have been developed, such as adversarial smoothing (Salman et al. 2019). denoised smoothing (Salman et al., 2020), smoothed ViT (Salman et al. 2021), and feature smoothing (Addepalli et al. 2021
Zeroth-order (ZO) optimization for adversarial ML. ZO optimization methods are gradient- free counterparts of first-order (FO) optimization methods (Liu et al., (2220)) They approximate the FO gradients through function value based gradient estimates. Thus, ZO optimization quite useful to solve black-box problems when explicit expressions of their gradients are difficult to compute or infeasible to obtain. In the area of adversarial ML, ZO optimization has become a principled approach to generate adversaria examples from black-box victim ML models (Chen et al. 2017 [lyss et al.)
2018alb bl Tu et al.] 2019; Liu et al. 2019 2020a Huang & Zhang 2020 Cai et al. 2020; Such 7o optimization- based attack generation methods can be as effective as ffaee-ofthhe-eee
white-box attacks, despite only having access to the inputs and outputs of the targeted model. For example, the work (Tu et al. (219)) leveraged the white-box decoder to map the generated low-dimension perturbations back to the original input dimension. Inspired by (Tu et al. 2019), we leverage the autoencoder architecture to tackle the high-dimension challenge of ZO optimization in black-box defense. Despite the widespread application of ZO optimization to black-box attack generation, few work studies the problem of black-box defense.
PROBLEM FORMULATION, BLACK-BOX DEFENSE
In this section, we formulate the problem of black-box defense, i.e., robustifying black-box ML models without having any model information such as architectures and parameters.
3

---

Published as a conference paper at ICLR 2022
Problem statement. Let fe... (x) denote pre-defined black-box (bb) predictive model, which can map an imput example x to a prediction. In our work, fe... can either an image classifier or an image reconstructor. For simplicity of notation, to black-box model. The threat model of our interest is given by norm-ball constrained adversarial attacks (Goodfellow et al.) 2013). To defend against these attacks, existing approaches commonly require the white-box assumption of f (Madry et al. 2018) or have access to white-box surrogate models of f (Salman et al. 2020). Different from the prior works, we study the problem of black-box defense when the owner of f is not able to share the model details. Accordingly, the only mode of interaction with the black-box system via submitting inputs and receiving the corresponding predicted outputs. The formal statement of black-box defense is given below:
will drop the model parameters bbb when referring
(Black-box defense) Given a black-box base model ʃ. can we develop defensive operation R using just input-output functitoo queries so as to produce the robustified model R((() against adversarial attacks?
Defensive operation. We next provide a concrete formulation of the defensive operation R. In the literature, two principled defensive operations were used: (R1) end-to-end AT (Madry al. 2018
Zhang et al. 2019b Cohen et al. 2019), and (R2) prepending a defensive component to a base model
(Meng & Chen 2017: Salman al. 2020 Aldahdooh et al. 202T). The former (R, ) has achieved
the state-of-the-an robustness performance (Athalye et al. 2018 Croce & Hein 2020) but is not applicable to black-box defense. By contrast, the latter (R2) is more compatible with black-box models. For example, denonised smoothing (DS), -eently-develope R2-type approach (Salman) jt al. 2020), gives a certified defense by prepending a custom-trained denoiser to the targeted model. In this work, choose DS as the backbone of our defensive operation (Fig. 2
In DS, a denoiser is integrated with a base model so that the augmented system becomes resilient to Gaussian noise and thus plays a role similar to the RS-based certified defense (Cohen et al. 2019). That is, DS yields
Inpot
Denoised example
Prediction
Dg White box deneiser
Blasks predictor
R(f(x)) == (x)))
(1)
((((X))
where De denotes the learnable denoiser (with Figure 2: DS-based black-bo defense.
parameters 0) prepended to the (black-box) predictor f. Once De is learned, then the DS-based smooth classifier, arg max, (o ( + 6)) = c). can achieve certified robustness, where is a class label, E (N o²), denotes the standard Gaussian noise with variance o², and arg max, [o
8) signifies a smooth version of f.
Based on ( the goal of black-box defense becomes to find the optimal denoiser Da so as to achieve satisfactory accuracy as well as adversarial robustness. In the FO learning paradigm, /allman et al.) (2020) proposed a stability regularized denoising loss to train De:
minimize 2 [|Do 8) x11 +s (CE(R(((x + 8)),
(
where U denotes the training dataset, the first objective term Denoise corresponds to the mean squared error (MSE) of image denoising, the second objective term E measures the prediction stability through the cross-entropy (CE) between the outputs of the denoised input and the original input, and 7 > 0 is regularization parameter that strikes a balance between Denoise and {
We remark that problem ( can be solved using the FO gradient descent method if the base model f is fully disclosed to the defender. However, the black-box nature of f makes the gradients of the stability loss Estb (0) infeasible obtain. Thus, we will develop a gradient-free DS-oriented defense.
4 METHOD: A SCALABLE ZEROTH-ORDER OPTIMIZATION SOLUTION
In this section, we begin by presenting brief background on ZO optimization, and elaborate on the challenge of black-box defense in high dimensions. Next, we propose a novel ZO optimization- based DS method that not only improve model query complexity but also lead to certified robustness.
4

---

Published as a conference paper at ICLR 2022
ZO optimization. In ZO optimization, the FO gradient of black-box function {(w) (with d-dimension variable w) is approximated by the difference of two function values along a set of random direction vectors. This leads to the randomized gradient estimate (RGE) (Liu et al. 2020b):
(w =
(f(w + uuu)) - ((w)) u;
where {u:}²=1 are q random vectors drawn independently and uniformly from the sphere of a unit ball, and fa is given small step size, known the smoothing parameter. The rationale behind
(3) is that it provides an unbiased estimate of the FO gradient of the Gaussian smoothing version of l (Gao Gao et al. 2018 with variance in the order of O (Liu al. 2020b). Thus, large-scale
problem (with large d) yields large variance of RGE (3) To reduce the variance, large number of querying directions (i.e., 9) is then needed, with the worst-case query complexity in the order of
Odd). If q = d, then the least estimation variance can be achieved by the coordinatewise gradient
estimate (CGE) (Lian et al. 2016 Liu al. 2018):
Ww((w) Σ f((w + ..c;) ((w)
(4)
1
where e; E Rd denotes the ith elementary basis vector, with at the ith coordinate and Os elsewhere. For any off-the-shelf FO optimizers, e.g., stochastic gradient descent (SGD), if we replace the FO gradient estimate with the ZO gradient estimate, then obtain the ZO counterpart of FO solver, e.g., ZO-SGD Ghadimi Lan 2013).
Warm-up: A direct application of ZO optimization. A straightforward method achieve the DS-based black-box defense is to solve problem (2) using ZO optimization directly. However, it will give rise to the difficulty of ZO optimization in high dimensions. Specifically, DS requires to calculate the gradient of the defensive operation [ With the aid of ZO gradient estimation, we obtain
(x) df(z)
dDo(x) V (zf(z) lx=
v)R(f(x)) = d0 dz
Dax))(x)
(5)
d0
where with an abuse of notation, let d denote the dimension of x (yielding Dø(x) E Rd and z E Rd)
and dg denote the dimension of 0, dDa(x) Rddxd is the Jacobian matrix of the vector-valued
de
function Do ()), and / (z) denotes the 70 gradient estimate of f. following 3 or (4). Since the dimension of input is typically large for image classification (e.g., d 3072 for CIFAR-10
image), it imposes two challenges: (a) The variance of RGE 33) will be ultra-large the query complexity stays low, i.e., small query number q is used; And (b)
Certinied rubusiness radios: 11.0)
accuracy (96
RO-D8
30.23
71,80
the variance-least CGE ) becomes impracticable due to the need of ultra-high querying cost (i.e., = d).
(RGE. g 192)
2T4DS
5.06 [) 25
44.81 [j 26.90)
Table I: Performance comparison between FO- DS (Salman et al. 2020) and its direct ZO imple- mentation ZO-DS (CIFAR-10, ResNet-110),
Indeed, Table shows that the direct application of (5) into the existing FO-DS solver (Salman et al. (2200). which we call ZO-DS, yields over 25% degradation in
both standard accuracy and certified robustness evaluated at input perturbations with & norm less than 128 255, where pixels of an input image are normalized to [0, 1]. We refer readers to Sec. for more details.
Zo autoencoder-based DS (Z0-AE-
DS): A scalable solution to black-box de- fense. The difficulty of ZO optimization in high dimensions prevents us from devel- oping effective DS-oriented provable defense for black-box ML models. To tackle such problem, we introduce Au- toencoder (AE) to connect the front-end denoiser De with the back-end black-box predictive model f so that ZO optimization can be conducted in a Nlowdimmnssion feature embedding space. To be concrete, let rhe 60.... denote AE consisting of the encoder (Enc) rhe and the
X
Excadier
Do
Black-box
White-ba
Figure Model architecture for ZE-AE-DS.
5

---

Published as a conference paper at ICLR 2022
decoder (Dec) @B where denotes the function composition operation. Plugging 6a between the denoiser Dg and the black-box. predictor f. we then extend the defensive operation to the following (see Fig. for illustration):
Ruew (f(x)) ==
: (00 (z))
2 Oprn (xx(x))
(6)
now black box
neW white box
where z E IR denotes the low oowdiimension feature embedding with d2 < d. In 6 we integrate the decoder d'epue with the black-box predictor to construct a new black-box model (' mwho (z)), which enables us to derive ZO gradient estimate of reduced dimension:
R2 d (Do(x)) (z) ( la ba= (Do(x))
new (f(x))
(7)
d0
Assisted by AE, RGE of VxJ' has a reduced variance from O to O(4)). Meanwhile, the least-variance CGE 44 also becomes feasible by setting the query number as q dz.
Note that the eventual ZO estimate 7 is a function of the dg x dz Jacobian matrix Ve @e (D(x))). For case of storing and computing the Jacobian matrix, derive the following computationally light alternative of (7) (see derivation in Appendix(A):
V Velstab ( ~ Vela aa @B (Do(x + 8)), a= (f)
l2=00pm. (Do(x+5))
(8)
where recall that (z) (Woraen (z)) and denotes the ZO gradient estimate given by 33 or (4).
The computation advantage of (8) is that the derivative operation Vo can be applied scalar-valued inner product built upon a pre-calculated ZO gradient estimate a.
Training ZO-AE-DS. Recall from Fig. 3 that the proposed defensive system involves three compo- nents: denoiser Dg. AE Vepe o @0 and pre-defined black-box predictor f. Thus, the parameters to be optimized include 0, 0Dee and Onee To train ZO-AE-DS, we adopt two-stage training protocol. @ White- White-box//e pre-training on AE: At the first stage, we pre-train the AE model by calling standard FO optimizer (e.g., Adam) to minimize the reconstruction loss Ex| Sxllo (We (x)) x 2- The resulting AE will be used as the initialization of the second-ster -ttge training. We remark that the denoising model Dg can also pre-trained. However, such a pre-training could hamper optimization, ie., making the second-stage training over & easily trapped at poor local optima. @ End-to-end training: At the second stage, we keep the pre-trained decoder óØD.. intact and merge into the black-box system as shown in Fig. 3 We then optimize O and Onne by minimizing the DS-based training loss (2), where the denoiser Dg and the defensive operation R are replaced by the o Do and Ruew (6). respectively. In (2). minimization over the stability loss Estab (0) calls the ZO estimate
of V Volstat given by In Appendix C.2 different training schemes are discussed.
EXPERIMENTS
In this section, demonstrate the effectiveness of our proposal through extensive experiments. We will show that the proposed ZO-AE-DS outperforms a series of baselines when robustifying black-box neural networks for secure image classification and image reconstruction.
5.1 EXPERIMENT SETUP
Datasets and model architectures. In the task of image classification, we focus on CIFAR-I0 and STL-10 datasets. In Appendix C.3 we demonstrate the effectiveness of ZO-AE-DS the high-dimension ImageNet images. In the task of image reconstruction, we consider the MNIST dataset. To build ZO-AE-DS and its variants and basclines, we specify the prepended denoiser Do as DnCNN (Zhang et al. 2017). We then implement task-specific AE for different datasets. Superficially. the dimension of encoded feature embedding, namely, d, in (6), is set as 192, 576 and 192 for CIFAR-10 10, STL-10 and MNIST, respectively. The architectures of AE are configured following MMo et al. 2016), and ablation study on the choice of AE shown in Appendix C.I To specify the black-box image classification model, we choose R 10 for CIFAR- 10 following (Salman et al. 2020). and ResNet- 18 for STL-10. It is worth noting that STL-10 contains 500 labeled 96 x 96 training images, and the pre-trained ResNeet 18 achieves 76.6% test accuracy that matches to state-of-te the-art performance. For image reconstruction, we adopt a reconstruction network consisting of convolution, deconvolution and ReL.U layers, following (Raj et al.! 2020).

---

Published as a conference paper at ICLR 2022
Baselines. We will consider two variants of our proposed ZO- ij ZO-AE-DS using RGE (3). ii ZO-AE-DS using CGE (4). In addition, we will compare ZO-AE-DS with i) FO-AE-DS. T.e., the first-order implementation of ZO-AE-DS, ti) FO-DS, which developed in (Salman et al. 2020), iä) RS-based certified training, proposed in (Cohen et al. 2019). and iv) ZO-DS, i.e., the 70 implementation of FO-DS using RGE. Note that CGE is not applicable to ZO-DS due to the obstacle of high dimensions. To our best knowledge, ZO-DS is the only query-based black-box defense baseline that can be directly compared with ZO-AE-DS.
Training setup. We build the training pipeline of the proposed 2O-AE-DS following "Training ZO-AE-DS' in Sec. To optimize the denoising model Do. we will cover two training schemes: training from scratch, and pre-training & fine-tuning. In the scenario of training from scratch, we use Adam optimizer with learning rate 10-3 to train the model for 200 epochs and then use SGD optimizer with learning rate 10 drop by a factor of 10 at every 200 epoch, where the total number of epochs is 600. As will be evident later, training from scratch over Dg leads to better performance of ZO-AE-DS. In the scenario of pre-training fine-tuning, we use Adam optimizer to pre-train the denoiser Do with the MSE loss Denoise in (2 for 90 epochs and fine-tune the denoiser with Estab for 200 epochs with learning rate 10-5 drop by a factor of 10 every 40 epochs. When implementing the baseline FO-DS, we use the best training setup provided by (Salman et al., 2020). When implementing ZO-DS, we reduce the initial learning rate to 10 for training from scratch and 10 6 for pre-training & fine-tuning to stabilize the convergence of ZO optimization. Furthermore,
set the smoothing parameter = 0.005 for RGE and CGE. And to achieve a smooth predictor, we set the Gaussian smoothing noise 5 E (O, o I) with 0² 0.25. With the help of matrix operations and the parallel computing power of the GPU, we optimize the training time to an acceptable range. The averaged one-epoch training time on a single Nvidia RTX A6000 GPU is about Imin and 29min for FO-DS and our proposed ZO method, AS (CGE, 9 = 192), on the CIFAR-10 dataset.
Evaluation metrics. In the task of robust image classification, the performance will be evaluated at standard test accuracy (SA) and certified accuracy (CA). Here CA is a provable robust guarantee of the Gaussian smoothing version of predictive model. Let us take ZO-AE-DS as an example, the resulting smooth image classifier is given by fsmooth (x) := arg max, P P& Rnnw ( (f(x + 8)) c). where Roew is given by (6). Further, a certified radius of fn-norm perturbation ball with respect
to an input example can be calculated following the RS approach provided in (Cohen et al., 2019).
As result, CA at a given f2-radius is the percentage of the correctly classified data points w whose certified radii are larger than r. Note that if
0, then CA reduces to SA.
RO
ZO-DS
20-AE-DS (Owrs)
fg-radius
RS
FO-DS
PO-AE-DS
q
(RGE)
7-
(RGE)
9-
(RGE)
g-
(RGE)
9 100
(RGE)
9- 192
(RGE)
g-
(CGE)
(SA)
76.41
71.80
75.97
44.81
42.72
38.61
0.25
60.64
$1.74
59.12
3.89
18:05
19.16
29.57
40.96
45.69
54.88
0.50
41.19
30.22
38.50
0.60
4.78
5.06
17.85
24.28
27.84
35.50
0.75
21.11
11.87
18.18
0.03
0.32
0.30
8.52
9,45
10.89
16.37
Table 2: SA (standard accuracy, %6 and CA (certified accuracy. %) versus different values of E2-radius r. Note that SA corresponds to the case of r = 0. In both FO and ZO blocks, the best accuracies for each lg fz-radius are highlighted in bold.
5.2 EXPERIMENT RESULTS ON IMAGE CLASSIFICATION
Performance on CIFAR-10. In Table we present certified accuracies of ZO-AE-DS and its variants/baselines versus different & radii in the setup of (CIFAR-10, ResNet-110). Towards a comprehensive comparison, different RGE- variants of ZO-AE-DS and ZO-DS are demonstrated using the query number q E [20, 100, 192). First, the comparison between ZO-AE-DS and ZO-DS shows that our proposal significantly outperforms ZO-DS ranging from the low query number q = 20 to the high query number q = 192 when RGE is applied. Second, we observe that the use of CGE yields the best CA and SA (corresponding to r 0). The application of CGE is benefited from AE, which reduces the dimension from d = 32 x 32 x 3 to d, = 192. In particular, CGEE-asee ZO-AE-DS improves the case studied in Table I from 5. 0.0 35.5% at the fy-radius
0.5. Third, although FO-AE-DS yields CA improvement over FO-DS in the white-box context, the improvement achieved by ZO-AE-DS (vs. ZO-DS) for black-box defense is much more significant. This implies
7

---

Published as a conference paper at ICLR 2022
that the performance of black-box defense relies proper solution (namely, ZO-AE-DS) to tackle the challenge of ZO optimization in high dimensions. Fourth, RS outperforms the ZO methods. This is not surprising since RS is known white-box certifiably robust training approach. In Appendix[B] we demonstrate the consistent effectiveness of ZO-AE-DS under different denoisier and classifiers.
Performance on 101.-10. In Table 3 we evaluate the performance of ZO-AE-DS for STL-10 image classification. For comparison, we also represent the performance of FO-DS, FO-AE-DS, and ZO-DS. Similar to T the improvement brought by our proposal over ZO-DS is evident, with at least 10% SA/CA improvement across different Ez-radii.
When comparing ZO-AE-DS with FO-DS, we observe that ours introduces 7% degradation in
STL-T0
fg-radius
FO-DS
FO-AE-DS
ZO.DS
ZO-AE-DS
(RGF, q 576)
(CGF, d, 576)
SA (at r = 0). This is differ- ent from CIFAR-10 classification. There might be two reasons for
(SA)
33,36
54.26
38.60
45.67
0.25
35.83
43.99
21.50
35.78
0.50
21.61
34.85
9.58
26.70
25.56
17.91
the degradation of SA in STL-10. First, the size of a STL-10 image is 9x larger than CIFAR-I0 im- age. Thus, the over-reduced feature dimension could hamper SA. In this example, we set d2 = 576. which is only 3x larger than d= = 192 used for CIFAR- 10 classification. Second, the variance of ZO gradient estimates has a larger effect on the performance of STL- than that of CIFAR-10, since the former only contains 500 labeled images, leading to challenging training task. Despite the degradation of SA, ZO-AE-DS outperforms FO-DS in CA, especially when facing large f--radius. This is consistent with Table The rationale is that AE can be regarded as an extra smoothing operation for the image classifee, and thus improves certified robustness over FO-DS, even if the latter is designed in a white-box setup. If we compare ZO-AE-DS with FO-AE-DS, then the FO approach leads to the best performance due to the high-aceuracy of gradient estimates.
Table 3: CA (certified accuracy, %) vs. different Eg-radi for image classification on STL--10.
Advantage of AE on ZO optimization. Ex- tended from Table 2 F presents the complete CA curve of non-AF-based and AF--based methods vs. the value of fg-raduus in the example of (CIFAR- 10, ResNet-110), As we can see, ZO-AE-DS using RGE with the smallest query number q 20 has outperformed ZO-DS using RGE with the largest query number q 192. This shows the vital role of AE on ZO optimization. Meanwhile, consistent with Table and Table the best model achieved by ZO-AE-DS using CGE could be even better than the FO bascline FO-DS since AE could play a sim- ilar role on the smoothing operation. Furthermore, as the query number increases, the improvement of ZO-AE-DS grows, towards the performance of FO-AE-DS.
FD-E:05
22 IKGE
251
20 AE (PGE.
150)
20 AE-35 (RGE.
192)
Certited Recurecy
PO-25
20-4C-35 (DGE. 192)
20 US (NGG, q 200
20: DS INGE, q 1000
70- [RGGF, . 192)
0.0
1.0
rodius Figure 4: Comparison between non-AF-based and AF--hased methods in CA vs. different Eg-radii. Dashed lines: Models obtained by non-AF-based methods: Solid lines: Models obtained by AE-based methods.
Effect of training scheme on ZO-AE-DS. In
Z0-AE-DS (CGE, 192)
T we present the impact of training scheme (over the denoiser Da) on the CA perfor- mance of ZO-AE-DS versus differrnnt fz-radii Two training schemes, training from scratch and pre-training + fine-tuning, are considered. As we can see, training from scratch for De leads to the better performance of ZO-AE-DS than pre-
(2-radius r
Training from scratch
Pre training + line tuning
72.23
59.74
0.25
54.87
42.61
0,50
35.50
26,26
0.75
16.37
11.13
Table 4: ZO-AE-DS using different denoiser training
schemes under (CIFAR- 10, ResNet-1100
training- + fine-tuning. This is because the application of pre-training to Do could make optimization easily get trapped at a local optima. We list other ablation studies Appendix
5.3 EXPERIMENT RESULTS ON IMAGE RECONSTRUCTION.
In what follows, we apply the proposed ZO-AE-DS to robustifying a black- image reconstruction network. The goal of image reconstruction is to recover the original image from noisy measurement.
8

---

Published as a conference paper at ICLR 2022
Following (Antun et al. 2020 Raj al. 2020), we generate the noisy measurement following linear observation model y Ax, where A a sub-sampling matrix (e.g.. Gaussian sampling). and is an original image. A pre-trained image reconstruction network (Raj al., 2020) then takes A T as the input to recover x. To evaluate the reconstruction performance, we adopt two metrics (Antun et al. 2020), the root SSIM is supplementary metric to RMSE, since it gives an accuracy indicator when evaluating the similarity between the true image and its estimate at fine-level regions. The vulnerability of image reconstruction networks to adversarial attacks, e.g. PGD attacks (Madry et al. 2018), has been shown Wolf 2019).
squared error (RMSE) and structural similarity (SSIM).
in (Antun et al. 2020 Raj et 2020
When the image reconstructor is given as a black-box model, spurred by above, Table presents the performance of image reconstruction using various training methods against adversarial attacks with different perturbation strengths. As we can see, compared to the normally trained image reconstructor (i.e., 'Standard' in T)ble(5). all robustification methods lead degraded standard image reconstruction performance in the non-adversarial context (i.e., ||0||2 0). But the worst performance is provided by ZO-DS. When the perturbation strength increases, the model achieved by standard training becomes over-sensitive to adversarial perturbations, yielding the highest RMSE and the lowest SSIM. Furthermore, we observe that the proposed black-box defense ZO-AE-DS yields very competitive and even better performance with respect FO defenses. In Fig.51 we provide visualizations of the reconstructed images using different approaches at the presence of reconstruction-evasion PGD) attacks. For example, the comparison between Fig. (f) and (b)(d) clearly shows the robustness gained by ZO-AE-DS.
Imige
MNIST
|6] -1
A1 1-3
|
Method
RMSE
SSIM
RMSE
SSIM
RMSE
SSIM
RMSE
SSIM
RMSE
SSIM
Standard
0.014
FO-DS
0.143
0.781
0.168
0.703
0.221
0.544
0.278
0.117
0.331
0.337
20-DS
0.197
0.521
0.217
0.474
0.262
0.378
0.313
0.284
00.356
0.225
FO-AE-DS
0.139
0.792
0.162
0.717
0.215
0.554
0.274
0.421
0.329
0.341
Z0-AE-DS
0.718
0.551
0.277
Table 5:
Performance of image reconstruction using different methods various attack scenarios. Here 'standard' refers the original image reconstructor without making any robustification. Four robustification methods are presented including FO-DS, ZO-DS (RGE, g = 192). FO-AE-DS and ZO-AE-DS (CGE, g = 192).
The performance metrics RMSE and SSIM are measured by adversarial example (x + 5), generated by 10-step & PGD attacks under different values of & perturbation norm ||0||2-
2
3
4
5
6
7
8
7
8
4
5
5
3
9
5
6
7
8
9
0
2
3
4
5
6
y
P
K
5
S
9 O
2
3
4
5
6
(a) Ground truth
(b) Standard
(c)
FO-DS
2
3
9
5
0
7
8
1 2
3
9
5
6
7 8
2 3
9
5
6
7
g
9
0
2
3
7
6
9
0
1
2
3
4
5 6
9 0
1
2
3
4
5
6
(d) ZO-DS
(e) FO-AE-DS
(i) ZO-AE-DS
Figure 5: Visualization for Image Reconstruction under l PGD attack (Step 40, 1.0). Original: base
reconstruction network. ZO-DS: RGE with q = 192. ZO-AE-DS: CGE with q = 192
CONCLUSION
In this paper, we study the problem of black-box defense, aiming to secure black-box models against adversarial attacks using only input-output model queries. The proposed black-box learning paradigm is new to adversarial defense, but is also challenging to tackle because of the black-box optimization nature. To solve this problem, integrate denoised smoothing (DS) with ZO (zeroth- order) optimization to build a feasible black-box defense framework. However, we find that the direct application of ZO optimization makes the defense ineffective and difficult to scale. We then propose ZO-AE-DS, which leverages autoencoder (AE) to bridge the gap between FO and ZO optimization. We show that ZO-AB-DS reduces the variance of ZO gradient estimates and improves the defense and optimization performance in significant manner. Lastly, we evaluate the superiority of our proposal series of baselines in both image classification and image reconstruction tasks.

---

Published as a conference paper at ICLR 2022
ACKNOWLEDGMENT
Yimeng Zhang, Yuguang Yao, Jinghan Jia, and Sijia Liu are supported by the DARPA RED program.
REFERENCES
Sravanti Addepalli, Samyak Jain, Gaurang Sriramanan, and R Venkatesh Babu. Boosting adversarial robustness using feature level stochastic smoothing. In Proceedings the IEEE/CVF Conference Computer Vision and Puttern Recognition, pp. 93-102, 2021.
Ahmed Aldahdooh, Wassim Hamidouche. Sid Ahmed Fezza, and Olivier Deforges. Adversarial example detection for dnn models: review. arXiv preprint arXiv:2105.0000 00203, 2021.
Maksym Andriushchenko and Nicolas Flammarion. Understanding and improving fast adversarial training.
arXiv preprint arXiv:2007.02617. 2020.
Vegard Antun, Francesco Renna, Clarice Poon. Ben Adcock, and Anders C Hansen. On instabilities of deep learning in image reconstruction and the potential costs of ai. Proceedings of the National Academy of Sciences, 117(48):30088 30095, 2020.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense security: Circumventing defenses to adversarial examples. arXiv preprint 1rXiv:/802.000 2018.
Tom B Brown, Dandelion Mané, Aurko Roy, Martín Abadi, and Justin Gilmer. Adversarial patch. arXiv preprint arXiv: 1XXiv:712.00663. 2017.
Rudy R Bunel, llker Turkaslan, Philip Torr, Pushmeet Kohli, and Pawan K Mudigonda. unified view of piecewise linear neural network verification. In Advances Neural Information Processing Systems, pp- 4790-4799, 2018
HanQin Cai, Daniel Mekenzie, Wotao Yin, and Zhenliang Zhang. Zeroth-order regularized optimization (zoro): Approximately sparse gradients and adaptive sampling. SIOPT, 2020.
HanQin Cai, Yuchen Lou, Daniel McKenzie, and Wotao Yin. A zeroth-order block coordinate descent algorithm for huge-scale black-box optimization. In International Conference on Machine Learning, pp. 1193-1203. PMLR, 2021.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In IEEE Symposium on S&P, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, Percy Liang, and John C Duchi. Unlabeled data improves adversarial robustness. arXiv preprint 19X.::1905.13736, 2019.
Pin- Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pp. 15-26. ACM, 2017.
T. Chen, S. Liu. S. Chang. Y. Cheng, L. Amini, and Z. Wang. Adversarial robustness: From self-supervised pretraining fine-tuning, In CVPR, 2020.
Chih-Hong Cheng, Georg Nührenberg, and Harald Ruess. Maximum resilience of artificial neural networks. In International Symposium Automated Technology for Verification and Analysis, pp. 251 2268. Springer. 2017.
Jeremy M Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. arXiv preprint arXiv: 02918. 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with ensemble of diverse parameter-free attacks. In International Conference on Machine Learning, pp. 2206-2216. PMLR, 2020.
Souradeep Dutta, Susmit Jha, Sriram Sanakaranarayanan, and Ashish Tiwari. Output range analysis for deep neural networks. arXiv preprint 1709 030. 2017.
Krishnamurthy Dvijotham, Robert Stanforth, Sven Gowal, Timothy Mann, and Pushmect Kohli. A dual approach to scalable verification of deep networks. UAI, 2018.
Ruediger Fhlers. Formal verification of piece-wise linear ford-forward neural networks. In International Symposium on Automated Technology for Verification and Analysis, pp. 269-286. Springer, 2017.
10

---

Published as a conference paper at ICLR 2022
Kevin Eykholt, Ivan Evtimov, Earlence Fernandes, Bo Li, Amir Rahmati, Chaowei Xiao, Atul Prakash, Ta- dayoshi Kohno, and Dawn Song. Robust physical-world attacks on deep learning models. arXiv preprint (tt::1707.08955 2017.
Samuel G Finlayson, John D Bowers, Joichi Ito, Jonathan I. Zittrain, Andrew Beam, and Isaac Kohane. Adversarial attacks on medical machine learning. Science, (68(6442):1288871889 2019.
Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence informa- tion and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pp. 1322 1333, 2015.
Xiang Gao, Bo Jiang. and Shuzhong Zhang. On the informmation-ddapllee variants of the admm: iteration complexity perspective. Journal of Scientific Computing, 76( 1332733333 2018.
Ghadimi and G. Lan. Stochastic first-and zeroth-ordo order methods nonconvex stochastic programming. SIAM Journal on Optimization, 23(4):2341-2368, 2013.
Zhitao Gong, Wenlu Wang, and Wei-Shinn Ku. Adversarial and clean data are not twins. arXiv preprint 2017.
Ian Goodfellow, Jonathon Shlens, and Christian Szegedy, Explaining and harnessing adversarial examples. ICLR, 2015.
Kathrin Grosse, Praveen Manoharan, Nicolas Papernot, Michael Backes, and Patrick McDanicl. On the (statistical) detection of adversarial examples. arXiv preprint arXiv: /X2.:0/022.6280 2017.
Chuan Guo, Mayank Rana, Moustapha Cissé, and Laurens van der Maaten. Countering adversarial images using input transformations. arXiv preprint 1rXiv:1711.001111 2017.
Zhichao Huang and Tong Zhang. Black-box adversarial attack with transferable model-based embedding. In International Conference on Learning Representations, 2020.
Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. Black-box adversarial attacks with limited queries and information. In International Conference on Machine Learning, pp. 2137-2146. PMLR, 2018a.
Andrew Iyas, Logan Engstrom, and Aleksander Madry. Prior convictions: Black-box adversarial attacks with bandits and priors. arXiv preprint arXiv: 1807.07978, 2018b.
Guy Katz, Clark Barrett, David L. Dill, Kyle Julian, and Mykel Kochenderfer. Reluplex: An efficient smt solver for verifying deep neural networks. In International Conference on Computer Aided Verification, pp. 97-117. Springer. 2017.
Aviral Kumar and Sergey Levine. Model inversion networks for model-based optimization. arXiv preprint 1xX1v:191..13466. 2019.
Juncheng Li. Frank Schmidt, and Zico Kolter. Adversarial camera stickers: A physical camera- attack on deep learning systems. In International Conference on Machine Learning, pp. 3896-3904, 2019.
X. Lian, H. Zhang, C.-J. Hsich, Y. Huang, and J. Liu. comprehensive linear speedup analysis for asynchronous stochastic parallel optimization from zeroth-order to first-order. In Advances in Neural Information Processing Systems, pp. 3054-3062, 2016.
Siyuan Liang. Baoyuan Wu, Yanbo Fan, Xingxing Wei, and Xiaochun Cao. Parallel rectangle flip attack: A query-based based black-box attack against object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 7697-7707, 2021.
S. Liu, B. Kailkhura, P.-Y. Chen, P. Ting, Chang, and Amini Zeroth-oree stochastic variance reduction for nonconvex optimization. Advances in Neural Information Processing Systems, 2018.
Liu, P.Y. Chen, X. Chen, M. Hong. signSGD via zeroth-order oracle. In International Conference on Learning Representations, 2019.
S. Liu, S. Lu, X. Chen, Y. Feng, K. Xu, A. Al-Dujaili, M. Hong, and U.-M. O O'Reilly. Min-max optimization without gradients: Convergence and applications to adversarial ML. In ICML, 2020a.
Sijia Liu, Pin-Yo Chen, Bhavya Kailkhura, Gaoyuan Zhang, Alfred O Hero III, and Pramod K Varshney, A primer zeroth-order optimization in signal processing and machine learning: Principals, recent advances, and applications. IEEE Signol Processing Magazine, 37(5):33-54, 2020b.
11

---

Published as a conference paper at ICLR 2022
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant adversarial attacks. ICLR, 2018.
Xiaojiao Mao, Chunhua Shen, and Yu-Bin Yang. Image restoration using very deep convolutional encoder- decoder networks with symmetric skip connections. Advances neural information processing systems, 29: 2802-2810, 2016.
Dongyu Meng and Hao Chen, Magnet: a two- evo-pronged defense against adversarial examples. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pp. 135-147, ACM, 2017.
Jan Hendrik Metzen, Tim Genewein, Volker Fischer, and Bastian Bischoff On detceting adversarial perturbations. arXiv preprint orXiv:/702 (72.:070..04267, 2017.
Seong Joon Oh, Bernt Schiele, and Mario Fritz. Towards reverse-engineering black-box neural networks. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pp. 121-144. Springer, 2019.
Nicolas Papemot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. Security and Privacy (EuroS&P), 2016 IEEE European Symposium on, pp. 372-387. IEEE, 2016.
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Berkay Celik, and Ananthram Swami. Practical black-box attacks against machine learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Cummunications Security, pp. 506-519. ACM, 2017.
Adnan Qayyum, Junaid Qadir, Muhammad Bilal, and Ala Al-Fuqaha. Secure and robust machine learning for healthcare: A survey. IEEE Reviews in Biomedical Engineering, 14:115-180, 2020.
Aditi Raghunathan, Jacob Steinhardt, and Percy Liang. Certified defenses against adversarial examples. In International Conference on Learning Representations, 2018, URL [tLL/ttts/////cecce/// n
Lorum? Dorrum?dd=Byaee Bd=BBy4ob- Rb
Ankit Raj. Yoram Bresler, and Bo Li. Improving robustness of deep-learning-bene image reconstruction. In International Conference on Machine Learning, pp. 7942. PMLR, 2020.
Hadi Salman, Greg Yang, Jerry Li, Pengchuan Zhang, Huan Zhang. Ilya Razenshteyn, and Sebastien Bubeck.
Provably robust deep learning via adversarially trained smoothed classifiers. classffirs.. arXiv preprint 1506:::90004455 2019.
Hadi Salman, Mingjic Sun, Greg Yang, Ashish Kapoor, and Zico Kolter. Denoised smoothing: A provable defense for pretrained classifiers. NeurIPS, 2020.
Hadi Salman, Saachi Jain, Eric Wong, and Aleksander Mądry. Certified patch robustness via smoothed vision transformers. arXiv preprint arXiv:21 2021.
Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! In Advances in Neural Information Processing Systems, pp. 3353-3364, 2019.
Sanjay M Sisodiya, Christopher D Whelan, Sean N Hatton, Khoa Huynh, Andre Altmann, Mina Ryten, Annamaria Vezzani, Maria Eugenia Caligiuri, Angelo Labate, and Antonio Gambardella. The enigma- epilepsy working group: Mapping disease from large data sets. Human brain mapping, 2020.
Robert Stanforth. Alhussein Fawzi, Pushmeet Kohli, et al. Are labels required for improving adversarial robustness? arXiv preprint arXiv:1905.13722 2019.
Jiachen Sun, Yulong Cao, Qi Alfred Chen, and Morley Mao. Towards robust lidar-based perception in autonomous driving: General black-box adversarial sensor attack and countermeasures. In 29th { {USENIX) Security Symposium, pp. 877-894, 2020.
Florian Tramer, Nicholas Carlini, Wieland Brendel, and Alcksander Madry, On adaptive attacks to adversarial example defenses. arXiv preprint arXiv:2002. 08347, 2020.
Dimitris Tsipras. Shibani Santurkar, Logan Engstrom. Alexander Turner, and Aleksander Madry. Robustness may be at odds with accuracy. In International Conference on Learning Representations, 2019. URL hctps://openrevieer
Chun Chen-hhe Tu, Paishun Ting, Pin- Yu Chen, Sijia Liu, Huan Zhang, Jinfeng Yi, Cho-Jui Jui Hsieh, and Shin-Ming Cheng. Autozoom: autoencoder-baseet zeroth order optimization method for attacking black-hox neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 742-749, 2019.
12

---

Published as a conference paper at ICLR 2022
Jelica Vasiljevié, Friedrich Feuerhake, Cédric Wemmert, and Thomas Lampert. Self adversarial attack augmentation method immunohistochemical stainings. In 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI), pp. 1939-1943. IEEE, 2021.
Yixiang Wang, Jiqiang Liu. Xiaolin Chang, Jianhua Wang, and Ricardo Rodríguez. Di-aa: An interpretable white-hox attack for fooling deep neural networks. arXiv preprint arXiv:2 10.07.30, 2021.
Tsui- Weii Weng, Huan Zhang, Hongge Chen, Zhao Song, Cho-Jui Hsich, Duane Boning, Inderjit Dhillon, and Luca Daniel. Towards fast computation certified robustness for relu networks. JCML, 2018a.
Tsui- Wei Weng, Huan Zhang. Pin-Y Chen, Jinfeng Yi. Dong Su, Yupeng Gao, Cho-Jui Hsieh, Luca Daniel. Evaluating the robustness of neural networks: An extreme value theory approach. JCLR, 2018b.
Adva Wolf. Making medical image reconstruction adversarially robust, 2019.
Eric Wong and J Zico Kolter. Provable defenses against adversarial examples via the convex outer adversarial polytope. arXiv preprint vrXiv:1111.00655 2017.
Eric Wong, Frank Schmidt, Jan Hendrik Metzen, and J Zico Kolter. Scaling provable adversarial defenses. In Advances Neural Information Processing Systems, pp.
2018.
Eric Wong, Leslie Rice, and Zico Kolter. Fast is better than free: Revisiting adversarial training. In ICLR, 2020.
Kaidi Xu, Sijia Liu,, Pu Zhao, Pin-Yu Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang. and Xue Lin. Structured adversarial attack: Towards general implementation and better interpretability. In ICLR, 2019.
Kaidi Xu, Gaoyuan Zhang, S. Liu, Quanfu Fan, Mengshu Sun, Hongge Chen, Pin-Yu Chen, Yanzhi Wang, and Xue Lin. Adversarial T-Shirt! evading person detectors physical world. ECCV, 2020.
Zheng Yuan, Jie Zhang, Yunpei Jía, Chuanqi Tan, Tao Xue, and Shiguang Shan. Meta gradient adversarial attack. In Proceedings of IEEF/CVF International Conference on Computer Vision, pp. 7748-7757, 2021.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing 7hu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. arXiv preprint arXiv: 1905.00877 2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. JCML., 2019b.
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE transactions on image processing, 26(7)3114-3116 2017.
Shudong Zhang. Haichang Gao, and Qingxun Rao. Defense against adversarial attacks by reconstructing images. IFFE Transactions on Image Processing, 30:61 17-6129, 2021.
13

---

further Published as a conference paper at ICLR 2022
A DERIVATION OF (8)
First, based on (2 and (6. the stability loss corresponding to ZO-AE-DS given by
Estab (0) tce (z) fxx)) g(z), where fz) (@0 (z)) z d'O (Do(x + +))
(9)
We then take the derivative of (stab (0) w.r.t. O. This yields
dz dg(z))
Volstab
(10)
d0 dz V0=ve (Do(x+8))
where dx Rdd and dalx) 30
de E RR4.
Since (z) involves the black-box function S. we first compute its 70 gradient estimate following 3
or 4 and obtain
dg(z)
Vzg(z) 1z=Vogmo (Do(x+6))
(11)
dz
2= (x+(x))))
Substituting the above into 110 we obtain
da d0;
da
V
o Stab (0)
dz
d82
da z
Vola Tola. (Do(x 8))]
(12)
d0
d0
da 2 d0
where the last equality holds based on (9) This completes the derivation.
B COMBINATION OF DIFFERENT DENOISERS AND CLASSIFIERS
Table presents the certified accuracies of our proposal using different denoiser models (Wide-
DnCnn vs. DnCnn) and image classifier (Vgg-16 16).
DnCán & VGG-16
Wide-DnCnn & VGG-16
ZO-AE-DS
ZO-AB-DS
E2-raduus
FO-DS
FO-AE-DS
FO-DS
FO-AE-DS
(CGE, y d, 192)
(CGE, d, 192)
0.00 (SA)
71.37
73.75
71.92
66.57
75.14
72.97
0.25
51.37
54.74
54.33
50.1
57.45
54,92
0.50
30.21
34.6
34.39
31.52
37.59
34.2
0.75
11.72
15.45
15.36
13.94
17.64
15.7
Table Al: CA (certified accuracy, %) different f2-radi for different combinations of denoisers and classifier.
C ADDITIONAL EXPERIMENTS AND ABLATION STUDIES
In what follows, we will show the ablation study on the choice of AE architectures in Appendix C Afterwards, we will show the performance of FO-AE-DS versus different training schemes in Appendix C.2 Finally, we will show the performance of our proposal on the high-dimension ImageNet images in Appendix C3
C.1 THE PERFORMANCE OF FO-AE-DS WITH DIFFERENT AUTOENCODERS.
Table. A2 presents the certified accuracy performance of FO-AE-DS with different autoencoders
(AE). As we can see, if AE-96 is used (namely, the encoded dimension is half AE-19 192 used in the paper), then observe slight performance drop. This is promising result reduce the query complexity by choosing a different autoencoder since the use of CGE has to be matched with the encoded dimension.
14

---

Published as a conference paper at ICLR 2022
E-ruuuus
AE-96
AE-192
0.00 (SA)
75.57
75.97
0.25
58.07
59.12
0.50
37.09
38.50 50
0.75
17.05
18.
Table A2: CA (certified accuracy, %) vs. different εz-radi for FO-AE-DS with different AutoEncoders.
C.2 THE PERFORMANCE OF FO-AE-DS WITH DIFFERENT TRAINING SCHEMES.
Table, A3 presents certified accuracy of FO-AE-DS (first-order implementation ZO-AE-DS) with different training schemes. Training both denoiser and encoder is the default setting, As we can see, only training the denoiser would bring performance degradation, and training both denoiser and AE does boost the performance. It is worth noting that FO-AE-DS with "train the denoiser and AE" training scheme can be regarded as the FO-DS treating the combination of the original denoiser and the same AE used in FO-AE-DS as new denoiser, which cannot be implemented for ZO-AF-DS since the decoder of ZO-AE-DS is merged into the black-box classifier and its parameters cannot be updated. Furthermore, the key of the introduced AE is to reduce the variable dimension for Zeroth-Order (ZO) gradient estimation.
FO-AF-DS (only train denoiser)
FO-AF-DS (train the denoiser encoder)
FO-AF-DS
f2-radius
FO-DS
(train the denoiser and the AE)
0.00 (SA)
71.80
73.34
75.97
75.76
0.25
51.74
55.61
59.12
58. 14
0.50
30.22
35.68
38.88
0.75
11.87
15.92
18.18
18.48
Table A3: CA (certified accuracy, %) vs. different f2-radi for FO-AE-DS with different training schemes.
C.3 THE PERFORMANCE OF ZO-AE-DS ON IMAGENET IMAGES.
To evaluate the performance of ZO-AE-DS on the Restricted ImageNet (R-ImageNet) dataset, 10-class subset of ImageNet with 38472 images for training and 1500 images for testing, similar to (Tiipaas et al. 2019). Due to our limited computing resources, we are not able to scale up our experiment to the full ImageNet dataset, but the purpose of evaluating on high-dimension images remains the same. In the implementation of ZO-AE-DS, we choose an AE with an aggressive compression (130:1), which is to compress the original 3 x 224 224 images into the 1152 1 x 1 feature dimension. We compare the certified accuracy (CA) performance our proposed ZO-AE-DS (using CGE) with the black-box baseline ZO-DS, and the white-box baselines FO-DS and FO-AE-DS. Results are summarized in the following table.
(1) when considering the black-box classifier, the proposed ZO-AE-DS still signif- icantly outperforms the direct ZO implementation of DS. This shows the importance of variance reduction of query-based gradient estimates. (2) Since ZO-AE-DS and FO-AE-DS used an aggressive AE structure, the performance drops compared to FO-DS. (3) the use of high-resolution images would make the black-box defense much more challenging. However, ZO-AE-DS is still a principled black-box defense method that can achieve reasonable performance.
As
ZO-AE-DS (RGE, q =1152 and encoder)
ZO-AE-DS (CGE, q=1152)
f2-radius r
FO-DS
FO-AE-DS
0.00 (SA)
89.33
71.07
26.93
63.60
0.25
81.67
63.40
18.40
52.80
0.50
68.87
53.60
11.67
43.13
0.75
49.80
42.87
5.53
32.73
Table A4: CA (certified accuracy, %) different fy-radi for FO-AE-DS on ImageNet Images.
15


Reviews:
[{"review_id": "d-XM7PAHnX8", "reviewer": "Reviewer_ULvk", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strength:\nThe the proposed ZO-AE-DS paradigm is novel to me. The idea of black-box defense is very interesting, which is actually a very important real-world application scenario. Compared to the competing method FO-DS, the authors eliminate the need for white-box model by applying zero-order optimization, and also use an autoencoder to avoid dimension issues for calculation. Then they demonstrate it with experiments that AE can also improve robustness. The paper is overall easy to follow, and clearly written. Experiments are thorough and well organized.\n\nWeakness:\nThe performance of ZO-AE-DS (zero-order) does not outperform FO-AE-DS (first-order) since the latter needs a surrogate model whereas ZO only requires access to model’s queries. The question is that it is not too hard to obtain a surrogate model if we have queries access. This paper can be further strengthened if benefit of avoiding surrogate models are articulated. \n\nTypo: “a recently-developed A2-type approach (Salman et al., 2020),” should be “R2-type”\n", "summary_of_the_paper": "This paper proposes a novel approach to robustify black-box models to address the problem of black-box defense, which arises due to the concerns of privacy. The approach proposed by the authors essentially incorporates zero-order optimization on a prepended denoising smoothing(DS) module, with an autoencoder connecting the DS and model, termed as “ZO-AE-DS”, which the authors claim that it can robustify models using only model queries.", "summary_of_the_review": "Question: Both ZO-AE-DS and FO-AE-DS outperforms their counterparts without AE module. Considering the denoising nature of AE, the DS+AE can be viewed as a type of stacked DS. Have the authors tried other combinations of stacked DS? Is it possible to also increase the performance like inserting AE to FO-DS? Overall I think the proposed zero-order optimization for black-box defense is a very promising application scenario. Although the method seems doesn’t outperform FO version with surrogates, I think it has broader applicability owning to its minimum assumptions.\n\n", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "bEZvIOScwM2", "reviewer": "Reviewer_RU4Z", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The contribution of this work may be relatively incremental. This is because the black-box defense setting has been considered in [1]. The differences between this work and [1] are: 1) query-based black-box defense is studied in this work 2) the authors introduce an AutoEncoder to reduce the variance of gradient estimation. Moreover, introducing AutoEncoder to promote gradient estimation has been studied in [2].\n\nMaybe, above comments are somewhat harsh. In fact, I have felt that. Thus, if the authors can highlight the contribution of this work, especially compared with [1], I am willing to raise my rating.\n\nI guess one contribution of this work may be that the authors have shown that query-based black-box defense is simple yet effective and improving the efficiency of gradient estimation is a good strategy for the query-based black-box defense setting.\n\n[1] Denoised Smoothing: A Provable Defense for Pretrained Classifiers. NeurIPS2020.\n[2] AutoZOOM: Autoencoder-based Zeroth Order Optimization Method for Attacking Black-box Neural Networks. AAAI2019.\n", "summary_of_the_paper": "This paper explores an exciting direction, i.e., black-box defense, where defense methods cannot access parameters of the target model when endowing the target model with certified adversarial robustness. However, the black-box defense setting considered in this paper has been studied in [1]. ", "summary_of_the_review": "I like this work, but I do hope the authors can further highlight the contribution, especially when comparing this work with [1].", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XXmBfknc8nJ", "reviewer": "Reviewer_Vo8C", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n- They show comparison with the baselines over a sufficiently wide range of experiments.\n- The argument for the use of the Autoencoder is intuitive and clearly works. The algorithm itself is simple, building on tools that have been known to work.\n- The algorithm itself is truly black-box in that it does not require knowledge of the model that is being robustified.\n\nConcerns/Comments:\n- Though it is fair that the algorithm is only compared against other black-box algorithms, it would be useful to show the certified accuracy(CA) of a whitebox algorithm like (Cohen et al., Madry et al.) as a comparison. This would illustrate the gap between the two regimes.\n- While the autoencoder reduces the variance of the gradient estimates, the fact that $2q$ forward passes need to be made for every gradient computation, should indicate that turning this method into a ZO-method increases the computational burden. I did not see this discussed in the paper. A comparison would be useful.\n- I believe there is a typo on page 4 where $\\mathcal{A}_2$ is used when it should be $\\mathcal{R}_2$. Either choice of nomenclature is fine as long as it is consistent.\n- Is there any justification for choosing $q=192$ in the experiments on Cifar10?\n- The authors mention that pre-training $D_{\\theta}$ causes it to get stuck at a local optima. However, this doesn't seem to happen to $\\theta_{ENC}$. I was also curious to see if any experiments were conducted where the autoencoder was fixed after pre-training and only $\\theta$ was trained. Does this perform poorly?\n- Is my understanding correct in that the accuracies reported in Table 2 are over different sets of data? It seems like the algorithm outputs a certified radius for each point and we compute the CA over only points which have a certified radius greater than $r$. I find this confusing because Salman et al. report a SA for every $r$ while in this setting SA only makes sense for $r=0$. Can you clarify?\n- In Table 2, RGE and CGE for $q=192$ are quite different. However, if the AE converges correctly then the random directions in RGE should all be linearly independent and provide an equivalent estimate to the standard basis directions. Is there any intuition for this difference?", "summary_of_the_paper": "This work provides an algorithm to ensure robust training of an ML model with just black-box knowledge of it i.e., input and output access. The algorithm relies on using Denoised Smoothing with zeroth-order optimization where the gradients are estimated using random perturbations (finite-differencing). They avoid the computational burden and high variance of these estimates by first training an auto-encoder to reduce the inputs to a low-dimensional subspace. They show over different architectures and multiple datasets (Cifar10, STL10, image reconstruction over MNIST) that the proposed algorithm performs better than the baseline which is the Denoised Smoothing algorithm by Salman et al.", "summary_of_the_review": "Overall, I find the algorithm to make sense intuitively and the experiments are quite thorough. The results are superior to the current baselines and therefore I am satisfied with this paper.\n\n~However, I have a few concerns and clarifications. If they are resolved, I would recommend the paper to be accepted.~\n[Update]: I am satisfied with the author's response. I am increasing my score.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "3DleckeK1OW", "reviewer": "Reviewer_ATPZ", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1.\tThe concept of black-box defense is novel and interesting, and this paper is the first work to tackle the problem of query-based black-box defense.\n2.\tThe author shows the idea of how to solve the black-box defense step by step. The paper ingeniously applies the zero-order optimization method to the DS method and leverages autoencoder to bridge the gap between first-order and zero-order optimization.\n3.  The paper is clearly written and easy to follow.\n\n\nCons:\n1.\tThe experiments are conducted on CIFAR-10 and STL-10. Also, It can be observed that the larger the picture, the worse the defense effect. Is this method still effective on ImageNet? If ZO-AE-DS cannot be applied to ImageNet, then the method is of little significance. After all, black-box defense is designed to solve the problem in real-world scenes.\n2.\tThe references are all before 2020. The authors should add more recent works. \n3.\tThe baselines are not strong enough. The experiments are all compared with DS and its variants.\n", "summary_of_the_paper": "The authors formulate the problem of black-box defense and propose a novel black-box defense approach called the Zero Order AutoEncoder-based Denoised Smoothing (ZO-AE-DS). Black-box defense corresponds to situations in which the defense model information cannot be obtained due to privacy protection in real scenarios. ZO-AE-DS introduces zero-order optimization on the structure of denoised smoothing (DS) to estimate the gradient and uses an Autoencoder (AE) to connect the denoiser with the model so that zero-order optimization can be conducted in a (low-dimension) feature embedding space.", "summary_of_the_review": "Overall, this paper formulates the problem of black-box defense and provides a feasible black-box defense framework. It is clearly written and easy to follow.The updated experiments are comprehensive and convincing.\n\n[Update]: Though the method still can not really handle ImageNet, the authors' rebuttal addresses most of my concerns. Also, the setting of black-box defenses is novel and should be encouraged. Hence, I am increasing my score to 8.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: All reviewers have converged to an unanimous rating of the paper, highlighting, in the paper or during the discussion, many strengths, including a compelling approach clearly relevant to applications and its solid range of experiments. 

A clear accept, and I would encourage the authors to push in the final version the experiments and discussions following the threads with reviewers (in particular, Vo8C and ULvk).

Thanks also to authors and reviewers for a thorough discussion which helped to strengthen further the paper's content.

AC.
Decision: Accept (Spotlight)