Original: H OW TO R OBUSTIFY B LACK -B OX ML M ODELS ? A Z EROTH -O RDER O PTIMIZATION P ERSPECTIVE
Revised: H OW TO R OBUSTIFY B LACK -B OX ML M ODELS ? A Z EROTH -O RDER O PTIMIZATION P ERSPECTIVE

Original: Anonymous authors

Original: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The lack of adversarial robustness has been recognized as an important issue for state-of-the-art machine learning (ML) models, e.g., deep neural networks (DNNs).
Revised: The lack of adversarial robustness has been recognized as an important issue for state-of-the-art machine learning (ML) models, e.g., deep neural networks (DNNs).

Original: Thereby, robustifying ML models against adversarial attacks is now a major focus of research.
Revised: Thereby, robustifying ML models against adversarial attacks is now a major focus of research.

Original: However, nearly all existing defense methods, particularly for robust training, made the white-box assumption that the defender has the access to the details of an ML model (or its surrogate alternatives if available), e.g., its architectures and parameters.
Revised: However, nearly all existing defense methods, particularly for robust training, made the white-box assumption that the defender has the access to the details of an ML model (or its surrogate alternatives if available), e.g., its architectures and parameters.

Original: Beyond existing works, in this paper we aim to address the problem of black-box defense : How to robustify a black-box model using just input queries and output feedback?
Revised: Beyond existing works, in this paper we aim to address the problem of black-box defense : How to robustify a black-box model using just input queries and output feedback?

Original: Such a problem arises in practical scenarios, where the owner of the predictive model is reluctant to share model information in order to preserve privacy.
Revised: Such a problem arises in practical scenarios, where the owner of the predictive model is reluctant to share model information in order to preserve privacy.

Original: To this end, we propose a general notion of defensive operation that can be applied to black-box models, and design it through the lens of denoised smoothing (DS), a ﬁrst-order (FO) certiﬁed defense technique.
Revised: To this end, we propose a general notion of defensive operation that can be applied to black-box models, and design it through the lens of denoised smoothing (DS), a ﬁrst-order (FO) certiﬁed defense technique.

Original: To allow the design of merely using model queries, we further integrate DS with the zeroth-order (gradient-free) optimization.
Revised: To allow the design of merely using model queries, we further integrate DS with the zeroth-order (gradient-free) optimization.

Original: However, a direct implementation of zeroth-order (ZO) optimization suffers a high variance of gradient estimates, and thus leads to ineffective defense.
Revised: However, a direct implementation of zeroth-order (ZO) optimization suffers a high variance of gradient estimates, and thus leads to ineffective defense.

Original: To tackle this problem, we next propose to prepend an autoencoder (AE) to a given (black-box) model so that DS can be trained using variance-reduced ZO optimization.
Revised: To tackle this problem, we next propose to prepend an autoencoder (AE) to a given (black-box) model so that DS can be trained using variance-reduced ZO optimization.

Original: We term the eventual defense as ZOAE-DS.
Revised: We term the eventual defense as ZO-AE-DS.

Original: In practice, we empirically show that ZO-AE-DS can achieve improved accuracy, certiﬁed robustness, and query complexity over existing baselines.
Revised: In practice, we empirically show that ZO-AEDS can achieve improved accuracy, certiﬁed robustness, and query complexity over existing baselines.

Original: And the effectiveness of our approach is justiﬁed under both image classiﬁcation and image reconstruction tasks.
Revised: And the effectiveness of our approach is justiﬁed under both image classiﬁcation and image reconstruction tasks.

Revised: Codes are available at https://github.com/damon-demon/Black-Box-Defense .

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: ML models, DNNs in particular, have achieved remarkable success owing to their superior predictive performance.
Revised: ML models, DNNs in particular, have achieved remarkable success owing to their superior predictive performance.

Original: However, they often lack robustness.
Revised: However, they often lack robustness.

Original: For example, imperceptible but carefully-crafted input perturbations can fool the decision of a well-trained ML model.
Revised: For example, imperceptible but carefully-crafted input perturbations can fool the decision of a well-trained ML model.

Original: These input perturbations refer to adversarial perturbations , and the adversarially perturbed (test-time) examples are known as adversarial examples or adversarial attacks (Goodfellow et al., 2015; Carlini & Wagner, 2017;
Revised: These input perturbations refer to adversarial perturbations , and the adversarially perturbed (test-time) examples are known as adversarial examples or adversarial attacks (Goodfellow et al., 2015; Carlini & Wagner, 2017;

Original: Papernot et al., 2016).
Revised: Papernot et al., 2016).

Original: Existing studies have shown that it is not difﬁcult to generate adversarial attacks.
Revised: Existing studies have shown that it is not difﬁcult to generate adversarial attacks.

Original: Numerous attack generation methods have been designed and successfully applied to (i) different use cases from the digital world to the physical world, e.g. , image classiﬁcation (Brown et al., 2017;
Revised: Numerous attack generation methods have been designed and successfully applied to (i) different use cases from the digital world to the physical world, e.g. , image classiﬁcation (Brown et al., 2017;

Original: Li et al., 2019; Xu et al., 2019; Yuan et al., 2021), object detection/tracking (Eykholt et al., 2017; Xu et al., 2020; Sun et al., 2020), and image reconstruction (Antun et al., 2020; Raj et al., 2020; Vasiljevi´c et al., 2021), and (ii) different types of victim models, e.g. , white-box models whose details can be accessed by adversaries (Madry et al., 2018; Carlini & Wagner, 2017; Tramer et al., 2020; Croce & Hein, 2020; Wang et al., 2021), and black-box models whose information is not disclosed to adversaries (Papernot et al., 2017;
Revised: Li et al., 2019; Xu et al., 2019; Yuan et al., 2021), object detection/tracking (Eykholt et al., 2017; Xu et al., 2020; Sun et al., 2020), and image reconstruction (Antun et al., 2020; Raj et al., 2020; Vasiljevi´c et al., 2021), and (ii) different types of victim models, e.g. , white-box models whose details can be accessed by adversaries (Madry et al., 2018; Carlini & Wagner, 2017; Tramer et al., 2020; Croce & Hein, 2020; Wang et al., 2021), and black-box models whose information is not disclosed to adversaries (Papernot et al., 2017;

Original: Tu et al., 2019; Ilyas et al., 2018a; Liang et al., 2021).
Revised: Tu et al., 2019; Ilyas et al., 2018a; Liang et al., 2021).

Original: Given the prevalence of adversarial attacks, methods to robustify ML models are now a major focus in research.
Revised: Given the prevalence of adversarial attacks, methods to robustify ML models are now a major focus in research.

Original: For example, adversarial training (AT) (Madry et al., 2018), which has been poised one of the most effective defense methods (Athalye et al., 2018), employed min-max optimization to minimize the worst-case (maximum) training loss induced by adversarial attacks.
Revised: For example, adversarial training (AT) (Madry et al., 2018), which has been poised one of the most effective defense methods (Athalye et al., 2018), employed min-max optimization to minimize the worst-case (maximum) training loss induced by adversarial attacks.

Original: Extended from AT, various empirical defense methods were proposed, ranging from supervised learning, semi-supervised learning, to unsupervised learning (Madry et al., 2018;
Revised: Extended from AT, various empirical defense methods were proposed, ranging from supervised learning, semi-supervised learning, to unsupervised learning (Madry et al., 2018;

Original: Zhang et al., 2019b; Shafahi et al., 2019; Zhang et al., 2019a;
Revised: Zhang et al., 2019b; Shafahi et al., 2019; Zhang et al., 2019a;

Original: Carmon et al., 2019; Chen et al., 2020; Zhang et al., 2021).
Revised: Carmon et al., 2019; Chen et al., 2020; Zhang et al., 2021).

Original: In addition to empirical defense, certiﬁed defense is another research focus, which aims to train provably robust ML models and provide certiﬁcates of robustness (Wong & Kolter, 2017; Raghunathan et al., 2018; Katz et al., 2017;
Revised: In addition to empirical defense, certiﬁed defense is another research focus, which aims to train provably robust ML models and provide certiﬁcates of robustness (Wong & Kolter, 2017; Raghunathan et al., 2018; Katz et al., 2017;

Original: Salman et al., 2019; 2020; 2021).
Revised: Salman et al., 2019; 2020; 2021).

Original: Although exciting progress has been made in adversarial defense, nearly all existing works ask a defender to perform over white-box ML models (assuming non-conﬁdential model architectures and parameters).
Revised: Although exciting progress has been made in adversarial defense, nearly all existing works ask a defender to perform over white-box ML models (assuming non-conﬁdential model architectures and parameters).

Original: However, the white-box assumption may restrict the defense application in practice.
Revised: However, the white-box assumption may restrict the defense application in practice.

Original: For example, a model owner may refuse to share the model details, since disclosing model information could hamper the owner’s privacy, e.g. , model inversion attacks lead to training data leakage (Fredrikson et al., 2015).
Revised: For example, a model owner may refuse to share the model details, since disclosing model information could hamper the owner’s privacy, e.g. , model inversion attacks lead to training data leakage (Fredrikson et al., 2015).

Original: Besides the privacy consideration, the white-box defense built upon the (end-to-end) robust training ( e.g. , AT) is computationally intensive, and thus is difﬁcult to scale when robustifying multiple models.
Revised: Besides the privacy consideration, the white-box defense built upon the (end-to-end) robust training ( e.g. , AT) is computationally intensive, and thus is difﬁcult to scale when robustifying multiple models.

Original: For example, in the medical domain, there exist massive pre-trained ML models for different diseases using hundreds of neuroimaging datasets (Sisodiya et al., 2020).
Revised: For example, in the medical domain, there exist massive pre-trained ML models for different diseases using hundreds of neuroimaging datasets (Sisodiya et al., 2020).

Original: Thus, robustly retraining all models becomes impractical.
Revised: Thus, robustly retraining all models becomes impractical.

Original: Taking the model privacy and the defense efﬁciency into consideration, we ask:
Revised: Taking the model privacy and the defense efﬁciency into consideration, we ask:

Original: Is it possible to design an adversarial defense over black-box models using only model queries?
Revised: Is it possible to design an adversarial defense over black-box models using only model queries?

Original: [Figure]
Revised: [Figure]

Original: Extending adversarial defense to the black-box regime (that we call ‘blackbox defense’) is highly non-trivial due to the challenge of black-box optimization ( i.e. , learning over blackbox models).
Revised: Extending adversarial defense to the black-box regime (that we call ‘blackbox defense’) is highly non-trivial due to the challenge of black-box optimization ( i.e. , learning over blackbox models).

Original: To tackle this problem, the prior work (Salman et al., 2020) leveraged surrogate models as approximations of the black-box models, over which defense can be conducted following the whitebox setup.
Revised: To tackle this problem, the prior work (Salman et al., 2020) leveraged surrogate models as approximations of the black-box models, over which defense can be conducted following the whitebox setup.

Original: Yet, this still requires to have access to the information on the victim model type and its function.
Revised: Yet, this still requires to have access to the information on the victim model type and its function.

Original: In practice, those conditions could be difﬁcult to achieve.
Revised: In practice, those conditions could be difﬁcult to achieve.

Original: For example, if the domain knowledge related to medicine or healthcare is lacking (Qayyum et al., 2020; Finlayson et al., 2019), then it will be difﬁcult to determine a proper surrogate model of a medical ML system.
Revised: For example, if the domain knowledge related to medicine or healthcare is lacking (Qayyum et al., 2020; Finlayson et al., 2019), then it will be difﬁcult to determine a proper surrogate model of a medical ML system.

Original: Even if a black-box model estimate can be obtained using the model inversion technique (Kumar & Levine, 2019), a signiﬁcantly large number of model queries are needed even just for tackling a MNIST-level prediction task (Oh et al., 2019).
Revised: Even if a black-box model estimate can be obtained using the model inversion technique (Kumar & Levine, 2019), a signiﬁcantly large number of model queries are needed even just for tackling a MNIST-level prediction task (Oh et al., 2019).

Original: Different from (Salman et al., 2020). We study an authentic blackbox scenario , in which the interaction between defender and model is only based on input-output function queries (see Fig. 1).
Revised: Different from (Salman et al., 2020), we study an authentic blackbox scenario , in which the interaction between defender and model is only based on input-output function queries (see Fig. 1).

Original: To our best knowledge, this is the ﬁrst work to tackle the problem of query-based black-box defense.
Revised: To our best knowledge, this is the ﬁrst work to tackle the problem of query-based black-box defense.

Original: Contributions.
Revised: Contributions.

Original: We summarize our contributions below.
Revised: We summarize our contributions below.

Original: (cid:172) (Formulation-wise)
Revised: (cid:172) (Formulation-wise)

Original: We formulate the problem of black-box defense and investigate it through the lens of zeroth-order (ZO optimization.
Revised: We formulate the problem of black-box defense and investigate it through the lens of zeroth-order (ZO) optimization.

Original: Different from existing work, our paper aims to design the restriction-least black-box defense and our formulation is built upon a query-based black
Revised: Different from existing works, our paper aims to design the restriction-least black-box defense and our formulation is built upon a query-based black-box setting, which avoids the use of surrogate models.

Original: -box setting, which avoids the

Original: use

Original: of surrogate models.

Original: (cid:173)
Revised: (cid:173)

Original: (Methodology-wise
Revised: (Methodology-wise

Original: ) We propose a novel black-box defense approach, ZO AutoEncoder-based
Revised: ) We propose a novel black-box defense approach, ZO AutoEncoder-based

Original: Denoised Smoothing (ZO-AE-DS), which is able to tackle the challenge of ZO optimization in high dimensions and convert a pre-trained non-robust ML model into a certiﬁably robust model using only function queries.
Revised: Denoised Smoothing (ZO-AE-DS), which is able to tackle the challenge of ZO optimization in high dimensions and convert a pre-trained non-robust ML model into a certiﬁably robust model using only function queries.

Original: (cid:174) (Experiment-wise
Revised: (cid:174) (Experiment -wise)

Original: ) We verify the efﬁcacy of our method through an extensive experimental study.
Revised: We verify the efﬁcacy of our method through an extensive experimental study.

Original: In the task of image classiﬁcation, the proposed ZO-AE-DS signiﬁcantly outperforms the ZO baseline built upon (Salman et al., 2020).
Revised: In the task of image classiﬁcation, the proposed ZO-AE-DS signiﬁcantly outperforms the ZO baseline built upon (Salman et al., 2020).

Original: For instance, we can improve the certiﬁed robust accuracy of ResNet110 on CIFAR-10 from 19 .
Revised: For instance, we can improve the certiﬁed robust accuracy of ResNet110 on CIFAR-10 from 19 .

Original: 16% (using baseline) to 54 .
Revised: 16% (using baseline) to 54 .

Original: 87% (using ZO-AE-DS) under adversarial perturbations with (cid:96) 2 norm less than 64 / 255 .
Revised: 87% (using ZO-AE-DS) under adversarial perturbations with (cid:96) 2 norm less than 64 / 255 .

Original: We also empirically show that our proposal stays effective even in the task of image reconstruction.
Revised: We also empirically show that our proposal stays effective even in the task of image reconstruction.

Original:  R ELATED WORK
Revised:  R ELATED WORK

Original: Empirical defense.
Revised: Empirical defense.

Original: An immense number of defense methods have been proposed, aiming to improve model robustness against adversarial attacks.
Revised: An immense number of defense methods have been proposed, aiming to improve model robustness against adversarial attacks.

Original: Examples include detecting adversarial attacks (Guo et al., 2017;
Revised: Examples include detecting adversarial attacks (Guo et al., 2017;

Original: Meng & Chen, 2017;
Revised: Meng & Chen, 2017;

Original: Gong et al., 2017; Grosse et al., 2017; Metzen et al., 2017) and training robust ML models (Madry et al., 2018; Zhang et al., 2019b; Shafahi et al., 2019;
Revised: Gong et al., 2017; Grosse et al., 2017; Metzen et al., 2017) and training robust ML models (Madry et al., 2018; Zhang et al., 2019b; Shafahi et al., 2019;

Original: Wong et al., 2020; Zhang et al., 2019a; Athalye et al., 2018; Cheng et al., 2017; Wong & Kolter, 2017;
Revised: Wong et al., 2020; Zhang et al., 2019a; Athalye et al., 2018; Cheng et al., 2017; Wong & Kolter, 2017;

Original: Salman et al., 2019; Raghunathan et al., 2018; Katz et al., 2017).
Revised: Salman et al., 2019; Raghunathan et al., 2018; Katz et al., 2017).

Original: In this paper, we focus on advancing the algorithm foundation of robust training over black-box models.
Revised: In this paper, we focus on advancing the algorithm foundation of robust training over black-box models.

Original: Robust training can be broadly divided into two categories: empirical defense and certiﬁed defense.
Revised: Robust training can be broadly divided into two categories: empirical defense and certiﬁed defense.

Original: In the former category, the most representative method is AT (adversarial training) that formulates adversarial defense as a two-player game (between attacker and defender)
Revised: In the former category, the most representative method is AT (adversarial training) that formulates adversarial defense as a two-player game (between attacker and defender)

Original: (Madry et al., 2018).
Revised: (Madry et al., 2018).

Original: Spurred by
Revised: Spurred by

Original: AT, empirical defense has developed rapidly.
Revised: AT, empirical defense has developed rapidly.

Original: For example, in (Zhang et al., 2019b), TRADES was proposed to seek the optimal trade-off between accuracy and robustness.
Revised: For example, in (Zhang et al., 2019b), TRADES was proposed to seek the optimal trade-off between accuracy and robustness.

Original: In (Stanforth et al., 2019;
Revised: In (Stanforth et al., 2019;

Original: Carmon et al., 2019), unlabeled data and self-training were shown effective to improve adversarial defense in both robustness and generalization.
Revised: Carmon et al., 2019), unlabeled data and self-training were shown effective to improve adversarial defense in both robustness and generalization.

Original: In (Shafahi et al., 2019; Wong et al., 2020; Zhang et al., 2019a; Andriushchenko & Flammarion, 2020), to improve the scalability of adversarial defense, computationally-light alternatives of AT were developed.
Revised: In (Shafahi et al., 2019; Wong et al., 2020; Zhang et al., 2019a; Andriushchenko & Flammarion, 2020), to improve the scalability of adversarial defense, computationally-light alternatives of AT were developed.

Original: Despite the effectiveness of empirical defense against adversarial attacks (Athalye et al., 2018), it lacks theoretical guarantee (known as ‘certiﬁcate’) for the achieved robustness.
Revised: Despite the effectiveness of empirical defense against adversarial attacks (Athalye et al., 2018), it lacks theoretical guarantee (known as ‘certiﬁcate’) for the achieved robustness.

Original: Thus, the problem of certiﬁed defense arises.
Revised: Thus, the problem of certiﬁed defense arises.

Original: Certiﬁed defense.
Revised: Certiﬁed defense.

Original: Certiﬁed defense seeks to provide a provably guarantee of ML models.
Revised: Certiﬁed defense seeks to provide a provably guarantee of ML models.

Original: One line of research focuses on post-hoc formal veriﬁcation of a pre-trained ML model.
Revised: One line of research focuses on post-hoc formal veriﬁcation of a pre-trained ML model.

Original: The certiﬁed robustness is then given by a ‘safe’ input perturbation region, within which any perturbed input will not fool the given model (Katz et al., 2017; Ehlers, 2017; Bunel et al., 2018; Dutta et al., 2017).
Revised: The certiﬁed robustness is then given by a ‘safe’ input perturbation region, within which any perturbed inputs will not fool the given model (Katz et al., 2017; Ehlers, 2017; Bunel et al., 2018; Dutta et al., 2017).

Original: Since the exact veriﬁcation is computationally intensive, a series of work (Raghunathan et al., 2018;
Revised: Since the exact veriﬁcation is computationally intensive, a series of work (Raghunathan et al., 2018;

Original: Dvijotham et al., 2018; Wong & Kolter, 2017; Weng et al., 2018a;b; Wong et al., 2018) proposed ‘incomplete’ veriﬁcation, which utilizes convex relaxation to over-approximate the output space of a predictive model when facing input perturbations.
Revised: Dvijotham et al., 2018; Wong & Kolter, 2017; Weng et al., 2018a;b; Wong et al., 2018) proposed ‘incomplete’ veriﬁcation, which utilizes convex relaxation to over-approximate the output space of a predictive model when facing input perturbations.

Original: Such a relaxation leads to fast computation in the veriﬁcation process but only proves a lower bound of the exact robustness guarantee.
Revised: Such a relaxation leads to fast computation in the veriﬁcation process but only proves a lower bound of the exact robustness guarantee.

Original: Besides the post-hoc model veriﬁcation with respect to each input example, another line of research focuses on in-processing certiﬁcation-aware training and prediction.
Revised: Besides the post-hoc model veriﬁcation with respect to each input example, another line of research focuses on in-processing certiﬁcation-aware training and prediction.

Original: For example, randomized smoothing (RS) transforms an empirical classiﬁer into a provably robust one by convolving the former with an isotropic Gaussian distribution.
Revised: For example, randomized smoothing (RS) transforms an empirical classiﬁer into a provably robust one by convolving the former with an isotropic Gaussian distribution.

Original: It was shown in (Cohen et al., 2019) that RS can provide formal guarantees for adversarial robustness.
Revised: It was shown in (Cohen et al., 2019) that RS can provide formal guarantees for adversarial robustness.

Original: Different types of RS-oriented provable defenses have been develped, such as adversarial smoothing (Salman et al., 2019), denoised smoothing (Salman et al., 2020), smoothed ViT (Salman et al., 2021), and feature smoothing (Addepalli et al., 2021).
Revised: Different types of RS-oriented provable defenses have been developed, such as adversarial smoothing (Salman et al., 2019), denoised smoothing (Salman et al., 2020), smoothed ViT (Salman et al., 2021), and feature smoothing (Addepalli et al., 2021).

Original: Zeroth-order (ZO) optimization for adversarial ML.
Revised: Zeroth-order (ZO) optimization for adversarial ML.

Original: ZO optimization methods are gradientfree counterparts of ﬁrst-order (FO) optimization methods (Liu et al., 2020b).
Revised: ZO optimization methods are gradientfree counterparts of ﬁrst-order (FO) optimization methods (Liu et al., 2020b).

Original: They approximate the
Revised: They approximate the

Original: FO gradients through function value based gradient estimates.
Revised: FO gradients through function value based gradient estimates.

Original: Thus, ZO optimization is quite useful to solve black-box problems when explicit expressions of their gradients are difﬁcult to compute or infeasible to obtain.
Revised: Thus, ZO optimization is quite useful to solve black-box problems when explicit expressions of their gradients are difﬁcult to compute or infeasible to obtain.

Original: In the area of adversarial ML, ZO optimization has become a principled approach to generate adversarial examples from black-box victim ML models (Chen et al., 2017; Ilyas et al., 2018a;b; Tu et al., 2019; Liu et al., 2019; 2020a; Huang & Zhang, 2020;
Revised: In the area of adversarial ML, ZO optimization has become a principled approach to generate adversarial examples from black-box victim ML models (Chen et al., 2017; Ilyas et al., 2018a;b; Tu et al., 2019; Liu et al., 2019; 2020a; Huang & Zhang, 2020;

Original: Cai et al.,  2021).
Revised: Cai et al., 2020; 2021).

Original: Such
Revised: Such

Original: ZO optimization-based attack generation methods can be as effective as state-of-the-art white-box attacks, despite only having access to the inputs and outputs of the targeted model.
Revised: ZO optimization-based attack generation methods can be as effective as state-of-the-art white-box attacks, despite only having access to the inputs and outputs of the targeted model.

Original: For example, the work (Tu et al., 2019) leveraged the white-box decoder to map the generated low-dimension perturbations back to the original input dimension.
Revised: For example, the work (Tu et al., 2019) leveraged the white-box decoder to map the generated low-dimension perturbations back to the original input dimension.

Original: Inspired by (Tu et al., 2019), we leverage the autoencoder architecture to tackle the high-dimension challenge of ZO optimization in black-box defense.
Revised: Inspired by (Tu et al., 2019), we leverage the autoencoder architecture to tackle the high-dimension challenge of ZO optimization in black-box defense.

Original: Despite the widespread application of ZO optimization to black-box attack generation, few work studies the problem of black-box defense.
Revised: Despite the widespread application of ZO optimization to black-box attack generation, few work studies the problem of black-box defense.

Original:  P ROBLEM F ORMULATION : B LACK -B OX D EFENSE
Revised:  P ROBLEM F ORMULATION : B LACK -B OX D EFENSE

Original: In this section, we formulate the problem of black-box defense, i.e. , robustifying black-box ML models without having any model information such as architectures and parameters.
Revised: In this section, we formulate the problem of black-box defense, i.e. , robustifying black-box ML models without having any model information such as architectures and parameters.

Original: Problem statement.
Revised: Problem statement.

Original: Let f θ bb ( x ) denote a pre-deﬁned black-box (bb) predictive model , which can map an input example x to a prediction.
Revised: Let f θ bb ( x ) denote a pre-deﬁned black-box (bb) predictive model , which can map an input example x to a prediction.

Original: In our work, f θ bb can be either an image classiﬁer or an image reconstructor.
Revised: In our work, f θ bb can be either an image classiﬁer or an image reconstructor.

Original: For simplicity of notation, we will drop the model parameters θ bb when referring to a black-box model.
Revised: For simplicity of notation, we will drop the model parameters θ bb when referring to a black-box model.

Original: The threat model of our interest is given by norm-ball constrained adversarial attacks (Goodfellow et al., 2015).
Revised: The threat model of our interest is given by norm-ball constrained adversarial attacks (Goodfellow et al., 2015).

Original: To defend against these attacks, existing approaches commonly require the white-box assumption of f (Madry et al., 2018) or have access to white-box surrogate models of f (Salman et al., 2020).
Revised: To defend against these attacks, existing approaches commonly require the white-box assumption of f (Madry et al., 2018) or have access to white-box surrogate models of f (Salman et al., 2020).

Original: Different from the prior works, we study the problem of black-box defense when the owner of f is not able to share the model details.
Revised: Different from the prior works, we study the problem of black-box defense when the owner of f is not able to share the model details.

Original: Accordingly, the only mode of interaction with the black-box system is via submitting inputs and receiving the corresponding predicted outputs.
Revised: Accordingly, the only mode of interaction with the black-box system is via submitting inputs and receiving the corresponding predicted outputs.

Original: The formal statement of black-box defense is given below: (Black-box defense)
Revised: The formal statement of black-box defense is given below: (Black-box defense)

Original: Given a black-box base model f , can we develop a defensive operation
Revised: Given a black-box base model f , can we develop a defensive operation

Original: R using just input-output function queries so as to produce the robustiﬁed model R ( f ) against adversarial attacks?
Revised: R using just input-output function queries so as to produce the robustiﬁed model R ( f ) against adversarial attacks?

Original: Defensive operation.
Revised: Defensive operation.

Original: We next provide a concrete formulation of the defensive operation R .
Revised: We next provide a concrete formulation of the defensive operation R .

Original: In the literature, two principled defensive operations were used: ( R 1 ) end-to-end AT (Madry et al., 2018;
Revised: In the literature, two principled defensive operations were used: ( R 1 ) end-to-end AT (Madry et al., 2018;

Original: Zhang et al., 2019b; Cohen et al., 2019), and ( R 2 ) prepending a defensive component to a base model (Meng & Chen, 2017; Salman et al., 2020; Aldahdooh et al., 2021).
Revised: Zhang et al., 2019b; Cohen et al., 2019), and ( R 2 ) prepending a defensive component to a base model (Meng & Chen, 2017; Salman et al., 2020; Aldahdooh et al., 2021).

Original: The former ( R 1 ) has achieved the state-of-the-art robustness performance (Athalye et al., 2018; Croce & Hein, 2020) but is not applicable to black-box defense.
Revised: The former ( R 1 ) has achieved the state-of-the-art robustness performance (Athalye et al., 2018; Croce & Hein, 2020) but is not applicable to black-box defense.

Original: By contrast, the latter ( R 2 ) is more compatible with black-box models.
Revised: By contrast, the latter ( R 2 ) is more compatible with black-box models.

Original: For example, denonised smoothing (DS), a recently-developed R 2 -type approach (Salman et al., 2020), gives a certiﬁed defense by prepending a custom-trained denoiser to the targeted model.
Revised: For example, denonised smoothing (DS), a recently-developed R 2 -type approach (Salman et al., 2020), gives a certiﬁed defense by prepending a custom-trained denoiser to the targeted model.

Original: In this work, we choose DS as the backbone of our defensive operation (Fig. 2).
Revised: In this work, we choose DS as the backbone of our defensive operation (Fig. 2).

Original: [Figure]
Revised: [Figure]

Original: In DS, a denoiser is integrated with a base model f so that the augmented system becomes resilient to Gaussian noise and thus plays a role similar to the RS-based certiﬁed defense (Cohen et al., 2019).
Revised: In DS, a denoiser is integrated with a base model f so that the augmented system becomes resilient to Gaussian noise and thus plays a role similar to the RS-based certiﬁed defense (Cohen et al., 2019).

Original: That is, DS yields
Revised: That is, DS yields

Original: [Equation]
Revised: [Equation]

Original: where D θ denotes the learnable denoiser (with parameters θ ) prepended to the (black-box) predictor f .
Revised: where D θ denotes the learnable denoiser (with parameters θ ) prepended to the (black-box) predictor f .

Original: Once D θ is learned, then the DS-based smooth classiﬁer, arg max c P δ ∈N ( 0 ,σ 2 I )
Revised: Once D θ is learned, then the DS-based smooth classiﬁer, arg max c P δ ∈N ( 0 ,σ 2 I )

Original: [ R ( f ( x + δ ))
Revised: [ R ( f ( x + δ ))

Original: =
Revised: =

Original: c ] , can achieve certiﬁed robustness, where c is a class label, δ ∈ N ( 0 , σ 2 I ) denotes the standard Gaussian noise with variance σ 2 , and
Revised: c ] , can achieve certiﬁed robustness, where c is a class label, δ ∈ N ( 0 , σ 2 I ) denotes the standard Gaussian noise with variance σ 2 , and

Original: arg max c P δ ∈N ( 0 ,σ 2 I )
Revised: arg max c P δ ∈N ( 0 ,σ 2 I )

Original: [ f ( x + δ )
Revised: [ f ( x + δ )

Original: =
Revised: =

Original: c ] signiﬁes a smooth version of f .
Revised: c ] signiﬁes a smooth version of f .

Original: Based on (1), the goal of black-box defense becomes to ﬁnd the optimal denoiser D θ so as to achieve satisfactory accuracy as well as adversarial robustness.
Revised: Based on (1), the goal of black-box defense becomes to ﬁnd the optimal denoiser D θ so as to achieve satisfactory accuracy as well as adversarial robustness.

Original: In the FO learning paradigm, Salman et al.
Revised: In the FO learning paradigm, Salman et al.

Original: (2020) proposed a stability regularized denoising loss to train D θ : minimize 
Revised: (2020) proposed a stability regularized denoising loss to train D θ : minimize θ

Original: [Equation]
Revised: [Equation]

Original: where U denotes the training dataset, the ﬁrst objective term (cid:96) Denoise ( θ ) corresponds to the mean squared error (MSE) of image denoising, the second objective term (cid:96) Stab ( θ ) measures the prediction stability through the cross-entropy (CE) between the  denoised input and the original input, and γ >is a regularization parameter that strikes a balance between (cid:96) Denoise and (cid:96) Stab .
Revised: where U denotes the training dataset, the ﬁrst objective term (cid:96) Denoise ( θ ) corresponds to the mean squared error (MSE) of image denoising, the second objective term (cid:96) Stab ( θ ) measures the prediction stability through the cross-entropy (CE) between the outputs of the denoised input and the original input, and γ > 0 is a regularization parameter that strikes a balance between (cid:96) Denoise and (cid:96) Stab .

Original: We remark that problem (2) can be solved using the FO gradient descent method if the base model f is fully disclosed to the defender.
Revised: We remark that problem (2) can be solved using the FO gradient descent method if the base model f is fully disclosed to the defender.

Original: However, the black-box nature of f makes the gradients of the stability loss (cid:96) Stab ( θ ) infeasible to obtain.
Revised: However, the black-box nature of f makes the gradients of the stability loss (cid:96) Stab ( θ ) infeasible to obtain.

Original: Thus, we will develop a gradient-free DS-oriented defense.
Revised: Thus, we will develop a gradient-free DS-oriented defense.

Original:  M ETHOD : A S CALABLE Z EROTH -O RDER O PTIMIZATION S OLUTION
Revised:  M ETHOD : A S CALABLE Z EROTH -O RDER O PTIMIZATION S OLUTION

Original: In this section, we begin by presenting a brief background on ZO optimization, and elaborate on the challenge of black-box defense in high dimensions.
Revised: In this section, we begin by presenting a brief background on ZO optimization, and elaborate on the challenge of black-box defense in high dimensions.

Original: Next, we propose a novel ZO optimization-based
Revised: Next, we propose a novel ZO optimization-based

Original: DS method that can not only improve model query complexity but also lead to certiﬁed robustness.
Revised: DS method that can not only improve model query complexity but also lead to certiﬁed robustness.

Original: ZO optimization.
Revised: ZO optimization.

Original: In ZO optimization, the FO gradient of a black-box function (cid:96) ( w ) (with a d -dimension variable w ) is approximated by the difference of two function values along a set of random direction vectors.
Revised: In ZO optimization, the FO gradient of a black-box function (cid:96) ( w ) (with a d -dimension variable w ) is approximated by the difference of two function values along a set of random direction vectors.

Original: This leads to the randomized gradient estimate (RGE) (Liu et al., 2020b):
Revised: This leads to the randomized gradient estimate (RGE) (Liu et al., 2020b):

Original: [Equation]
Revised: [Equation]

Original: where { u i } qi =1 are q random vectors drawn independently and uniformly from the sphere of a unit ball, and µ > 0 is a given small step size, known as the smoothing parameter.
Revised: where { u i } qi =1 are q random vectors drawn independently and uniformly from the sphere of a unit ball, and µ > 0 is a given small step size, known as the smoothing parameter.

Original: The rationale behind (3) is that it provides an unbiased estimate of the FO gradient of the Gaussian smoothing version of (cid:96) (Gao et al., 2018), with variance in the order of O ( dq ) (Liu et al., 2020b).
Revised: The rationale behind (3) is that it provides an unbiased estimate of the FO gradient of the Gaussian smoothing version of (cid:96) (Gao et al., 2018), with variance in the order of O ( dq ) (Liu et al., 2020b).

Original: Thus, a large-scale problem (with large d ) yields a large variance of RGE (3).
Revised: Thus, a large-scale problem (with large d ) yields a large variance of RGE (3).

Original: To reduce the variance, a large number of querying directions ( i.e. , q ) is then needed, with the worst-case query complexity in the order of
Revised: To reduce the variance, a large number of querying directions ( i.e. , q ) is then needed, with the worst-case query complexity in the order of

Original: O ( d ) .
Revised: O ( d ) .

Original: If q = d , then the least estimation variance can be achieved by the coordinatewise gradient estimate (CGE)
Revised: If q = d , then the least estimation variance can be achieved by the coordinatewise gradient estimate (CGE)

Original: (Lian et al., [Equation] Liu [Equation]
Revised: (Lian et al., 2016; Liu et al., 2018):

Original: [Equation]
Revised: [Equation]

Original: where e i ∈ R
Revised: where e i ∈ R

Original: d denotes the i th elementary basis vector, with 1 at the i th coordinate and 0 s elsewhere.
Revised: d denotes the i th elementary basis vector, with 1 at the i th coordinate and 0 s elsewhere.

Original: For any off-the-shelf FO optimizers, e.g. , stochastic gradient descent (SGD), if we replace the FO gradient estimate with the ZO gradient estimate, then we obtain the ZO counterpart of a FO solver, e.g. , ZO-SGD (Ghadimi & Lan, 2013).
Revised: For any off-the-shelf FO optimizers, e.g. , stochastic gradient descent (SGD), if we replace the FO gradient estimate with the ZO gradient estimate, then we obtain the ZO counterpart of a FO solver, e.g. , ZO-SGD (Ghadimi & Lan, 2013).

Original: Warm-up: A direct application of ZO optimization.
Revised: Warm-up: A direct application of ZO optimization.

Original: A straightforward method to achieve the
Revised: A straightforward method to achieve the

Original: DS-based black-box defense is to solve problem (2) using ZO optimization directly.
Revised: DS-based black-box defense is to solve problem (2) using ZO optimization directly.

Original: However, it will give rise to the difﬁculty of ZO optimization in high dimensions .
Revised: However, it will give rise to the difﬁculty of ZO optimization in high dimensions .

Original: Speciﬁcally, DS requires to calculate the gradient of the defensive operation (1).
Revised: Speciﬁcally, DS requires to calculate the gradient of the defensive operation (1).

Original: With the aid of ZO gradient estimation, we obtain
Revised: With the aid of ZO gradient estimation, we obtain

Original: [Equation]
Revised: [Equation]

Original: where with an abuse of notation, let d denote the dimension of x (yielding D θ ( x ) ∈ R d and z ∈ R d ) and d θ denote the dimension of θ , dD θ ( x ) d θ ∈ R d θ × d is the Jacobian matrix of the vector-valued function D θ ( x ) , and ˆ ∇ z f ( z ) denotes the ZO gradient estimate of f , following (3) or (4).
Revised: where with an abuse of notation, let d denote the dimension of x (yielding D θ ( x ) ∈ R d and z ∈ R d ) and d θ denote the dimension of θ , dD θ ( x ) d θ ∈ R d θ × d is the Jacobian matrix of the vector-valued function D θ ( x ) , and ˆ ∇ z f ( z ) denotes the ZO gradient estimate of f , following (3) or (4).

Original: Since the dimension of an input is typically large for image classiﬁcation ( e.g. , d = 3072 for a CIFAR-
Revised: Since the dimension of an input is typically large for image classiﬁcation ( e.g. , d = 3072 for a CIFAR-

Original: [Table]
Revised: [Table]

Original: image)
Revised: image)

Original: , it imposes two challenges : (a) The variance of RGE (3) will be ultra-large if the query complexity stays low, i.e. , a small query number q is used; And (b) the variance-least CGE (4) becomes impracticable due to the need of ultra-high querying cost ( i.e. , q = d ).
Revised: , it imposes two challenges : (a) The variance of RGE (3) will be ultra-large if the query complexity stays low, i.e. , a small query number q is used; And (b) the variance-least CGE (4) becomes impracticable due to the need of ultra-high querying cost ( i.e. , q = d ).

Original: Indeed, Table 1 shows that the direct application of (5) into the existing FO-DS solver (Salman et al., 2020), which we call ZO-DS, yields over 25% degradation in both standard accuracy and certiﬁed robustness evaluated at input perturbations with (cid:96) 2 norm less than 128 / 255 , where pixels of an input image are normalized to [ 0 , 1 ] .
Revised: Indeed, Table 1 shows that the direct application of (5) into the existing FO-DS solver (Salman et al., 2020), which we call ZO-DS, yields over 25% degradation in both standard accuracy and certiﬁed robustness evaluated at input perturbations with (cid:96) 2 norm less than 128 / 255 , where pixels of an input image are normalized to [ 0 , 1 ] .

Original: We refer readers to Sec.
Revised: We refer readers to Sec.

Original: 5 for more details.
Revised: 5 for more details.

Original: [Figure]
Revised: [Figure]

Original: E n code r
Revised: n c

Original: [Figure]
Revised: [Figure]

Original: ode
Revised: r

Original: [Figure]
Revised: [Figure]

Original: ZO autoencoder-based DS (ZO-AEDS):
Revised: ZO autoencoder-based DS (ZO-AEDS):

Original: A scalable solution to black-box defense.
Revised: A scalable solution to black-box defense.

Original: The difﬁculty of ZO optimization in high dimensions prevents us from developing an effective DS-oriented provable defense for black-box ML models.
Revised: The difﬁculty of ZO optimization in high dimensions prevents us from developing an effective DS-oriented provable defense for black-box ML models.

Original: To tackle such problem, we introduce an Autoencoder (AE) to connect the front-end denoiser D θ with the back-end black-box predictive model f
Revised: To tackle such problem, we introduce an Autoencoder (AE) to connect the front-end denoiser D θ with the back-end black-box predictive model f

Original: so that ZO optimization can be conducted in a (low-dimension) feature embedding space.
Revised: so that ZO optimization can be conducted in a (low-dimension) feature embedding space.

Original: To be concrete, let ψ
Revised: To be concrete, let ψ

Original: θ Dec ◦ φ θ Enc denote AE consisting of the encoder (Enc) ψ θ Enc and the decoder (Dec) φ θ Dec , where ◦ denotes the function composition operation.
Revised: θ Dec ◦ φ θ Enc denote AE consisting of the encoder (Enc) ψ θ Enc and the decoder (Dec) φ θ Dec , where ◦ denotes the function composition operation.

Original: Plugging φ θ AE between the denoiser D θ and the black-box predictor f , we then extend the defensive operation (1) to the following (see Fig.
Revised: Plugging φ θ AE between the denoiser D θ and the black-box predictor f , we then extend the defensive operation (1) to the following (see Fig.

Original: 3 for illustration):
Revised: 3 for illustration):

Original: [Equation]
Revised: [Equation]

Original: where z ∈ R d z denotes the low-dimension feature embedding with d z < d .
Revised: where z ∈ R d z denotes the low-dimension feature embedding with d z < d .

Original: In (6), we integrate the decoder ψ θ Dec with the black-box predictor f to construct a new black-box model f (cid:48)
Revised: In (6), we integrate the decoder ψ θ Dec with the black-box predictor f to construct a new black-box model f (cid:48)

Original: ( z ) :
Revised: ( z ) :

Original: = f ( ψ θ Dec ( z )) , which enables us to derive a ZO gradient estimate of reduced dimension : 
Revised: = f ( ψ θ Dec ( z )) , which enables us to derive a ZO gradient estimate of reduced dimension : [Equation]

Original: [Equation]
Revised: [Equation]

Original: Assisted by AE, RGE of ˆ ∇ z f (cid:48) has a reduced variance from O ( dq ) to O
Revised: Assisted by AE, RGE of ˆ ∇ z f (cid:48) has a reduced variance from O ( dq ) to O

Original: ( d z q ) .
Revised: ( d z q ) .

Original: Meanwhile, the least-variance CGE (4) also becomes feasible by setting the query number as q = d z .
Revised: Meanwhile, the least-variance CGE (4) also becomes feasible by setting the query number as q = d z .

Original: Note that the eventual ZO estimate (7) is a function of the d θ × d z Jacobian matrix ∇ θ [ φ θ Enc
Revised: Note that the eventual ZO estimate (7) is a function of the d θ × d z Jacobian matrix ∇ θ [ φ θ Enc

Original: ( D θ ( x ))] .
Revised: ( D θ ( x ))] .

Original: For ease of storing and computing the Jacobian matrix, we derive the following computationally-light alternative of (7) (see derivation in Appendix A):
Revised: For ease of storing and computing the Jacobian matrix, we derive the following computationally-light alternative of (7) (see derivation in Appendix A):

Original: [Equation]
Revised: [Equation]

Original: where recall that f (cid:48) ( z )
Revised: where recall that f (cid:48) ( z )

Original: =
Revised: =

Original: f ( ψ θ Dec ( z )) , and ˆ ∇ denotes the ZO gradient estimate given by (3) or (4).
Revised: f ( ψ θ Dec ( z )) , and ˆ ∇ denotes the ZO gradient estimate given by (3) or (4).

Original: The computation advantage of (8) is that the derivative operation ∇ θ can be applied to a scalar-valued inner product built upon a pre-calculated ZO gradient estimate a .
Revised: The computation advantage of (8) is that the derivative operation ∇ θ can be applied to a scalar-valued inner product built upon a pre-calculated ZO gradient estimate a .

Original: Training ZO-AE-DS.
Revised: Training ZO-AE-DS.

Original: Recall from Fig.
Revised: Recall from Fig.

Original: 3 that the proposed defensive system involves three components: denoiser D θ , AE ψ θ Dec ◦ φ θ Enc , and pre-deﬁned black-box predictor f .
Revised: 3 that the proposed defensive system involves three components: denoiser D θ , AE ψ θ Dec ◦ φ θ Enc , and pre-deﬁned black-box predictor f .

Original: Thus, the parameters to be optimized include θ , θ
Revised: Thus, the parameters to be optimized include θ , θ

Original: Dec and θ Enc .
Revised: Dec and θ Enc .

Original: To train ZO-AE-DS, we adopt a two-stage training protocol.
Revised: To train ZO-AE-DS, we adopt a two-stage training protocol.

Original: (cid:172) White-box pre-training on AE : At the ﬁrst stage, we pre
Revised: (cid:172) White-box pre-training on AE : At the ﬁrst stage, we pre

Original: -train the AE model by calling a standard FO optimizer ( e.g. , Adam) to minimize the reconstruction loss E x (cid:107) φ θ Dec ( ψ θ Enc ( x ))
Revised: -train the AE model by calling a standard FO optimizer ( e.g. , Adam) to minimize the reconstruction loss E x (cid:107) φ θ Dec ( ψ θ Enc ( x ))

Original: − x (cid:107) 22 .
Revised: − x (cid:107) 22 .

Original: The resulting AE will be used as the initialization of the second-stage training.
Revised: The resulting AE will be used as the initialization of the second-stage training.

Original: We remark that the denoising model D θ can also be pre-trained.
Revised: We remark that the denoising model D θ can also be pre-trained.

Original: However, such a pre-training could hamper optimization, i.e. , making the second-stage training over θ easily trapped at a poor local optima.
Revised: However, such a pre-training could hamper optimization, i.e. , making the second-stage training over θ easily trapped at a poor local optima.

Original: (cid:173)
Revised: (cid:173)

Original: End-to-end training : At the second stage, we keep the pre-trained decoder φ θ Dec intact and merge it into the black-box system as shown in Fig.
Revised: End-to-end training : At the second stage, we keep the pre-trained decoder φ θ Dec intact and merge it into the black-box system as shown in Fig.

Original: We then optimize θ and θ Enc by minimizing the DS-based training loss (2), where the denoiser D θ and the defensive operation R are replaced by ψ θ Enc ◦ D θ and R new (6), respectively.
Revised: We then optimize θ and θ Enc by minimizing the DS-based training loss (2), where the denoiser D θ and the defensive operation R are replaced by ψ θ Enc ◦ D θ and R new (6), respectively.

Original: In (2), minimization over the stability loss (cid:96) Stab ( θ ) calls the ZO estimate of ∇ θ (cid:96) Stab ( θ ) , given by (7).
Revised: In (2), minimization over the stability loss (cid:96) Stab ( θ ) calls the ZO estimate of ∇ θ (cid:96) Stab ( θ ) , given by (7).

Original: In Appendix C.2, different training schemes are discussed.
Revised: In Appendix C.2, different training schemes are discussed.

Original:  E XPERIMENTS
Revised:  E XPERIMENTS

Original: In this section, we demonstrate the effectiveness of our proposal through extensive experiments.
Revised: In this section, we demonstrate the effectiveness of our proposal through extensive experiments.

Original: We will show that the proposed ZO-AE-DS outperforms a series of baselines when robustifying black-box neural networks for secure image classiﬁcation and image reconstruction.
Revised: We will show that the proposed ZO-AE-DS outperforms a series of baselines when robustifying black-box neural networks for secure image classiﬁcation and image reconstruction.

Original: 5. E XPERIMENT SETUP
Revised: 5. E XPERIMENT SETUP

Original: Datasets and model architectures.
Revised: Datasets and model architectures.

Original: In the task of image classiﬁcation, we focus on CIFARand STL-10 datasets.
Revised: In the task of image classiﬁcation, we focus on CIFARand STL-10 datasets.

Original: In Appendix C.3, we demonstrate the effectiveness of ZO-AE-DS on the high-dimension ImageNet images.
Revised: In Appendix C.3, we demonstrate the effectiveness of ZO-AE-DS on the high-dimension ImageNet images.

Original: In the task of image reconstruction, we consider the MNIST dataset.
Revised: In the task of image reconstruction, we consider the MNIST dataset.

Original: To build ZO-AE-DS and its variants and baselines, we specify the prepended denoiser
Revised: To build ZO-AE-DS and its variants and baselines, we specify the prepended denoiser

Original: D θ as DnCNN (Zhang et al., 2017).
Revised: D θ as DnCNN (Zhang et al., 2017).

Original: We then implement task-speciﬁc AE for different datasets.
Revised: We then implement task-speciﬁc AE for different datasets.

Original: Superﬁcially, the dimension of encoded feature embedding, namely, d z in (6), is set as 192,and 192 for CIFAR-10, STL-10 and MNIST, respectively.
Revised: Superﬁcially, the dimension of encoded feature embedding, namely, d z in (6), is set as 192,and 192 for CIFAR-10, STL-10 and MNIST, respectively.

Original: The architectures of AE are conﬁgured following (Mao et al., 2016), and ablation study on the choice of AE is shown in Appendix C.1.
Revised: The architectures of AE are conﬁgured following (Mao et al., 2016), and ablation study on the choice of AE is shown in Appendix C.1.

Original: To specify the black-box image classiﬁcation model, we choose ResNet-110 for CIFAR-10 following (Salman et al., 2020), and ResNet-18 for STL-10.
Revised: To specify the black-box image classiﬁcation model, we choose ResNet-110 for CIFAR-10 following (Salman et al., 2020), and ResNet-18 for STL-10.

Original: It is worth noting that STL-10 contains 500 labeled 96 × 96 training images, and the pre-trained ResNet-18 achieves 76.6% test accuracy that matches to state-of-the-art performance.
Revised: It is worth noting that STL-10 contains 500 labeled 96 × 96 training images, and the pre-trained ResNet-18 achieves 76.6% test accuracy that matches to state-of-the-art performance.

Original: For image reconstruction, we adopt a reconstruction network consisting of convolution, deconvolution and ReLU layers, following (Raj et al., 2020).
Revised: For image reconstruction, we adopt a reconstruction network consisting of convolution, deconvolution and ReLU layers, following (Raj et al., 2020).

Original: Baselines.
Revised: Baselines.

Original: We will consider two variants of our proposed ZO-AE-DS: i) ZO-AE-DS using RGE (3), ii) ZO-AE-DS using CGE (4).
Revised: We will consider two variants of our proposed ZO-AE-DS: i) ZO-AE-DS using RGE (3), ii) ZO-AE-DS using CGE (4).

Original: In addition, we will compare ZO-AE-DS with i) FO-AE-DS , i.e. , the ﬁrst-order implementation of ZO-AE-DS, ii) FO-DS , which developed in (Salman et al., 2020), iii) RS -based certiﬁed training, proposed in (Cohen et al., 2019), and iv) ZO-DS , i.e. , the ZO implementation of FO-DS using RGE.
Revised: In addition, we will compare ZO-AE-DS with i) FO-AE-DS , i.e. , the ﬁrst-order implementation of ZO-AE-DS, ii) FO-DS , which developed in (Salman et al., 2020), iii) RS -based certiﬁed training, proposed in (Cohen et al., 2019), and iv) ZO-DS , i.e. , the ZO implementation of FO-DS using RGE.

Original: Note that CGE is not applicable to ZO-DS due to the obstacle of high dimensions.
Revised: Note that CGE is not applicable to ZO-DS due to the obstacle of high dimensions.

Original: To our best knowledge, ZO-DS is the only query-based black-box defense baseline that can be directly compared with ZO-AE-DS.
Revised: To our best knowledge, ZO-DS is the only query-based black-box defense baseline that can be directly compared with ZO-AE-DS.

Original: Training setup.
Revised: Training setup.

Original: We build the training pipeline of the proposed ZO-AE-DS following ‘Training
Revised: We build the training pipeline of the proposed ZO-AE-DS following ‘Training

Original: ZO-AE-DS’ in Sec. 4.
Revised: ZO-AE-DS’ in Sec. 4.

Original: To optimize the denoising model D θ , we will cover two training schemes: training from scratch, and pre-training & ﬁne-tuning.
Revised: To optimize the denoising model D θ , we will cover two training schemes: training from scratch, and pre-training & ﬁne-tuning.

Original: In the scenario of training from scratch, we use Adam optimizer with learning rate 10 − 3 to train the model for 200 epochs and then use SGD optimizer with learning rate 10 − 3 drop by a factor of 10 at every 200 epoch, where the total number of epochs is 600.
Revised: In the scenario of training from scratch, we use Adam optimizer with learning rate 10 − 3 to train the model for 200 epochs and then use SGD optimizer with learning rate 10 − 3 drop by a factor of 10 at every 200 epoch, where the total number of epochs is 600.

Original: As will be evident later, training from scratch over D θ leads to better performance of ZO-AE-DS.
Revised: As will be evident later, training from scratch over D θ leads to better performance of ZO-AE-DS.

Original: In the scenario of pre-training & ﬁne-tuning, we use Adam optimizer to pre-train the denoiser D θ with the MSE loss (cid:96) Denoise in (2) for 90 epochs and ﬁne-tune the denoiser with (cid:96) Stab for 200 epochs with learning rate 10 − 5 drop by a factor of 10 every 40 epochs.
Revised: In the scenario of pre-training & ﬁne-tuning, we use Adam optimizer to pre-train the denoiser D θ with the MSE loss (cid:96) Denoise in (2) for 90 epochs and ﬁne-tune the denoiser with (cid:96) Stab for 200 epochs with learning rate 10 − 5 drop by a factor of 10 every 40 epochs.

Original: When implementing the baseline FO-DS, we use the best training setup provided by (Salman et al., 2020).
Revised: When implementing the baseline FO-DS, we use the best training setup provided by (Salman et al., 2020).

Original: When implementing
Revised: When implementing

Original: ZO-DS, we reduce the initial learning rate to 10 − 4 for training from scratch and 10 − 6 for pre-training & ﬁne-tuning to stabilize the convergence of ZO optimization.
Revised: ZO-DS, we reduce the initial learning rate to 10 − 4 for training from scratch and 10 − 6 for pre-training & ﬁne-tuning to stabilize the convergence of ZO optimization.

Original: Furthermore, we set the smoothing parameter µ = 0 .
Revised: Furthermore, we set the smoothing parameter µ = 0 .

Original: 005 for RGE and CGE.
Revised: 005 for RGE and CGE.

Original: And to achieve a smooth predictor, we set the Gaussian smoothing noise as δ ∈ N ( 0 , σ 2 I ) with σ 2 = 0 .
Revised: And to achieve a smooth predictor, we set the Gaussian smoothing noise as δ ∈ N ( 0 , σ 2 I ) with σ 2 = 0 .

Original: With the help of matrix operations and the parallel computing power of the GPU, we optimize the training time to an acceptable range.
Revised: With the help of matrix operations and the parallel computing power of the GPU, we optimize the training time to an acceptable range.

Original: The averaged one-epoch training time on a single Nvidia RTX A6000 GPU is about ∼ 1min and ∼ 29min for FO-DS and our proposed ZO method, ZO-AE-DS (CGE, q = 192 ), on the CIFAR-10 dataset.
Revised: The averaged one-epoch training time on a single Nvidia RTX A6000 GPU is about ∼ 1min and ∼ 29min for FO-DS and our proposed ZO method, ZO-AE-DS (CGE, q = 192 ), on the CIFAR-10 dataset.

Original: Evaluation metrics.
Revised: Evaluation metrics.

Original: In the task of robust image classiﬁcation, the performance will be evaluated at standard test accuracy (SA) and certiﬁed accuracy (CA).
Revised: In the task of robust image classiﬁcation, the performance will be evaluated at standard test accuracy (SA) and certiﬁed accuracy (CA).

Original: Here CA is a provable robust guarantee of the
Revised: Here CA is a provable robust guarantee of the

Original: Gaussian smoothing version of a predictive model.
Revised: Gaussian smoothing version of a predictive model.

Original: Let us take ZO-AE-DS as an example, the resulting smooth image classiﬁer is given by f smooth ( x ) :
Revised: Let us take ZO-AE-DS as an example, the resulting smooth image classiﬁer is given by f smooth ( x ) :

Original: = arg max c P δ ∈N ( 0 ,σ 2 I )
Revised: = arg max c P δ ∈N ( 0 ,σ 2 I )

Original: [ R new ( f ( x + δ ))
Revised: [ R new ( f ( x + δ ))

Original: =
Revised: =

Original: c ] , where R new is given by (6).
Revised: c ] , where R new is given by (6).

Original: Further, a certiﬁed radius of (cid:96) 2 -norm perturbation ball with respect to an input example can be calculated following the RS approach provided in (Cohen et al., 2019).
Revised: Further, a certiﬁed radius of (cid:96) 2 -norm perturbation ball with respect to an input example can be calculated following the RS approach provided in (Cohen et al., 2019).

Original: As a result, CA at a given (cid:96) 2 -radius r is the percentage of the correctly classiﬁed data points whose certiﬁed radii are larger than r .
Revised: As a result, CA at a given (cid:96) 2 -radius r is the percentage of the correctly classiﬁed data points whose certiﬁed radii are larger than r .

Original: Note that if r = 0 , then CA reduces to SA.
Revised: Note that if r = 0 , then CA reduces to SA.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: 5. E XPERIMENT RESULTS ON IMAGE CLASSIFICATION
Revised: 5. E XPERIMENT RESULTS ON IMAGE CLASSIFICATION

Original: Performance on CIFAR-10.
Revised: Performance on CIFAR-10.

Original: In Table 2, we present certiﬁed accuracies of ZO-AE-DS and its variants/baselines versus different (cid:96) 2 -radii in the setup of (CIFAR-10, ResNet-110).
Revised: In Table [Table] we present certiﬁed accuracies of ZO-AE-DS and its variants/baselines versus different (cid:96) 2 -radii in the setup of (CIFAR-10, ResNet-110).

Original: Towards a comprehensive comparison, different RGE-based variants of ZO-AE-DS and ZO-DS are demonstrated using the query number q ∈ { 20 , 100 , 192 } .
Revised: Towards a comprehensive comparison, different RGE-based variants of ZO-AE-DS and ZO-DS are demonstrated using the query number q ∈ { 20 , 100 , 192 } .

Original: First, the comparison between ZO-AE-DS and ZO-DS shows that our proposal signiﬁcantly outperforms ZO-DS ranging from the low query number q =to the high query number q = 192 when RGE is applied. 
Revised: First, the comparison between ZO-AE-DS and ZO-DS shows that our proposal signiﬁcantly outperforms ZO-DS ranging from the low query number q =to the high query number q = 192 when RGE is applied. Second

Original: Second, we observe that the use of CGE yields the best CA and SA (corresponding to r = 0 ).
Revised: [Table] we observe that the use of CGE yields the best CA and SA (corresponding to r = 0 ).

Original: The application of CGE is beneﬁted from
Revised: The application of CGE is beneﬁted from

Original: AE, which reduces the dimension from d = 32 × 32 × 3 to d z = 192 .
Revised: AE, which reduces the dimension from d = 32 × 32 × 3 to d z = 192 .

Original: In particular, CGE-based
Revised: In particular, CGE-based

Original: ZO-AE-DS improves the case studied in Table 1 from 5 .
Revised: ZO-AE-DS improves the case studied in Table 1 from 5 .

Original: 06% to 35 .
Revised: 06% to 35 .

Original: 5% at the (cid:96) 2 -radius r = 0 . 5 .
Revised: 5% at the (cid:96) 2 -radius r = 0 . 5 .

Original: Third, although FO-AE-DS yields CA improvement over FO-DS in the white-box context, the improvement achieved by ZO-AE-DS (vs. ZO-DS) for black-box defense is much more signiﬁcant.
Revised: Third, although FO-AE-DS yields CA improvement over FO-DS in the white-box context, the improvement achieved by ZO-AE-DS (vs. ZO-DS) for black-box defense is much more signiﬁcant.

Original: This implies that the performance of black-box defense relies on a proper solution (namely, ZO-AE-DS) to tackle the challenge of ZO optimization in high dimensions.
Revised: This implies that the performance of black-box defense relies on a proper solution (namely, ZO-AE-DS) to tackle the challenge of ZO optimization in high dimensions.

Original: Fourth, RS outperforms the ZO methods.
Revised: Fourth, RS outperforms the ZO methods.

Original: This is not surprising since RS is a known white-box certiﬁably robust training approach.
Revised: This is not surprising since RS is a known white-box certiﬁably robust training approach.

Original: In Appendix B, we demonstrate the consistent effectiveness of ZO-AE-DS under different denoisier and classiﬁers.
Revised: In Appendix B, we demonstrate the consistent effectiveness of ZO-AE-DS under different denoisier and classiﬁers.

Original: Performance on STL-10.
Revised: Performance on STL-10.

Original: In Table 3, we evaluate the performance of ZO-AE-DS for STLimage classiﬁcation.
Revised: In Table 3, we evaluate the performance of ZO-AE-DS for STLimage classiﬁcation.

Original: For comparison, we also represent the performance of FO-DS, FO-AE-DS, and
Revised: For comparison, we also represent the performance of FO-DS, FO-AE-DS, and

Original: ZO-DS.
Revised: ZO-DS.

Original: Similar to Table 2, the improvement brought by our proposal over ZO-DS is evident, with at least 10% SA/CA improvement across different (cid:96) 2 -radii.
Revised: Similar to Table 2, the improvement brought by our proposal over ZO-DS is evident, with at least 10% SA/CA improvement across different (cid:96) 2 -radii.

Original: [Table]
Revised: [Table]

Original: When comparing ZO-AE-DS with FO-DS, we observe that ours introduces a 7% degradation in
Revised: When comparing ZO-AE-DS with FO-DS, we observe that ours introduces a 7% degradation in

Original: SA (at r = 0 ).
Revised: SA (at r = 0 ).

Original: This is different from CIFAR-10 classiﬁcation.
Revised: This is different from CIFAR-10 classiﬁcation.

Original: There might be two reasons for the degradation of SA in STL-10.
Revised: There might be two reasons for the degradation of SA in STL-10.

Original: First, the size of a STL-10 image is 9 × larger than a CIFAR-10 image.
Revised: First, the size of a STL-10 image is 9 × larger than a CIFAR-10 image.

Original: Thus, the over-reduced feature dimension could hamper SA.
Revised: Thus, the over-reduced feature dimension could hamper SA.

Original: In this example, we set d z = 576 , which is only 3 × larger than d z = 192 used for CIFAR-10 classiﬁcation.
Revised: In this example, we set d z = 576 , which is only 3 × larger than d z = 192 used for CIFAR-10 classiﬁcation.

Original: Second, the variance of ZO gradient estimates has a larger effect on the performance of STL-10 than that of CIFAR-10, since the former only contains 500 labeled images, leading to a challenging training task.
Revised: Second, the variance of ZO gradient estimates has a larger effect on the performance of STL-10 than that of CIFAR-10, since the former only contains 500 labeled images, leading to a challenging training task.

Original: Despite the degradation of SA, ZO-AE-DS outperforms FO-DS in CA, especially when facing a large (cid:96) 2 -radius.
Revised: Despite the degradation of SA, ZO-AE-DS outperforms FO-DS in CA, especially when facing a large (cid:96) 2 -radius.

Original: This is consistent with Table 2.
Revised: This is consistent with Table 2.

Original: The rationale is that AE can be regarded as an extra smoothing operation for the image classiﬁer, and thus improves certiﬁed robustness over FO-DS, even if the latter is designed in a white-box setup.
Revised: The rationale is that AE can be regarded as an extra smoothing operation for the image classiﬁer, and thus improves certiﬁed robustness over FO-DS, even if the latter is designed in a white-box setup.

Original: If we compare ZO-AE-DS with FO-AE-DS, then the FO approach leads to the best performance due to the high-accuracy of gradient estimates.
Revised: If we compare ZO-AE-DS with FO-AE-DS, then the FO approach leads to the best performance due to the high-accuracy of gradient estimates.

Original: Advantage of AE on ZO optimization.
Revised: Advantage of AE on ZO optimization.

Original: Extended from Table 2, Fig.
Revised: Extended from Table 2, Fig.

Original: 4 presents the complete
Revised: 4 presents the complete

Original: CA curve of non-AE-based and AE-based methods vs. the value of (cid:96) 2 -radius in the example of (CIFAR10, ResNet-110).
Revised: CA curve of non-AE-based and AE-based methods vs. the value of (cid:96) 2 -radius in the example of (CIFAR10, ResNet-110).

Original: As we can see, ZO-AE-DS using
Revised: As we can see, ZO-AE-DS using

Original: RGE with the smallest query number q = 20 has outperformed ZO-DS using RGE with the largested query number q = 192 .
Revised: RGE with the smallest query number q = 20 has outperformed ZO-DS using RGE with the largest query number q = 192 .

Original: This shows the vital role of AE on ZO optimization.
Revised: This shows the vital role of AE on ZO optimization.

Original: Meanwhile, consistent with Table 2 and Table 3, the best model achieved by ZO-AE-DS using CGE could be even better than the FO baseline FO-DS since AE could play a similar role on the smoothing operation.
Revised: Meanwhile, consistent with Table 2 and Table 3, the best model achieved by ZO-AE-DS using CGE could be even better than the FO baseline FO-DS since AE could play a similar role on the smoothing operation.

Original: Furthermore, as the query number q increases, the improvement of ZO-AE-DS grows, towards the performance of
Revised: Furthermore, as the query number q increases, the improvement of ZO-AE-DS grows, towards the performance of

Original: FO-AE-DS.
Revised: FO-AE-DS.

Original: [Figure]
Revised: [Figure]

Original: [Table]
Revised: [Table]

Original: Effect of training scheme on ZO-AE-DS.
Revised: Effect of training scheme on ZO-AE-DS.

Original: In
Revised: In

Original: Table 4, we present the impact of training scheme (over the denoiser D θ ) on the CA performance of ZO-AE-DS versus different (cid:96) 2 -radii.
Revised: Table 4, we present the impact of training scheme (over the denoiser D θ ) on the CA performance of ZO-AE-DS versus different (cid:96) 2 -radii.

Original: Two training schemes, training from scratch and pre-training + ﬁne-tuning, are considered.
Revised: Two training schemes, training from scratch and pre-training + ﬁne-tuning, are considered.

Original: As we can see, training from scratch for D θ leads to the better performance of ZO-AE-DS than pretraining + ﬁne-tuning.
Revised: As we can see, training from scratch for D θ leads to the better performance of ZO-AE-DS than pretraining + ﬁne-tuning.

Original: This is because the application of pre-training to D θ could make optimization easily get trapped at a local optima.
Revised: This is because the application of pre-training to D θ could make optimization easily get trapped at a local optima.

Original: We list other ablation studies in Appendix C.
Revised: We list other ablation studies in Appendix C.

Original: 5. E XPERIMENT RESULTS ON IMAGE RECONSTRUCTION .
Revised: 5. E XPERIMENT RESULTS ON IMAGE RECONSTRUCTION .

Original: In what follows, we apply the proposed ZO-AE-DS to robustifying a black-box image reconstruction network.
Revised: In what follows, we apply the proposed ZO-AE-DS to robustifying a black-box image reconstruction network.

Original: The goal of image reconstruction is to recover the original image from a noisy measurement.
Revised: The goal of image reconstruction is to recover the original image from a noisy measurement.

Original: Following (Antun et al., 2020; Raj et al., 2020), we generate the noisy measurement following a linear observation model y
Revised: Following (Antun et al., 2020; Raj et al., 2020), we generate the noisy measurement following a linear observation model y

Original: =
Revised: =

Original: A (cid:62) x , where A is a sub-sampling matrix ( e.g. , Gaussian sampling), and x is an original image.
Revised: Ax , where A is a sub-sampling matrix ( e.g. , Gaussian sampling), and x is an original image.

Original: A pre-trained image reconstruction network (Raj et al., 2020) then takes A (cid:62) y as the input to recover x .
Revised: A pre-trained image reconstruction network (Raj et al., 2020) then takes A (cid:62) y as the input to recover x .

Original: To evaluate the reconstruction performance, we adopt two metrics (Antun et al., 2020), the root mean squared error (RMSE) and structural similarity (SSIM).
Revised: To evaluate the reconstruction performance, we adopt two metrics (Antun et al., 2020), the root mean squared error (RMSE) and structural similarity (SSIM).

Original: SSIM is a supplementary metric to RMSE, since it gives an accuracy indicator when evaluating the similarity between the true image and its estimate at ﬁne-level regions.
Revised: SSIM is a supplementary metric to RMSE, since it gives an accuracy indicator when evaluating the similarity between the true image and its estimate at ﬁne-level regions.

Original: The vulnerability of image reconstruction networks to adversarial attacks, e.g. , PGD attacks (Madry et al., 2018), has been shown in (Antun et al., 2020;
Revised: The vulnerability of image reconstruction networks to adversarial attacks, e.g. , PGD attacks (Madry et al., 2018), has been shown in (Antun et al., 2020;

Original: Raj et al., 2020; Wolf, 2019).
Revised: Raj et al., 2020; Wolf, 2019).

Original: When the image reconstructor is given as a black-box model, spurred by above, Table 5 presents the performance of image reconstruction using various training methods against adversarial attacks with different perturbation strengths.
Revised: When the image reconstructor is given as a black-box model, spurred by above, Table 5 presents the performance of image reconstruction using various training methods against adversarial attacks with different perturbation strengths.

Original: As we can see, compared to the normally trained image reconstructor ( i.e. , ‘Standard’ in Table 5), all robustiﬁcation methods lead to degraded standard image reconstruction performance in the non-adversarial context ( i.e. , (cid:107) δ (cid:107) 2 = 0 ).
Revised: As we can see, compared to the normally trained image reconstructor ( i.e. , ‘Standard’ in Table 5), all robustiﬁcation methods lead to degraded standard image reconstruction performance in the non-adversarial context ( i.e. , (cid:107) δ (cid:107) 2 = 0 ).

Original: But the worst performance is provided by ZO-DS.
Revised: But the worst performance is provided by ZO-DS.

Original: When the perturbation strength increases, the model achieved by standard training becomes over-sensitive to adversarial perturbations, yielding the highest RMSE and the lowest SSIM.
Revised: When the perturbation strength increases, the model achieved by standard training becomes over-sensitive to adversarial perturbations, yielding the highest RMSE and the lowest SSIM.

Original: Furthermore, we observe that the proposed black-box defense ZO-AE-DS yields very competitive and even better performance with respect to FO defenses.
Revised: Furthermore, we observe that the proposed black-box defense ZO-AE-DS yields very competitive and even better performance with respect to FO defenses.

Original: In Fig.
Revised: In Fig.

Original: 5, we provide visualizations of the reconstructed images using different approaches at the presence of reconstruction-evasion PGD attacks.
Revised: 5, we provide visualizations of the reconstructed images using different approaches at the presence of reconstruction-evasion PGD attacks.

Original: For example, the comparison between Fig.
Revised: For example, the comparison between Fig.

Original: 5-(f) and (b)/(d) clearly shows the robustness gained by ZO-AE-DS.
Revised: 5-(f) and (b)/(d) clearly shows the robustness gained by ZO-AE-DS.

Original: [Table]
Revised: [Table]

Original: [Table] [Table]
Revised: [Figure] [Table]

Original:  C ONCLUSION
Revised:  C ONCLUSION

Original: In this paper, we study the problem of black-box defense, aiming to secure black-box models against adversarial attacks using only input-output model queries.
Revised: In this paper, we study the problem of black-box defense, aiming to secure black-box models against adversarial attacks using only input-output model queries.

Original: The proposed black-box learning paradigm is new to adversarial defense, but is also challenging to tackle because of the black-box optimization nature.
Revised: The proposed black-box learning paradigm is new to adversarial defense, but is also challenging to tackle because of the black-box optimization nature.

Original: To solve this problem, we integrate denoised smoothing (DS) with ZO (zerothorder) optimization to build a feasible black-box defense framework.
Revised: To solve this problem, we integrate denoised smoothing (DS) with ZO (zerothorder) optimization to build a feasible black-box defense framework.

Original: However, we ﬁnd that the direct application of ZO optimization makes the defense ineffective and difﬁcult to scale.
Revised: However, we ﬁnd that the direct application of ZO optimization makes the defense ineffective and difﬁcult to scale.

Original: We then propose
Revised: We then propose

Original: ZO-AE-DS, which leverages autoencoder (AE) to bridge the gap between FO and ZO optimization.
Revised: ZO-AE-DS, which leverages autoencoder (AE) to bridge the gap between FO and ZO optimization.

Original: We show that ZO-AE-DS reduces the variance of ZO gradient estimates and improves the defense and optimization performance in a signiﬁcant manner.
Revised: We show that ZO-AE-DS reduces the variance of ZO gradient estimates and improves the defense and optimization performance in a signiﬁcant manner.

Original: Lastly, we evaluate the superiority of our proposal to a series of baselines in both image classiﬁcation and image reconstruction tasks.
Revised: Lastly, we evaluate the superiority of our proposal to a series of baselines in both image classiﬁcation and image reconstruction tasks.

Revised: A CKNOWLEDGMENT

Revised: Yimeng Zhang, Yuguang Yao, Jinghan Jia and Sijia Liu are supported by the DARPA RED program

Original: A D ERIVATION OF (8)
Revised: A D ERIVATION OF (8)

Original: First, based on (2) and (6), the stability loss corresponding to ZO-AE-DS is given by (cid:96) Stab ( θ ) =
Revised: First, based on (2) and (6), the stability loss corresponding to ZO-AE-DS is given by (cid:96) Stab ( θ ) =

Original: (cid:96) CE
Revised: (cid:96) CE

Original: ( f (cid:48)
Revised: ( f (cid:48)

Original: ( z ) , f ( x )) :
Revised: ( z ) , f ( x )) :

Original: = g ( z ) ,
Revised: = g ( z ) ,

Original: where f (cid:48) ( z ) =
Revised: where f (cid:48) ( z ) =

Original: f ( φ θ Dec ( z )) , z = ψ θ Enc
Revised: f ( φ θ Dec ( z )) , z = ψ θ Enc

Original: ( D θ ( x + δ )) .
Revised: ( D θ ( x + δ )) .

Original: We then take the derivative of (cid:96) Stab ( θ ) w.r.t.
Revised: We then take the derivative of (cid:96) Stab ( θ ) w.r.t.

Original: θ .
Revised: θ .

Original: This yields
Revised: This yields

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: where d z d θ ∈ R d θ × d and dg ( z ) d z ∈ R d .
Revised: where d z d θ ∈ R d θ × d and dg ( z ) d z ∈ R d .

Original: Since g ( z ) involves the black-box function f , we ﬁrst compute its ZO gradient estimate following (3) or (4) and obtain
Revised: Since g ( z ) involves the black-box function f , we ﬁrst compute its ZO gradient estimate following (3) or (4) and obtain

Original: [Equation] [Equation]
Revised: [Equation] [Equation]

Original: Substituting the above into (10), we obtain
Revised: Substituting the above into (10), we obtain

Original: [Equation]
Revised: [Equation]

Original: where the last equality holds based on (9).
Revised: where the last equality holds based on (9).

Original: This completes the derivation.
Revised: This completes the derivation.

Original: B C OMBINATION OF D IFFERENT D ENOISERS AND C LASSIFIERS
Revised: B C OMBINATION OF D IFFERENT D ENOISERS AND C LASSIFIERS

Original: Table A1 presents the certiﬁed accuracies of our proposal using different denoiser models (WideDnCnn vs. DnCnn) andimage classiﬁer (Vgg-16)
Revised: Table A1 presents the certiﬁed accuracies of our proposal using different denoiser models (WideDnCnn vs. DnCnn) and

Original: [Table]
Revised: [Table]

Original: C A DDITIONAL EXPERIMENTS AND ABLATION STUDIES
Revised: C A DDITIONAL EXPERIMENTS AND ABLATION STUDIES

Original: In what follows, we will show the ablation study on the choice of AE architectures in Appendix C.1.
Revised: In what follows, we will show the ablation study on the choice of AE architectures in Appendix C.1.

Original: Afterwards, we will show the performance of FO-AE-DS versus different training schemes in
Revised: Afterwards, we will show the performance of FO-AE-DS versus different training schemes in

Original: Appendix C.2.
Revised: Appendix C.2.

Original: Finally, we will show the performance of our proposal on the high-dimension
Revised: Finally, we will show the performance of our proposal on the high-dimension

Original: ImageNet images in Appendix C.3.
Revised: ImageNet images in Appendix C.3.

Original: C. T HE PERFORMANCE OF FO-AE-DS WITH DIFFERENT A UTO E NCODERS .
Revised: C. T HE PERFORMANCE OF FO-AE-DS WITH DIFFERENT A UTO E NCODERS .

Original: Table.
Revised: Table.

Original: A2 presents the certiﬁed accuracy performance of FO-AE-DS with different autoencoders (AE).
Revised: A2 presents the certiﬁed accuracy performance of FO-AE-DS with different autoencoders (AE).

Original: As we can see, if AE-96 is used (namely, the encoded dimension is half of AE-192 used in the paper), then we observe a slight performance drop.
Revised: As we can see, if AE-96 is used (namely, the encoded dimension is half of AE-192 used in the paper), then we observe a slight performance drop.

Original: This is a promising result as we can further reduce the query complexity by choosing a different autoencoder since the use of CGE has to be matched with the encoded dimension.
Revised: This is a promising result as we can further reduce the query complexity by choosing a different autoencoder since the use of CGE has to be matched with the encoded dimension.

Original: [Table]
Revised: [Table]

Original: T HE PERFORMANCE OF FO-AE-DS WITH DIFFERENT TRAINING SCHEMES .
Revised: T HE PERFORMANCE OF FO-AE-DS WITH DIFFERENT TRAINING SCHEMES .

Original: Table.
Revised: Table.

Original: A3 presents the certiﬁed accuracy of FO-AE-DS (ﬁrst-order implementation of ZO-AE-DS) with different training schemes.
Revised: A3 presents the certiﬁed accuracy of FO-AE-DS (ﬁrst-order implementation of ZO-AE-DS) with different training schemes.

Original: Training both denoiser and encoder is the default setting.
Revised: Training both denoiser and encoder is the default setting.

Original: As we can see, only training the denoiser would bring performance degradation, and training both denoiser and AE does boost the performance.
Revised: As we can see, only training the denoiser would bring performance degradation, and training both denoiser and AE does boost the performance.

Original: It is worth noting that FO-AE-DS with "train the denoiser and AE" training scheme can be regarded as the FO-DS treating the combination of the original denoiser and the same AE used in FO-AE-DS as a new denoiser, which cannot be implemented for
Revised: It is worth noting that FO-AE-DS with "train the denoiser and AE" training scheme can be regarded as the FO-DS treating the combination of the original denoiser and the same AE used in FO-AE-DS as a new denoiser, which cannot be implemented for

Original: ZO-AE-DS since the decoder of ZO-AE-DS is merged into the black-box classiﬁer and its parameters cannot be updated.
Revised: ZO-AE-DS since the decoder of ZO-AE-DS is merged into the black-box classiﬁer and its parameters cannot be updated.

Original: Furthermore, the key of the introduced AE is to reduce the variable dimension for
Revised: Furthermore, the key of the introduced AE is to reduce the variable dimension for

Original: Zeroth-Order (ZO) gradient estimation
Revised: Zeroth-Order (ZO) gradient estimation

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: C. T HE PERFORMANCE OF ZO-AE-DS ON I MAGE N ET I MAGES .
Revised: C. T HE PERFORMANCE OF ZO-AE-DS ON I MAGE N ET I MAGES .

Original: To evaluate the performance of ZO-AE-DS on the Restricted ImageNet (R-ImageNet) dataset, a 10-class subset of ImageNet with 38472 images for training and 1500 images for testing, similar to (Tsipras et al., 2019).
Revised: To evaluate the performance of ZO-AE-DS on the Restricted ImageNet (R-ImageNet) dataset, a 10-class subset of ImageNet with 38472 images for training and 1500 images for testing, similar to (Tsipras et al., 2019).

Original: Due to our limited computing resources, we are not able to scale up our experiment to the full ImageNet dataset, but the purpose of evaluating on high-dimension images remains the same.
Revised: Due to our limited computing resources, we are not able to scale up our experiment to the full ImageNet dataset, but the purpose of evaluating on high-dimension images remains the same.

Original: In the implementation of ZO-AE-DS, we choose an AE with an aggressive compression (130:1), which is to compress the original 3 × 224 × 224 images into the 1152 × 1 ×feature dimension.
Revised: In the implementation of ZO-AE-DS, we choose an AE with an aggressive compression (130:1), which is to compress the original 3 × 224 × 224 images into the 1152 × 1 ×feature dimension.

Original: We compare the certiﬁed accuracy (CA) performance of our proposed ZO-AE-DS (using CGE) with the black-box baseline ZO-DS, and the white-box baselines FO-DS and FO-AE-DS.
Revised: We compare the certiﬁed accuracy (CA) performance of our proposed ZO-AE-DS (using CGE) with the black-box baseline ZO-DS, and the white-box baselines FO-DS and FO-AE-DS.

Original: Results are summarized in the following table.
Revised: Results are summarized in the following table.

Original: As we can see, (1) when considering the black-box classiﬁer, the proposed ZO-AE-DS still significantly outperforms the direct ZO implementation of DS.
Revised: As we can see, (1) when considering the black-box classiﬁer, the proposed ZO-AE-DS still significantly outperforms the direct ZO implementation of DS.

Original: This shows the importance of variance reduction of query-based gradient estimates.
Revised: This shows the importance of variance reduction of query-based gradient estimates.

Original: (2) Since ZO-AE-DS and FO-AE-DS used an aggressive
Revised: (2) Since ZO-AE-DS and FO-AE-DS used an aggressive

Original: AE structure, the performance drops compared to FO-DS.
Revised: AE structure, the performance drops compared to FO-DS.

Original: (3) the use of high-resolution images would make the black-box defense much more challenging.
Revised: (3) the use of high-resolution images would make the black-box defense much more challenging.

Original: However, ZO-AE-DS is still a principled black-box defense method that can achieve reasonable performance.
Revised: However, ZO-AE-DS is still a principled black-box defense method that can achieve reasonable performance.

Original: [Table]
Revised: [Table]