How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

1 INTRODUCTION . ML models , DNNs in particular , have achieved remarkable success owing to their superior predictive performance . However , they often lack robustness . For example , imperceptible but carefully-crafted input perturbations can fool the decision of a well-trained ML model . These input perturbations refer to adversarial perturbations , and the adversarially perturbed ( test-time ) examples are known as adversarial examples or adversarial attacks ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016 ) . Existing studies have shown that it is not difficult to generate adversarial attacks . Numerous attack generation methods have been designed and successfully applied to ( i ) different use cases from the digital world to the physical world , e.g. , image classification ( Brown et al. , 2017 ; Li et al. , 2019 ; Xu et al. , 2019 ; Yuan et al. , 2021 ) , object detection/tracking ( Eykholt et al. , 2017 ; Xu et al. , 2020 ; Sun et al. , 2020 ) , and image reconstruction ( Antun et al. , 2020 ; Raj et al. , 2020 ; Vasiljević et al. , 2021 ) , and ( ii ) different types of victim models , e.g. , white-box models whose details can be accessed by adversaries ( Madry et al. , 2018 ; Carlini & Wagner , 2017 ; Tramer et al. , 2020 ; Croce & Hein , 2020 ; Wang et al. , 2021 ) , and black-box models whose information is not disclosed to adversaries ( Papernot et al. , 2017 ; Tu et al. , 2019 ; Ilyas et al. , 2018a ; Liang et al. , 2021 ) . Given the prevalence of adversarial attacks , methods to robustify ML models are now a major focus in research . For example , adversarial training ( AT ) ( Madry et al. , 2018 ) , which has been poised one of the most effective defense methods ( Athalye et al. , 2018 ) , employed min-max optimization to minimize the worst-case ( maximum ) training loss induced by adversarial attacks . Extended from AT , various empirical defense methods were proposed , ranging from supervised learning , semi-supervised learning , to unsupervised learning ( Madry et al. , 2018 ; Zhang et al. , 2019b ; Shafahi et al. , 2019 ; Zhang et al. , 2019a ; Carmon et al. , 2019 ; Chen et al. , 2020 ; Zhang et al. , 2021 ) . In addition to empirical defense , certified defense is another research focus , which aims to train provably robust ML models and provide certificates of robustness ( Wong & Kolter , 2017 ; Raghunathan et al. , 2018 ; Katz et al. , 2017 ; Salman et al. , 2019 ; 2020 ; 2021 ) . Although exciting progress has been made in adversarial defense , nearly all existing works ask a defender to perform over white-box ML models ( assuming non-confidential model architectures and parameters ) . However , the white-box assumption may restrict the defense application in practice . For example , a model owner may refuse to share the model details , since disclosing model information could hamper the owner ’ s privacy , e.g. , model inversion attacks lead to training data leakage ( Fredrikson et al. , 2015 ) . Besides the privacy consideration , the white-box defense built upon the ( end-to-end ) robust training ( e.g. , AT ) is computationally intensive , and thus is difficult to scale when robustifying multiple models . For example , in the medical domain , there exist massive pre-trained ML models for different diseases using hundreds of neuroimaging datasets ( Sisodiya et al. , 2020 ) . Thus , robustly retraining all models becomes impractical . Taking the model privacy and the defense efficiency into consideration , we ask : Is it possible to design an adversarial defense over black-box models using only model queries ? Extending adversarial defense to the black-box regime ( that we call ‘ blackbox defense ’ ) is highly non-trivial due to the challenge of black-box optimization ( i.e. , learning over blackbox models ) . To tackle this problem , the prior work ( Salman et al. , 2020 ) leveraged surrogate models as approximations of the black-box models , over which defense can be conducted following the whitebox setup . Yet , this still requires to have access to the information on the victim model type and its function . In practice , those conditions could be difficult to achieve . For example , if the domain knowledge related to medicine or healthcare is lacking ( Qayyum et al. , 2020 ; Finlayson et al. , 2019 ) , then it will be difficult to determine a proper surrogate model of a medical ML system . Even if a black-box model estimate can be obtained using the model inversion technique ( Kumar & Levine , 2019 ) , a significantly large number of model queries are needed even just for tackling a MNIST-level prediction task ( Oh et al. , 2019 ) . Different from ( Salman et al. , 2020 ) . We study an authentic blackbox scenario , in which the interaction between defender and model is only based on input-output function queries ( see Fig . 1 ) . To our best knowledge , this is the first work to tackle the problem of query-based black-box defense . Contributions . We summarize our contributions below . ¬ ( Formulation-wise ) We formulate the problem of black-box defense and investigate it through the lens of zeroth-order ( ZO ) optimization . Different from existing work , our paper aims to design the restriction-least black-box defense and our formulation is built upon a query-based black-box setting , which avoids the use of surrogate models . ( Methodology-wise ) We propose a novel black-box defense approach , ZO AutoEncoder-based Denoised Smoothing ( ZO-AE-DS ) , which is able to tackle the challenge of ZO optimization in high dimensions and convert a pre-trained non-robust ML model into a certifiably robust model using only function queries . ® ( Experiment-wise ) We verify the efficacy of our method through an extensive experimental study . In the task of image classification , the proposed ZO-AE-DS significantly outperforms the ZO baseline built upon ( Salman et al. , 2020 ) . For instance , we can improve the certified robust accuracy of ResNet110 on CIFAR-10 from 19.16 % ( using baseline ) to 54.87 % ( using ZO-AE-DS ) under adversarial perturbations with ` 2 norm less than 64/255 . We also empirically show that our proposal stays effective even in the task of image reconstruction . 2 RELATED WORK . Empirical defense . An immense number of defense methods have been proposed , aiming to improve model robustness against adversarial attacks . Examples include detecting adversarial attacks ( Guo et al. , 2017 ; Meng & Chen , 2017 ; Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) and training robust ML models ( Madry et al. , 2018 ; Zhang et al. , 2019b ; Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ; Athalye et al. , 2018 ; Cheng et al. , 2017 ; Wong & Kolter , 2017 ; Salman et al. , 2019 ; Raghunathan et al. , 2018 ; Katz et al. , 2017 ) . In this paper , we focus on advancing the algorithm foundation of robust training over black-box models . Robust training can be broadly divided into two categories : empirical defense and certified defense . In the former category , the most representative method is AT ( adversarial training ) that formulates adversarial defense as a two-player game ( between attacker and defender ) ( Madry et al. , 2018 ) . Spurred by AT , empirical defense has developed rapidly . For example , in ( Zhang et al. , 2019b ) , TRADES was proposed to seek the optimal trade-off between accuracy and robustness . In ( Stanforth et al. , 2019 ; Carmon et al. , 2019 ) , unlabeled data and self-training were shown effective to improve adversarial defense in both robustness and generalization . In ( Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ; Andriushchenko & Flammarion , 2020 ) , to improve the scalability of adversarial defense , computationally-light alternatives of AT were developed . Despite the effectiveness of empirical defense against adversarial attacks ( Athalye et al. , 2018 ) , it lacks theoretical guarantee ( known as ‘ certificate ’ ) for the achieved robustness . Thus , the problem of certified defense arises . Certified defense . Certified defense seeks to provide a provably guarantee of ML models . One line of research focuses on post-hoc formal verification of a pre-trained ML model . The certified robustness is then given by a ‘ safe ’ input perturbation region , within which any perturbed input will not fool the given model ( Katz et al. , 2017 ; Ehlers , 2017 ; Bunel et al. , 2018 ; Dutta et al. , 2017 ) . Since the exact verification is computationally intensive , a series of work ( Raghunathan et al. , 2018 ; Dvijotham et al. , 2018 ; Wong & Kolter , 2017 ; Weng et al. , 2018a ; b ; Wong et al. , 2018 ) proposed ‘ incomplete ’ verification , which utilizes convex relaxation to over-approximate the output space of a predictive model when facing input perturbations . Such a relaxation leads to fast computation in the verification process but only proves a lower bound of the exact robustness guarantee . Besides the post-hoc model verification with respect to each input example , another line of research focuses on in-processing certification-aware training and prediction . For example , randomized smoothing ( RS ) transforms an empirical classifier into a provably robust one by convolving the former with an isotropic Gaussian distribution . It was shown in ( Cohen et al. , 2019 ) that RS can provide formal guarantees for adversarial robustness . Different types of RS-oriented provable defenses have been develped , such as adversarial smoothing ( Salman et al. , 2019 ) , denoised smoothing ( Salman et al. , 2020 ) , smoothed ViT ( Salman et al. , 2021 ) , and feature smoothing ( Addepalli et al. , 2021 ) . Zeroth-order ( ZO ) optimization for adversarial ML . ZO optimization methods are gradientfree counterparts of first-order ( FO ) optimization methods ( Liu et al. , 2020b ) . They approximate the FO gradients through function value based gradient estimates . Thus , ZO optimization is quite useful to solve black-box problems when explicit expressions of their gradients are difficult to compute or infeasible to obtain . In the area of adversarial ML , ZO optimization has become a principled approach to generate adversarial examples from black-box victim ML models ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; b ; Tu et al. , 2019 ; Liu et al. , 2019 ; 2020a ; Huang & Zhang , 2020 ; Cai et al. , 2021 ) . Such ZO optimization-based attack generation methods can be as effective as state-of-the-art white-box attacks , despite only having access to the inputs and outputs of the targeted model . For example , the work ( Tu et al. , 2019 ) leveraged the white-box decoder to map the generated low-dimension perturbations back to the original input dimension . Inspired by ( Tu et al. , 2019 ) , we leverage the autoencoder architecture to tackle the high-dimension challenge of ZO optimization in black-box defense . Despite the widespread application of ZO optimization to black-box attack generation , few work studies the problem of black-box defense .

Summary: The authors formulate the problem of black-box defense and propose a novel black-box defense approach called the Zero Order AutoEncoder-based Denoised Smoothing (ZO-AE-DS). Black-box defense corresponds to situations in which the defense model information cannot be obtained due to privacy protection in real scenarios. ZO-AE-DS introduces zero-order optimization on the structure of denoised smoothing (DS) to estimate the gradient and uses an Autoencoder (AE) to connect the denoiser with the model so that zero-order optimization can be conducted in a (low-dimension) feature embedding space.