How to Robustify Black-Box ML Models? A Zeroth-Order Optimization Perspective

1 INTRODUCTION . ML models , DNNs in particular , have achieved remarkable success owing to their superior predictive performance . However , they often lack robustness . For example , imperceptible but carefully-crafted input perturbations can fool the decision of a well-trained ML model . These input perturbations refer to adversarial perturbations , and the adversarially perturbed ( test-time ) examples are known as adversarial examples or adversarial attacks ( Goodfellow et al. , 2015 ; Carlini & Wagner , 2017 ; Papernot et al. , 2016 ) . Existing studies have shown that it is not difficult to generate adversarial attacks . Numerous attack generation methods have been designed and successfully applied to ( i ) different use cases from the digital world to the physical world , e.g. , image classification ( Brown et al. , 2017 ; Li et al. , 2019 ; Xu et al. , 2019 ; Yuan et al. , 2021 ) , object detection/tracking ( Eykholt et al. , 2017 ; Xu et al. , 2020 ; Sun et al. , 2020 ) , and image reconstruction ( Antun et al. , 2020 ; Raj et al. , 2020 ; Vasiljević et al. , 2021 ) , and ( ii ) different types of victim models , e.g. , white-box models whose details can be accessed by adversaries ( Madry et al. , 2018 ; Carlini & Wagner , 2017 ; Tramer et al. , 2020 ; Croce & Hein , 2020 ; Wang et al. , 2021 ) , and black-box models whose information is not disclosed to adversaries ( Papernot et al. , 2017 ; Tu et al. , 2019 ; Ilyas et al. , 2018a ; Liang et al. , 2021 ) . Given the prevalence of adversarial attacks , methods to robustify ML models are now a major focus in research . For example , adversarial training ( AT ) ( Madry et al. , 2018 ) , which has been poised one of the most effective defense methods ( Athalye et al. , 2018 ) , employed min-max optimization to minimize the worst-case ( maximum ) training loss induced by adversarial attacks . Extended from AT , various empirical defense methods were proposed , ranging from supervised learning , semi-supervised learning , to unsupervised learning ( Madry et al. , 2018 ; Zhang et al. , 2019b ; Shafahi et al. , 2019 ; Zhang et al. , 2019a ; Carmon et al. , 2019 ; Chen et al. , 2020 ; Zhang et al. , 2021 ) . In addition to empirical defense , certified defense is another research focus , which aims to train provably robust ML models and provide certificates of robustness ( Wong & Kolter , 2017 ; Raghunathan et al. , 2018 ; Katz et al. , 2017 ; Salman et al. , 2019 ; 2020 ; 2021 ) . Although exciting progress has been made in adversarial defense , nearly all existing works ask a defender to perform over white-box ML models ( assuming non-confidential model architectures and parameters ) . However , the white-box assumption may restrict the defense application in practice . For example , a model owner may refuse to share the model details , since disclosing model information could hamper the owner ’ s privacy , e.g. , model inversion attacks lead to training data leakage ( Fredrikson et al. , 2015 ) . Besides the privacy consideration , the white-box defense built upon the ( end-to-end ) robust training ( e.g. , AT ) is computationally intensive , and thus is difficult to scale when robustifying multiple models . For example , in the medical domain , there exist massive pre-trained ML models for different diseases using hundreds of neuroimaging datasets ( Sisodiya et al. , 2020 ) . Thus , robustly retraining all models becomes impractical . Taking the model privacy and the defense efficiency into consideration , we ask : Is it possible to design an adversarial defense over black-box models using only model queries ? Extending adversarial defense to the black-box regime ( that we call ‘ blackbox defense ’ ) is highly non-trivial due to the challenge of black-box optimization ( i.e. , learning over blackbox models ) . To tackle this problem , the prior work ( Salman et al. , 2020 ) leveraged surrogate models as approximations of the black-box models , over which defense can be conducted following the whitebox setup . Yet , this still requires to have access to the information on the victim model type and its function . In practice , those conditions could be difficult to achieve . For example , if the domain knowledge related to medicine or healthcare is lacking ( Qayyum et al. , 2020 ; Finlayson et al. , 2019 ) , then it will be difficult to determine a proper surrogate model of a medical ML system . Even if a black-box model estimate can be obtained using the model inversion technique ( Kumar & Levine , 2019 ) , a significantly large number of model queries are needed even just for tackling a MNIST-level prediction task ( Oh et al. , 2019 ) . Different from ( Salman et al. , 2020 ) . We study an authentic blackbox scenario , in which the interaction between defender and model is only based on input-output function queries ( see Fig . 1 ) . To our best knowledge , this is the first work to tackle the problem of query-based black-box defense . Contributions . We summarize our contributions below . ¬ ( Formulation-wise ) We formulate the problem of black-box defense and investigate it through the lens of zeroth-order ( ZO ) optimization . Different from existing work , our paper aims to design the restriction-least black-box defense and our formulation is built upon a query-based black-box setting , which avoids the use of surrogate models . ( Methodology-wise ) We propose a novel black-box defense approach , ZO AutoEncoder-based Denoised Smoothing ( ZO-AE-DS ) , which is able to tackle the challenge of ZO optimization in high dimensions and convert a pre-trained non-robust ML model into a certifiably robust model using only function queries . ® ( Experiment-wise ) We verify the efficacy of our method through an extensive experimental study . In the task of image classification , the proposed ZO-AE-DS significantly outperforms the ZO baseline built upon ( Salman et al. , 2020 ) . For instance , we can improve the certified robust accuracy of ResNet110 on CIFAR-10 from 19.16 % ( using baseline ) to 54.87 % ( using ZO-AE-DS ) under adversarial perturbations with ` 2 norm less than 64/255 . We also empirically show that our proposal stays effective even in the task of image reconstruction . 2 RELATED WORK . Empirical defense . An immense number of defense methods have been proposed , aiming to improve model robustness against adversarial attacks . Examples include detecting adversarial attacks ( Guo et al. , 2017 ; Meng & Chen , 2017 ; Gong et al. , 2017 ; Grosse et al. , 2017 ; Metzen et al. , 2017 ) and training robust ML models ( Madry et al. , 2018 ; Zhang et al. , 2019b ; Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ; Athalye et al. , 2018 ; Cheng et al. , 2017 ; Wong & Kolter , 2017 ; Salman et al. , 2019 ; Raghunathan et al. , 2018 ; Katz et al. , 2017 ) . In this paper , we focus on advancing the algorithm foundation of robust training over black-box models . Robust training can be broadly divided into two categories : empirical defense and certified defense . In the former category , the most representative method is AT ( adversarial training ) that formulates adversarial defense as a two-player game ( between attacker and defender ) ( Madry et al. , 2018 ) . Spurred by AT , empirical defense has developed rapidly . For example , in ( Zhang et al. , 2019b ) , TRADES was proposed to seek the optimal trade-off between accuracy and robustness . In ( Stanforth et al. , 2019 ; Carmon et al. , 2019 ) , unlabeled data and self-training were shown effective to improve adversarial defense in both robustness and generalization . In ( Shafahi et al. , 2019 ; Wong et al. , 2020 ; Zhang et al. , 2019a ; Andriushchenko & Flammarion , 2020 ) , to improve the scalability of adversarial defense , computationally-light alternatives of AT were developed . Despite the effectiveness of empirical defense against adversarial attacks ( Athalye et al. , 2018 ) , it lacks theoretical guarantee ( known as ‘ certificate ’ ) for the achieved robustness . Thus , the problem of certified defense arises . Certified defense . Certified defense seeks to provide a provably guarantee of ML models . One line of research focuses on post-hoc formal verification of a pre-trained ML model . The certified robustness is then given by a ‘ safe ’ input perturbation region , within which any perturbed input will not fool the given model ( Katz et al. , 2017 ; Ehlers , 2017 ; Bunel et al. , 2018 ; Dutta et al. , 2017 ) . Since the exact verification is computationally intensive , a series of work ( Raghunathan et al. , 2018 ; Dvijotham et al. , 2018 ; Wong & Kolter , 2017 ; Weng et al. , 2018a ; b ; Wong et al. , 2018 ) proposed ‘ incomplete ’ verification , which utilizes convex relaxation to over-approximate the output space of a predictive model when facing input perturbations . Such a relaxation leads to fast computation in the verification process but only proves a lower bound of the exact robustness guarantee . Besides the post-hoc model verification with respect to each input example , another line of research focuses on in-processing certification-aware training and prediction . For example , randomized smoothing ( RS ) transforms an empirical classifier into a provably robust one by convolving the former with an isotropic Gaussian distribution . It was shown in ( Cohen et al. , 2019 ) that RS can provide formal guarantees for adversarial robustness . Different types of RS-oriented provable defenses have been develped , such as adversarial smoothing ( Salman et al. , 2019 ) , denoised smoothing ( Salman et al. , 2020 ) , smoothed ViT ( Salman et al. , 2021 ) , and feature smoothing ( Addepalli et al. , 2021 ) . Zeroth-order ( ZO ) optimization for adversarial ML . ZO optimization methods are gradientfree counterparts of first-order ( FO ) optimization methods ( Liu et al. , 2020b ) . They approximate the FO gradients through function value based gradient estimates . Thus , ZO optimization is quite useful to solve black-box problems when explicit expressions of their gradients are difficult to compute or infeasible to obtain . In the area of adversarial ML , ZO optimization has become a principled approach to generate adversarial examples from black-box victim ML models ( Chen et al. , 2017 ; Ilyas et al. , 2018a ; b ; Tu et al. , 2019 ; Liu et al. , 2019 ; 2020a ; Huang & Zhang , 2020 ; Cai et al. , 2021 ) . Such ZO optimization-based attack generation methods can be as effective as state-of-the-art white-box attacks , despite only having access to the inputs and outputs of the targeted model . For example , the work ( Tu et al. , 2019 ) leveraged the white-box decoder to map the generated low-dimension perturbations back to the original input dimension . Inspired by ( Tu et al. , 2019 ) , we leverage the autoencoder architecture to tackle the high-dimension challenge of ZO optimization in black-box defense . Despite the widespread application of ZO optimization to black-box attack generation , few work studies the problem of black-box defense .

Summary: This work provides an algorithm to ensure robust training of an ML model with just black-box knowledge of it i.e., input and output access. The algorithm relies on using Denoised Smoothing with zeroth-order optimization where the gradients are estimated using random perturbations (finite-differencing). They avoid the computational burden and high variance of these estimates by first training an auto-encoder to reduce the inputs to a low-dimensional subspace. They show over different architectures and multiple datasets (Cifar10, STL10, image reconstruction over MNIST) that the proposed algorithm performs better than the baseline which is the Denoised Smoothing algorithm by Salman et al.