On the Certified Robustness for Ensemble Models and Beyond

1 INTRODUCTION . Deep neural networks ( DNN ) have been widely applied in various applications , such as image classification ( Krizhevsky , 2012 ; He et al. , 2016 ) , face recognition ( Sun et al. , 2014 ) , and natural language processing ( Vaswani et al. , 2017 ; Devlin et al. , 2019 ) . However , it is well-known that DNNs are vulnerable to adversarial examples ( Szegedy et al. , 2013 ; Carlini & Wagner , 2017 ; Xiao et al. , 2018a ; b ; Bhattad et al. , 2020 ; Bulusu et al. , 2020 ) , and it has raised great concerns especially when DNNs are deployed in safety-critical applications such as autonomous driving and facial recognition . To defend against such attacks , several empirical defenses have been proposed ( Papernot et al. , 2016b ; Madry et al. , 2018 ) ; however , many of them have been attacked again by strong adaptive attackers ( Athalye et al. , 2018 ; Tramer et al. , 2020 ) . To end such repeated game between the attackers and defenders , certified defenses ( Wong & Kolter , 2018 ; Cohen et al. , 2019 ) have been proposed to provide the robustness guarantees for given ML models , so that no additional attack can break the model under certain adversarial constraints . For instance , randomized smoothing has been proposed as an effective defense providing certified robustness ( Lecuyer et al. , 2019 ; Cohen et al. , 2019 ; Yang et al. , 2020a ) . Among different certified robustness approaches ( Weng et al. , 2018 ; Xu et al. , 2020 ; Li et al. , 2020a ) , randomized smoothing provides a model-independent way to smooth a given ML model and achieves state-of-the-art certified robustness on large-scale datasets such as ImageNet . Currently , all the existing certified defense approaches focus on the robustness of a single ML model . Given the observations that ensemble ML models are able to bring additional benefits in standard learning ( Opitz & Maclin , 1999 ; Rokach , 2010 ) , in this work we aim to ask : Can an ensemble ML model provide additional benefits in terms of the certified robustness compared with a single model ? If so , what are the sufficient and necessary conditions to guarantee such certified robustness gain ? Empirically , we first find that standard ensemble models only achieve marginally higher certified robustness by directly appling randomized smoothing : with L2 perturbation radius 1.5 , a single model achieves certified accuracy as 21.9 % , while the average aggregation based ensemble of three models achieves certified accuracy as 24.2 % on CIFAR-10 ( Table 2 ) . Given such observations , next we aim to answer : How to improve the certified robustness of ensemble ML models ? What types of conditions are required to improve the certified robustness for ML ensembles ? In particular , from the theoretical perspective , we analyze the standard Weighted Ensemble ( WE ) and Max-Margin Ensemble ( MME ) protocols , and prove the sufficient and necessary conditions for the certifiably robust ensemble models under model-smoothness assumption . Specifically , we prove that : ( 1 ) an ensemble ML model is more certifiably robust than each single base model ; ( 2 ) diversified gradients and large confidence margins of base models are the sufficient and necessary conditions for the certifiably robust ML ensembles . We show that these two key factors would lead to higher certified robustness for ML ensembles . We further propose Ensemble-before-Smoothing as the model smoothing strategy and prove the bounded model-smoothness with such strategy , which realizes our model-smoothness assumption . Inspired by our theoretical analysis , we propose Diversity-Regularized Training ( DRT ) , a lightweight regularization-based ensemble training approach . DRT is composed of two simple yet effective and general regularizers to promote the diversified gradients and large confidence margins respectively . DRT can be easily combined with existing ML approaches for training smoothed models , such as Gaussian augmentation ( Cohen et al. , 2019 ) and adversarial smoothed training ( Salman et al. , 2019 ) , with negligible training time overhead while achieves significantly higher certified robustness than state-of-the-art approaches consistently . We conduct extensive experiments on a wide range of datasets including MNIST , CIFAR-10 , and ImageNet . The experimental results show that DRT can achieve significantly higher certified robustness compared to baselines with similar training cost as training a single model . Furthermore , as DRT is flexible to integrate any base models , by using the pretrained robust single ML models as base models , DRT achieves the highest certified robustness so far to our best knowledge . For instance , on CIFAR-10 under L2 radius 1.5 , the DRT-trained ensemble with three base models improves the certified accuracy from SOTA 24.2 % to 30.3 % ; and under L2 radius 2.0 , DRT improves the certified accuracy from SOTA 16.0 % to 20.3 % . Technical Contributions . In this paper , we conduct the first study for the sufficient and necessary conditions of certifiably robust ML ensembles and propose an efficient training algorithm DRT to achieve the state-of-the-art certified robustness . We make contributions on both theoretical and empirical fronts . • We provide the necessary and sufficient conditions for robust ensemble ML models including Weighted Ensemble ( WE ) and Max-Margin Ensemble ( MME ) under the model-smoothness assumption . In particular , we prove that the diversified gradients and large confidence margins of base models are the sufficient and necessary conditions of certifiably robust ensembles . We also prove the bounded model-smoothness via proposed Ensemble-before-Smoothing strategy , which realizes our model-smoothness assumption . • To analyze different ensembles , we prove that when the adversarial transferability among base models is low , WE is more robust than MME . We also prove that the ML ensemble is more robust than a single base model under the model-smoothness assumption . • Based on the theoretical analysis of the sufficient and necessary conditions , we propose DRT , a lightweight regularization-based training approach that can be easily combined with different training approaches and ensemble protocols with small training cost overhead . • We conduct extensive experiments to evaluate the effectiveness of DRT on various datasets , and we show that to the best of your knowledge , DRT can achieve the highest certified robustness , outperforming all existing baselines . Related work . DNNs are known vulnerable to adversarial examples ( Szegedy et al. , 2013 ) . To defend against such attacks , several empirical defenses have been proposed ( Papernot et al. , 2016b ; Madry et al. , 2018 ) . For ensemble models , existing work mainly focuses on empirical robustness ( Pang et al. , 2019 ; Li et al. , 2020b ; Cheng et al. , 2021 ) where the robustness is measured by accuracy under existing attacks and no certified robustness guarantee could be provided or enhanced ; or certify the robustness for a standard weighted ensemble ( Zhang et al. , 2019 ; Liu et al. , 2020 ) using either LP-based ( Zhang et al. , 2018 ) verification or randomized smoothing without considering the model diversity ( Liu et al. , 2020 ) to boost their certified robustness . In this paper , we aim to prove that the diversified gradient and large confidence margin are the sufficient and necessary conditions for certifiably robust ensemble ML models . Moreover , to our best knowledge , we propose the first training approach to boost the certified robustness of ensemble ML models . Randomized smoothing ( Lecuyer et al. , 2019 ; Cohen et al. , 2019 ) has been proposed to provide certified robustness for a single ML model . It achieved the state-of-the-art certified robustness on large-scale dataset such as ImageNet and CIFAR-10 under L2 norm . Several approaches have been proposed to further improve it by : ( 1 ) choosing different smoothing distributions for different Lp norms ( Dvijotham et al. , 2019 ; Zhang et al. , 2020 ; Yang et al. , 2020a ) , and ( 2 ) training more robust smoothed classifiers , using data augmentation ( Cohen et al. , 2019 ) , unlabeled data ( Carmon et al. , 2019 ) , adversarial training ( Salman et al. , 2019 ) , regularization ( Li et al. , 2019 ; Zhai et al. , 2019 ) , and denoising ( Salman et al. , 2020 ) . In this paper , we compare and propose a suitable smoothing strategy to improve the certified robustness of ML ensembles . 2 CHARACTERIZING ML ENSEMBLE ROBUSTNESS . In this section , we prove the sufficient and necessary robustness conditions for both general and smoothed ML ensemble models . Based on these robustness conditions , we discuss the key factors for improving the certified robustness of an ensemble , compare the robustness of ensemble models with single models , and outline several findings based on additional theoretical analysis . 2.1 PRELIMINARIES . Notations . Throughout the paper , we consider the classification task with C classes . We first define the classification scoring function f : Rd → ∆C , which maps the input to a confidence vector , and f ( x ) i represents the confidence for the ith class . We mainly focus on the confidence after normalization , i.e. , f ( x ) ∈ ∆C = { p ∈ RC≥0 : ‖p‖1 = 1 } in the probability simplex . To characterize the confidence margin between two classes , we define fy1/y2 ( x ) : = f ( x ) y1 − f ( x ) y2 . The corresponding prediction F : Rd → [ C ] is defined by F ( x ) : = arg maxi∈ [ C ] f ( x ) i . We are also interested in the runner-up prediction F ( 2 ) ( x ) : = arg maxi∈ [ C ] : i 6=F ( x ) f ( x ) i. r-Robustness . For brevity , we consider the model ’ s certified robustness , against the L2-bounded perturbations as defined below . Our analysis can be generalizable for L1 and L∞ perturbations , leveraging existing work ( Li et al. , 2019 ; Yang et al. , 2020a ; Levine & Feizi , 2021 ) . Definition 1 ( r-Robustness ) . For a prediction function F : Rd → [ C ] and input x0 , if all instance x ∈ { x0 + δ : ‖δ‖2 < r } satisfies F ( x ) = F ( x0 ) , we say model F is r-robust ( at point x0 ) . Ensemble Protocols . An ensemble model contains N base models { Fi } Ni=1 , where Fi ( x ) and F ( 2 ) i ( x ) are their top and runner-up predictions for given input x respectively . The ensemble prediction is denoted by M : Rd → [ C ] , which is computed based on outputs of base models following certain ensemble protocols . In this paper , we consider both Weighted Ensemble ( WE ) and Maximum Margin Ensemble ( MME ) . Definition 2 ( Weighted Ensemble ( WE ) ) . Given N base models { Fi } Ni=1 , and the weight vector { wi } Ni=1 ∈ RN+ , the weighted ensembleMWE : Rd → [ C ] is defined by MWE ( x0 ) : = arg max i∈ [ C ] N∑ j=1 wjfj ( x0 ) i . ( 1 ) Definition 3 ( Max-Margin Ensemble ( MME ) ) . Given N base models { Fi } Ni=1 , for input x0 , the max-margin ensemble modelMMME : Rd → [ C ] is defined by MMME ( x0 ) : = Fc ( x0 ) where c = arg max i∈ [ N ] ( fi ( x0 ) Fi ( x0 ) − fi ( x0 ) F ( 2 ) i ( x0 ) ) . ( 2 ) The commonly-used WE ( Zhang et al. , 2019 ; Liu et al. , 2020 ) sums up the weighted confidence of base models { Fi } Ni=1 with weight vector { wi } Ni=1 , and predicts the class with the highest weighted confidence . The standard average ensemble can be viewed as a special case of WE ( where all wi ’ s are equal ) . MME chooses the base model with the largest confidence margin between the top and the runner-up classes , which is a direct extension from max-margin training ( Huang et al. , 2008 ) . Randomized Smoothing . Randomized smoothing ( Lecuyer et al. , 2019 ; Cohen et al. , 2019 ) provides certified robustness by constructing a smoothed model from a given model . Formally , let ε ∼ N ( 0 , σ2Id ) be a Gaussian random variable , for any given model F : Rd → [ C ] ( can be an ensemble ) , we define smoothed confidence function gεF : Rd →∆C such that gεF ( x ) j : = E ε∼N ( 0 , σ2Id ) I [ F ( x+ ε ) = j ] = Pr ε∼N ( 0 , σ2Id ) ( F ( x+ ε ) = j ) . ( 3 ) Intuitively , gεF ( x ) j is the probability of base model F ’ s prediction on the jth class given Gaussian smoothed input . The smoothed classifier GεF : Rd → [ C ] outputs the class with highest smoothed confidence : GεF ( x ) : = arg maxj∈ [ C ] g ε F ( x ) j . Let cA be the predicted class for input x0 , i.e. , cA : = G ε F ( x0 ) . Cohen et al . show that G ε F is ( σΦ −1 ( gεF ( x0 ) cA ) ) -robust at input x0 , i.e. , the certified radius is σΦ−1 ( gεF ( x0 ) cA ) where Φ −1 is the inverse cumulative distribution function of standard normal distribution . In practice , we will leverage the smoothing strategy together with Monte-Carlo sampling to certify ensemble robustness . More details can be found in Appendix A .

Summary: This paper studies the following problem: How to train a certifiably robust classifier with ensemble methods? The authors considered two types of ensembles: the weighted-average ensemble and large-margin ensemble. They first derived theoretically sufficient and necessary conditions for robustness under two types of ensembles, with the conclusion that large confidence margin and diversified gradients are two factors which contributes to the robustness of ensemble models. Diversity-regularized training, a method of designing loss functions for training ensemble models, is proposed motivated by their theoretical findings. They applied this methodology to randomized smoothing, performed extensive experiments and showed non-trivial improvement over single model methods.