Collaborate to Defend Against Adversarial Attacks

Adversarially robust learning methods require invariant predictions to a small neighborhood of its natural inputs , thus often encountering insufficient model capacity . Learning multiple sub-models in an ensemble can mitigate this insufficiency , further improving both generalization and robustness . However , an ensemble still wastes the limited capacity of multiple models . To optimally utilize the limited capacity , this paper proposes to learn a collaboration among multiple sub-models . Compared with the ensemble , the collaboration enables the possibility of correct predictions even if there exists a single correct sub-model . Besides , learning a collaboration could enable every sub-model to fit its vulnerability area and reserve the rest of the sub-models to fit other vulnerability areas . To implement the idea , we propose a collaboration framework—CDA2 the abbreviation for Collaborate to Defend against Adversarial Attacks . CDA2 could effectively minimize the vulnerability overlap of all sub-models and then choose a representative sub-model to make correct predictions . Empirical experiments verify that CDA2 outperforms various ensemble methods against black-box and white-box adversarial attacks . 1 INTRODUCTION . Safety-critical applications ( such as in medicine and finance ) require the adversarial robustness of deep models ( Goodfellow et al. , 2015 ; Szegedy et al. , 2014 ) . An adversarially robust learning method requires invariant predictions to a small neighborhood of its natural inputs , thus often encountering insufficient model capacity ( Zhang et al. , 2021 ; Yu et al. , 2021a ) . This limits the further improvement of robustness and has the undesirable degradation of generalization ( Madry et al. , 2018 ) . Learning multiple sub-models in an ensemble ( Breiman , 1996 ; Freund et al. , 1996 ) can mitigate this insufficiency ( Pang et al. , 2019 ; Kariyappa & Qureshi , 2019 ; Yang et al. , 2020a ) . Remarkably , Pang et al . ( 2019 ) , Kariyappa & Qureshi ( 2019 ) and Yang et al . ( 2020a ) minimized the vulnerability overlaps between each pair of sub-models and improved both robustness and generalization over a single model . However , an ensemble wastes the limited capacity of multiple models . In the example of three sub-models ( see Figure 1 ( b ) ) , the adversarial input that lies in the black areas can fool the ensemble successfully , i.e. , more than half of sub-models must correctly classify the adversarial input . Therefore , the ensemble ’ s voting-based strategy excludes the possibility that true predictions remain with the minority . Besides , learning an ensemble requires more than half of the sub-models to fit the same vulnerability areas , which leaves the following question unanswered whether we could only leverage a single sub-model to fit a vulnerability area and reserve the rest of the sub-models to fit other vulnerability areas . To optimally utilize the limited capacity , this paper proposes to learn a collaboration among multiple sub-models . As shown in Figure 1 ( c ) , the adversarial input that lies in the vulnerability overlaps of all sub-models can undoubtedly fool the collaboration . Compared with the ensemble in Figure 1 ( b ) ) , collaboration enables the possibility of correct predictions even if there exists a single correct submodel merely . Besides , learning a collaboration could enable every sub-model to fit its vulnerability area , which could collectively fix broader vulnerability areas than the ensemble does . Then , submodels could collaboratively choose trustworthy ones to make the final predictions . To realize the idea , we propose a collaboration framework—Collaborate to Defend against Adversarial Attacks ( CDA2 ) ( Algorithms 1 and 2 ) . In CDA2 , each sub-model has dual heads : one outputs a vector of predicted probability fθ ( · ) ; another outputs a scalar that measures posterior probability density ( PPD ) of the prediction . In the training phase , given a natural or adversarial input x , each sub-model chooses an easy one ( s ) to feed itself . The PPD head is meanwhile updated by comparing the predicted probability on the true label—fyθ ( · ) ( a scalar ) . In the inference phase , given an input , CDA2 chooses a sub-model with the largest PPD value as the representative to output the prediction . We highlight our key contributions as follows . • We provide a new perspective on learning multiple sub-models for defending against adversarial attacks . We theoretically show the collaboration makes better decisions than the ensemble , which implies collaboration may fix broader vulnerability areas . • We propose a novel collaboration framework—CDA2 ( see Section 3.2 ) . In the training phase , CDA2could effectively minimize the vulnerability overlap of all sub-models ; In the inference phase , CDA2 could effectively choose a representative sub-model to make correct predictions . We also provide a comprehensive analysis illustrating the rationale of CDA2 . • Empirical experiments verify that CDA2 outperforms various ensemble methods against black-box and white-box adversarial attacks . 2 RELATED WORKS . Adversarial attack Adversarial attacks aim to craft the human-imperceptible adversarial input to fool the deep models . Adversarial attacks could be roughly divided into white-box attacks in which the adversary is fully aware of the model ’ s structures ( Goodfellow et al. , 2015 ; Moosavi-Dezfooli et al. , 2016 ; Carlini & Wagner , 2017b ; Chen et al. , 2018 ; Athalye et al. , 2018 ; Xiao et al. , 2018 ; Zheng et al. , 2019 ; Wong et al. , 2019 ; Mopuri et al. , 2019 ; Alaifari et al. , 2019 ; Sriramanan et al. , 2020 ; Wu et al. , 2020b ; Croce & Hein , 2020 ; Yu et al. , 2021b ) and black-box attacks in which the deep models are treated as black boxes to the adversary ( Cheng et al. , 2019 ; 2020 ; Wu et al. , 2020a ; Chen et al. , 2020a ; Li et al. , 2020a ; Rahmati et al. , 2020 ; Yan et al. , 2021b ; Hendrycks et al. , 2021 ; Dong et al. , 2018 ; Xie et al. , 2019 ) . This paper focuses on building effective defense and select both white-box and black-box attack methods as our robustness evaluation metrics . Adversarial defense Defending adversarial attacks is a challenging task and researchers have proposed various solutions . Certified defense tries to learn provably robust deep models against norm-bounded ( e.g. , ` 2 and ` ∞ ) perturbations ( Wong & Kolter , 2018 ; Tsuzuku et al. , 2018 ; Weng et al. , 2018 ; Mirman et al. , 2018 ; Hein & Andriushchenko , 2017 ; Lécuyer et al. , 2019 ; Xiao et al. , 2019 ; Cohen et al. , 2019 ; Balunovic & Vechev , 2020a ; Zhang et al. , 2020a ; Singla & Feizi , 2020 ; Balunovic & Vechev , 2020b ; Zou et al. , 2021 ) . Empirical defense leverages adversarial data to build effective defense such as adversary detection ( Metzen et al. , 2017 ; Li & Li , 2017 ; Carlini & Wagner , 2017a ; Tian et al. , 2018 ; Ma et al. , 2018b ; Lee et al. , 2018 ; Pang et al. , 2018 ; Smith & Gal , 2018 ; Roth et al. , 2019 ; Liu et al. , 2019 ; Yin & Rohde , 2020 ; Sperl et al. , 2020 ; Cohen et al. , 2020 ; Sheikholeslami et al. , 2021 ; Chen et al. , 2021a ; Yang et al. , 2020b ; Qin et al. , 2020 ; Tian et al. , 2021 ; Wu et al. , 2021 ) and adversarial training ( AT ) , in which AT stands out as the most effective defense . Researchers have investigated various aspects of AT , such as improving AT ’ s robustness or generalization ( Madry et al. , 2018 ; Yan et al. , 2018 ; Wu et al. , 2018 ; Cai et al. , 2018 ; Najafi et al. , 2019 ; Alayrac et al. , 2019 ; Carmon et al. , 2019 ; Farnia et al. , 2019 ; Song et al. , 2019 ; Zhang et al. , 2019b ; Wang et al. , 2019 ; Tramèr & Boneh , 2019 ; Zhang & Wang , 2019 ; Stutz et al. , 2020 ; Pang et al. , 2020 ; Gan et al. , 2020 ; Dong et al. , 2020 ; Zhang et al. , 2020b ; Chen et al. , 2020b ; Song et al. , 2020 ; Ding et al. , 2020 ; Wang et al. , 2020b ; Zhang et al. , 2021 ) , fixing AT ’ s undesirable robust overfitting ( Rice et al. , 2020 ; Chen et al. , 2021b ) , improving AT ’ s training efficiency ( Zhang et al. , 2019a ; Shafahi et al. , 2019 ; Zheng et al. , 2020 ; B.S . & Babu , 2020 ; Andriushchenko & Flammarion , 2020 ; Wong et al. , 2020 ) , understanding/interpreting AT ’ s unique traits ( Nakkiran , 2019 ; Yin et al. , 2019 ; Gao et al. , 2019 ; Cranko et al. , 2019 ; Zhang et al. , 2019c ; Liu et al. , 2020 ; Roth et al. , 2020 ; Wang et al. , 2020a ; Zhang et al. , 2020c ; Li et al. , 2020b ; Zou et al. , 2021 ; Mehrabi et al. , 2021 ; Xu et al. , 2021 ) , etc . Besides , researchers have alao actively investigated robust-structured models ( Cisse et al. , 2017 ; Xie et al. , 2020 ; Moosavi-Dezfooli et al. , 2019 ; Xie & Yuille , 2020 ; Yan et al. , 2021a ; Du et al. , 2021 ; Pang et al. , 2021 ) . Nevertheless , the above research thoroughly investigated a single model ; this paper focuses on the collaboration among multiple models for adversarial defense . Ensemble methods for adversarial robustness The most relevant studies are the ensemble methods . Ensemble methods such as bagging ( Breiman , 1996 ) and boosting ( Freund et al. , 1996 ) have been investigated for significantly improving the model ’ s generalization . Motivated by the benefits of ensemble methods in improving generalization , researchers introduced an ensemble to improve the model robustness ( Yang et al. , 2020a ; Kariyappa & Qureshi , 2019 ; Pang et al. , 2019 ; Tramèr et al. , 2018 ) . Tramèr et al . ( 2018 ) proposed to reduce the adversarial transferability by training a single model with adversarial examples from multiple pretrained sub-models . Pang et al . ( 2019 ) introduce a regularization method—ADP—to encourage high diversity in the non-maximal predictions of sub-models . Kariyappa & Qureshi ( 2019 ) improved the ensemble diversity by maximizing the introduced cosine distance between the gradients of sub-models with respect to the input . Yang et al . ( 2020a ) proposed to distill non-robust features in the input and diversify the adversarial vulnerability . These methods reduced overlaps of vulnerability areas between sub-models ( Yang et al. , 2020a ) . To further improve the ensembles , mixture-of-experts ( MOE ) assume that the problem space can be divided into multiple sub-problems through a gate module ; the gate module specifies each sub-model on a specific sub-problem ( Jacobs et al. , 1991 ; Ma et al. , 2018a ) . Nevertheless , to the best of our knowledge , MOE-based methods have been not applied to help adversarial robustness . Inspired by MOE , we propose the collaboration framework to defend against adversary attacks .

Summary: This paper proposes an ensemble or mixture-of-experts method to defend against adversarial examples. Though the authors prefer to use the term collaboration method to highlight its difference from vanilla ensemble. The main idea is that, during adversarial training, the sub-models are trained on each other's adversarial examples.  Specifically: * for each training image, one adversarial example is generated per sub-model by carrying out an attack on each sub-model. * each adversarial example is (softly) assigned to the sub-model that has the lowest loss on it as a training image. * each sub-model has a second output called PPD that quantifies its confidence. * at inference time, for each input, the sub-model with the highest PPD produces the output.  The rationale is that, because each sub-model only needs to cover part of the adversarial example space, they can do a better job. Experiments on CIFAR-10 with L_inf attacks are reported.