Rethinking Uncertainty in Deep Learning: Whether and How it Improves Robustness

1 INTRODUCTION . Deep neural networks ( DNNs ) have achieved great success in image recognition ( Russakovsky et al. , 2015 ) , audio recognition ( Graves & Jaitly , 2014 ) , etc . However , as shown by ( Szegedy et al. , 2013 ) , DNNs are vulnerable to adversarial attacks , where slightly perturbed adversarial examples can easily fool DNN classifiers . The ubiquitous existence of adversarial examples ( Kurakin et al. , 2016 ) casts doubts on real-world DNN applications . Therefore , many techniques are proposed to enhance the robustness of DNNs to adversarial attacks ( Papernot et al. , 2016 ; Kannan et al. , 2018 ) . Regarding defenses to adversarial attacks , adversarial training ( AT ) ( Goodfellow et al. , 2014 ; Madry et al. , 2017 ) is commonly recognized as the most effective defense . However , it illustrates poor performance on clean examples and under attacks stronger than what they are trained on ( Song et al. , 2018 ) . Recent works ( Rice et al. , 2020 ) attribute such shortcomings to overfitting , but solutions , or even mitigation to them remain open problems . Meanwhile , many regularization techniques are proposed to improve robustness , among which two closely related regularizers , entropy maximization ( EntM ) and label smoothing ( LS ) show performance improvements under weak attacks ( Pang et al. , 2019 ; Shafahi et al. , 2019 ) without compromising accuracy on clean examples . However , as stated by ( Uesato et al. , 2018 ; Athalye et al. , 2018 ) , strong and various kinds of attacks should be used to better approximate the adversarial risk of a defense . Therefore , it remains doubtful how they perform under stronger attacks by themselves and how their adversarial risk is . One appealing property of both EntM and LS is that they both penalize over-confident predictions and promote prediction uncertainty ( Pereyra et al. , 2017 ; Szegedy et al. , 2016 ) . Therefore , both of them are used as regularizers that mitigate overfitting in multiple applications ( Müller et al. , 2019 ) . However , it remains unclear whether EntM and LS can effectively regularize AT , where many other regularizers are ineffective ( Rice et al. , 2020 ) . Therefore , in this paper , we perform an extensive empirical study on uncertainty promotion regularizers , i.e . entropy maximization and label smoothing , in the domain of adversarial machine learning . We carry out experiments on both regularizers , with and without AT , on multiple datasets and under various adversarial settings . We found out that , although neither EntM and LS are able to provide consistent robustness by themselves , both regular- izers complement AT and improve its performances consistently . Specifically , we observe not only better accuracy on clean examples , but also better robustness , especially under attacks with larger perturbations ( e.g . 9 % improvement under perturbation ε = 16/255 for models trained with 8/255 AT . See Table 1 and 3 ) . In addition , we investigate the underlying mechanisms about how EntM and LS complement AT , and attribute the improvements to the shrunken norm of Jacobian matrix ‖∇Xf ( X ; θ ) ‖ ( ∼10 times . See Sect . 6 ) , which indicates better numerical stability and hence better adversarial robustness than AT . To summarize , we make the following contributions : 1 . We carry out an extensive empirical study about whether uncertainty promotion regularizers , i.e . entropy maximization ( EntM ) and label smoothing ( LS ) , provide better robustness , and find out that while neither of them provides consistent robustness under strong attacks alone , both techniques serve as effective regularizers that improve the performance of AT consistently on multiple datasets , especially under large perturbations . 2 . We provide further analysis on uncertainty promotion regularizers from the perspective of Jacobian matrices , and find out that by applying such regularizers , the norm of Jacobian matrix ‖∇Xf ( X ; θ ) ‖ is significantly shrunken , leading to better numerical stability and adversarial robustness . 2 RELATED WORK AND DISCUSSIONS . Adversarial Attacks and Defenses . ( Szegedy et al. , 2013 ; Goodfellow et al. , 2014 ) pointed out the vulnerability of DNNs to adversarial examples , and proposed an efficient attack , the FGSM , to generate such examples . Since then , as increasingly strong adversaries Carlini & Wagner ( 2017 ) are proposed , AT ( Madry et al. , 2017 ) is considered as the most effective defense remaining . More recently , Zhang et al . ( 2019 ) proposes TRADES , extending AT and showing better robustness than Madry et al . ( 2017 ) . We refer readers to ( Yuan et al. , 2019 ; Biggio & Roli , 2018 ) for more comprehensive surveys on adversarial attacks and defenses . However , all AT methods illustrate common drawbacks . Specifically , Rice et al . ( 2020 ) shows that AT suffers from severe overfitting that can not be easily mitigated , which leads to multiple undesirable consequences . For example , AT performs poorly on clean examples , and Song et al . ( 2018 ) shows that AT overfits on the adversary it is trained on and performs poorly under other attacks . Uncertainty Promotion Techniques in Machine Learning . The principle of maximum entropy is widely accepted in not only physics , but also machine learning , such as reinforcement learning ( Ziebart et al. , 2008 ; Haarnoja et al. , 2018 ) . In supervised learning , uncertainty promotion techniques , including entropy maximization ( Pereyra et al. , 2017 ) and label smoothing ( Szegedy et al. , 2016 ) act on model outputs and can improve generalization in many applications , such as image classification and language modeling ( Pereyra et al. , 2017 ; Müller et al. , 2019 ; Szegedy et al. , 2016 ) . There are only several works that study uncertainty promotion techniques in the field of adversarial machine learning . ADP ( Pang et al. , 2019 ) claims that diversity promotion in ensembles can promote robustness , for which entropy maximization is used . Essentially , ADP consists of multiple mutually independent models that in total mimic the behavior of entropy maximization . However , ADP only shows improvements under weak attacks , e.g . PGD with 10 iterations . Shafahi et al . ( 2019 ) claims that label smoothing can improve robustness beyond AT by itself , yet the claim remains to be further investigated under a wider range of adversaries , e.g . adaptive attacks and black-box attacks . As stated by ( Uesato et al. , 2018 ; Athalye et al. , 2018 ) that , one should use strong and various attackers for better approximation of the adversarial risk ( Uesato et al. , 2018 ) , it remains unknown how much robustness EntM or LS alone can provide under strong and diverse attacks . Moreover , neither of these works study the relationship between AT and uncertainty promotion , which is an important open problem since many regularizers fail to mitigate the overfitting of AT ( Rice et al. , 2020 ) . We also discuss knowledge distillation ( KD ) ( Hinton et al. , 2015 ) as another line of work that introduces uncertainty . KD is a procedure that trains a student model to fit the outputs of a teacher network , which are also ’ soft ’ labels and introduce uncertainty . However , KD alone has been shown not to improve robustness ( Carlini & Wagner , 2016 ) . Also , while ( Goldblum et al. , 2020 ) proposed adversarially robust distillation ( ARD ) , combining AT and KD , ARD primarily aims to build robust models upon robust pre-trained teacher models , while we primarily study uncertainty and do not rely on pre-trained teacher models . Therefore we consider ARD to be orthogonal to our work . 3 PRELIMINARIES . Notations . We denote a dataset D = { X ( i ) , y ( i ) } ni=1 , where X ( i ) ∈ Rd are input data , y ( i ) ∈ RC are one-hot vectors for labels . We denote a DNN parameterized by θ as f ( X ; θ ) : Rd → RC , which outputs logits on C classes . We denote fσ ( X ; θ ) = σ ◦ f ( X ; θ ) , the network followed by softmax activation . We denote the cross entropy loss as CE ( ŷ , y ) = − ∑C i=1 yi log ŷi , the Shannon entropy as H ( p ) = − ∑C i=1 pi log pi and the Kullback-Leibler divergence as DKL ( p‖q ) = ∑C i=1 pi log pi qi . Adversarial Attacks and Adversarial Training . Given a data-label pair ( X , y ) and a neural network f ( X ; θ ) , adversarial attacks aim to craft an adversarial example X ( adv ) , ‖X ( adv ) −X‖ ≤ ε , so as to fool f ( X ; θ ) . Such goal can be formulated as the problem : X ( adv ) = argmax X′ Latk ( fσ ( X ′ ; θ ) , y ) , subject to‖X ′ −X‖ ≤ ε . ( 1 ) whereLatk is some loss function , generally taken as CE . Eqn . 1 can generally be solved via iterative optimization , such as Projected Gradient Descent ( PGD ) ( Madry et al. , 2017 ) . Adversarial Training ( AT ) is generally considered the most effective defense against adversarial attacks , where the training set consists of both clean examples and adversarial examples , and the following objective is minimized ( Goodfellow et al. , 2014 ) , min θ E ( X , y ) ∼D [ αCE ( fσ ( X ; θ ) , y ) + ( 1− α ) max ‖X ( adv ) −X‖≤ε CE ( fσ ( X ( adv ) ; θ ) , y ) ] , ( 2 ) where the inner maximization is commonly solved via PGD . We specifically denote AT which uses PGD to solve the inner problem as PAT ( Madry et al. , 2017 ) . TRADES ( Zhang et al. , 2019 ) extends PAT by optimizing Eqn . 3 in similar min-max styles , leading to better robustness than PAT . min θ E ( X , y ) ∼D [ CE ( fσ ( X ; θ ) , y ) + β max ‖X ( adv ) −X‖≤ε DKL ( fσ ( X ( adv ) ; θ ) ‖fσ ( X ; θ ) ) ] . ( 3 ) 4 METHODOLOGY . 4.1 ENTROPY MAXIMIZATION . Given an example ( X , y ) and a neural network f ( X ; θ ) , the proposed entropy maximization ( EntM ) minimizes the cross entropy loss , while maximizing the Shannon entropy of the output probability . Lentm ( X , y ) = CE ( fσ ( X ; θ ) , y ) − λH ( fσ ( X ; θ ) ) , ( 4 ) where λ is a hyperparameter controlling the strength of entropy maximization . For normal training , we minimize over E ( X , y ) ∼D [ Lentm ( X , y ) ] ; for AT , such as PAT and TRADES , we replace CE with Lentm in Eqn . 2 , which for PAT yields Eqn . 5 , and similarly for TRADES : min θ E ( X , y ) ∼D [ αLentm ( fσ ( X ; θ ) , y ) + ( 1− α ) max ‖X ( adv ) −X‖≤ε Lentm ( fσ ( X ( adv ) ; θ ) , y ) ] , ( 5 ) The entropy maximization term penalizes for over-confident outputs and encourages uncertain predictions , which is different from CE encouraging one-hot outputs . Such a formulation is also seen in ( Pereyra et al. , 2017 ; Dubey et al. , 2018 ) in general deep learning and fine-grained classification .

Summary: The authors combine adversarial training with two methods that increase the entropy of the output distribution of neural networks (label smoothing and entropy maximization). The authors find that this combination of ideas increases adversarial robustness on standard benchmarks, especially in the regime of large perturbation budgets (e.g., 16/255 on CIFAR-10). The authors also investigate the effect of their methods on the classification margin to understand the increase in adversarial accuracy.