Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution Data

1 INTRODUCTION . The last decade has seen remarkable progress in deep learning ( DL ) , which has prompted widescale integration of DL frameworks into myriad application domains ( LeCun et al. , 2015 ) . In many of these applications , and in particular in safety-critical domains , it is essential that the DL systems are robust and trustworthy ( Dreossi et al. , 2019 ) . However , it is now well-known that DL is fragile to seemingly innocuous changes to the input data ( Szegedy et al. , 2013 ) . Indeed , well-documented examples of fragility to carefully-designed noise can be found in a variety of contexts , including image classification ( Madry et al. , 2017 ) , clinical trials ( Papangelou et al. , 2018 ) , and robotics ( Melis et al. , 2017 ) . Accordingly , a number of adversarial training algorithms ( Goodfellow et al. , 2014b ; Wong & Kolter , 2017 ) as well as certifiable defenses ( Raghunathan et al. , 2018 ; Fazlyab et al. , 2019a ) have recently been proposed , which have provided a rigorous framework for improving the robustness of DL against norm-bounded perturbations ( Fazlyab et al. , 2019b ; Dobriban et al. , 2020 ) . Despite this encouraging progress , very recent papers have unanimously shown that DL is also fragile to unbounded shifts in the data-distribution due to a wide range of natural phenomena ( Djolonga et al. , 2020 ; Taori et al. , 2020 ; Hendrycks et al. , 2020 ; Hendrycks & Dietterich , 2019 ) . For example , in image classification , such shifts include changes due to lighting , blurring , or weather conditions ( Pei et al. , 2017 ; Chernikova et al. , 2019 ) . However , there are remarkably few general , principled techniques that have been shown to provide robustness against these forms of out-of-distribution , natural variation ( Hendrycks et al. , 2019a ) . Furthermore , as these unseen distributional shifts are arguably more common in safety-critical domains , the task of designing algorithms that generalize to natural , out-of-distribution data is an important and novel challenge for the DL community . In this paper , we propose a paradigm shift from perturbation-based adversarial robustness to modelbased robust deep learning . Our goal is to provide principled , general algorithms that can be used to train neural networks to be robust against natural , out-of-distribution shifts in data . Our experiments show that across a variety of challenging , naturally-occurring conditions , such as variation in lighting , haze , rain , and snow , and across various datasets , including SVHN , GTSRB , CURE-TSR , and ImageNet , classifiers trained with our model-based algorithms significantly outperform standard DL baselines , adversarially-trained classifiers , and , when applicable , domain adaptation methods . Contributions . The contributions of this paper can be summarized as follows : • Paradigm shift . We propose a paradigm shift from perturbation-based robustness to model-based robust deep learning , where models of natural variation express changes due to natural conditions . • Optimization-based formulation . We formulate a novel model-based robust training problem by constructing a general robust optimization procedure to search for challenging natural variation . • Models of natural variation . For many challenging forms of natural variation , we use deep generative models to learn models of natural variation that are consistent with realistic conditions . • Novel algorithms . We propose a family of novel robust training algorithms that exploit models of natural variation to improve the robustness of DL against worst-case natural variation . • Out-of-distribution robustness . We show that our algorithms are the first to consistently provide robustness against natural , out-of-distribution shifts that frequently occur in real-world environments , including snow , rain , fog , and brightness on SVHN , GTSRB , CURE-TSR , and ImageNet . • ImageNet-c robustness . We show that our algorithms can significantly improve the robustness of classifiers trained on ImageNet and tested on ImageNet-c by as much as 30 percentage points . • Robustness to simultaneous distributional shifts . We show that our methods are composable and can improve robustness to multiple simultaneous sources of natural variation . To evaluate this feature , we curate several new datasets , each of which has two simultaneous distributional shifts . • Robustness to unseen domains . We show that models of natural variation can be reused on datasets that are entirely unseen during training to improve out-of-distribution generalization . 2 PERTURBATION-BASED ROBUSTNESS : APPROACHES AND LIMITATIONS . In this paper , we consider a standard classification task in which training data ( x , y ) ⇠ D is distributed according to a joint distribution D over instances x 2 Rd and labels y 2 [ k ] : = { 0 , 1 , . . . , k } . In this setting , given a finite training sample drawn i.i.d . from D , the goal of the learning problem is to obtain a classifier fw parameterized by weights w 2 Rp such that fw can correctly predict the labels y corresponding to new instances x drawn i.i.d . from D. In practice , one can learn fw by approximately solving the non-convex empirical risk-minimization ( ERM ) problem argminw E [ ` ( x , y ; w ) ] where ` is a suitable loss-function . However , neural networks trained using ERM are known to be susceptible to adversarial attacks . This means that given a datum x with a corresponding label y , one can find another datum xadv such that x is close to xadv in a given Euclidean norm and xadv is predicted by the learned classifier as belonging to a different class c 6= y . If such a datum xadv exists , it is called an adversarial example . This is illustrated in Figure 1a ; although these pandas look identical , they were classified differently in ( Goodfellow et al. , 2014b ) . ( a ) Models take the form G ( x , ) , where is a nuisance parameter that describes how the output image x0 : = G ( x , ) is varied . ( b ) Input image x and corresponding generated images for a learned model of natural variation on ImageNet . Figure 2 : In this paper , we introduce models of natural variation to describe natural transformations . The dominant paradigm toward improving robustness against adversarial examples relies on a robust optimization perspective wherein neural networks are trained to correctly classify worst-case perturbations of data ( Madry et al. , 2017 ; Wong & Kolter , 2017 ) . This can be formulated as follows : argmin w E ( x , y ) ⇠D h max 2 ` ( x+ , y ; w ) i ( 1 ) We can think of ( 1 ) as comprising two coupled optimization problems : an inner maximization problem in which we seek a challenging perturbation and an outer minimization problem in which we seek weights that lead to strong classification performance . Limitations of perturbation-based robustness . Despite remarkable progress toward improving the robustness of DL against norm-bounded perturbations , there are significant limitations to adversarial training . Notably , DL is known to be fragile to many forms of natural variation , which can not be described by small perturbations x 7 ! x+ . In image classification , such natural variation includes changes in weather or background color ( Eykholt et al. , 2018 ; Hendrycks et al. , 2019b ; Hosseini & Poovendran , 2018 ) , spatial transformations such as rotation or scaling ( Xiao et al. , 2018b ; Karianakis et al. , 2016 ) , and sensor-based attacks ( Kurakin et al. , 2016 ) . Because such transformations frequently arise in safety-critical domains , it is critically important for the DL community to develop algorithms that are robust against out-of-distribution , natural variation in data . In this paper , we specifically address this challenge by proposing a principled , optimization-based framework which can be used in general settings to provide robustness against arbitrary sources of natural variation . 3 A NEW ROBUSTNESS PARADIGM : MODEL-BASED ROBUST DEEP LEARNING . Underlying the task of improving the robustness of neural networks against natural , out-ofdistribution data are two fundamental challenges . Firstly , unlike in the adversarial robustness community , in real-world , safety-critical environments , data can vary in unknown and highly nonlinear ways . Thus , the first step toward building a robust training procedure must be to design a mechanism that accurately describes how data varies in such environments . Next , assuming a suitable model of natural variation , the second challenge is to formulate a training procedure that exploits this model toward improving robustness . In this section , we present novel solutions to each of these challenges . 3.1 MODELS OF NATURAL VARIATION . In order to effectively model sources of natural variation in a domain-agnostic setting , we will abstractly define models of natural variation . Concretely , a model of natural variation G ( x , ) is a map that describes how an input datum x can be naturally varied by a nuisance parameter resulting in a new datum x0 : = G ( x , ) . Ideally , for a fixed datum x , varying the nuisance parameter should vary the severity of the natural conditions in the generated datum x0 . An example of such a model is shown in Figure 2 , where an image x on the left ( in this case , in sunny weather ) can be naturally varied by and consequently transformed into the image x0 on the right ( in snowy weather ) . In the remainder of this subsection , we consider cases in which ( 1 ) a model G is known a priori , and ( 2 ) a model G is unknown and therefore must be learned offline from data . In this second case in which models of natural variation must be learned , we propose a method for obtaining these models . Known models of natural variation . In many problems , a model G ( x , ) is known a priori due to intrinsic geometric structure . For example , there is underlying structure that describes how data can be rotated , translated , or scaled ; models for rotating an image can be characterized by G ( x , ) = R ( ) x where R ( ) is a rotation matrix and 2 : = [ 0 , 2⇡ ) . In prior work , this idea has been used to train classifiers to be robust to rotation and scaling ( Engstrom et al. , 2017 ; Kamath et al. , 2020 ) . Learning models of natural variation from data . In many situations , models natural variation are not known a priori or are too costly to obtain . For example , consider Figure 2 in which a model G ( x , ) takes an image x of a street sign in sunny weather and maps it to an image x0 : = G ( x , ) in snowy weather . Even though there is a relationship between the two images , obtaining a model G relating these two domains is extremely challenging if we resort to geometric structure . For such problems we advocate for learning the model G from data . To do so , we assume that we have access to two unpaired domains A and B that are drawn from a common distribution . Domain A contains the original data , such as the images with sunny weather , and domain B contains naturally transformed data , such as images with snow . Ideally , a model of natural variation should learn to map images from domain A to corresponding images with different levels of natural variation captured by the images of domain B . In our experiments section , we rely on the MUNIT framework ( Huang et al. , 2018 ) , which combines two autoencoders and two generative adversarial networks ( Goodfellow et al. , 2014a ) , to learn mappings between domains A and B . Furthermore , we note that many choices unpaired , unconditional image-to-image translation networks satisfy our criteria for G , and in future work we plan to investigate the efficacy of these architectures . In Appendix A , we describe parallel experiments that we carried out with two other architectural choices for G , and we fully characterize the MUNIT architecture used in our experiments .

Summary: The paper extends current adversarial learning approaches beyond imperceptible L_p norm perturbations. The proposed approach can handle many models of natural variation, such as a change in brightness. The main idea behind the approach is to use unsupervised approaches such as GANs to model the natural variation. Given this model of natural variation, the paper replaces the adversarial learning objective of finding the worst example in an L_p norm ball around a data point to finding the worst example based on the model of natural variation. This is expensive, so the paper also proposes more computationally efficient approaches based on data augmentation. The experimental results demonstrate that the proposed approach performs well on a variety of tasks.