Adversarial Robustness as a Prior for Learned Representations

1 INTRODUCTION . Beyond achieving remarkably high accuracy on a variety of tasks ( Krizhevsky et al. , 2012 ; He et al. , 2015 ; Collobert & Weston , 2008 ) , a major appeal of deep learning is the ability to learn effective representations of data . Specifically , deep neural networks can be thought of as linear classifiers acting on learned feature representations ( also known as feature embeddings ) . A major goal in representation learning is for these embeddings to encode high-level , interpretable features of any given input ( Goodfellow et al. , 2016 ; Bengio et al. , 2013 ; Bengio , 2019 ) . Indeed , learned representations turn out to be quite versatile—in computer vision , for example , they are the driving force behind transfer learning ( Girshick et al. , 2014 ; Donahue et al. , 2014 ) , and image similarity metrics such as VGG distance ( Dosovitskiy & Brox , 2016a ; Zhang et al. , 2018 ) . Still , deep networks ’ feature embeddings exhibit some shortcomings that are at odds with our idealized model of a linear classifier on top of interpretable high-level features . For example , the existence of adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) —and the fact that they may correspond to flipping predictive features ( Ilyas et al. , 2019 ) —suggests that deep neural networks make predictions based on features that are vastly different from what humans use , or even recognize . ( This message has been also corroborated by several recent works ( Brendel & Bethge , 2019 ; Geirhos et al. , 2019 ; Jetley et al. , 2018 ; Zhang & Zhu , 2019 ) . ) An more direct example of such a shortcoming is pinpointed by Jacobsen et al . ( 2019 ) , who show that one can find image pairs that appear completely different to a human but are nearly identical in terms of their feature embeddings . 1https : //github.com/cantankerousdolphin/robust-learned-representations . Our contributions . Motivated by the limitations of standard representations , we propose using robust optimization ( in particular , adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) ) to enforce a prior on features that models learn ( and thus on their learned feature representations ) . It turns out that the resulting “ robust representations ” ( the embeddings learned by adversarially trained neural networks ) are significantly better-behaved than their standard counterparts . We demonstrate this fact by looking at two well-studied tasks that are typically used to study representations : • Representation inversion ( Section 5 ) : In contrast to standard representations—for which representation inversion is a significant challenge , e.g. , ( Mahendran & Vedaldi , 2015 ) — robust representations are naturally approximately invertible . In particular , adversarially robust networks provide an embedding of the input such that images with similar representations are semantically similar , and the salient features of an image are easily recoverable from its robust feature representation . This property also naturally enables feature interpolation between arbitrary inputs . • Feature visualization ( Section 6 ) : Direct maximization of the coordinates of robust representations suffices to visualize easily recognizable features of the model . This is again a significant departure from standard representations where ( a ) without explicit regularization at visualization-time , feature visualization often produces unintelligible results ; and ( b ) even with regularization , visualized features in the representation layer are scarcely humanrecognizeable ( Olah et al. , 2017 ) . As a result of this property , robust representations enable the addition of specific features to images through direct first-order optimization . The tasks above and their respective applications are illustrated in Figure 2 . Broadly , our results suggest robust optimization as a promising avenue for learning better-behaved image representations . 2 RELATED WORK . Our work studies the feature representations of adversarially robust networks . As discussed in Section 3 , these are networks trained with the robust optimization framework ( Wald , 1945 ; Goodfellow et al. , 2015 ; Madry et al. , 2018 ) and were originally proposed in the context of defending against adversarial perturbations ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) . Adversarial robustness has been studied extensively in the context of machine learning security ( see e.g. , Carlini & Wagner ( 2017 ) ; Athalye et al . ( 2018b ; a ) ; Papernot et al . ( 2017 ) ) , and as an independent phenomenon ( see e.g. , Gilmer et al . ( 2018 ) ; Schmidt et al . ( 2018 ) ; Jacobsen et al . ( 2019 ) ; Ilyas et al . ( 2019 ) ; Tsipras et al . ( 2019 ) ; Su et al . ( 2018 ) . Several recent works have studied the qualitative properties of robust models . Zhang & Zhu ( 2019 ) find that adversarially robust models behave more predictably in the face of various outof-distribution data , appearing to use more global features . Tsipras et al . ( 2019 ) observe that large adversarial perturbation constructed for robust networks actually resemble instances of the target class . Santurkar et al . ( 2019 ) leverage this fact and use robust classifiers for a wide array of image synthesis tasks . Our work is complementary to this line of research , and focuses on understanding properties of robust representations through the lens of “ benchmark ” representation learning tasks ( namely , inversion and component visualization ) . There is also a large body of work dedicated to studying each of the representation learning tasks we study below ( inversion and feature visualization ) —we have embedded discussions of the related work for each task within the relevant sections . 3 BACKGROUND AND MOTIVATION . 3.1 ADVERSARIAL EXAMPLES AND ROBUST TRAINING . In standard settings , supervised machine learning models are trained by minimizing the expected loss with respect to a set of parameters θ , i.e. , by solving an optimization problem of the form : θ∗ = min θ E ( x , y ) ∼D [ Lθ ( x , y ) ] . ( 1 ) We refer to ( 1 ) as the standard training objective—finding the optimum of this objective should guarantee high performance on unseen data from the distribution . It turns out that deep neural networks trained with the standard objective are vulnerable to adversarial examples ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) —by changing a natural input imperceptibly , one can easily manipulate the predictions of a deep network to be arbitrarily incorrect . A natural approach ( and one of the most successful ) for defending against these adversarial examples is to use the robust optimization framework : a classical framework for optimization in the presence of uncertainty ( Wald , 1945 ; Danskin , 1967 ) . In particular , instead of just finding parameters which minimize the expected loss ( as in the standard objective ) , a robust optimization objective also requires that the model induced by the parameters θ be robust to worst-case perturbation of the input : θ∗ = arg min θ E ( x , y ) ∼D [ max δ∈∆ Lθ ( x+ δ , y ) ] . ( 2 ) This robust objective is in fact common in the context of machine learning security , where ∆ is usually chosen to be a simple convex set , e.g. , an ` p-ball . Canonical instantiations of robust optimization such as adversarial training ( Goodfellow et al. , 2015 ; Madry et al. , 2018 ) ) have arisen as practical ways of obtaining networks that are invariant to small ` p-bounded changes in the input while maintaining high accuracy ( though a small tradeoff between robustness and accuracy has been noted by prior work ( Tsipras et al. , 2019 ; Su et al. , 2018 ) ( also cf . Appendix Tables 4 and 5 for a comparison of accuracies of standard and robust classifiers ) ) . 3.2 ROBUST TRAINING AS A FEATURE PRIOR . Traditionally , adversarial robustness in the deep learning setting has been explored as a goal predominantly in the context of ML security and reliability ( Biggio & Roli , 2018 ) . In this work , we consider an alternative perspective on adversarial robustness—we cast it as a prior on the features that can be learned by a model . Specifically , models trained with objective ( 2 ) must be invariant to a set of perturbations ∆ . Thus , selecting ∆ to be a set of perturbations that humans are robust to ( e.g. , small ` p-norm perturbations ) results in models that share more invariances with ( and thus are encouraged to use similar features to ) human perception . Note that incorporating human-selected priors and invariances in this fashion has a long history in the design of ML models—convolutional layers , for instance , were introduced as a means of introducing an invariance to translations of the input ( Fukushima , 1980 ) . In what follows , we will explore the effect of the prior induced by adversarial robustness on models ’ learned representations , and demonstrate that representations learned by adversarially robust models are significantly better-behaved , and enable many previously-infeasible modes of direct interaction with feature embeddings . It is worth noting that despite the value of ε used for training being quite small , we find that robust optimization globally affects the behavior of learned representations . As we demonstrate in this section , the benefits of robust representations extend to out-of-distribution inputs and far beyond ε-balls around the training distribution . 3.3 STANDARD AND ROBUST REPRESENTATIONS . Our work is primarily focused on studying the representations of trained neural networks . Throughout this work , we define the representation function R ( · ) as a function induced by a neural network which maps inputs x ∈ Rn to vectors R ( x ) ∈ Rk in the representation layer of that network ( the penultimate layer ) . In what follows , we refer to “ standard representations ” as the representation functions induced by standard ( non-robust ) networks , trained with the objective ( 1 ) —analogously , “ robust representations ” refer to the representation functions induced by ` 2-adversarially robust networks , i.e . networks trained with the objective ( 2 ) with ∆ being the ` 2 ball : θ∗robust = arg min θ E ( x , y ) ∼D [ max ‖δ‖2≤ε Lθ ( x+ δ , y ) ] . 4 EXPERIMENTAL SETUP . We train robust and standard ResNet-50 ( He et al. , 2016 ) networks on the Restricted ImageNet ( Tsipras et al. , 2019 ) and ImageNet ( Russakovsky et al. , 2015 ) datasets . Datasets specifics are in in Appendix A.1 , training details are in in Appendices A.2 and A.3 , and the performance of each model is reported in Appendix A.4 . In the main text , we present results for Restricted ImageNet , and link to ( nearly identical ) results for ImageNet in Appendices ( B.1.4 , B.3.2 ) . Unless explicitly noted otherwise , our optimization method of choice for any objective function will be ( projected ) gradient descent ( PGD ) , a first-order method which is known to be highly effective for minimizing neural network-based loss functions for both standard and adversarially robust neural networks ( Athalye et al. , 2018a ; Madry et al. , 2018 ) .

Summary: The paper looks at  favorable properties of feature representations of an adversarially robust model. In particular, the authors look at a model trained with PGD training with an $\ell_p$ adversary. In terms of favourable properties, the authors look at representation inversion and feature manipulation and with experimental evidences claim that adversarially robust models are naturally better at it,