Interpreting Adversarial Robustness: A View from Decision Surface in Input Space

1 INTRODUCTION . It is commonly believed that a neural network ’ s generalization is correlated to its geometric properties of the loss surface , i.e . the flatness of the local minima in parameter space ( Kawaguchi ( 2016 ) ; Im et al . ( 2017 ) ) . For example , Keskar et al . ( 2017 ) showed that a sharp local minima brings poorer neural network generalization . Li et al . ( 2017 ) visualized and demonstrated that the outstanding performance of ResNet comes from its “ wide valley ” of the local minima on the loss surface . Chaudhari et al . ( 2017 ) then proposed Entropy-SGD , which utilizes the “ wide valleys ” property to guide the neural network training for better generalization . However , such a generalization estimation approach is challenged by adversarial examples recently ( Szegedy et al . ( 2013 ) ) : even models with good generalization may still suffer from adversarial examples , resulting in extremely low accuracy . For example , ResNet model usually converges to a wide and flat local minima on loss surface in parameter space as visualized in Fig . 1 ( a ) , which indicates good test accuracy according to the mentioned evaluation approach . When testing adversarial examples , the model ’ s accuracy is significantly defected by the adversarial noises , which are small , but can cause dramatically loss increments . Therefore , the conventional generalization estimation in parameter space fails under adversarial settings , and how to estimate the generalization over adversarial examples , i.e . the adversarial robustness , remains a significant challenge . Fortunately , the loss increments introduced by adversarial noises can be well reflected in the loss surface in input space as visualized in Fig . 1 ( b ) , where the non-smoothness indicates high sensitivity to the adversarial noises . The differences in Fig . 1 suggest the ineffectiveness of generalization estimation in parameter space and the potential in input space . Therefore , different from prior works focusing on parameter space , we will explore the robustness estimation mainly in input space . In this work , we have the following contributions : • We analyze the geometric properties of the loss surface in both parameter and input space . We demonstrate that the input space is more essential in evaluating the generalization and adversarial robustness of a neural network ; • We reveal the shared mechanisms of various adversarial attack methods . To do so , we first extend the concept of loss surface to decision surface for clearer geometry visualization . By visualizing the adversarial attack trajectory on decision surfaces in input space , we reveal that various adversarial attacks are all utilizing the decision surface geometry properties to cross the decision boundary within least distance . • We then formalize the adversarial robustness indicator by involving the geometric properties of Jacobian and Hessian ’ s eigenvalues . Such an indicator can effectively evaluate a neural network ’ s intrinsic robustness property against various adversarial attacks without field accuracy testing of massive adversarial examples ; It also concludes that the wide and flat plateau of decision surface in input space enables better generalization and robustness . • We also propose a robust training method guided by our adversarial robustness indicator , which aims to smooth the decision surfaces and enhances adversarial robustness by regulating the Jacobian in training process . Our robustness estimation approach has provable relationship with neural network ’ s robustness performance and geometry properties in input space . This enables us to evaluate adversarial robustness of a neural network , without conducting massive adversarial attacks for field test . Guided by such an estimation approach , our robust training method could also effectively enhance the neural network against various adversarial attacks without involving the time-consuming adversarial training . 2 INEFFECTIVENESS OF ADVERSARIAL ROBUSTNESS ESTIMATION FROM THE LOSS SURFACE IN PARAMETER SPACE . In this section , we compare the effectiveness of the adversarial robustness estimation from the loss surface in both parameter and input space with an effective visualization method . 2.1 VISUALIZATION NEURAL NETWORK LOSS SURFACE BY PROJECTION . Given a neural network with a loss function F ( θ , x ) , where θ is neural network parameters ( weight and bias ) and x is the input . As the function inputs are usually in high-dimensional space , direct visualization analysis on the loss surface is impossible . Therefore , following the methods proposed by Goodfellow et al . ( 2015 ) and Li et al . ( 2017 ) , we project the high-dimensional loss surface into a low-dimensional space , e.g . 2D hyperplane to visualize it . In such methods , two projection vectors α and β are chosen and normalized as the base vectors for x and y axes . Then given an starting point o , the points around it are interpolated and corresponding loss values can be calculated : V ( i , j , α , β ) = F ( o+ i · α+ j · β ) ( 1 ) Here , the original point o in function F could be either in parameter space , which is mostly studied by prior work ( e.g . Li et al . ( 2017 ) ; Im et al . ( 2017 ) ) , or in input space , which is our major focus in this paper . The coordinate ( i , j ) denotes how far the original point moves along α and β direction . After calculating enough points ’ loss values , the function F with high-dimensional inputs could be projected to the chosen hyperplane formed by vector α and β . Fig . 1 has already shown two visualized loss surface examples in both parameter space and input space , which give an intuition of the dramatic difference between the two approaches . In the next section , we will further demonstrate that the loss surface geometry in input space is more essential regarding the neural network robustness properties . 2.2 THE LOSS SURFACE IN PARAMETER SURFACE VS . INPUT SPACE . To prove our statement , we examine the robustness of a pair of neural networks with the same ResNet model setting , but trained with natural process and Min-Max robust training scheme respectively . Both neural networks could achieve optimal accuracy ( ∼ 90 % ) on CIFAR10 dataset . However , their adversarial robustness degrees are significantly different : 0.0 % and 44.71 % accuracy under ` ∞ = 8 adversarial attacks . To analyze such a difference , the loss surfaces and corresponding contour maps are visualized in both parameter space ( as shown in Fig . 2 ) and input space ( as shown in Fig . 3 ) . Here the z-axis of 3D visualization denotes the loss values ( as well as the numbers on contour lines in 2D visualization ) , and the x and y axes are the corresponding projection vectors α and β . When illustrated in parameter space , both neural networks ’ loss surfaces on local minima are wide and flat , which align well with their high accuracy as stated in ( Li et al . ( 2017 ) ) . But comparing Fig . 2 ( a ) and ( b ) , even though the two neural networks have distinct degrees of robustness , there is no obvious difference between their loss surfaces in parameter space . More examples could be found in Appendix 8.5 , which also indicates the limitations of parameter space loss to describe robustness . However when illustrated in input space as shown in Fig . 3 , obvious differences emerge between the natural and robust neural networks : ( 1 ) Based on the 3D surface visualization , the natural neural network ’ s loss surface in input space has a deep and sharp bottom , while the local minima one the loss surface of the robust neural network is much flatter ; ( 2 ) Based on the contour map visualization , we show that the original inputs in the natural neural network ’ s surface locate in a very small valley , while the robust one shows a wide area . Thus , in the natural neural network ’ s case , once some small perturbations are injected into the inputs and move the inputs out of the small valley , the function loss will significantly increase and the prediction result could be easily flipped . By comparison of parameter space and input space loss surfaces , we demonstrate the potential advantages of using loss surfaces in input space than in parameter space . Therefore , we clarify that in terms of generalization and adversarial robustness , we should not only focus on the so-called “ wide valley ” of loss surfaces in parameter space , but also in input space . In the next section , we will visualize the adversarial attack trajectory and further demonstrate the close relation between adversarial vulnerability and decision surface geometry in input space . 3 REVEALING ADVERSARIAL ATTACKS ’ MECHANISM THROUGH DECISION SURFACE ANALYSIS . Previously , we showed the potential of adversarial robustness estimation in input space . In this section , we further explore the adversarial attacks ’ mechanism with input space decision surface . 3.1 EXTEND LOSS SURFACE TO DECISION SURFACE . Directly visualizing loss surfaces in input space has certain disadvantages : The major issue is that there is no explicit decision boundary for the correct or wrong prediction for a given input image ; Another deficiency of cross entropy based loss surface is that it can not well demonstrate the geometry , especially when the loss is relatively low 1 . This usually causes large blank regions with no useful information in visualization . To resolve these problems , we extend the concept of loss surface to decision surface , which enriches the geometry information and offers clear decision boundary . Here , we introduce the definition of neural network decision boundary and decision surface . For one input image x with label yt , the decision boundary of a neural network is defined as following : L ( x ) = Z ( x ) t − max { Z ( x ) i , i 6= t } = 0 . ( 2 ) Z ( x ) is the logit layer output before softmax layer , and t is the true class of input x . The decision function L ( x ) evaluates the confidence of prediction , i.e . how much correct logit output exceeds the max incorrect logit . In correct prediction cases , L ( x ) should always be positive and higher value is often better ( different from cross-entropy loss that lower is better ) . L ( x ) = 0 indicates the equal confidence in correct and wrong class , and thus is the decision boundary between correct and wrong prediction . The surface formed by function L ( x ) is called the decision surface to distinguish from cross entropy based loss surface , also because it contains the explicit decision boundary . Fig . 4 compares the cross entropy based loss surface visualization ( first row ) and the decision surface visualization ( second row ) . The decision surfaces can well resemble the loss surfaces in their informative areas but also include the confidence information in the blank area of the loss surfaces . Meanwhile , the explicit decision boundary , i.e . contour line L ( x ) = 0 , enables us to clearly see when network decision changes , which is very useful in analyzing adversarial examples as we will show next . In the following paper , we will use decision surface visualization as default settings .

Summary: This paper argues that analyzing loss surfaces in parameter space for the purposes of evaluating adversarial robustness and generalization is ineffective, while measuring input loss surfaces is more accurate. By converting loss surfaces to decision surfaces (which denote the difference between the max and second highest logit), the authors show that all adversarial attack methods appear similar wrt the decision surface. This result is then related to the statistics of the input Jacobian and Hessian, which are shown to differ across adversarially sensitive and robust models. Finally, a regularization method based on regularizing the input Jacobian is proposed and evaluated. All of these results are shown through experiments on MNIST and CIFAR-10.