Empirical Studies on the Properties of Linear Regions in Deep Neural Networks

1 INTRODUCTION . In the past few decades , deep neural networks ( DNNs ) have achieved remarkable success in various difficult tasks of machine learning ( Krizhevsky et al. , 2012 ; Graves et al. , 2013 ; Goodfellow et al. , 2014 ; He et al. , 2016 ; Silver et al. , 2017 ; Devlin et al. , 2019 ) . Albeit the great progress DNNs have made , there are still many problems which have not been thoroughly studied , such as the expressivity and optimization of DNNs . High expressivity is believed to be one of the most important reasons for the success of DNNs . It is well known that a standard deep feedforward network with piecewise linear activations can partition the input space into many linear regions , where different linear functions are fitted ( Pascanu et al. , 2014 ; Montufar et al. , 2014 ) . More specifically , the activation states are in one-to-one correspondence with the linear regions , i.e. , all points in the same linear region activate the same nodes of the DNN , and hence the hidden layers serve as a series of affine transformations of these points . As approximating a complex curvature usually requires many linear regions ( Poole et al. , 2016 ) , the expressivity of a DNN is highly relevant to the number of the linear regions . Studies have shown that the number of the linear regions increases more quickly with the depth of the DNN than with the width ( Montufar et al. , 2014 ; Poole et al. , 2016 ; Arora et al. , 2018 ) . Serra et al . ( 2018 ) detailed the trade-off between the depth and the width , which depends on the number of neurons and the size of the input . However , a deep network usually leads to difficulties in optimization , such as the vanishing/exploding gradient problem ( Bengio et al. , 1994 ; Hochreiter , 1998 ) and the shattered gradients problem ( Balduzzi et al. , 2017 ) . Batch normalization ( BN ) can alleviate these by repeatedly normalizing the outputs to zero-mean and unit standard deviation , so that the scale of the weights can no longer affect the gradients through the layers ( Ioffe & Szegedy , 2015 ) . Another difficulty is that the high complexity caused by the depth can easily result in overfitting . Srivastava et al . ( 2014 ) proposed dropout to reduce overfitting , which allows a DNN to randomly drop some nodes during training and work like an ensemble of several thin networks during testing . Despite the empirical benefits of these techniques , their effects on the trained model and the reasons behind their success are still unclear . Previous studies focused on explaining why these techniques ∗Dongrui Wu is the corresponding author . can help the optimization during training ( Wager et al. , 2013 ; Santurkar et al. , 2018 ; Bjorck et al. , 2018 ) . Different from theirs , our study is trying to answer the following question : What properties do these techniques introduce to the linear regions after training ? We present an intuitive view of different techniques ’ effects on the linear regions in Figure 1 . We may observe that : 1 . BN and dropout help the DNN partition the input space into many more linear regions than the vanilla DNN . 2 . The linear regions resulted from the BN model are more uniform in size than those from the dropout DNN . 3 . Many transition boundaries of the dropout DNN share similar norm directions , and concentrate around the decision boundaries . Figure 1 illustrates that different optimization techniques can lead to completely different linear regions even using the same DNN architecture , which may influence the behaviors of the DNN , such as its adversarial robustness ( Biggio et al. , 2013 ; Szegedy et al. , 2014 ) or some other undiscovered ones ( Appendix A shows another example on a two-dimensional toy dataset ) . Therefore , it is important to probe the properties of the linear regions introduced by these frequently-used optimization techniques , instead of just looking at the learning curves . This paper introduces mathematical tools of polyhedral computation into linear region analysis , and systematically compares various properties of the linear regions after DNNs being trained with different optimization techniques ( BN , dropout ) . Though the models have similar classification accuracies , we do observe significant differences among their linear regions : 1 . BN can help DNNs partition the input space into smaller linear regions , whereas dropout helps around the decision boundaries . 2 . BN can make the norm directions of the hyperplanes orthogonal , whereas dropout the opposite . 3 . Dropout makes regions in which data points lie less likely to contain the decision boundaries . 4 . The gradient information of a linear region has high relevance to its surrounding regions ; however , BN slightly reduces the relevance . It should be noted that our approach can also be applied to analyzing the preferences introduced by other optimization techniques , such as different initialization approaches ( Glorot & Bengio , 2010 ; He et al. , 2015 ) , different optimizers ( Duchi et al. , 2011 ; Kingma & Ba , 2014 ) , some widely-used operators like skip-connection ( He et al. , 2016 ) , and even different hyperparameters . Since training deep learning models is related to the complex interaction among the training dynamics , data manifold and model architectures , we believe it is better to decouple it into two subtasks : 1 ) figure out what kinds of linear regions can best approximate the data manifold ; and , 2 ) design optimization techniques to achieve such linear regions . Our work provides tools to analyze the linear regions , which helps study both subtasks . The remainder of the paper is organized as follows : Section 2 presents the details of the linear regions of DNNs with ReLU activation . Section 3 shows how BN and dropout influence the linear regions . Section 4 draws conclusions . 2 LINEAR REGIONS . This section describes the detailed approach1 for searching for the linear regions of a given input , and introduces some basic concepts of convex polytopes . 2.1 SEARCH FOR THE LINEAR REGIONS . A linear region is the intersection of a finite number of halfspaces . Therefore , to search for the linear region in which a given input point lies , we only need to search for its corresponding halfspaces , which are determined by the activation state of the DNN . Let ’ s consider a vanilla fully-connected DNN with L hidden layers and ReLU activation . Mathematically , let x ∈ Rd denote the d-dimensional vectorized input , hl ( x ) the pre-activation outputs of the l-th hidden layer with nl nodes ( l = 1 , 2 , ... , L ) , z ( x ) the logits of the output layer , and M the number of classes . Given an input example x∗ , because of the one-to-one correspondence relationship between a linear region and an activation state , its corresponding linear region is a set of inputs which lead to the same activation state of the DNN . Let Sl denote a set of the inputs whose activation states of the first l hidden layers are the same as those of x∗ . Obviously , Sl+1 ⊆ Sl . Consider the first layer . Since h1 : Rd → Rn1 is simply an affine transform of x , according to the activation state , S1 can be written as : S1 = { x | wTi x+ bi ≥ 0 , ∀i ∈ { 1 , ... , n1 } } , ( 1 ) where wi = sgn ( h1i ( x ∗ ) ) ∇xh1i ( x∗ ) , ( 2 ) bi = sgn ( h1i ( x ∗ ) ) [ h1i ( x ∗ ) − ( ∇xh1i ( x∗ ) ) Tx∗ ] . ( 3 ) Next , we select the points from S1 to construct S2 , which activate the same nodes of the second layer as x∗ does . Recall that the points in S1 lead to the same activation state of the first layer , hence h2 : S1 → Rn2 is also an affine transform of x . Therefore , S2 can be constructed by adding linear inequality constraints to S1 , in the same way as ( 1 ) . Due to the linearity of hl : Sl−1 → Rnl ( where S0 = Rd ) for all l , this procedure can be repeated till the last hidden layer . The detailed deduction is presented in Appendix B , and the depth-wise exclusion process to search for a linear region of x∗ is illustrated in Figure 2 . 1We developed the approach for finding the linear regions independently , but later found that the same approach has been proposed in ( Lee et al. , 2019 ) . However , the two papers use the linear regions for different purposes . Let C∗ = { ( wi , bi ) } ∑L l=1 nl i=1 be the set of parameters of all the linear inequality constraints . Then SL , which is the linear region that x∗ lies in , can be represented as the intersection of ∑L l=1 nl halfspaces : SL = { x | wTi x+ bi ≥ 0 , ∀ ( wi , bi ) ∈ C∗ } , ( 4 ) where the DNN is a locally linear model , as z : SL → RM is an affine transform of x . Though we mainly discuss the standard fully-connected DNNs , the form to describe a linear region can be extended to other situations with little modification . For instance , BN is just a reparameterization of the weights , a convolutional neural network ( CNN ) can be viewed as a sparse fully-connected DNN , and max-pooling is simply adding more linear inequalities to identify the maximum . 2.2 CONVEX POLYTOPES . A linear region can be represented as the set of solutions to a finite set of linear inequalities in ( 4 ) , which is exactly the H-representation of a convex polyhedron . With the natural bounds of the input value , these linear regions are convex polytopes2 . We omit the word convex in the rest of the paper to avoid repetition . A polytope can also be represented as a convex hull of a finite set of points , which is known as the V-representation . Different representations may lead to different complexities when dealing with the same problem ; nevertheless , it is challenging to convert one representation into the other ( Toth et al. , 2017 ) . As the dimensionality increases , it is even harder to calculate the number of vertices/facets with a single H-/V- representation ( Linial , 1986 ) , or to verify the equivalence between an H-representation and a V-representation ( Freund & Orlin , 1985 ) . More information about polyhedral computation can be found here3 . It should be noted that convex optimization is capable of probing the properties of the linear regions , because H-representation is a natural description of a convex feasible region , and a DNN ’ s behavior is completely linear in this feasible region . 3 PROPERTIES OF THE LINEAR REGIONS . This section presents various properties of the linear regions of DNNs , which were trained using different optimization settings . 3.1 MODELS . We used a fully-connected DNN as our vanilla model , which consisted of three hidden layers with 1,024 rectified linear nodes in each . The BN model was the vanilla model with BN added after the 2Here we use the term polytope to denote a bounded polyhedron . 3https : //inf.ethz.ch/personal/fukudak/polyfaq/polyfaq.html pre-activations of each hidden layer , and the dropout model added dropout after the activations . We trained our models on the vectorized MNIST dataset , which were rescaled to [ −1 , 1 ] . All models were trained using Adam optimizer ( Kingma & Ba , 2014 ) with the default setting ( β1 = 0.9 , β2 = 0.999 ) but different learning rates ( 1e-3 and 1e-4 ) , and the batch size was 256 . We used the Xavier uniform initializer for weight initialization , with biases set to zero ( Glorot & Bengio , 2010 ) . Early stopping was used to reduce overfitting . The classification accuracies are shown in Table 1 . Since it is unlikely to explore the influence of all optimization settings , this paper emphasizes two optimization techniques , i.e. , BN and dropout , and compares the influence of different learning rates . As shown in Section 2 , the number of constraints in ( 4 ) equals the number of hidden nodes ; hence , it is hard to handle the constraints as the scale of DNN increases , such as removing the redundant inequalities . In our experiments , the number of constraints was 1 , 024 × 3 = 3 , 072 . In addition to the natural bounds for the 28× 28 = 784 pixel values ( MINx ≤ x ≤ MAXx ) , each linear region is defined by 3 , 072 + 784× 2 = 4 , 640 linear inequalities . Our analysis of CNNs trained on the CIFAR-10 dataset is presented in Appendix E .

Summary: This paper addresses the following: how do batch normalization and dropout affect the number of linear regions present in a deep network? It does so by devising a search procedure for enumerating a set of linear inequalities that define the linear region around a particular input. The linear region is defined as the region of input space that activates the same units/nodes in the network. The authors compute these linear regions for three different types of fully connected networks trained with: vanilla (nothing added), batch norm, and dropout. Given these linear regions, the authors studied a number of their properties, such as the radii of inscribed spheres, angles between hyperplanes, and number of unique surrounding regions.