Overfitting Detection of Deep Neural Networks without a Hold Out Set

1 INTRODUCTION . Deep neural networks have shown superior performance for a wide range of machine learning task such as speech recognition ( Graves & Jaitly ( 2014 ) ) , image classification ( Krizhevsky et al . ( 2012 ) ) , playing board games ( Silver et al . ( 2016 ) ) ; machine translation ( Kalchbrenner et al . ( 2016 ) ) ; beating previous methods by orders of magnitudes . To apply neural networks to safety critical problems such as autonomous driving it is necessary to evaluate their performance on new previously unseen data . One of the major problems of neural networks is their vulnerability to adversarial attacks . It has been shown that tiny unrecognizable changes of the input can fool the network to predict any class the attacker has chosen . One way to interpret this vulnerability is that the neural network overfits to the training data , with the output varying rapidly around each training point and thus slight changes of the input can lead to big changes in the output . It is thus highly desirable to prevent the network from overfitting during training . Previously reported methods reduce the chance of overfitting by evaluating the neural network on some holdout set , or by penalizing the complexity of the model class . This has the disadvantage that a holdout set can only be used once . By using design choices proven to be successful in the past the model becomes dependent on the holdout set . Penalizing the model class is only a heuristic remedy to overfitting . In the present paper we devise a method which prevents overfitting by relying on the training data only . We motivate l1-regularization of the kernel weights as a preferable choice to control the network complexity . Using no holdout set requires an alternative notion of overfitting . In the paper , we say that a model overfits if it is able to learn noise . 2 LITERATURE REVIEW . Heuristics . There exists several well known heuristics which reduce the chance of overfitting . Typically one reduces the hypothesis space , or one increases the data set . We can directly control the size of the hypothesis space by the number of parameters of the model . Typical choices are the width , the depth and the filter size of the network . Dropout introduced by Srivastava et al . ( 2014 ) at each training step , individual nodes and their incoming and outgoing edges are ignored ( dropped out ) with probability p. This reduces the dependency of the model on individual nodes . In early stopping ( Prechelt ( 1996 ) ) the model complexity is controlled by the number of iterations , in weight decay ( Krogh & Hertz ( 1991 ) ) an l2 penalty on the weights is added to the loss function . Data augmentation . Data augmentation is a commonly used technique to prevent overfitting . Here the size of the input data is artificially enlarged by applying a transformation to the input signal which is supposed to keep the output fixed . In practice this is done by cropping , adding input noise , applying affine transformations or small deformation to the input and so on . A more indirect data augmentation technique is input normalization see for example Jaderberg et al . ( 2015 ) . Detecting overfitting . There are several known approaches to detect overfitting . In holdout testing the data set is split in train and test sets . The models are learned on the training data only . After training the models performance is tested with the test data , thereby providing an empirical estimate of the true risk . If data is scarce cross validation techniques can be applied . Generalization theory . Generalization theory relates the minimization of the empirical risk to the minimization of the ( unknown ) expected risk . The difference between both risks is called the generalization gap , see Kawaguchi et al . ( 2017 ) . An important result by Bartlett ( 1998 ) proves that the l1-norm of the weight matrices is more important for generalization than the number of weights . Furthermore , in Bartlett et al . ( 2017 ) it was shown that the spectral complexity of the neural network bounds the generalization gap . This norm based bound is independent of the architecture parameters , except for log terms . Overfitting and neural networks . For neural network not all of these techniques behave as one would expect . For example it has been reported that deeper networks generalize better than shallower networks . It seems that the number of parameters of a network is of lesser importance to generalization as measured on the test set . Several large scale experiments supporting this claim are reported in Figure 1 of Novak et al . ( 2018 ) . Randomization experiments . Several papers used randomized labels to investigate generalization and memorization in neural networks . Most notably Zhang et al . ( 2017 ) showed that a neural network can memorize randomized training data well , concluding that a theory based on the complexity of the model space alone can not explain the generalization puzzle . Arpit et al . ( 2017 ) analyzed memorization in deep networks based on randomization of the training data . 3 METHODS . 3.1 COMPLEXITY AND l1-REGULARIZATION OF THE KERNEL WEIGHTS The complexity of a neural network is controlled by its hyper parameter and the hyper parameter of the training algorithm . We propose to control the model class by adding the l1 norm of the kernel weights ||Wj ||1 multiplied with a regularization factor λ to the loss function . Some additional background for this section is provided in the appendix . Notation . This paper considers feed forward networks f : Rd0 → Rk which maps the input space V0 = R d0 to some target space VL = Rk . This is followed by an argmax function which picks as output the coordinate of the largest value . The margin measures the gap between the output for the correct label and the other labels , γ = f ( x ) y −maxj 6=y f ( x ) j . A positive margin means a correct classification , whereas a negative margin means an incorrect classification . Each layer of the network consists of a linear blockAk : Vk−1 →Wk and a non linearity φk : Wk → Vk . The networks are thus written as : f ( x ) = φL ◦AL ( · · · ( φ1 ◦A1 ( x ) ) ) Here of course φ ◦ A ( x ) is just another way of writing φ ( A ( x ) ) . The concatenation ηk = φk ◦ Ak will be called a block of the network . In this paper only the standard relu nonlinearity φ : R→ R+ , which is defined by φ ( x ) = x ∨ 0 = max { x , 0 } = x+ , is considered . The input data is denoted by x0 ∈ Rd0 . Further , the output of layer k of the network is denoted by yk = fk ( x ) : = φk ◦ Ak ( · · · ( φ1 ◦ A1 ( x ) ) ) ∈ Rdk and the network which outputs the k-th layer by fk : Rd0 → Rdk . Finally , the width of the network is defined by d = max { d0 , . . . , dL } . In the paper we will call the data yk , passing through the layers , signal . Finally , we arrange all data points and signals in matrices , denoted by X ∈ Rd0×n and Y k ∈ Rdk×n . So we write by slightly abusing notation , Y k = fk ( X ) = fk ( x1 , . . . , xn ) = [ fk ( x1 ) , . . . , fk ( xn ) ] . l1-regularization A typical convolutional kernel W is determined by the filter size , and the number of incoming and outgoing features . If the convolution is written as matrix operation y = Ax and zero padding is being assumed , then the matrix A can be arranged as a vertically stacked block matrix each subblock Ai representing one outgoing feature . The entries of each these blocks are determined by the weights of the i-th filter . Due to zero padding and weight sharing , each weight occurs precisely once in each row and each column of Ai . It follows that the filter matrix A contains in each column each weight precisely once . Lemma 3.1.1 . The spectral norm of a convolution matrix A is bounded by the l1-norm of its kernel weights . ||A||σ ≤ ||W ||1 Proof . The inequality follows as the spectral norm can be bounded by the row and columns norms of A , which can be estimated by the weight matrix W . ||A||σ ≤ √ ||A||1→1||A||∞→∞ = √√√√ max j∈ { 1 , ... , n } m∑ i=1 |aij | max i∈ { 1 , ... , m } n∑ i=1 |aij | ≤ ||W ||1 ( 1 ) Recently shown generalization bounds are dominated by RA||X||2nγ ( Bartlett et al . ( 2017 ) ) see also Theorem A.1.1 . Here γ > 0 is the margin , X denotes the training data arranged in a matrix , and RA is the spectral complexity . We will use a simplified version of RA defined by RA : = L∏ j=1 ||Aj ||σ ( L∑ i=1 ||ATi || 2/3 1 ||Ai||2/3σ ) 3/2 . ( 2 ) Lemma 3.1.2 . The spectral complexity can be bounded by the l1-norm of the kernel weights Wj . RA ≤ dL3/2 L∏ i ||Wi||1 Proof . RA = L∑ i=1 L∏ j 6=i ||Aj ||2/3σ ||Ai|| 2/3 1 3/2 ≤ L3/2 L L∑ i=1 L∏ j 6=i ||Aj ||σ||Ai||1 ≤ d √ L L∑ i=1 L∏ i ||Wi||1 ( 3 ) ≤ dL3/2 L∏ i ||Wi||1 ( 4 ) Here the first inequality holds because of inequalities between the generalized 2/3-mean and 1- mean . The second inequality follows as the spectral norm can be bounded by ||W ||1 by Lemma 3.1.1 . Both lemmas show that it is beneficial to l1-norm of the kernel weights . In fact Lemma 3.1 shows by looking at RA||X||2nγ that the margin scales as ||W ||1 . Since we do not use a bias in our model , we may rescale each kernel matrix by ||W ||−11 . This is compensated by decreasing the margin accordingly . So in order to achieve better generalization bounds we can penalize the kernel weights by λ||W ||1 . Here λ > 0 is a regularization parameter to be determined . 3.2 ACCURACY CURVES . Assumptions . To simplify the analysis we make three assumptions . First , we assume that the data is independent and identically distributed . This implies that the with an increasing level of randomness the complexity of the data also increases . In dependent data , this is not necessarily the case . As in that case correlation in the data can be destroyed by the introduction of randomness making the data faster to learn . Second , we assume that the model complexity is controlled by a regularization parameter in such a way that an increase of the regularization parameter implies a strict decrease of the complexity of the model . Third , we assume that the regularization parameter and the randomness are on a similar scale . To explain this , note that the accuracy is a function of the regularization parameter and randomness . The assumption simply means that the norm of the gradient of the accuracy taking in direction of the regularization is of the same order as the norm of the gradient taken in direction of randomness . Creating randomized data . In this paper we consider a classification problem . Each data point consists of a pair ( x , y ) which is typically an image and a corresponding class . For a fixed level of randomness p ∈ [ 0 , 1 ] we split the training data Z = ( X , Y ) in two parts . The first split Zp contains a random sample of p-percent of each class of the training data . The second part Z1−p contains the rest of the training data . The classes of the first set are randomly permuted to give the set Z̃p . The randomized data is obtained by joining Dp = Z̃p ∪ Zp−1 . With this D0 is equal to the original data set Z and D1 is obtained by randomly permuting all class labels Y of the data set Z . Accuracy curves . Central to our paper are accuracy plots . Here we plot the accuracy as computed on the training data over the randomness of the training data . Alternatively the plot of the accuracy as computed on the training data over the regularization parameter . In the paper we call such curves in the plots accuracy over randomness curves and accuracy over regularization curves . To generate the plots we keep everything fixed except the randomness of the training data or the regularization parameter . Monotony . Let us assume that we successfully trained our model on the unperturbed data set D0 = Z . This means that the accuracy over randomness curve starts in the left upper corner at some point close to 1 . As we increase the level of randomness the training data Dp becomes more complex and it is thus more difficult to learn , which means our algorithm will take longer to achieve the same training error . Thus , we expect that the accuracy drops . In other words the accuracy curve is strictly monotonically decreasing for increasing randomness . Further , if we increase the regularization parameter the model complexity drops . Thus we also expect that accuracy drops if the regularization of the model is increased . This shows that our assumption imply that the accuracy is strictly monotonically decreasing as a function of randomness and regularization . Figure 1 shows the qualitative behavior of accuracy over randomness curves which follows the assumption we made . Real accuracy curves To compare these idealized curves with accuracy curves of real data we computed the accuracy curves for different data sets , see Figure 2 . In each subfigure we trained a neural network on either mnist ( a ) , cifar10 ( b ) , and patched-noise ( c ) - a generated data set . For each curve we varied the l1 regularization parameter . Furthermore , for each randomness value on the x-axis , the network was trained for five different randomization of the labels . More details on the networks , the data sets and more plots can be found in the appendix .

Summary: This paper is about detecting overfitting of deep neural networks without using a validation set. This is an interesting research problem. However, it is not clear how this paper contributes to solve the problem. My understanding is that this is a preliminary work, put into a paper in haste. There are more research efforts required to turn it into a good paper, theoretically as well empirically.