Predicting Deep Neural Network Generalization with Perturbation Response Curves

1 Introduction . Neural networks have produced state-of-the-art and human-like performance across a variety of tasks . This rapid progress has led to wider-spread adoption and deployment . Given their prevalence and increasing applications , it is important to estimate how well a trained net will generalize . Additionally , specific tasks often require models to be invariant to certain transformations or perturbations of the data . This can be achieved either through data augmentation that changes the underlying statistics of the training sets or through inductive architectural biases , such as translation invariance that is inherent in convolutional neural networks . It is important as well to understand how and when a network has been able to learn task-dependent invariances . Various attempts at bounding and predicting neural network generalization are well summarized and analyzed in the recent survey [ 2 ] . While both theoretical and empirical progress has been made , there remains a gap in the literature for an efficient and intuitive measure that can predict generalization given a trained network and its corresponding data post hoc . Aiming to fill this gap , the recent Predicting Generalization in Deep Learning ( PGDL ) NeurIPS 2020 competition [ 1 ] encouraged participants to provide complexity measures that would take into account network weights and training data to predict generalization gaps , i.e. , the difference between training and test set accuracy . In this work , we propose a new framework that presents progress towards this goal . Our methodology consists of first building an estimate of how the accuracy of a network changes as a function of 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . varying levels of perturbation present in training samples . To do so , we evaluate a trained network ’ s accuracy on a subset of the training dataset that has been perturbed to some degree . Using multiple observations of accuracy vs. perturbation magnitude , we develop a perturbation response ( PR ) curve for each model . From the PR curves , we derive two new measures called the Gi-score and the Pal-score , which compare a given network ’ s PR curve to that of an idealized network that is unaffected by all perturbation magnitudes . When applying our framework to inter and intra class Mixup [ 3 ] perturbations , we are able to achieve better generalization prediction scores on a majority of the tasks than the current state-of-the-art proposal from the PGDL competition . Because our framework can be applied to any parametric perturbation , we also demonstrate how it can be used to predict the degree to which a network has learned to be invariant to a given perturbation . 2 Related work . The PGDL competition resulted in several proposals of complexity measures that aim to predict neural network generalization gaps . While several submissions build off the work of [ 4 ] and rely on margin-based measures , we will focus on those submissions that measure perturbation response , specifically to Mixup , since this is most relevant to our work . Mixup , first introduced in [ 3 ] , is a novel training paradigm in which training occurs not just on the given training data , but also on linearly interpolated points . Manifold Mixup training extends this idea to interpolation of intermediate network representations [ 5 ] . Perhaps most closely related to our work is that of the winning submission , [ 6 ] . While [ 6 ] presents several proposed complexity measures , the authors explore accuracy on Mixup and Manifold Mixup training sets as potential predictors of generalization gap , and performance on mixed up data is one of the inputs into their winning submission . While this closely resembles our work , in that the authors are using performance on a perturbed dataset , namely a mixed up one , the key difference is that [ 6 ] only investigates a network ’ s response to a single magnitude of interpolation , 0.5 . Additionally , we investigate between-class interpolation as well , while [ 6 ] only interpolates between data points or intermediate representations within a class , not between classes . Our proposed Gi-score and Pal-score therefore provide a much more robust sense for how invariant a network is to this mixing up perturbation and can easily be applied to other perturbations as well . In the vein of exploring various transformations / perturbations , the second place submission [ 7 ] performs various augmentations , such as color saturation , applying Sobel filters , cropping and resizing , and others , to create a composite penalty score based on how a network performs on these perturbed data points . Our work , in addition to achieving better generalizatiton gap prediction scores , can be thought of as an extension of [ 7 ] , because as above , rather than looking at a single perturbation level , the Gi-score and Pal-score provide a summary of how a model reacts to a spectrum of parameterized perturbations . In this work , we also extend our proposed Gi and Pal-scores to predict generalization performance on different data transformations . [ 8 ] shows that even for architectures that have inductive biases that should render a network invariant to certain perturbations , in practice these networks are not actually invariant due to how they are trained . This is true when data augmentation is not carefully employed [ 8 ] , which highlights the need for our predicting invariance line of work . Unlike works , such as [ 9 ] , that measure how individual layer and neuron outputs respond to input perturbations , e.g. , rotation , translation , and color-jittering , we measure how a network ’ s overall accuracy responds to the these perturbations and compare that to an idealized network that is fully invariant . Consistent prediction on invariant data transformations is a desired property for neural networks [ 10 ] and it has been widely used as a data augmentation or regularization tool during training for improving generalization [ 11 ] . Our scores are inspired by the Gini coefficient and Palma ratio - most commonly used in economics as measures of income inequality [ 12–15 ] . In economics , the Gini coefficient measures income inequality by ordering a population by income and plotting the percentage of the total national income on the vertical axis vs. the percentage of the population total on the horizontal axis . For an idealized economy , wealth distribution lies on a 45 line from the origin , which means each percent of the population holds the same percentage of wealth . Plotting this distribution for an actual economy allows for the calculation of the Gini coefficient by taking a ratio of the area between the idealized and actual economy wealth distribution curves and the total area below the idealized economy ’ s curve . The Palma ratio is also calculated from this plot by taking a ratio of some top x % of the population ’ s wealth ( area below the actual economy ’ s curve ) divided by that of some bottom y % . In addition to income inequality , the Gini coefficient has been used in a wide variety of applications in different domains , e.g , measuring biodiversity in ecology [ 16 ] , quality of life in health [ 17 ] , and protein selectivity in chemistry [ 18 ] . It is also used in machine learning for measuring classification performance , as twice the area between the ROC curve ( curve of false positive rate vs. true positive rate for varying classifier threshold ) and the diagonal [ 19 , 20 ] and has also been used as a criteria for feature selection [ 21 ] . Our approach is the first to use this metric together with our PR curve framework for measuring generalization , as well as to use the Pal-score in a machine learning setting . 3 Methodology . 3.1 Notation . We begin by defining a network for a classification task as f : Rd ! k ; that is , a mapping of real input signals x of dimension d to discrete distributions , with k being the space of all k-simplices . We also define the intermediate layer mappings of a network as f ( ` ) : Rd ` 1 ! Rd ` , where ` refers to a layer ’ s depth with dimension d ` . The output of each layer is defined as x ( ` ) = f ( ` ) ( x ( ` 1 ) ) , with inputs defined as x ( 0 ) . Additionally , let f ` : Rd ` ! k be the function that maps intermediate representations x ( ` ) to the final output of probability distributions over classes . For a dataset D , consisting of pairs of inputs x 2 Rd and labels y 2 [ k ] , a network ’ s accuracy is defined as A = P x , y2D ( maxi2 [ k ] f ( x ) [ i ] = y ) / |D| , i.e . the fraction of samples where the predicted class matches the ground truth label , where ( · ) is an indicator function and f ( x ) [ i ] refers to the probability weight of the ith class . We define perturbations of the network ’ s representations as T↵ : Rd ` ! Rd ` , where ↵ controls the magnitude of the perturbation . For example , changing the intensity of an image by ↵ percent can be represented as T↵ ( x ( 0 ) ) = ↵x ( 0 ) . To measure a network ’ s response to a perturbation T↵ applied at the ` th layer output , we calculate the accuracy of the network for a sample of the training data on which the perturbation has been applied : A ( ` ) ↵ = X x , y⇠Dsample ( max i2 [ k ] f ` ( T↵ ( x ( ` ) ) ) [ i ] = y ) / |Dsample| . ( 1 ) The greater the gap A A ( ` ) ↵ , the less the network is resilient or invariant to the perturbation T↵ when applied to the ` th layer . Perturbations at deeper network layers can be viewed as perturbations in an implicit feature space learned by the network . 3.2 Calculating Perturbation Response curves . To measure a network ’ s robustness to a perturbation , one could simply choose a fixed ↵ and measure the network ’ s response . However , a more complete picture is provided by sampling the network ’ s response to various magnitudes of ↵ . In Figure 1 , we show this in practice . For example , letting T↵ refer to image rotation , we vary ↵ from a minimum ↵min degree of rotation , to a maximum ↵max degree . For each ↵ , we calculate accuracy A ( ` ) ↵ to measure the network ’ s response to the perturbation of magnitude ↵ applied at depth ` . Plotting A ( ` ) ↵ on the vertical axis and ↵ on the horizontal axis gives us the PR curves . In Figure 1 , we display rotation applied to training images ( ` = 0 ) from the SVHN dataset . This methodology is summarized in Algorithm 1 in Appendix A.3 . 3.3 Calculating the Gi-score and Pal-score . To extract a single statistic from the PR curves in Figure 1 , we draw inspiration from the Gini coefficient and Palma ratio . Namely , we compare a network ’ s response to varying magnitudes of perturbations with an idealized network : one whose accuracy is unaffected by the perturbations . The idealized network therefore has a PR curve that starts and remains at accuracy 1.0 . This comparison is achieved by creating a new graph that plots the cumulative density integral under the PR curves against the magnitudes ↵i 2 [ ↵min , ↵max ] : R ↵i 0 A↵d↵ . This produces what we call perturbation cumulative density ( PCD ) curves seen in Figure 1 . For the idealized network whose PR is identically equal to 1 for all ↵ , this PCD curve is just the 45 line passing through the origin . Finally , the Gi-score ( named for the Gini coefficient it draws inspiration from ) is calculated by taking a ratio of the area between the idealized network ’ s PCD curve and that of the actual network and the total area below the idealized network PCD . We summarize this in Algorithm 2 in Appendix A.4 . The Pal-score ( named for the Palma ratio it draws inspiration from ) calculates the area under the PCD curve and takes a ratio of the area for top 60 % of perturbation magnitudes divided by the area for the bottom 10 % . This allows us to focus on variations on the upper and lower ends of the perturbation magnitude spectrum , ignoring the middle perturbations that might not vary as widely across networks . We give the pseudocode for the Pal-score in Algorithm 4 in Appendix A.6 .

Summary: The authors propose a new framework to evaluate the generalizability of trained networks. The framework is based on evaluating the perturbation response curve on intra and inter-class sample mixup. It uses two novel statistical measures: the Gi-score and the Pal-score to predict the generalization gap. The framework is able to achieve a state-of-the-art predictive measure for several tasks in the PGDL competition and can be used to capture invariants to parametric input transformations.