On the Reflection of Sensitivity in the Generalization Error
Authors: Mahsa Forouzesh
Farnood Salehi
Patrick Thiran
Venue:  (2020)
Keywords: Generalization Error
Sensitivity Analysis
Deep Neural Networks
Bias-variance Decomposition

Abstract: Even though recent works have brought some insight into the performance improvement of techniques used in state-of-the-art deep-learning models, more work is needed to understand the generalization properties of over-parameterized deep neural networks. We shed light on this matter by linking the loss function to the output’s sensitivity to its input. We find a rather strong empirical relation between the output sensitivity and the variance in the bias-variance decomposition of the loss function, which hints on using sensitivity as a metric for comparing generalization performance of networks, without requiring labeled data. We find that sensitivity is decreased by applying popular methods which improve the generalization performance of the model, such as (1) using a deep network rather than a wide one, (2) adding convolutional layers to baseline classifiers instead of adding fully connected layers, (3) using batch normalization, dropout and max-pooling, and (4) applying parameter initialization techniques.

Under review as a conference paper at ICLR 2020
ON THE REFLECTION OF SENSITIVITY IN THE GENER- ALIZATION ERROR
Anonymous authors Paper under double-blind review
ABSTRACT
Even though recent works have brought some insight into the performance im- provement of techniques used in state-of-the-an deep-learning models, more work is needed to understand the generalization properties of parameterrzee deep neural networks. We shed light on this matter by linking the loss function to the output's sensitivity its input. We find a rather strong empirical relation between the output sensitivity and the variance in the bias-variance decomposition of the loss function, which hints using sensitivity alization performance of networks, without requiring labeled data. We find that sensitivity is decreased by applying popular methods which improve the general- ization performance of the model, such as (1) using deep network rather than a wide one, (2) adding convolutional layers baseline classifiers instead of adding fully connected layers, (3) using batch normalization, dropout and max mao-lio,ing, and (4) applying parameter initialization techniques.
metric for comparing gener-
INTRODUCTION
In muchine-learning tasks, the main challenge network designer faces is to find a model that learns the training data and that is able to predict the output of unseen data with high accuracy. The first part easily achievable in current peramearaeteeiiee deep neural networks. Yet the second part, referred to as generalization, demands careful expert hand-tuning (LeCun et al. (2015), Goodfellow al. (2016)). Modern convolutional neural network (CNN) architectures that achieve state-of-the-an results in computer-vision tasks, such as ResNet (He et al., 2016) and VGG (Simonyan & Zisserman 2014), attain high- geggeeeelllliiiinnn performance. Part of their success due recent advances in hardware and the availability of large amounts of data. However, among models applying these resources, some outperform others. Hence, the key questions about when and why some models generalize, still remain mystery (Neyshabur et al. (017).
In this paper, by investigating the link between sensitivity and generalization, we get one step closer to understanding the generalization properties of neural networks. Our findings suggest a relation between the sensitivity metric and the variance term the bias-variance decomposition of the test loss (Geman et al. (1992), Tibshirani (199)). Neal et al. (2018)). This relation provides us with some intuition about the link, when the bias is small, between sensitivity and loss. We present strong empirical results the link between the sensitivity and the test loss in well-established deep neural networks with ReL.U Naarr & Hinton 2010) non-linearity.
Building this relation, can the sensitivity metric examine which network is more prone to overfitting, and we rediscover the generalization properties of certain structures. We provide numerical evidence that significant decrease the network output's sensitivity input perturbations can be a good predictor of the generalization capability of certain techniques used in state-of-the-art models, such batch normalization (Totfe & Szegedy 2015) and He initialization (He et al. 2015). Sensitivity also supports the commonly ly accepted observation that depth is better than width in deep neural networks (Bengio Delalleau (20TT), Montufar al. (2014). Telgarsky (2016). Mhaskar et al.) (2017)). Furthermore, in certain settings, the sensitivity of untrained networks can be used as proxy the generalization error of trained networks.

---

Under review as a conference paper at ICLR 2020
2 RELATED WORK
Recently, Sokolić et al. (017)) suggested bounding the generalization error of deep neural networks with the spectral norm of the input-output Jacobian matrix, measure of output sensitivity to its inputs. Empirical support for their conclusions is provided in Novak et al. (2018) through experiments on fully connected neural networks. Further research is needed to determine the conditions that are required to compare the generalization performance of models by using sensitivity metrics, For this purpose, we provide thcoretical and empirical insights to the link between sensitivity and generalization by studying the relation between sensitivity and the variance term in the vias-variance decomposition of the loss function. To the best of our knowledge, this work is the first find such a relation. In our work, we not only elaborate on the relation between sensitivity and loss for wide range networks (not limited to fully connected) but also empirically show that many state-of-the-art methods improve performance alongside reducing the sensitivity of the network.
To avoid overfitting in deep-learning architectures, regularization techniques are applied: for example, weight decay, early stopping, dropout (Srivastava et al. 2014), and batch normalization (BN) (Ioffe & Szegedy 2015). loffe & Szegedy (2013) argues that the reason for the success of BN is that addresses the iiternal-oovaiintt this belief and explains that the success of BN is due to its ability to make the optimization landscape smoother. Here, we look at the success of dropout and BN from another perspective. These methods decrease the output sensitivity to random input perturbations in a same manner as decreasing the test loss, resulting in better generalization performance.
phenomenon. However, Sannturar et al. (018)) argues against
Designing neural network architectures is one of the main challenges in machine-learning tasks. One major line of work in this regard comparing deep and shallow networks (Bengio & Delalleau (201) Mhaskar et al. (2117). Wu et al. (2019), Ba & Caruana (0014). Montular et al. (2014), and [Simonyan & Zisserman 2014). ). is shown in Telgarsky (2016) that to approximate a deep network, a shallow network requires an exponentially larger number of units per layer. After finding satisfactory architecture, the trainability of the network needs to be carefully assessed. order to avoid exploding or vanishing gradients, Glorot & Bengio (2010) and He et al. (2015) introduce parameter initialization techniques that are widely used current frameworks. By linking sensitivity and generalization, we present new viewpoint understanding the success of current sute-of-the-art architectures and techniques.
Previous theoretical studies attempting to solve the mystery of generalization mostly include gener- alization error (GE) bounds that use complexity measures such as VC-dimension and Rademacher complexities (Xu & Mannor (2122) KKwaguuhh et al. (017). Arora et al. (2018), and (Sokoić et al.] (2077). GE depends not only on the architecture of the network, but also on the structure of the data itself] This is why the same network has lower GE when trained with structured data than with random data (Zhang et al. 2016)). Here, we do not study GE bounds; however, in order to find complexity measures to bound GE, we should take note of the link between sensitivity and generalization.
There has been research on sensitivity analysis in neural networks with sigmoid and tanh activation functions (Dimopoulos et al. (1995), & Chen (1993), Znd & Yeung (2001)). Yang et al. (2013) introduce a sensitivity-basee ensemble approach which selects individual networks with diverse sensitivity values from pool of trained networks. |Pihe (1995) performs sensitivity analysis in neural networks to determine the required precision of the weight updates in each iteration. this work, we extend these results to networks with ReLU non- non-lincarity with different goal; to study the relation between sensitivity and generalization error in the state-of-the-at deep neural networks.
(200g)
The metrics used for evaluating the network performance on training data cannot reflect generalization, because an over-paramenn parameterrie model can achieve zero training loss for networks trained with randomly labeled data (Zhang et al. (0116). There have been recent attempts to predict the test loss for snpervised-learningg tasks (Novak et al. (2018), Jiang et al. (2018), and Wang et al. (2018)) |Phllipp) & Carbonell (2018) introduces the non-linearity coefficient (NLC) measure of the neural network, which is empirically shown to be a predictor of the test error for fully connected neural networks. According our results on both fully connected and convolutional neural networks, sensitivity is also a predictor of the test error, even before the networks are trained, which suggests sensitivity as a computationally inexpensive architecture-selccte metric.
gradient- based complexity
In this work, we do not investigate the structure of the data set.
2

---

Under review as a conference paper at ICLR 2020
Paper Outline. We formally define loss and sensitivity metrics in Section 3.1 and Section 3.2
respectively. In Section 4 we state the main findings of the paper and present the numerical and analytical results supporting that. Later in Section []] we propose possible proxy for generalization properties of certain structures and certain methods. Finally in Section 66, we further discuss the observations followed up by conclusion in Section 7 The empirical results presented in the paper are for an image-llassification task on the CIFAR-IO dataset. In Appendix [8.6] we present the empirical results for MNIST and CIFAR-I00 datasets. In Appendix 8.7 results for a regression task with the Boston housing dataset.
we present the empirical
PRELIMINARIES
Consider a superviee-learninng task, where the model predicts ground-truth output y E y = RK for an input x É X' == DR The predictor Fo X - y is a deep neural network parameterized by the parameter vector l that learned on the training dataset 7, by using the stochastic learning algorithm A. The training dataset 7 and the testing dataset т, consist of i.i.d. samples drawn from the data distribution p. With some abuse of notation, we use when the samples are uniformly drawn both from set of samples and from probability distribution.
3.1 Loss
Our main focus is classification task where the loss function is the cross-entropy criterion. The average test loss can be defined as
y log F (x:) 1]
L = Eg- [To.]] = Eo- Blaws ~uo
where 8* is the random parameter vector found by A minimizing the train loss defined on Tr, K is the number of classes and F/.. is the k-th entry of the vector Fø-, which is the output of the softmax layer, i.e., Fg- softmax (z)). where fo- (x) is the output of the last layer of the network. In classification tasks, the output space is y = [0, 1] and the output vector is the probability assigned to each class.
3.2 SENSITIVITY
Let us inject external noise to the input of the network and compute the noise in the output. If the original input vector is x E X' and we add an i.i.d. normal noise vector हn ~ (0, o² I) to the input, then the output noise due the input noise e,r X is E, Jo(x + Ex) fa(x) E y. We the variance of the output noiseF] averaged over its K entries, measure of sensitivity: So Var(ey). The average sensitivity is
1 K
S = S=E,(So) Es [Var(ey)] Eg Vara.e
where ety is the k-th entry of the vector Ey. In the following sections, we use Spefore = Eg [Sp] and Saner Eg- [So-] when the expectation is over the network parameters before and after training, respectively. We consider unspecific sensitivity (the average over the entries of the output noise), which requires unlabeled data samples, as opposed to specific sensitivity (the sensitivity of the output of the desired class) (Tartaglione et al. 2018). By expanding Equation ( up to the first order, the sensitivity S' is approximated by the product of the variance of the input noise vector o² and the square of the Frobenius norm of the Jacobian of the output of the network (Novak et al., 2018). In experiments, prefer S to the Jacobian, because in order to compute S it is enough to look the network as a black box that given an input, generates an output, without requiring further knowledge of the model.
"hhe randomness is introduced by the stochastic optimization algorithm A and the randomized parameter initialization technique.
"Note that consider the output of the neural network and not the output of the softmax layer. The softmax layer can be interpreted normalizing layer pushing the values to he hetween and 1.
3

---

Under review as a conference paper at ICLR 2020
4 SENSITIVITY vs. Loss
4.1 MAIN OBSERVATION
After training, an ideal predictor is expected to have a good generalization ability. An ideal predictor should also be robust; given similar inputs, the outputs should be close to each other. Assuming that the unseen data is drawn from the same distribution as the training data, there should be a link between the two concepts of robustness and generalization. Robustness here is the average-case robustness, not the worst-case robustness (adversarial robustness). We measure the robustness by computing S (Equation 2 and considering near-zero train loss, we refer to the test loss L (Equation (I) as the generalization error. According to our observations on wide set of experiments, we find a rather strong relation between the sensitivity and the generalization error L. State-of-the-an networks decrease the generalization error alongside with the sensitivity of the output of the network with respect to the input (Figure
20
0.9707
Aleuet
layer CNN standard normal
Lo
VGG16
10
VOG13 standard normal
ResNet18
ResNet18 standard nomnal
Equation
0
10
20
30
40
log( Sater)
Figure 1: Sensitivity Sater versus test loss L for popular CNN architectures. The parameter initialization is Xavier (Glorot Bengio (010) with uniform distribution unless stated as standard normal distribution. The networks are trained and fully converged on a subset the CIFAR 10 training dataset and are evaluated on the entire CIFAR 10 testing dataset. Each point indicates a network with different numbers of channels and hidden units, and is averaged over multiple runs. For more details on configurations refer to Appendix 8.1 The Pearson correlation coefficient, between data points indicated in the figure.
4.2 NUMERICAL EXPERIMENTS
Many factors influence the generalization performance of deep-learning models, among which network topology, initialization technique, and regularization method. We study the influence of these three factors on the sensitivity S and keep all the other factors, including the learning algorithm, the same. We refer to fully connected neural networks as FC and convolutional neural networks as CNN. First, we examine the relation between Sater and T. for different architectures, different initialization techniques, and different regularizations. We observe a strong link between Sater and L; lower sensitivity has lower test loss (Figures 2 and 3) This strong link suggests that, by comparing the sensitivity Safeer we can simply compare generalization performances of multiple neural networks that fully trained (with various hyper- apper-parameters. types of layer and regularization methods). Using sensitivity as a proxy for test loss is particularly advantageous in settings where labeled training data limited; assessing generalization performance can be then done without having sacrifice training data for the validation set. Next, when initialization is fixed to the standard normal distribution and no explicit regularization technique is applied, we investigate the link between Sbefore and L (Figure in Section I. We see again a clear relation between Siefore and L; Siefore can potentially be used as an seshiteectur-sslleette metric before training the models.
4.3 BIAS- VARIANCE DECOMPOSITION
In this section, a crude approximate relation between sensitivity and generalization error established through the link between sensitivity and the variance term in the bias-variance decomposition of the mean square error. First, we find the link between the cross-entropp loss and the mean square error. Then, present the relation between sensitivity and the variance term in the decomposition of the mean square error. Finally, the link between sensitivity and generalization emror I is established.
4

---

Under review as a conference paper at ICLR 2020
4 Eigerit ApperiCaddpppy
Saye EC layer
liyer
13
log
layer PC
logs L.
$
10
Equalin
0.9625
0.9127
0.9658
10
10 lagee
10
21
(a)
log (Sate))
logs Jogg.
layer CNN 9 loger CNN--IN
laaer PC
de Wharye
Mayerr PC2 layer loger)
10
layer CNNeena-pnsinng
laaer CNN-sdropot
byer CNSS CNNV lalayers Lane
log. (.)
Equation
X
log (L)
0.9460
20
10
(d)
log( (tee
log/ Sance)
Figure 2: Test loss versus sensitivity for networks trained on a subset of the CIFAR-10 10 training dataset where the network parameters are initially drawn from standard normal distribution. Each point indicates network with different numbers of channels and hidden units and is the average over multiple runs. The interval indicates the minimum maximum values sensitivity over multiple runs. (a) Fully connected neural networks. (b) 4-layer FC trained with or without regularization. (c) Convolutional neural networks. (d) 3-layer CNN trained with or without regularization. (e) Comparison between adding convolutional layer and adding fully connected layer baseline classifier that is a fully connected neural network with one hidden layer.
When the predictor Fg. (x) assigns the probability Fý. (z) to the correct class e and 1 Fý- (2) to another class (see Appendix 8.3 for details), the cross-entropy loss L can be approximated as
L (2
F (F) (x) yh)²
2
We roughly approximate the eight-hand side in Equation with v 1 where LMSE is the mean square error (MSE) criteria and is defined as
L
Eg- [Lor usa.] = Eg E (a.y)~
|1 (x) v1
(4)
Consider the classic notion bias- variance decomposition for the MSE loss (Geman et al. (1992)), where the generalization еттor is the sum of three terms: bias, variance and noise, i.e., LMSE
Ebius + variance + Epoise. this work, we consider the labels to be noiseless and neglect the third term Epoise- The bias term is formally defined
|IB8- [Fa- (x)] - v/2]
bias
Ez.v
and the variance term is formulated as
K Ex
E Evariance
Varg (FØ (x))
(5)
A=
Let us now draw an again crude approximate relation between E and undder strong assumptions on the probability distributions of 0, x, and Ex (refer to Appendix 8.2 for more details). Given feed-forward neural network with M hidden layers and H,, < S M, units per layer, where the
non-linear activation function is positive homogeneous with parameters a and (Equation (10) in Appendix 8.2). we have
Evariance
K K
S
+ x
(6)
*ReL.U is positive homogencous function with and 8 0.
5

---

Under review as a conference paper at ICLR 2020
where
M
M a² 32
II i=++1
o H,
and K is the number of units in the output of the softmax layer and بw.. of oz and o² the second moment of weights and biases of layer I, input x and imput noise Ex, respectively. Equation (7) can be extended to convolutional neural networks by replacing H, with fan,n of weights of layer
Given an infinite amount of training data, the bias represents the best performance of our model which can be approximated by the training loss (Mehta et al. (2019), Ng Fortmann- Roe (2012) ). In our experiments, the training loss is close to zero, hence if we neglect the bias term Ebis in the test loss we have,
(
K
2
+x).
L
S
where X is given by Equation 77) In experiments, we observe that o is usually very small or zero (for instance in ResNets 6; 0), making X = 0.
According to Equation (6) and the relation between LMSE and L, to compare networks with the small value of Ebas (which is usually the case in deep neural networks where the bias is approximated with the near-zero training loss), the test loss can be approximated by the sensitivity following Equation (8) Despite the strong assumptions and crude approximations numerical experiments show a rather surprisingly good match with Equation (8) (Figures [ and
get Equation 88) the
when X is neglected in Equation (8). It is interesting note that the right-hand side of Equation (8) computed without requiring labeled data points.
If Sins can no longer be approximated by the training loss, which may in part explain the poorer match in lower values of Figure[] we need more training data to make this approximation valid. In our experiments. we train the networks with only subset of the training dataset, we also show in Appendix 8.8 that by training with more data samples, numerical results become closer to Equation (8)
5 SENSITIVITY AS A PROXY FOR GENERALIZATE
In this section, we argue that methods improving the generalization performance of neural networks remarkably reduce the sensitivity S. In particular, we revisit some common notions regarding generalization in deep neural networks: adding convolutional layers instead of fully connected layers, using deeper networks instead of wider ones, adding dropout, batch normalization (BN) and max-pooling layers, and initializing the network parameters by techniques such as Xavier and He.
5.1 COMPARING DIFFERENT ARCHITECTURES
Convolutional vs Fully Connected Layers. The relation between the sensitivity and the general- ization error T. supports the common view that CNNs outperform FCs in image-classification tasks. We empirically observe that, given a CNN and an FC with the same number of parameters, the CNN has lower sensitivity and test loss than the FC. Moreover, some CNNs with more parameters than FCs have both lower sensitivity and lower test loss, even though they are more over-parameterized Let us start from baseline classifier with one hidden layer (2 layers in total), and we compare the effect of adding another fully connected layer versus adding a convolutional layer in Figure ( We vary the number of parameters of 1-layer CNNs (which consist of layers and conv layer) from 450k to 10M by increasing the number of channels and hidden units, whereas the number of parameters for 3-layer FCs varies from 320k 1.7M. Despite the large number of parameters of CNNs, they suffer from less overfitting and have lower sensitivity S than FCs. Next, let compare a FC to a CNN with the same number of parameters in Figure [(:): a 3-layer FC with 140 units in each layer (yellow mark) and a I-layer CNN with 5 channels and 100 units (green mark), both have 450k parameters. The CNN has remarkably lower sensitivity and test loss than the FC, which indicates better performance compared to the FC with the same number of parameters.
Depth vs Width. Consider a feedforward FC with ReLU activation function where all the network parameters follow the standard normal distribution and are independent from each other and from the

---

Under review as a conference paper at ICLR 2020
imput. If we have M layers with H units in each hidden layer, K units in the output layer and the input layer with D units, then (see Appendix 8.4 for details)
D K
H
S
(9)
According to Equation ( considering two neural networks with the same value for HM, one being deep and narrow (higher M and lower H), and the other being shallow and wide (lower M and higher H), subject to the same noise level of the deeper network has lower sensitivity S. Assuming both networks have near-zero training losses, then, depth is better than width regarding generalization in fully connected neural networks. The empirical results in Figure 2(a) support Equation (9). For instance, 4-layer FC with 500 units per layer (the last point among 4-layer FCs), has the same value for (2) as a 5-layer FC with 165 units per layer (the 4th point among 5-layer FCs). In Figure these two networks have the same value for both sensitivity and loss, and all narrower 5-layer networks (with 100, 120, and 140 units) have better performance than the wide 4-layer network (with 500 units). Note that Equation ( only holds for networks with parameters drawn from the standard normal distribution and does not hold when the parameter initialization is Xavier or He.
5.2 REGULARIZATION AND INITIALIZATION METHODS
Figures 2 b) and d) show the sensitivity Sater versus the test loss L, for different regularization methods. In particular, we study the effect dropout and BN on the sensitivity in the FCs; and apply dropout, BN and max-pooling for the CNNs. The results are consistent with the relation between sensitivity S and loss L. For all mentioned regularization techniques, we observe a shift of the points towards the bottom left; this shift suggests that these techniques improve generalization by decreasing the network sensitivity to input perturbations. This is especially noticeable in the BN case, where both the sensitivity and test loss decrease dramatically. This suggests that batch normalization improves performance by making the network less sensitive to input perturbations.
Another interesting observation is the effect of various parameter initialization techniques on the sensitivity and loss values, after the networks are trained (Figure 3). We consider four initialization techniques for network parameters in our experiments: (i) Standard Normal distribution (SN), (ii) Xavier (Glorot & Bengio 2010) initialization method with uniform distribution (XU), (iii) He (He) al. 2015) initialization method with uniform distribution (HU), and (iv) He initialization method with normal distribution (HN). As shown in Figure 3 the relation between sensitivity Saner and test loss provides us a new view-point of the success of the state-of-the-ee initialization techniques; the He initialization method with normal distribution has the best generalization performance, alongside the lowest sensitivity value.
layer FC SN
0.9622
6-7 layer PC XU
2.8
0.9284
(b)(L)
6-7 byyer PC HI!
log(L)
3.6
100
6.7 layer PC HN
Equition
3.4
2:2
log(S
Figure 3: Test loss versus sensitivity for networks trained on subset samples of the CIFAR-10 dataset where networks are initialized with different methods. On the right, we have zoom in plot of the left figure.
6 DISCUSSION
6.1 SENSITIVITY OF UNTRAINED NETWORKS AS A PROXY FOR THE TEST LOSS
similar trend, as in Section is observed for neural networks that are not yet trained. In Figure 4 the sensitivity Stefore is measured before the networks trained, and the test loss I. is measured after
7

---

Under review as a conference paper at ICLR 2020
the networks are trained subset of the training dataset. The parameters in the fully connected and convolutional networks are initialized by sampling from the standard normal distribution, and no explicit regularization (dropout, BN, and max-pooling) is used in the training process. These two conditions are necessary, because regularization techniques only affect the training process, hence Shefore is the same for networks with or without regularization layers, and the He and Xavier initialization techniques force the sensitivity to be fixed regardless of the number of units in hidden layers. Hence, under these two conditions, the generalization performance of untrained networks with different architectures can be compared. The strong link between the sensitivity of untrained networks Sectore and the test loss L suggests that generalization of neural networks can be compared before the networks are even trained, making sensitivity computationally inexpensive architecture- selection method.
Figure 4: Test loss of trained models versus sensitivity of untrained models for networks whose parameters tially drawn from the standard normal distribution. Note that the regulariza- tion techniques BN, dropout and max- pooling are removed from Alexnet, VGGI1, and VGG13 configurations.
layer FC
40
1-3 layer CNN Alexnet
0.9737
log (L)
ini-
VGGH
20
VGGI3
Equation
20
40
60
80
6.2 FINAL REMARKS
is important to note that sensitivity is not the only factor affecting generalization. The sensitivity metric does not address the challenges brought by Zhang et al. (2016); the sensitivity of random- labeled input is the same as the sensitivity of a true-labeled input, with very different loss values. The pixel-wise linear input perturbations, considered in our experiments, might not be realistic; ideally, we would like to perturb the input in the latent space of the generative model of the input image. Also, the relation between S and L, requires the non-linearity be positive homogeneous, hence the generalization properties of networks with sigmoid and tanh activation functions remain unexplained. not possible to find single metric to explain the generalization properties across a space as large as deep neural networks and across the non-convex landscapes of deep-learning problems. But the sensitivity metric presented this work is strongly related to generalization, and even though Equation was derived under very strong assumptions and crude approximations, it is surprisingly quite close to the empirical results in a wide set of settings.
The sensitivity metric can be affected by re-scaling. If we re-scale all the indices of the very last layer of the neural network, the accuracy stays the same as the network predicts the same class, however, both the test loss and sensitivity are multiplied by the square of the scaling factor. Therefore, the relation highlighted in this paper between sensitivity and generalization error cannot always be extended relation between sensitivity and accuracy. Also note that, if we re-scale the input æ, define a new input Knew = az, and de-scale the output fnew = f(-)/a, then due to homogeneity of f, we have Tnww L,, however, Snow S/a². Hence sensitivity cannot replace test loss and to compare the generalization eтror of models by using their sensitivity, the comparison needs to be made fairly: the input of both networks should be drawn from the same distribution.
CONCLUSION
We find that the sensitivity metric is a strong indicator of overfitting. Given multiple networks, with near-zero training loss, to choose from with various hyper-parameters, the best architecture appears to be the one with the lowest sensitivity value. Sensitivity can potentially be used as an early stopping criterion and as an aechitetureeselectiie computing the sensitivity to the given input, the best model is the one with the lowest sensitivity. One of the advantages of the sensitivity metric is that it can provide a loose prediction of the test loss without the use of any labeled data. This is especially important in applications where generating labels requires expensive experiments.
method; given multiple networks to choose from by only
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Sanjeev Arora, Rong Gc, Behnam Neyshabur, and Yi Zhang. Stronger gencralization bounds for deep nets via compression approach. arXiv preprint 1rXivv1802.083 2018.
Jimmy Rich Caruana. Do deep nets really need to be deep? In Advances neural information processing systems. pp. 2654 2662226622222 2014.
Bellido and Emile Fiesler. Do backpropagation trained neural networks have normal weight distributions? In International Conference on Artificial Neural Nenworks, pp. 772-775. Springer, 1993.
Yoshua Bengio and Olivier Delalleau. On the expressive power of deep architectures. In International Conference on Algorithmic Learning Theory, pp. 18-36. Springer, 2011.
Léon Bottou and Olivier Bousquet. The tradcoffs of large scale learning. In Advances in neural information processing systems, pp. 161-168, 2008.
Yannis Dimopoulos, Paul Bourret, and Sovan Lek. Use of some sensitivity criteria for choosing networks with good generalization ability. Neural Processing Letters, 2(6): 141 1995.
S. Moremann-Roe.. Understanding the bias-variance tradeoff.. httt:://scor fortmanr roe. com/
docs Beecs/iasVVarrancee.oe html 2012.
Li Fu and Tinghuai Chen. Sensitivity analysis for input vector in multilayer feedforward neural networks. In Neural Networks, 1993., IEEE International Conference on, pp. 215-218. TEEE, 1993.
Stuart Geman, Elie Bienenstock, and René Doursat. Neural networks and the bias/variance dilemma. Neural computation, 4(1):1 58, 1992.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249-256, 2010.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
Kaiming Hc, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026-1034, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pp. 770-778, 2016.
Sobol' IM. Sensitivity estimates for nonlinear mathematical models. Math. Model. Comput. Exp. 1(4):477441, 1993.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint A 1502.03167, 2015.
Yiding Jiang, Dilip Krishnan, Hossein Mobahi, and Samy Bengio. Predicting the generalization gap in deep networks with margin distributions. arXiv preprint 1rX1v:1810.00113. 2018
Kenji Kawaguchi, Leslie Pack Kaclbling, and Yoshua Bengio. Generalization in deep learning. arXiv preprint 2017.
Nitish Shirish Keskar, Dheevatsa Mudigere, Jorge Nocedal, Mikhail Smelyanskiy, and Ping Tak Peter Tang. larch-batch training for deep learning: Generalization gap and sharp minima. arXiv preprint arXiv: 1609.04836 2016.
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097-1105, 2012.
Yann LeCun, Yoshua Bengio, et al. Convolutional networks for images, speech, and time series. The handbook of brain theory and neural networks, 3361 (10))199999 1995.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521((533):436, 2015.
Pankaj Mehta, Marin Bukov, Ching-Hao Hao Wang, Alexandre GR Day. Clint Richardson. Charles K Fisher, and David J Schwab. A high-bias, low-variance introduction to machine learning for physicists. Physics reports, 2019.

---

Under review as a conference paper at ICLR 2020
Hrushikesh Mhaskar, Qianli Liao, and Tomaso Poggio. When and why are deep networks better than shallow ones? Thirty-First AAAI Conference on Artificial Intelligence, 2017.
Guido F Montufar, Razvan Pascanu, Kyunghyun Cho, and Yoshua Bengio. On the number of lincar regions of deep neural networks. In Advances in nearal information processing systems, pp. 2924-2932, 2014.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th international conference on machine learning (ICML-I0), pp. 807-814, 2010.
Brady Neal, Sarthak Mittal, Aristide Baratin, Vinayak Tantia, Matthew Scicluna, Simon Lacoste-Julien, and Toannis Mitliagkas. modern take the bias-variance tradcoff in neural networks. arXiv preprint arXiv: 1810.08511, 2018.
Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generalization in deep learning. In Advances in Neural Information Processing Systems, pp. 5947-5956 2017.
Andrew Ng.
Learning
Thoery
Lecture Notes.
http://cs229. http:://s.22..stafadddddddd
22929- notes4. pdff
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl-Dickstein. Sensitivity and generalization in neural networks: an empirical study, arXiv preprint arXiv: 1x0vv:802.08777 1802.08760, 2018.
George Philipp and Jaime G Carbonell. The nonlincarity pefficient-prediccing overfitting in deep neural networks, arXiv preprint arXiv: 1806.80077777 2018.
Steve Piche. The selection of weight accuracies for madalines. IEEE Transactions on Neural Networks, 6(2): 432-445, 1995.
Shihani Santurkar, Dimitris Tsipras, Andrew Ilyas, and Aleksander Madry. How does hatch normalization help optimization? In Advances in Neural Information Processing Systems, pp. 2483-2493, 2018.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint Mxiv:1409,1556 2014.
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel RD Rodrigues. Robust large margin deep neural networks. IEEE Transactions on Signal Procesingg 65(16):4265-42888 2017.
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: simple way prevent neural networks from overfitting. The Journal Machine Learning Research, 15(1): 1929 1958, 2014.
Xu Sun, Xuancheng Ren, Shuming Ma, and Houfeng Wang. meprop: Sparsified back propagation for accelcrated deep lcarning with reduced overfitting, arXiv preprint arXiv: 1706.06177 2017.
Enzo Tartaglione, Skjalg Lepsoy. Atilio Fiandroti,, and Gianluca Francini. Learning sparse neural networks via sensitivity-drive regularization. In Advances in Neural Information Processing Syxtems, pp. 3878-3888, 2018.
Matus Telgarsky. Benefits of depth in neural networks. arXiv preprint arXiv: 1602..4402444 2016.
Robert Tihshirani. Bias, variance and prediction error for classification rules. Citeseer, 1996.
René Vidal, Joan Bruna, Raja Giryes, and Stefano Soatto. Mathematics of deep learning. CoRR, abs/1712.04741, 2017.
owg./ abs /tts/////l.... 04741
Timothy E Wang, Jack Gu, Dhagash Mehta, Xiaojun Zhao, and Edgar A Bernal. Towards robust deep neural networks. arXiv preprint arXiv: 18/0. /176,, 2018.
Zifeng Wu, Chunhua Shen, and Anton Van Den Hengel. Wider or deeper: Revisiting the resnet model for visual recognition. Pattern Recognition, 90:1 19-133, 2019.
Huan Xu and Shie Mannor. Robustness and generalization. Machine learning, 86(3):391-423, 2012.
Jing Yang, Xiaoqin Zeng. Shuiming Zhong, and Shengli Wu. Effective neural network ensemble approach for improving generalization performance. IEEE transactions on neural networks and learning systems, 24(6): 878-887 887, 2013.
Xiaoqin Zeng and Daniel S Yeung. Sensitivity analysis of multilayer perceptron to input and weight perturbations. IEEE Transactions on Neural Networks, 138611135 2001.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. arXiv preprint arXiv:1611.05550 2016.
10

---

Under review as a conference paper at ICLR 2020
8 APPENDIX
8.1 EXPERIMENTAL DETAILS
The CIFAR-IO datase is used for the experiments presented in the paper: The fully connected neural networks have the same number of units in the hidden layers, varying from 100 to 500 with step size of 20. For the convolutional neural networks the number of channels in convolutional layers vary from 5 to 25 with a step size of (note that each time a channel added in the convolutional layers, an extra units is added to the fully connected layers of the CNN accordingly). As it is computationally expensive to reach zero training loss for the entire dataset, we choose subset of the training set containing 1000 samples of the CIFAR-10 dataset. Zero training loss is necessary for fair comparison between different networks since we would like to have the same value for Ehiss and Eoose among them. For the optimization algorithm, choose the Adam optimizer with its default parameters and we initialize the weights and biases with random values drawn from distribution according to the initialization technique stated in each figure. The nonlinearity is set to be the ReLU function throughout the experiments. We stop the training when the training loss reaches below the threshold 10 for 10 times. In case this condition is not met, stop the training after 2000 epochs (each epech is iterations over the mini-batches of the entire training set). The noise added to the input image is random tensor with the same size as the input noise is first averaged over all its K entries (for CIFAR-10 10 the number of classes is K = 10), then we take its variance over inputs of the testing dataset and the input noise, finally it is averaged over multiple runs.
is drawn from the Gaussian distribution with zero mean and 0.1 standard deviation. The output
Using the notations:
Conv(number of filters, kernel size, stride, padding)
Maxpnol(kemel size)
Linear(number of units)
Dropout(dropout rate)
for layers of a convolutional neural network where Conv and Lincar layers also include the ReLU non-lincarity except the very last linear layer. The configurations that are used are:
The Alexnet (Kizzevsky al.
2012): Conv(h, 2, 1)- Maxpool()) Conv(3*h, 3, 1. 1) Maxpool(2) Conv( 6Th, 1) Conv(4 h, 3, 1) Conv(4 3, 1, 1) Maxpool(2) Dense layer- (00)out(0.5)
Lincar(4096)) Dropout(0.5) Lincar(4096) Linear(K)
where h E [16, 32, 48, 64, 80)
VGG13 (Simonyan & Zisserman 2014): 2 Conv(64*s, 3, 1, I)- D)-Mxxooo(22-- Conv(128*s, 1, 1)- - M2xpool(2)-
Conv(256's, T. M2xpool()- x Conv(512*s, 3, 1) Maxpool(2)-
Conv(512%s, 3. 1, 1) Maxpool(2) Avgpool(2) Dense layer- - Linear(K) where s E [0.25, 0.5, 1, 1.5,2) and all Conv layers have batch normalization
Each block of ResNet (He al. 2016) configuration: 2 Conv(h, 3, 1, 1) + Conv(h, 1, which Conv layers include BN and ReI.U and the result the summation goes into ReLU layer and h the number of channels.
VGG16 is the same as VGG13 with the difference that it has three layers in the last three blocks. VGG11 configuration is the same tVGG13 except that in the first and second block it has one convolutional layer instead 2. VGG19 is the same as VGG13 except that there ResNet18 has blocks with h=64*s, blocks with h=128*s, 2 blocks with h=256*s, and 2 blocks with h=512* where s E [0.25, 0.5, 1, 1.5, 2]. ResNet34 has 3 blocks with h=64*s, 4 blocks with h=128*s, 6 blocks with h=256*s, and blocks with h=122 where [0.25, 0.5, 1, 1.5.2).
conv layers instead of 2 in the last three blocks.
8.2 COMPUTATION OF EQUATION
6
Computations of this section do not depend on the stage of the training. hence 0 denotes the parameter vector at any stage of training. Let recall the sensitivity metric (Equation (2) definition
S Eg Varzer [svl].
where Ey = 1/ IK ot and aú is the k-tth entry of output noise vector Ey and is
+ (+ 1 (x) _
where apply first order Taylor expansion of the output. For one hidden layer neural network with D input
units, H hidden units, and K output units, we have 8 {wi E I DxH wy e R Hx by R. by R
htm1
11

---

Under review as a conference paper at ICLR 2020
where wy and by are the weights and biases of layer 1 (1 1 is the hidden layer and = 2 is the output layer) which are independently drawn from zero mean normal distribution: wi (O o Л), W2 NO o /), b1 [O, oe I), and b2 ~ (O of 7) (which has been studied in Bellido & Fiesler 1993)). We have
(a Σ hkk a(p") 62.
where wit is the weight connecting unit in layer I to unit & in layer 1 + 1, bf is the bias term added to unit h in layer + 1, p" is the output of the linear transformation in the hidden unit h, ie.,
D
dh W1
and the non-lineas activation function a(-) is positive homogeneous function of degree 1; i.c.
a(x)
ox
> O, otherwise,
(10)
where a and B are non non-negative hyper syper-parameters. ReLU follows Equation 10 with a 1 and 8 0. By
applying the chain rule we obtain
Ord
d=
da(p")
u2 wi
8p"
d=1
Therefore, we have
H
4 nk an Sa(p²) wi 8y"
Σ
K
k=1 h=1d=1
The network parameters are assumed to be independent, and it is assumed that 1 0, and ez T (0,2) 2). Moreove, the entries of the input vector æ independent from each other with the same second moment. i... of { " for S d S Consider the imput noise e, to be a vector zero mean random variables. hence 5 = Rg. ((e)) ). Then sensitivity becomes
(Dalp") 27
s
E.
[(
[(=2)
(w)
(
K2
Oph
D
a² + 8
K
ww 2
(1I)
A=
h=1dd d=1
where the second equation is followed by computing the expectation for zero-men mean normal parameters, Let
var Ex [Var=[out]],
(12)
where out Ba(p")
1/K Σx 12 (æ). Because of the homogeneity of the non non-linearity a(-). we have a(p")
Hence
Op
1 hkk [ with +b2
Da(ph)) Oph
out
+ 69
K
Because the parameters are zero-mean, var = Egg [outt] and we have
k
(Dalp")
(ad)) (wuthi
var
Oph
(D(pp")
E
(wo
E
(bh)2
E
[bb)
K2
Oph
K
K
a²
0
K2
km Jmm d
32
K
+
Frg
K3
12

---

Under review as a conference paper at ICLR 2020
which is followed by taking the expectations over the parameters with zero-mean normal distributions, Therefore, we obtain
o² + K var S
a²
g
Th1
K
where o² is notation used for second moment a random variable. Following same computations for a neural network with M hidden layers, we have
var S oz o2
L
K
II
a² + B² o H 2
(13)
where K is the number of units of the output layer M 1. We refer to second term in the right-hand side of Equation (13 X- value a very rough approximation given the numerous assumptions made above, but practice can often be neglected because oh, is very small or zero (the ResNet configurations do not have biases) in most of our experiments.
The first order Taylor expansion for an arbitrary function the average of the input is g(z) as (([xx)) +
(El E E.z.). Taking the variance of this equation we have
Var(g(>)) (g' (B[x])) Var(x))
Here the function g(-) is the softmax function
(x)(
(x) Σx exp(f((=))
The input of the softmax function is K K-dimmnsional vector, the first order Taylor expansion includes the rector-matrix multiplication of the covariance independent from each other, so the covariance matrix considered a diagonal matrix. Because the parameters are considered to be zero-min meon, the input of the softmax has zero mean, Elfă (z)] 0 for S & < K, then
the input vector. We assume outputs the last layer are
(z)
Var(F*
Var(Jō(z))
af((a)
1
# )) Var (z))
Σ igek
K Var Var(f((>=),
K2
since softmax(0) = 1/K. Therefore,
Svariince
E. Var(T+ (z))
(K - 1)²
K"
K - 1)
K4
K2 KRear-
K K-11
S
o
a
+
x
which completes the computations.
8.3
THE RELATION BETWEEN THE CROSS ENTROPY LOSS AND THE MEAN SQUARE ERROR
We rewrite the cross-entropy loss (Equation
( as
L = Eg- [Lo = Ez.c, log(
where S c K is the index of the true class for the input æ, i.e., y and y* 0 for * c. For simplicity we use the notation Py. instead of F3. (x) in this section. For the MSE loss we have
LMSE
E.,,0.
Σ r
x*)
K Fo.
K Fl the summation inside the above expectation can be rewritten
Because Σ km
F
K Σ
K
x²) = =(1 Fg. )
1=1
3#0
K
= (1 - FG-) 22 + (1 - F7.
Fz.
K K
2 F3.)2
F Fi
1=1 ==
1 fafiic
13

---

Under review as a conference paper at ICLR 2020
K
Since $ Fg- - (1- Fg-
for Σ K,j j # and Σ Fg-
Fge, the above equation bounded as
K
(1 Fg.
Fgg - yy 5 - Fg-)².
(14)
The minimum occurs when FV. (1 (1-FF)))/( (1-
for < S K.J * and the maximum occurs when all the remaining probability (i.c., Pg. is given class besides the class e, and the of the classes are assigned with zero probability, By applying the first order Taylor expansion for logarithm we have
log(FF))) =1 Fö-,
where following the inequality in Equation we have
K ( R=1
K Σ
3 Fg.
S1
(F yt)?
2
K
Intuitively, the upper bound above is preferable in practice, because would like to be less confidant in assigning prohabilities to wrong classes. If we take the expectation the above inequality because Jenson's inequality,
the upper bound is upper bounded by v K /TASE. However, here we consider the worst approximation and approximate F.- with the lower bound above. In practice, observe that this approximation holds, the network overfits very confidently and assigns the probability F.. wrong class and zero to remaining classes. Hence, by roughly approximating the expectation of squared root with the squared root of expectation, we have
LMSE
L R
8.4 COMPUTATION OF EQUATION
Consider feedforward FC with ReLU activation function where i.i.d. zero mean random noise with variance oi, is added to the input. Then, assuming the output noise entries independent from each other, have
K
Var
R [E))]]
K2
we have M hidden layers with H₁, SIS M units per layer, assuming the parameters are i.i.d. and independent from the input noise ex. and are drawn from the standard normal distribution, following the same computations as in Equation [II) for network with M hidden layers, D input units and K output units,
H: 2
S
Do
K
hidden layers have the same number of units, H1 = Hg
HM = H, then,
D
H
S
K
2
c..
8.5 CIFAR-10 EXPERIMENTS
Figure presents the effect of different initialization techniques, and of adding dropout and batch normalization layers to fully connected and convolutional neural networks trained 1000 samples the CIFAR-10 10 training dataset, and evaluated on the entire CIFAR- 10 testing datasct. We observe again the strong relation between sensitivity Sater and generalization error L and the effect of these techniques on both Sater and L. In Figure we
present the empirical results the relation between vur defined in Equation 12] and S defined in Equation 2
2 We experiment for 5 cases, where we change the second moment of the input o and the input noise 2 In
Figures a) and b)), the original CIFAR-10 10 images are considered and Figures ((c). ()d) and 6e), we
normalize the inputs accordingly. In all tigures, the empirical relation between var and S shows good match
with Equation [13 We further experiment on image classification tasks for the MNIST and CIFAR- 100 datasets
in Appendix 8.6 We also provide numerical experiments for a regression task for the Boston house price
prediction in Appendix 8.7
14

---

Under review as a conference paper at ICLR 2020
16
20
networks SN
networks networkssdrppoul
networks XU:
0.8845
10
Equazoo
Equation
(2)
pm 119111
Logg
10
20
20
(Sa
log(S
(a) Effect of initialization
(b) Effect of adding dropout
20
networks
NetworksRN
Fiuution
Logt&)
0,9642
20
]og(S
(c) Effect of adding batch normalization
Figure 5: Test loss versus sensitivity for networks trained on 1000 samples of the CIFAR-10 10 training dataset presenting the effect of initialization, dropout and batch normalization. Each point the average over multiple runs and indicates different architecture. (a) The networks are 5 layer FC, 2-4 layer CNN where the parameters are initially drawn from either Xavier uniform distribution (XU) or standard normal distribution (SN). (h) The networks are 3, 5, 7 layer FC and 1-4 layer CNN. The top most right pink point is the same network architecture as the top most right teal point when dropout is added to the configuration. Hence, for all network architectures we observe shift of the numerical points towards bottom left of the figure when dropout applied. (c) The networks are 3, 5, 7 layer FC and 1-4 layer CNN. (b) and (c) the networks parameters are initially drawn from the standard normal distribution.
8.6 MNIST AND CIFAR- EXPERIMENTS
In this section, we present the experimental results for networks trained on 6000 samples of the MNIST training datasct and evaluated on the entire MNIST testing datasct. Figures a and (7)) show the results for fully connected neural networks with different number of layers and units and using regularization techniques batch normalization and dropout. In Figures (c) ddd) the results for convolutional neural networks are presented. Finally, in Figures 7 [e) 7 [f. the results on the comparison of sensitivity of untrained networks Shetore with the test loss after the networks are trained, are presented. Figure 8 presents the sensitivity S versus the loss L for networks trained on the 1000 samples of the CIFAR- 100 datase The results on these two datasets are consistent with the rest of the paper, and once again we observe the relation between sensitivity and generalization and the effect of state-of-the--nn techniques on hoth sensitivity and generalization.
8.7 REGRESSION TASK AND MSE LOSS
this section, we investigate the relation between sensitivity and generalization error for regression tasks with mean square error criteria. The loss function in this setting is defined as
LMSE = Eo- [Lo= Eg- [E(ra)),, [x)o - 3)
where 0" is found by minimizing the mean square algorithm A. Note that in this setting the output the last layer of the neural network, the softmax layer is not added in this setting and the output layer has unit, i... K and y is a scalar. The bias and variance term are
training dataset using the stochastic learning
http:: //yt:n///aan.... wwwp://ccc.///////////.e.e
https:: cs htt.:ccs..oooooooooooeeeeee
far. htm1
15

---

Under review as a conference paper at ICLR 2020
EC SN
XU
layer
Tayer
(og()
layer PC drogout
layor
0.9670
10
layer FC XU
layer FC
4 layer SC BN
8 HN
ResNer34 HN
ResNer50 HN
VGGLI IIN
log(
VGG13 HN
VGGI6 HN
VGG19 IN
Equation
(a) 0- = 0.2648 and o² = 0.01
3 layer HC SN
layer
8 layer PC XU
layer
20
layer EC HIN
layer
(og(eur)
layer FC SN BN
[ox(var)
layer FC SN BN
0.8910
layer FC SN dropout
- 0.9501
loyer PC SN dropoux
4 layer FC SN
4 loyer FC SN
layer
layer FC XU
layer FC HN
layer
layor RC BN
10
20
-10
20
VGGL1 HN Eqquation
log (Saeer
Equation
(Sane))
(b) - = 0.2648 and o² 0.04
(c) 0 = and o² 0.01
80
layer
layer
20
layer
layer PC HN
(bwn)
layer PC
20
layer PC HN
log(van
layer dropout
layer FC SN drupout
4 layer FC SN
10
0.0537
4 layer FC SN
/0
layer FC XU
layer FC XII
layer PC HN
layer FC HN
layer FC SN BN
FC Eqqattion
VGG11 IIN
10
20
log(S
Equition
log( Sutas)
23 and a? 0.04
(e) o 1.25 and of 0.01
Figure 6: var (Equation 12 versus S (Fquation for networks trained on 1000 samples of the CIFAR-10 training dataset. The non- coneliiy is ReLU, and is neglected in the computation of Equation ( in the figures. (a). (b) The nor-normalieed original CIFAR-10 input images. (c), (d) Normalized input images with eero-mean and unit variance. (e) We normalize inputs have unit variance and mean kept same the original images.
defined as and
Ebins
Ez.y
[(Bo- [fo- (x)] - y)
Evariance
12 [Varo- (fo- (z:))
respectively. consider the Boston housing datase where the objective to predict the price of house given 14 features (including crime rate, distance to employment centers, etc.). Figure shows the results for comparing sensitivity and test loss among fully connected neural networks with 3-8 layers and 100-500 hidden units per
.html
16

---

Under review as a conference paper at ICLR 2020
layer; the networks are trained on 70% of the dataset and then evaluated the remaining 30%. The networks could not get to zero training loss, so we stopped the training after 1000 epochs. The results are consistent on the relation between sensitivity and generalization error which for the regression task is LuSE (Figure (e)). For a more detailed view, we observe that scnsitivity is related to the variance in the bias-variane decomposition of the MSE loss (Figure()d)), and the MSE loss is the summation of the bias term and the variance term (Figure ())).
8.8 DISCUSSION REGARDING ERIAS
In this section, present the results comparing Saner vs. L when the networks trained on different numbers
training samples (Figures bb]b) and [o]c)) and different stages training (Figure dD to validate approximation made in Equation 8 where we neglect Ehis and Enoise- Figure (oja) shows sensitivity versus
loss after the networks are trained on a subset of 1000 samples of the CIFAR- 10 training dataset. Clearly, the
approximations made in the computation of Equation @ do not hold for ResNet18 and ResNet34 networks
and the match is poor, contrary the other part. Figure 10(b shows that this problem can explained (at least in part) by training the networks with more samples. For instance, Figure ()) the yellow mark ResNet18 88 with s=1, and the green mark is Resnet34 with s=0.5, and the transition of the results, towards the linear relation between log S and log L, is clearly observed as we add more training data samples in the training process. Therefore. the larger the number of training samples, the better the approximation Ehis trainloss becomes Evaiance becomes the dominant term the test loss. Figure[10 shows the effect of the stage the training. Both the sensitivity S and the loss are computed at different stages of training (with some abuse of notation for L). In this figure, we observe the effect of the stage of training on the approximations made for computing Equation (8) At initial stages of the training, the bias Ebs and noisc term in the test loss cannot be neglected.
17

---

Under review as a conference paper at ICLR 2020
20
Alesnet
Aloxner SN
layer CNN SN
log (1.)
VGGI6
VGGI1
Resnel
Equation
(Sate))
(a) Replicate of Figure
for MNIST
layer
layer
PC dmppuu
layer
layer
Jayer
Eqquation $
logt..)
layer
log(L)
10
Jayer
Equation
0.9065
0.9612
log( Synre)
log Sunee)
(b) Fully connected neural networks
(c) 4-layer fully connected neural network trained with without regularization
layer CNN
10
2 layer CNN layer CNN BN 2 layer CNN dropout
layer CNN
layer CNN
Equation
2 layer CNN max-pvoling Equation
Log(E)
M
0.5099
0.9386
10
20
10
20
Tog( 1og(5
log Sutee)
(d) Convolutional neural networks
(e) 2-layer convolutional neural networks trained with or without regularization tech- niques
20
Jayer
layer
layer
2ayer CNN
Jayer FC
3 layer CNN Equation
egtee Sed as
layer PC
Lo
layer PC
10
layer PC
Equatioa R
113039
- 0.0300
20
40
1.0
20
(Seetion)
(1 Sensitivity before training for FCs
(g) Sensitivity before training for CNNs
Figure 7: Test loss versus sensitivity for networks trained on 6000 samples of the MNIST training dataset. Each point indicates network with different width and the sensitivity and test loss are averaged over multiple runs.
18

---

Under review as a conference paper at ICLR 2020
20
Alexnet
layer
VGGI3 SN
log(L)
p 013890
Reszell8
Resset| 8 SN
Equation
20
40
(S
(a) Replicate of Figure for CIFAR-I00
2 layer
3 laye: FC
laye: FC
20
P
0.5718
S layer FC
6 layer FC
log(A.)
logt
layor FC
layer CNN
0.9624
10
layer CNN
10
layer PC
7 layer FC drogout
Equation
layer FC BN
Equation
40
Sater
(og(SSne:)
(b) Sensitivity after training vs test loss
(c) Effect regularization on fully connected neural networks
15
layer
layer
log(A)
layer
[og(L))
10
To
Jayer
0,9277
layer CNN
2 layer CNN
layer CNN
3 layer CNN
layer dropout layer
Equation
Equation
40
20
Tog( Seeese
Tog (rge))))e))
(d) Sensitivity before training test loss
(e) Effect of regularization convolutional neu- ral networks
Figure 8: L versus S for networks trained on 1000 samples of the CIFAR- 100 training dataset. Each point indicates a network with a different width and the sensitivity and test loss are averaged over multiple runs.
19

---

Under review as a conference paper at ICLR 2020
40
layer
layer
layer
layer
Jayer
layer
layer
log (MSS)
6 layer
layer FC
7 layer
S layer PC
8. layer PC
10
10
log(Svariance)
Fbsiu) (b)
(a)
3 layer FC
3 layer FC
layer
Tayer
layer
layer
layer
layer
layer
log (gg(Cran
log
* layer
layer
10
20
300
40
log (fggeb + Cuarramene
Tog(S
(c)
(d)
3 layer PC
4 layer FC
layer FC
30
layer
log
layer
0,7881
20
40
log( (Snrr)
Figure 9: Variance, bias and sensitivity versus test loss for a regression task using the MSE loss. The fully connected neural networks are trained and evaluated on the Boston house price dataset. Each point indicates an average over multiple runs of network with different widths H.
20

---

Under review as a conference paper at ICLR 2020
layer Resner34
Equation
X
log(
host C
traiised on 1000 samples
tmiined 2000 sumples
uninedd on 5000 sumples
10000 samples
-5
0
10
15
(Ster)
(a) Networks trained on 1000 samples the training dataset
(b) ResNet18 ResNet34
12
trineed on 100 samples trained 1000 samples
10
trained 2000 samples
trined 5000 sumples
Equasion (
log
epoctss tminning
after 100 epochs tnining
fully trined
Equition
20
Log (Sate)
log(:)
(c) 1-2 layer CNN
(d) 1-2 layer CNN
Figure 10: Sensitivity versus test loss for networks at different stages of training and trained on different numbers of training samples. Each point indicates average multiple runs of network with different width and depth. (b) is the zoom of (a) on bottom left, and we add the results the same networks trained with different numbers of samples. In (b) The network parameters are drawn from a normal distribution by using the He technique. (c) and (d) are the zoom in of (a) on the top left, and we add the results for the same networks trained with different number of training samples in (c) and different stages of training in (d). In (c) and (d) the network parameters drawn from the standard normal distribution.
21


Reviews:
[{"review_id": "Ske_OjzaYr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper studies the connection between sensitivity and generalization where sensitivity is roughly defined as the variance of the output of the network when gaussian noise is added to the input data (generated from the same distribution as the training error).\n\nThe paper is well-written and the experiments are very comprehensive. There are however 3 major issues with the current approach:\n\n1- Novelty: Novak et al. 2018 suggests a very similar notation of sensitivity and they show correlation with generalization. Even though the authors site this work, they don't discuss the connection very clearly. In light of that work, there is very limited novelty in this paper.\n\n2- Definition of test loss: Authors define the test loss to be cross-entropy but in almost all these tasks, what we care about is the task-loss which is 0/1 classification error on the test data and not the cross-entropy loss. These two loss behave very differently. In particular, the cross-entropy loss is very sensitive to the of variance of the output while 0/1 classification loss does not depend on it. Therefore, it is not surprising that there is high correlation between the output variance and the cross-entropy loss but it is not clear if this has anything to do with the test error.\n\n3- Using test data in the complexity measure: The goal of understanding generalization is not just to get correlation with the test error. One can always use a validation set to get a very good correlation. Even when we have limited data, we can always put a small portion of the data for validation without loosing much in the final performance. The main goal is to predict generalization without using any access to the distribution. In particular, we need properties that show how networks behave on new data instead of simply measuring a property on the new data. Therefore, using a measure that is evaluated on new data is not really helpful.\n\n\n********************************\n\nAfter author rebuttals:\n\nAuthors have addressed one of my concerns (no 3) but the other two concerns are not addressed adequately. I increase my score to \"weak reject\" but not higher because of my concern about the novelty of the work in light of Novak et al. 2018.", "extra_scores": {"experience_assessment": "I have published in this field for several years.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BJgIJnMy5S", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper examines generalization performance of various neural network architectures in terms of a sensitivity metric that approximates how the error responds to perturbations of the input. A crude argument is presented for how the proposed sensitivity metric captures the variance term in the standard bias-variance decomposition of the loss. A number of experimental results are presented that show strong correlation between the sensitivity metric and the empirical test loss.\n\nUnderstanding the distinguishing characteristics of networks that generalize well versus networks that generalize poorly is a central challenge in modern deep learning research, so the topic and analyses presented in this paper are salient and will be of interest to most of the community. The experimental results are intriguing and the presentation is clear and easy to read. While some may object to the egregious simplifications utilized in \"deriving\" the sensitivity metric, I believe this kind of analysis should be welcomed if it produces new insights and helps explain otherwise opaque empirical phenomena. All told, if taken in isolation from prior work, I think the insights and empirical results presented in this paper are quite interesting and certainly sufficient for acceptance to ICLR.\n\nHowever, there is significant overlap with prior work that severely detracts from the novelty of the results presented here, and I think the community is already familiar with the paper's main conclusions. From the empirical viewpoint, [1] performs a very similar (and actually quite a bit more thorough) analysis, and reaches very similar conclusions. The authors do cite [1], but unless I missed something, their main argument for uniqueness is basically \"in experiments, we prefer S to the Jacobian, because in order to compute S it is enough to look at the network as a black box that given an input, generates an output, without requiring further knowledge of the model.\" While this may be useful from the practical standpoint for some non-differentiable models, I'm not convinced that this distinction is really significant in terms of building insights or new understanding. \n\nOne additional way this paper is distinct from [1] is that it includes a theoretical \"derivation\" for the sensitivity metric. While I found the argument interesting, from the theoretical perspective, [2] gives much more rigorous and insightful arguments that help explain the observed phenomena. \n\nOverall, I'm just not convinced this paper is novel enough to merit publication. But perhaps I've overlooked something, in which case I hope the author's response can highlight their unique contributions relative to prior work.\n\n[1] Novak, Roman, et al. \"Sensitivity and generalization in neural networks: an empirical study.\" arXiv preprint arXiv:1802.08760 (2018).\n[2] Arora, Sanjeev, et al. \"Stronger Generalization Bounds for Deep Nets via a Compression Approach.\" International Conference on Machine Learning. 2018.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes a definition of the sensitivity of the output to random perturbations of the input and its link to generalization.

While both reviewers appreciated the timeliness of this research, they were taken aback by the striking similarity with the work of Novak et al. I encourage the authors to resubmit to a later conference with a lengthier analysis of the differences between the two frameworks, as they started to do in their rebuttal.
Decision: Reject