Relative stability toward diffeomorphisms indicates performance in deep nets
Authors: Leonardo Petrini
Alessandro Favero
Mario Geiger
Matthieu Wyart
Venue: NeurIPS 2021 Poster (2021)
Keywords: deep learning
stability
diffeomorphisms
image classification
learning invariants

Abstract: Understanding why deep nets can classify data in large dimensions remains a challenge. It has been proposed that they do so by becoming stable to diffeomorphisms, yet existing empirical measurements support that it is often not the case. We revisit this question by defining a maximum-entropy distribution on diffeomorphisms, that allows to study typical diffeomorphisms of a given norm. We confirm that stability toward diffeomorphisms does not strongly correlate to performance on benchmark data sets of images. By contrast, we find that the stability toward diffeomorphisms relative to that of generic transformations $R_f$ correlates remarkably with the test error $\epsilon_t$. It is of order unity at initialization but decreases by several decades during training for state-of-the-art architectures. For CIFAR10 and 15 known architectures, we find $\epsilon_t\approx 0.2\sqrt{R_f}$, suggesting that obtaining a small $R_f$ is important to achieve good performance. We study how $R_f$ depends on the size of the training set and compare it to a simple model of invariant learning.

Relative stability toward diffeomorphisms indicates performance in deep nets
Leonardo Petrini,
Alessandro Favero,
Mario Geiger,
Matthieu Wyart
Institute of Physics École Polytechnique Fédérale de Lausanne
1015 Lausanne, Switzerland {name. (wr.aameeeepf]le ch
Abstract
Understanding why deep nets can classify data in large dimensions remains chal- lenge. has been proposed that they do so by becoming stable to diffeomorphisms. yet existing empirical measurements support that it is often not the case. We revisit this question by defining maximum-entree allows to study typical diffeomorphisms of a given norm. We confirm that stability toward diffeomorphisms does not strongly correlate to performance on benchmark data sets of images. By contrast, we find that the stability toward diffeomorphisms relative to that of generic transformations Ry correlates remarkably with test error Eç. It is of order unity at initialization but decreases by several decades during training for oftte-of-thh-art architectures. For CIFAR 10 and 15 known architectures we find & 0.2. Ry, suggesting that obtaining small R important achieve good performance. We study how Ry depends on the size of the training set and compare it to simple model of invariant learning.
distribution on diffeomorphisms, that
Introduction
Deep learning algorithms LeCun al. (2015) are now remarkably successful wide range of tasks Amodei et al. (2016); Huval et al. (2015); Mnih et al. (2013); Shi et al. (2016); Silver et al. (2017). Yet, understanding how they can classify data in large dimensions remains challenge. In particular, the curse of dimensionality associated with the geometry of space in large dimension prohibits learning in generie setting Luxburg and Bousquet (2004). ditt--mimmsiinall data can be learnt, then they must be highly structured.
A popular idea is that during training, hidden layers of neurons learn representation Le (2013) that insensitive to aspects of the data unrelated to the task, effectively reducing the input dimension and making the problem tractable Ansuini al. (2019); Recanatesi et al. (2019); Shwartz-Ziv and Tishby (2017). Several quantities have been introduced to study this effect empirically. It includes (i) the mutual information between the hidden and visible layers of neurons Saxe al. (2019); Shwartz-Ziv and Tishby (2017), (ii) the intrinsic dimension of the neural representation of the data Ansuini et al. (2019); Recanatesi et al. (2019) and (iii) the projection the label of the data on the main features of the network Kopitkov and Indelman (2020); Oymak et al. (2019); Paccolat et al. (2021a), the latter being defined from the top eigenvectors of the Gram matrix of the neural tangent kernel (NTK) Jacot et al. (2018). All these measures support that the neuronal representation of the data indeed becomes well-suited to the task. Yet, they are agnostic the nature of what varies in the data that need not being represented by hidden neurons, and thus do not specify what it is.
Recently, there has been a considerable effort understand the benefits of learning features for one- hidden-layer fully connected nets. Learning features can occur and improve performance when the
35th Conference on Neural Information Processing Systems (NcurIPS 2021).

---

true function is highly anisotropic. in the sense that it depends only on a linear subspace of the input space Bach (2017); Chizat and Bach (2020); Ghorbani et al. (2019, 2020); Paccolat et al. (2021a); Refinetti et al. (2021); Yehudai and Shamir (2019). For image classification, such an anisotropy would occur for example if pixels on the edge of the image are unrelated to the task. Yet, fully-connected nets (unlike CNNs) acting on images tend to perform best in training regimes where features are not learnt Geiger et al. (2021, 2020); Lee et al. (2020), suggesting that such a linear invariance in the data is not central to the success of deep nets.
Instead, it has been proposed that images can be classified in high dimensions because classes are invariant to smooth deformations or diffeomorphisms of small magnitude Bruna and Mallat (2013); Mallat (2016). Specifically, Mallat and Bruna could handcraft convolution networks, the scattering transforms, that perform well and are stable to smooth transformations, in the sense that (((x))[((aa)) || is small if the norm of the diffeomorphism during training deep nets learn to become stable and thus less sensitive to these deformations, thus improving performance. More recent works generalize this approach to more common CNNs and discuss stability at initialization Bietti and Mairal (2019a.b). Interestingly, enforcing such stability can improve performance Kayhan and Gemert (2020).
small too. They hypothesized that
Answering if deep nets become more stable to smooth deformations when trained and quantifying how affects performance remains a challenge. Recent empirical results revealed that small shifts of images can change the output a lot Azulay and Weiss (2018); Dieleman et al. (2016); Zhang (2019), in apparent contradiction with that hypothesis. Yet in these works, image transformations (i) led to images whose statistics were very different from that of the training set or (ii) were cropping the image, thus are not diffeophormisms. In Ruderman et al. (2018), a class of diffeomorphisms (low-pass filter in spatial frequencies) was introduced to show that stability toward them can improve during training, especially in architectures where pooling layers are absent. Yet, these studies do not address how stability affects performance, and how it depends on the size of the training set. To quantify these properties and to find robust empirical behaviors across architectures, we will argue that the evolution of stability toward smooth deformations needs to be compared relatively to that of any deformation, which turns out to vary significantly during training.
Note that in the context of adversarial robustness, attacks that are geometric transformations of small norm that change the label have been studied Alaifari et al. (2018); Alcorn et al. (2019); Athalye et al. (2018); Engstrom et al. (2019); Fawzi and Frossard (2015); Kanbak et al. (2018); Xiao et al. (201 8). These works differ for the literature above and from out study below in the sense that they consider worst-case perturbations instead of typical ones.
1.1 Our Contributions
We introduce maximum entropy distribution of diffeomorphisms, that allow us to generate typical diffeomorphisms of controlled norm. Their amplitude governed by a "temperature" parameter T.
We define the relative stability to diffeomorphisms index R2 that characterizes the square magnitude of the variation of the output function I with respect to the input when it is transformed along diffeomorphism, relatively to that of random transformation of the same amplitude. It is averaged on the test set as well as on the ensemble of diffeomorphisms considered.
We find that at initialization, Ry is close to unity for various data sets and architectures, indicating that initially the output is as sensitive to smooth deformations as it is to random perturbations of the image.
Our central result is that after training. R, correlates very strongly with the test error & during training, Ry is reduced by several decades in current State Of The Art (SOTA) archi- tectures on four benchmark datasets including MNIST Lecun al. (1998), FaShionMNIST Xiao et al. (2017). CIFAR- 10 Krizhevsky (2009) and ImageNet Deng et al. (2009). For more primitive architectures (whose test error is higher) such as fully connected nets or simple CNNs, Ry remains of order unity. For CIFAR1O 10 we study 15 known architectures and find empirically that e, R /Rg.
R2 decreases with the size of the training set P. We compare to an inverse power 1/P expected in simple models of invariant learning Paccolat et al. (2021a).
2

---

The library implementing diffeomorphisms on images is available online at github.com/pesl- eptldiffeomoophism.
The code for training neural nets be found at gatnuu.coo//ennnntoorrrriiffffos and the corresponding pre-trained models at doi.org/10.528 0ttp://.5528///cmmodo00058883999
2
Maximum-entropy model of diffeomorphisms
2.1 Definition of maximum entropy model
We consider the case where the input vector ær an image. It can be thought as a function a(s) describing intensity in position (u, v) [0. 1]2, where and are the horizontal and vertical coordinates. To simplify notations we consider a single channel, in which case x(s) is a scalar (but our analysis holds for colored images as well). We denote by the image deformed by т, i.e. [rx] (s) = 2(s T(8)). 7(s) is a vector field of components (Tu()), (s)))). The deformation amplitude is measured by the norm
||~||² = / ((Vr.)² + (VTT)
(1)
0.11
To test the stability of deep nets toward diffeomorphisms, we seek build typical diffeomorphisms of controlled norm V/ We thus consider the distribution over diffeomorphisms that maximizes the entropy with a norm constraint. It can be solved by introducing Lagrange multiplier T' and by decomposing these fields on their Fourier components, see e.g. Kardar (2007) or Appendix A. In this canonical ensemble, one finds that Ta and T. are independent with identical statisties. For the picture frame not to be deformed, we impose fixed boundary conditions: T = if u 0, or 0, 1. One then obtains:
Σ Cig sin(iTu) sin(j
(2)
iJEN
where the Cig are Gaussian variables of zero mean and variance (C TT((2 +5²). If the picture is made n x n pixels, the result is identical except that the sum runs on < <i,j < n. For large n, the norm then reads || ( /2) /22nnn nT? and is dominated by high spatial frequency modes. It is useful to add another parameter e to cut-off the effect of high spatial frequencies, which can be simply done by constraining the sum in Eq.2 to 12 + 1²
2 < d² one then has ||Vr|2 (T* (8)/8) eT.
Once T is generated, pixels are displaced to random positions. A new pixelated image can then be obtained using standard interpolation methods. We use two interpolations, Gaussian and bi-linear', as described in Appendix C. As we shall see below, this choice does not affect our result as long as the diffeomorphism induced a displacement of order of the pixel size, or larger. Examples are shown in Fig. as function of T and c.
2.2 Phase diagram of acceptable diffeomorphisms
Diffcomorphisms are bijective, which is not the case for our transformations if T is too large. When this condition breaks down, single domain of the picture can break into several pieces, as apparent in Fig.1. It can be expressed as a condition on VT that must be satisfied in every point in space Lowe (2004), as recalled in Appendix B. This satisfied locally with high probability if || < 1, corresponding to T < (8/ (8/x2)/c2. In Appendix, we extract empirically curve of similar form in the (T, c) plane at which diffeommrphism is obtained with probability at least 1/2 For much smaller T, diffeomorphisms are obtained almost surely.
Finally, for diffeomorphisms to have noticeable consequences, their associated displacement must be of the order of magnitude of the pixel size. Defining 5² as the average square norm of the pixel displacement the center the image in the unit of pixel size, is straightforward to obtain from Eq.2 that asymptotically for large e (cf. Appendix B for the derivation),
82 n Tin()...
The line 5 1/2 is indicated in Fig. using empirical measurements that add pre-asymptotic terms to Eq.3. Overall, the green region corresponds to transformations that (i) are diffcomorphisms with high probability and (ii) produce significant displacements at least of the order of the pixel size.
'Thooughout the paper, if not specified otherwise, bi-linear interpolation is employed.
3

---

10
T- le 11.
Average pixel displacement
05,
T 06, 10
7 06, c 30
10 2
relevant diffoo region
Figure I: Samples of max- entropy diffeomorphisms for different temperatures T and high-frequency cut- off's c for an ImageNet data- point of resolution 320 320. The green region cor- responds well behav- ing diffeomorphisms (see Section 2.2). The dashed line corresponds to d = 1. The colored points on the line are those we focus our study in Section 3.
de D1. p-
T= de 04
10
[
10-1
T- 1e 03, c
16,
T - Ge 06, 100
10-3
10-6
10"
102
cut-off c
3 Measuring the relative stability to diffeomorphisms
Relative stability to diffeomorphisms To quantify how a deep net learns become less sen- sitive to diffeomorphisms than to generic data transformations, we define the relative stability to diffeomorphisms Ry as:
f(x)) R ()) f
(4)
where the notation (y can indicate alternatively the mean or the median with respect the distribution of y. In the numerator, this operation made over the test set and over the ensemble of diffeomor- phisms of parameters (T.c) e) (on which R, implicitly depends). In the denominator, the average is on the test set and on the vectors 1 sampled uniformly on the sphere of radius ||>|| = ([|Tæ - ell) 2. An illustration of what R, captures is shown in Fig.2. In the main text, consider median quantities, as they reflect better the typical values of distribution. In Appendix E.3 we show that our results for mean quantities, for which our conclusions also apply.
Dependence of Ry on the diffeomorphism magnitude Ideally, R2 could be defined for infinites- imal transformations, as it would then characterize the magnitude of the gradient of f along smooth deformations of the images, normalized by the magnitude of the gradient in random directions. How- ever, infinitesimal diffeomorphisms move the image much less than the pixel size, and their definition thus depends significantly the interpolation method used. It is illustrated the left panels of Fig.3, showing the dependence of Ry in terms of the diffeomorphism magnitude (here characterised by the mean displacement magnitude the center of the image 8) for several interpolation methods. We do see that R becomes independent of the interpolation when becomes of order unity. In what follows we thus focus on R,y (5 1). which we denote Ry.
SOTA architectures become relatively stable diffeomorphisms during training, but are not at initialization The central panels of Fig.3 show R, at initialization (shaded), and after training (full) for two SOTA architectures on four benchmark data sets. The first key result is that, at initial- ization, these architectures are as sensitive to diffeomorphisms Relative stability to diffeomorphisms initialization (guaranteed theoretically in some cases Bietti and Mairal (2019a,b)) thus does not appear to be indicative of successful architectures.
they are to random transformations.

---

I x+N
TL
data-space
Figure 2: Illustrative drawing of the data-space R around a data-point x (black point). We focus here perturbations of fixed magnitude ie. the sphere of radius centered in The intersection between the images x transformed via typical diffeomorphisms and the sphere is represented in dashed green. By contrast, the red point is example of random transformation. For large n, it is equívalent to adding an i.i.d. Gaussian noise to all the pixel values of x. Figures on the right illustrate these transformations, the color of the dot labelling them corresponds to that of the left illustration. The relative stability to diffeomorphisms Ry characterizes how a net f varies in the green directions, normalized by random ones.
By contrast, for these SOTA architectures, relative stability toward diffeomorphisms builds up during training on all the data sets probed. It is sigmificant effect, with values of Ry after training generally found in the range R E [10-?, 10-1)
Standard data augmentation techniques (translations, crops, and horizontal flips) are employed for training. However, the results we find only mildly depend on using such techniques, see Fig. 12 in Appendix.
Learning relative stability to diffeos requires large training sets How many data are needed to learn relative stability toward diffeomorphismss To answer this question, newly initialized networks are trained on different training-sets of size P. Ry is then measured for CIFARIO, as indicated in the right panels of Fig.3. Neural nets need a certain number of training points (P ~ 10 in order to become relatively stable toward smooth deformations. Past that point, R2 monotonically decreases with P. In a range of P, this decrease is approximately compatible with the an inverse behavior R2 1/P found in the simple model of Section 6. Additional results for MNIST and FaSH can be found in Fig. 13, Appendix E.3.
Simple architectures do not become relatively stable to diffeomorphisms To test the universal- ity of these results, focus two simple architectures: (i) 4-hidden-layer fully connected (FC) network (FullConn-L4) where each hidden layer has 64 neurons and (ii) LeNet LeCun et al. (1989) that consists of two convolutional layers followed by local max-pooling and three fully-connected layers.
Measurements of Ry for these networks are shown in Fig.4. For the FC net, R at initialization (as observed for SOTA nets) but grows after training on the full data set, showing that FC nets do not learn to become relatively stable to smooth deformations. It is consistent with the modest evolution of (P) with P. suggesting that huge training sets would be required to obtain Ry < The situation is similar for the primitive CNN LeNet, which only becomes slightly insensitive (R, z= 0.6) in single data set (CIFARIO), and otherwise remains larger than unity.
Layers' relative stability monotonically increases with depth Up to this point, we measured the relative stability of the output function for any given architecture. We now study how relative stability builds up as the input data propagate through the hidden layers. In Fig. 14 of Appendix E.3, we report R2 as function of depth for both simple and deep nets. What we observe is R s 1 independently
With the only exception of the ImageNet results (central panel) in which only one trained network is considered.
5

---

18 on CIEARIO
ReeNet
ResNe518 on CIPARTO
10
redused to diffee
10a
10-1 1
R
10-1
MNST F-MNIST CITARCO ImaggeNet
10
P
10a
Edferint Net CIEARI
Esficient Net
Ne:BBe CIPAR1O
10
10'
R
109
Inated
tc. Ioar ceur. jusn intepp
10
1ilxma
10-1-
10
10
MNIST F-MNEST CHEARIO ImageNett
10a
P
10'
Figure 3: Relative stability to diffeomorphisms Ry for SOTA architectures. Left panels: Ry vs. diffeomorphism displacement magnitude 6 at initialization (dashed lines) and after training (full lines) on the full data set of CIFAR1O (P = 50k) for several cut-off parameters c and two interpolations methods, indicated in legend. ResNet is shown the top and EfficientNet the bottom. Central panels: (5 = 1) for four different data-sets (x-axis) axis) and two different architectures at initialization (shaded histograms) and after training (full histograms). The values of (in different colors) (3, 15) and (3, 10, 30) for the first three data-sets and ImageNet, respectively. ResNet18 and EfficientNetBO employed for MNIST, F-MNIST and CIFARIO. ResNer101 and EfficientNetB2 for ImageNet. Right panels: ( = 1) vs. training set size at = 3 for ResNet18 (top) and EfficientNetBO (bottom) trained on CIFARIO. The value of R at initialization indicated with dashed lines. The triangles indicate the predicted slope R P-1 in simple model of invariant learning, see Section 6. Statistics: Each point in the graphs is obtained by training 16 differently initialized networks on 16 different subsets of the data-sets; each network then probed with 500 test samples in order to measure stability to diffeomorphisms and Gaussian noise. The resulting Ry obtained by log-averaging the results from single realizations.
FallConn-IA
LeNet
3 10
2
10"
insined
15
10P
109
MINSST
F-MNIST
CIFARIO
F.NNIST
CIFARIO
MINSST
10a
10
R1
CIRARIO
10P
109
10
10
P
P
Figure 4: Relative stability to diffeomorphisms Ry in primitive architectures. Top panels: R, at initialization (shaded) for trained nets (full) for fully connected net (left) primitive CNN (right) at P 50k. Bottom panels: (P) for and different data sets as indicated in legend. Statistics: see caption in the previous figure.
of depth at initialization, and monotonically decreases with depth after training. Overall, the gain in relative stability appears be well-spread through the net, is also found for stability alone Ruderman et al. (2018).

---

Relative stability to diffeomorphisms indicates performance
Thus, SOTA architectures appear to become relatively stable to diffeomorphisms after training. unlike primitive architectures. This observation suggests that high performance requires such relative stability to build up. To test further this hypothesis, we select set of architectures that have been relevant in the state of the progress the past decade; systematically train them in order to compare R to their test error 6t- Apart from fully connected nets, we consider the already cited LeNet (5 layers and 60k parameters); then AlexNet Krizhevsky et al. (2012) and VGG Simonyan and Zisserman (2015). deeper (8-19 layers) and highly over over-parametrizee (10-20M (million) params.) versions of the latter. We introduce batch-normalizztion in VGGs and skip connections with ResNets. Finally, we to EfficientNets, that have all the advancements introduced in previous models and achieve SOTA performance with relatively small number of parameters (<10M); this is accomplished by designing an efficient small network and properly scaling it up. Further details about these architectures can be found in Table Appendix E.2.
The results are shown in Fig.5. The correlation between Rg and e, is remarkably high (corr. coeff.". 3 0.97), suggesting that generating low relative sensitivity diffeomorphisms Ry important obtain good performance. In Appendix E.3 we also report how changing the train set size P affects the position of network in the (et, R2) plane, for the four architectures considered in the previous section (Fig.18) We also show that our results are robust to changes of 6, (Fig.2) and data sets (Fig.20).
What architectures enable low Ry value? The latter can be obtained with skip connections or not, and for quite different depths indicated in Fig.5. Also, the same architecture (EfficientNetBO) trained by transfer learning from ImageNet - instead of directly on CIFARIO - shows a large improvement both in performance and in diffeomorphisms invariance. Clearly, Ry is much better predicted by e than by the specific features of the architecture indicated in Fig.5.
FollCoun12 O PullComiL
CIFAR10 nets performance vs relative diffeo stability
FullConaL6
30%
O LeNet
AlexNet
10%
Nei
VGGI1
number of layers
vGgi
VGCL O
VGGLlba
batch-uoru
Net30
ResNotas
skip connections
ResNet34
Nel
Lrausfer learning from ImageNet.
8%
O **
Emiiient Not
= 0.24/
10-2
10-1
100
R
Figure 5: Test error & relative stability to diffeomorphisms Ry computed at 5 = and = 3 for common architectures when trained on the full 10-classes CIFAR10 dataset (P= 50k) with SGD and the cross-entropy loss; the EfficientNets achieving the best performance are trained by transfer learning from ImageNet (x) - more details on training procedures can found Appendix E. The color scale indicates depth, and the symbols the presence of batch-norm (o) and skip connections (t). Dashed grey line: power low fit e, s 0.2 R, strongly correlates to much less so to depth or the presence of skip connections. Statistics: Each point obtained by training differently initialized networks; each network is then probed with 500 test samples in order to measure Ry. The results are obtained by log- over single realizations. Error bars omitted here are shown in Fig. 19, Appendix E.3.

---

Stability toward diffeomorphisms vs. noise
The relative stability to diffcomorphisms Ry can be written R2 Dr/G, where G characterizes the stability with respect to additive noise and Dj the stability toward diffeomorphisms:
G fl 1)
- [(>)))
<I][(Tx)) - f(x))) Dj (Ilf(z) - f()))))
(5)
Here, chose to normalize these stabilities with the variation of f over the test set (to which both and 2 belong), and r a random noise whose magnitude prescribed as above. Stability toward additive noise has been studied previously in fully connected architectures Novak al. (2018) and for CNNS as a function of spatial frequency in Tsuzuku and Sato (2019); Yin et al. (2019).
The decrease of Ry with growing training set size P could thus be due to increase in the stability toward diffeomorphisms (i.e. D, decreasing with P) or a decrease stability toward noise (G₃ show in Fig.6 G (P(P), Dy and (P(P) for
increasing with P). To test these possibilities, MNIST, Fashion MNIST and CIFARIO for two SOTA architectures. The central results are that (i) stability toward noise is always reduced for larger training sets. This observation is natural: when more data needs to be fitted, the function becomes rougher. (ii) Stability toward diffeomorphisms does not behave universally: it can increase with P decrease depending the architecture and the training set. Additionally, G and D, alone show a much smaller correlation with performance than R-- see 15g in Appendix E.3.
MNIST
F-MNIST
CIFAR10
10F
10a1
RosNet18
10-1
10-2
10a1
10-4
G;
c 3
10-1
10a1
10-1
10
10a
P
10
10
P
10
10
10
19P
104
Elficient
10-3
10:1
10-3
10-1
10:3
10P
104
10a
104
10
10
10
Figure 6: Stability toward Gaussian noise (G and diffeomorphisms (Dy) alone, and the rela- tive stability Ry. Columns correspond to different data-sets (MNIST, FashionMNIST and CIFARIO) and rows to architectures (ResNet18 and EfficientNetBO). Each panel reports Gy (blue), D, (orange) and R (green) as a function of P and for different cut-off values c, as indicated in the legend. Statistics: ef. caption in Fig.3. Error bars omitted here are shown in Fig.22, Appendix E.3.
minimal model for learning invariants
In this section, we discuss the simplest model of invariance in data where stability transformation builds up, that can be compared with our observations of R above. Specifically, we consider the "stripe" model Paccolat et al. (2021b), corresponding to binary classification task for Gaussian- distributed data points x = (I), xI where the label function depends only on one direction in data space, namely y(z) = y(z)). ). Layers of y = +1 and y = -1 regions alternate along the direction zij separated by parallel planes. Hence, the data present 1 invariant directions in input-space denoted by æ as illustrated in 7-g.7.left.
When this model is learnt by one-hidde see-hidden-llan layer fully connected net, the first layer of weights can be shown align with the informative direction Paccolat et (2021a). The projection of these weights
"Corelation coefficient: Cov(log es log Ry)
Varglog es Vaagg R

---

Stripe Model
Figure 7: Left: example the stripe model. Dots are data- points, the vertical lines rep- resent the decision boundary and the color the class label. Right: Relative stability R for the stripe model in d 30. The slope of the curve is -1, as predicted.
2
10"1
10-3
on the orthogonal space vanishes with the training set size as 1/VP, an effect induced by the sampling noise associated to finite training sets.
In this model, Ry can be defined as:
Ry (1 +V) [1 11) (I 7) - f(x)]1))
where we made explicit the dependence of f on the two linear subspaces. Here, the isotropic noise is added only in the invariant directions. Again, impose ||v|| = |||||| (P) is shown in Fig. 7-right. We observe that (P(P) p-1, expected from the weight alignment mentioned above.
Interestingly, Fig.3 for CIFARIO and SOTA architectures support that the 1/P2 behavior compatible with the observations for some range of P. In Appendix E.3, Fig. 13, we show analogous results for MNIST and Rashion-MNIST. We observe the 1/P power-law scaling for ResNets. It suggests that for these architectures, learning to become invariant to diffeomorphisms may be limited by a naive measure of sampling noise as well. By contrast for EfficientNets, which the decrease in Ry is more limited, 1/ behavior cannot be identified.
7 Discussion
common belief is that stability to random noise (small Gý) and to diffeomorphisms (small D2) are desirable properties of neural nets. Its underlying assumption is that the true data label mildly depends on such transformations when they are small. Our observations suggest an alternative view:
1. Figs.6,16 better predictors are more sensitive to small perturbations in input space.
2. As a consequence, the notion that predictors are especially insensitive to diffeomorphisms is not captured by stability alone, but rather by the relative stability R Dr/G,-
3. We propose the following interpretation Fig.5: to perform well, the predictor must build large gradients in input space near the decision boundary leading to a large Gy overall. Networks that are relatively insensitive to diffeomorphisms (small R2) can discover with less data that strong gradients must be there and generalize them to larger regions of input space, improving performance and increasing Gj.
This last point can be illustrated in the simple model of Section 6, see Fig. 7-lfft panel. Imagine two data points of different labels falling close the c.g. - left true decision boundary. These two points can be far from each other if their orthogonal coordinates differ. Yet, if R2 = 0 (now defined in 6q. then the output does not depend on the orthogonal coordinates, and will need to build a strong gradient in input space along the parallel coordinate to fit these two data. This strong gradient will exist throughout that entire decision boundary, improving performance but also increasing Gy. Instead, if Ry- = 1. fitting these two data will not lead to strong gradient, since they can be far from each other in input space. Beyond this intuition, in this model decreasing Ry can quantitatively be shown to increase performance, Paccolat et al. (2021b).

---

8 Conclusion
We have introduced novel empirical framework to characterize how deep nets become invariant to diffeomorphisms. It is jointly based on a maximum- maximum-entropy distribution for diffeomorphisms, and on the realization that stability of these transformations relative to generic ones Ry strongly correlates to performance, instead of just the diffeomorphisms stability considered in the past.
The ensemble of smooth deformations we introduced may have interesting applications. It could serve as a complement to traditional data-augmentation techniques (whose effect on relative stability discussed in Fig. 12 of the Appendix). A similar idea present Hauberg et al. (2016); Shen et al. (2020) but our deformations have the advantage of being easier to sample and data agnostic. Moreover, the ensemble could be used to build adversarial attacks along smooth transformations, in the spirit of Alaifari et al. (2018); Engstrom et al. (201 1); Kanbak et al. (2018) 8). It would be interesting to test if networks robust such attacks are more stable in relative terms, and how such robustness affects their performance.
Finally, the tight correlation between relative stability Ry and test error e, suggests that if predictor displays a given Ry, its performance may be bounded from below. The relationships we observe e ( may then be indicative of this bound, which would be a fundamental property of given data set. Can it be predicted in terms of simpler properties of the data? Introducing simplified models of data with controlled stability to diffcomorphisms beyond the toy model of Section would be useful to investigate this key question.
Acknowledgements
We thank Alberto Bietti, Joan Bruna, Francesco Cagnetta, Pascal Frossard, Jonas Paccolat, Antonio Sclocchi and Umberto M. Tomasini for helpful discussions. This work supported by grant from the Simons Foundation (#454953 Matthieu Wyart).
References
Alaifari, R., Alberti, G. S., and Gauksson, T. (2018). ADef: an Iterative Algorithm to Construct Adversarial Deformations.
Alcorn, M. A., Li, Q., Gong, Z., Wang, C., Mai, L.., Ku, W.-S., and Nguyen, A. (2019). Strike (With) Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects. In 2019 JEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1840-4849 Long Beach, CA, USA. IEEE.
Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., Casper, J., Catanzaro, B., Cheng, Q., Chen, G., al. (2016). Deep speech 2: End-to-end speech recognition in english and mandarin. In International conference on machine learning, pages 173-182.
Ansuini, A., Laio, A., Macke, J. H., and Zoccolan, D. (2019). Intrinsic dimension of data represen- tations deep neural networks. In Advances in Neural Information Processing Systems, pages 6111-6122.
Athalye, A., Engstrom, L., Ilyas, A., and Kwok, K. (2018). Synthesizing Robust Adversarial Examples. In International Conference on Machine Learning, pages 284-293. PMLR. ISSN: 2640-3498.
Azulay, A. and Weiss, Y. (2018). Why do deep convolutional networks generalize so poorly to small image transformations? arXiv preprint urXiv:1805. 12.77.
Bach, F. (2017). Breaking the curse of dimensionality with convex neural networks. The Journal of Machine Learning Research, 18(1):629-683
Beale, P. (1996). Statistical Mechanics. Elsevier Science.
Bietti, A. and Mairal, J. (2019a). Group invariance, stability to deformations, and complexity of deep convolutional representations. The Journal of Machine Learning Research, 20(1):876-224
10

---

Bietti, A. and Mairal, J. (2019b). On the inductive bias of neural tangent kernels. arXiv preprint arXiv:1905.1211 12173.
Bruna, J. and Mallat, S. (2013). Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence, 35(8):1872-1886
Chizat, L. and Bach, F. (2020). Implicit Bias of Gradient Descent for Wide Two-layer Neural Networks Trained with the Logistic Loss. In Conference on Learning Theory, pages 1305-1338.
PMLR. ISSN: 2640-3498.
Deng, J., Dong, W. Socher, R., Li. L., Kai Li, and Li Fei-Fei (2009). ImageNet: A large-scale hierar- chical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248 255. ISSN: 1063-6919.
Dieleman, S., De Fauw, J., and Kavukeuoglu, K. (2016). Exploiting cyclic symmetry in convolutional neural networks. arXiv preprint arXiv:1602.02660.
Engstrom, L., Tran, B., Tsipras, D.. Schmidt, L., and Madry, A. (2019). Exploring the Landscape of Spatial Robustness. In International Conference on Machine Learning, pages 1802-1811. PMLR. ISSN: 2640-3498.
Fawzi, A. and Frossard, (2015). Manitest: Are classifiers really invariant? In Procedings of the British Machine Vision Conference 2015, pages 106.1-106.13, Swansea. British Machine Vision Association.
Geiger, M., Petrini, L., and Wyart, M. (2021). Landscape and training regimes in deep learning. Physics Reports.
Geiger, M., Spigler, S., Jacot, A., and Wyart, M. (2020). Disentangling feature and lazy training in deep neural networks. Journal of Statistical Mechanics: Theory and Experiment, (120(11):11301 Publisher: IOP Publishing.
Ghorbani, B., Mei, S., Misiakiewicz, T., and Montanari, A. (2019). Limitations of lazy training of two-layers neural network. In Advances in Neural Information Processing Systems, pages 9111-9121.
Ghorbani, B., Mei, S., Misiakiewicz, T., and Montanari, A. (2020). When Do Neural Networks Outperform Kernel Methods? Advances in Neural Information Processing Systems, 33.
Hauberg, S., Freifeld, O., Larsen, A. B. L., Fisher J. W., and Hansen, L. K. (2016). Dreaming More Data: Class-dependent Distributions over Diffcomorphisms for Learned Data Augmentation. arX1v:1510.02795 [cs]. arXiv: 1510.02795.
He, K., Zhang, X., Ren, S., and Sun, (2016). Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Puttern Recognition (CVPR), pages 770-778. ISSN: 1063-6919.
Huval, B., Wang, Tandon, S., Kiske, J.. Song, W., Pazhayampallil,, J., Andriluka, M., Rajpurkar, P., Migimatsu, T., Cheng- Yue, R., et al. (2015). An empirical evaluation of deep learning on highway driving, arxiv preprint wrXiv:/504.00716
Jacot, A., Gabriel, F., and Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. In Proceedings of the 32Nd International Conference Neural Information Processing Systems, NIPS' 18, pages 8580-8589 USA. Curran Associates Inc.
Kanbak, C., Moosavi--ezzfolll S.,M., and Frossard, P. (2018). Geometric Robustness of Deep Networks: Analysis and Improvement. In 2018 IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 4441-4449, Salt Lake City, UT. IEEE.
Kardar, M. (2007). Statistical physics of fields. Cambridge University Press.
Kayhan, O. S. and Gemert, J. C. v. (2020). On translation invariance in cnns: Convolutional layers can exploit absolute spatial location. Proceedings of the IEFF/CVF Conference on Computer Vision and Pattern Recognition, pages 14274-14285.

---

Kopitkov, D. and Indelman, V. (2020). Neural Spectrum Alignment: Empirical Study. Artificial Neural Networks and Machine Learning ICANN 2020.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Technical report.
Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). Imagenet classification with deep convolu- tional neural networks. In Pereira, F., Burges, J. C., Bottou, I.,, and Weinberger, K. Q-. editors, Advances in Neural Information Processing Systems 25, pages 1097-1105. Curran Associates, Inc.
Le, Q. V. (2013). Building high-level features using large scale unsupervised learning. In 2013 IEEE international conference on acoustics, speech and signal processing, pages 8595-8598. IEEE
LeCun, Y., Bengio, Y., and Hinton, G. (2015). Deep learning. Nature, 521(7553):436.
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., and Jackel, D. (1989). Backpropagation Applied to Handwritten Zip Code Recognition. Neural Computation, 1(4):541-551.
Lecun, Y.. Bottou, L.,, Bengio, Y., and Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 (6(11))2278-2324 Conference Name: Proceedings of the IEEE.
Lee, J., Schoenholz, S. S., Pennington, J., Adlam, B., Xiao, L.,, Novak, R., and Sohl-Dickstein, J. (2020). Finite versus infinite neural networks: an empirical study. arXiv preprint (kdi::2007.188///
Loshchilov, I. and Hutter, F. (2016). SGDR: Stochastic Gradient Descent with Warm Restarts.
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. International journal of computer vision, 60(2)911110.
Luxburg, U. v. and Bousquet, O. (2004). Distance-based classification with lipschitz functions. Journal of Machine Learning Research, 5(un))669-6655
Mallat, S. (2016). Understanding deep convolutional networks. Philosophical Transactions of the Royal Society A: Mathematical,, Physical and Engineering Sciences, 374(2065):2015020..
Mnih, V. Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, L., Wierstra, D., and Riedmiller, M. (2013). Playing atari with deep reinforcement learning. arXiv preprint arXiv: 1312.5602.
Novak, R., Bahri, Y., Abolafia, A., Pennington, J., and Sohl-Dickstein, J. (2018). Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv: /rXiv:/802.0000 1802.08760.
Oymak, S., Fabian, Z., Li, M., and Soltanolkotabi, M. (2019). Generalization guarantees for neural networks via harnessing the low-rank structure of the jacobian. arXiv preprint arXiv:1906.00 39..
Paccolat, Petrini, L., Geiger, M., Tyloo, K., and Wyart, M. (2021a). Geometric compression of invariant manifolds in neural networks. Journal of Statistical Mechanics: Theory and Experiment, 2021(4):044001, Publisher: IOP Publishing.
Paccolat, J., Spigler, S., and Wyart, M. (2021b). How isotropic kernels perform on simple invariants. Machine Learning: Science and Technology, 2(2):025020. Publisher: IOP Publishing.
Recanatesi, S., Farrell, M., Advani, M., Moore, T., Lajoie, G., and Brown, E. (2019). Dimen- sionality compression and expansion in deep neural networks. arXiv preprint arXiv:1906.0044 00443.
Refinetti, M., Goldt, S., Krzakala, F,, and Zdeborová, L. (2021). Classifying high-dimensional gaussian mixtures: Where kernel methods fail and neural networks succeed. arXiv preprint 1r2iv:2102.11142.
Ruderman, A., Rabinowitz, N. C., Morcos, A. S., and Zoran, D. (2018). Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs. arXiv:1804.0444 [cs, stat]. arXiv: 1804.04438.
Saxe, A. M., Bansal, Y., Dapello. J., Advani, M., Kolchinsky, A., Tracey, B. D., and Cox, D. D. (2019). On the information bottleneck theory of deep learning. Journal of Statistical Mechanics: Theory and Experiment, 2019(1)):22020)
12

---

Shen, Z., Xu, Z.. Olut, S., and Niethammer, M. (2020). Anatomical Data Augmentation via Fluid- Based Image Registration. In Martel, L., Abolmaesumi, P., Stoyanov, D., Mateus, D., Zuluaga. M. A., Zhou, S. K., Racoceanu, D., and Joskowicz, I.., editors, Medical Image Computing and Computer Assisted Intervention MICCAI 2020, Lecture Notes in Computer Science, pages 318-328 Cham. Springer International Publishing.
Shi, B., Bai, X., and Yao, C. (2016). An end-to-end trainable neural network for image- baage- sequence recognition and its application to scene text recognition. IEEE transactions on pattern analysis and machine intelligence, 19(11):2298-2)
Shwartz-Ziv, R. and Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv preprint arXiv: /00.:0000000000000
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker,
L., Lai, M., Bolton, A., et al. (2017). Mastering the game of go without human knowledge. nature, 550(7676):354-33 359.
Simonyan, K. and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR.
Tan, M. and Le, Q. (2019). EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In International Conference Machine Learning, pages 6105-6114, PMLR. ISSN: 2640-3498.
Tsuzuku, Y. and Sato, (2019). On the structural sensitivity of deep convolutional networks to the directions of fourier basis functions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 51-60.
Xiao, C., Zhu, J.-Y., Li, B., He, W., Liu, M., and Song, D. (2018). Spatially Transformed Adversarial Examples.
Xiao, H., Rasul, K., and Vollgraf, R. (2017). Fashion-MNIST: Novel Image Dataset for Bench- marking Machine Learning Algorithms. arXiv:1708.0772: [cs, star]. arXiv: 1708.07747.
Yehudai, G. and Shamir, O. (2019). On the power and limitations of random features for understanding neural networks. In Advances in Neural Information Processing Systems, pages
Yin, D., Lopes, R. G., Shlens, J., Cubuk, D., and Gilmer, J. (2019). A fourier perspective on model robustness in computer vision. arXiv preprint arXiv:1906.0m 08988.
Zhang, R. (2019). Making convolutional networks shift-invariant again. arXiv: 1186.
arXiv preprint
13


Reviews:
[{"review_id": "wqyZj85fG-2", "reviewer": "Reviewer_YhtQ", "summary": "This paper introduces a new quantity called \"relative stability\" that strongly correlates with the classification performance of state of the art architectures on standard datasets. This quantity is a ratio between the variation of the features with respect to diffeomorphic transformation of the images and simple additive transformations. \nTo do so, the authors propose to use a parametrized set of diffemorphisms with bounded L2 norm of the derivative. \nThe dependence on the training set is highlighted, in particular a given amount of data is necessary to see the \"relative stability\" decrease with the size training set.\nLast, a simple synthetic experiment is proposed that illustrates the expected behaviour.\n\n\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The main contribution of this work is twofold: (1) highlight the fact that stability to diffeomorphisms perturbations (or more precisely small deformation setting here) is not observed when increasing the dataset size. (2) introduce a quantity that is shown experimentally to decrease with the training size, at least for more advanced architectures.\n\nThe writing of the paper is clear and the main message (the relative stability indicator and its link with classification error) is new to my knowledge.\n\nMy opinion is that this contribution is definitively interesting however, it lacks a clear application, for instance in data augmentation, or a better theoretical understanding for a venue such as NeurIPS. \n\n\nOther: \n- There are many ways to generate diffeomorphisms (see references below and references therein). These have been used for data augmentation as is proposed in the conclusion of this paper.\n- some references are missing:\n   -- \"Dreaming More Data: Class-dependent Distributions\nover Diffeomorphisms for Learned Data Augmentation\", by Hauberg et al.\n   -- Anatomical Data Augmentation via Fluid-Based Image Registration, Shen et al.\n\n\nTypos:\n- page 5: \"the an inverse\"", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Since the work is essentially descriptive in nature there is not likely direct negative societal impact.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3.5"}, "extra_text": {}}, {"review_id": "i1KiUVc2JSC", "reviewer": "Reviewer_cGvy", "summary": "This work contributes a framework for characterizing the invariance of image classifiers to diffeomorphisms. It is based on the empirical index $R_f$, which measures the stability to random diffeomorphisms relative to stability to additive noise. The authors find that $R_f$ is strongly correlated with classifier performance for a wide array of image classifiers. The results suggest that an image classifier's generalization performance is strongly related to its invariance to certain diffeomorphisms, as opposed to additive noise.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The maximum entropy distribution of diffeomorphisms as well as the relative stability index $R_f$ were clearly explained, and the various figures made the ideas very easy to understand intuitively. The paper then thoroughly investigates the correlation between error and $R_f$ from various angles. In particular, the fact that $R_f$ is computed relative to noise makes the importance of diffeomorphisms more convincing. This paper has the potential to inspire further study of invariance to diffeomorphisms in deep nets, and the maximum entropy diffeomorphism distribution/$R_f$ measure are likely to be of independent interest for future work on data augmentation and adversarial robustness.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "UAhwzYKCex4", "reviewer": "Reviewer_gEsS", "summary": "This paper investigates the role of stability to the action of diffeomorphisms in the classification performances of deep neural networks. Surprisingly, it shows that stability measurements are good indicators to compare the performance of deep neural network classifiers even for complex image data bases. The paper introduces stability measurements of diffeomorphism actions over images and shows that these measurement correlate strongly with the classification errors obtained by different CNN architectures over MNIST and CIFAR. The paper also shows that the stability to diffeomorphisms decreases when the training sizes decreases and gives a simple model to explain this phenomenon. ", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Stability to diffeomorphisms had been shown to have an important role  in order to establish appropriate similarity measures between images for classification, which are stable to deformations. However, has mostly been studied over non-convolutional neural networks based on wavelets or kernel representations. The remarkable element of this paper is to demonstrate that it is an important element present in all CNN for image classification. These results appear clearly in Figure 3 which demonstrates the increased stability to diffeomorphism actions of ResNEt et EfficientNet as the learning proceeds and in Figure 5 for CIFAR 10.  However, let us mention that the impressive result on Figure 5 is obtained for only one type of deformation for c=3 which is relatively lower frequency, one one data set (CIFAR 10).  However, larger displacement are a priori also valid and it would be interesting to verify wether the result holds when displacements are larger and with a larger c to see in what sense this property is robust.\n\nThe fact that learning stability to diffeomorphisms requires large data sets is also really interesting, specially when compared with stability with additive noise perturbations. The authors analysis indicating that precise network classifiers learns a sophisticated metrics which have large gradients and leading larger errors to additive white noise but small errors to action of diffeomorphisms is also very interesting. It is a first attempt to characterise the non-linear metrics constructed by such networks. In summary this is a very investing paper, demonstrating a new idea, that may lead to more fruitful research in this area.", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"ethical_concerns": "No", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4h"}, "extra_text": {}}, {"review_id": "BPeHfBsF7VO", "reviewer": "Reviewer_iL72", "summary": "The authors consider the relationship between the performance of modern deep\nneural network architectures trained on standard benchmark natural image\ndatasets and their relationship to stability with respect to smooth\ndeformations of the input image. These issues have been studied previously in\nthe context of (natural) adversarial examples and in theoretically-constructed\n(i.e. non-trainable) networks -- the motivation and contribution here is to\nexamine them through the lens of trained networks as well. To study this issue\nempirically, the authors introduce a maximum-entropy distribution on\ndiffeomorphisms, which (I believe) amounts to constructing the deformation\nvector field as a (gaussian) random combination of periodic Fourier basis\nfunctions with frequency-dependent weights; they undertake an empirical\ninvestigation of the properties of this distribution in order to find parameter\nregimes where it produces realistic small-magnitude diffeomorphisms of the\nimage plane (rather than distorting it beyond recognition, or doing nothing).\nThey then use this tool to construct a computational \"relative diffeomorphism\nstability\" metric, which is the average stability of a predictor with respect to\nsuch diffeomorphisms divided by the predictor's average stability with respect\nto scale-appropriate gaussian noise perturbations, called $R_f$. With $R_f$,\nseveral findings are reported: across numerous modern architectures (ResNet18;\nEfficientNet) and datasets (CIFAR10; ImageNet), randomly-initialized networks\nhave $R_f$ near unity and it decreases after training; $R_f$ decreases with the\nsize of the training set (measured by subsampling); and test set performance\ncorrelates well with $R_f$ after training in the context of the CIFAR10\ndataset. The authors present a toy theoretical model to give a theoretical\ncorroboration of the $R_f$ parameter's behavior for a shallow network, and\nprovide some ablation studies in the context of presence or not of data\naugmentation in the training process and the type of interpolation used to\napply the diffeomorphisms.\n\n", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "## Summary wrt Decision\n\nThe paper studies an important problem in deep learning practice/theory that\nseems to be somewhat underrepresented in the literature, namely the\nrelationship between a well-trained deep net's performance and its stability to\ncertain transformations of the input that specifically respect the structure of\nnatural image data. Progress on this issue may lead to new developments and\ninsights in (say) robust vision applications, in a way that other\ntheoretical/experimental investigations that do not specifically consider the\nstructure of the data in these applications may not. I have rated the paper as\nI have due to certain confusions I have about the authors' interpretations of\ntheir results and their significance; I would tend to increase the rating to\nborderline accept or more if these are resolved (either by correcting me or\nproposing clarifications) in the discussion phase.\n\nI will provide more detailed comments below.\n\n## Strengths\n- The paper is very well-referenced, and code is provided for reproducibility.\n  Most details are well-motivated theoretically, and the descriptive figures in\n  the early sections give a good intuitive sense of the image plane\n  diffeomorphisms that are being studied.\n- The max-entropy distribution over diffeomorphisms is simple to compute and\n  may well be useful more broadly as a tool for experimentation, as the authors\n  point out.\n- The theoretical component of the paper in section 6 is simple, but nontheless\n  a good illustration of the central ideas of the paper. It could also be a\n  good problem for future study (perhaps with deeper networks, more complicated\n  data models, etc.)\n\n## Issues\n\n- I am unclear about the broad significance of the authors' findings, beyond\n  the introduction of the maximum entropy distribution on diffeomorphisms as an\n  experimental tool. I will frame this issue in terms of my understanding of\n  the paper's motivation in the introduction -- understanding the relationship\n  between *average case* deformation stability of modern deep networks and\n  their generalization performance on benchmark datasets -- and on its stated\n  implications for future research in the conclusion (section 7). Although the\n  results in Figure 5 on correlation between $R_f$ and test error for CIFAR10 \n  are striking, and definitely more pronounced in terms of correlation\n  coefficient than the corresponding plots for diffeomorphism stability and\n  $\\ell^2$ perturbation in Figs 15 and 16 in the appendices, the trends in\n  Figures 15 and 16 are roughly the same -- the most performant modern\n  architectures find themselves in the bottom-left of the plots, with a rough\n  'upward' trend towards the top-right as one decreases the capacity of the\n  network (modulo some outliers and inconsistencies). As a result I find the\n  interpretation of these results in Sections 4 and 5 lacking -- section\n  4 does not provide any interpretation of the results in Figure 5 beyond\n  discussing some architectural/training factors, and the discussion in section\n  5 is mostly restricted to considerations with respect to $P$ (which does not\n  seem to be directly relevant to the results in Figure 5), and the important\n  discussion in lines 228-238 seems hard to parse (I would appreciate some\n  clarification here). It would be helpful if the authors interpreted these\n  results in the context of discussions of (say) clean vs.\\ robust accuracy in\n  the literature, or more broadly other implications for practice. Without such\n  interpretations, it is hard to see how the results in Figure 5 shed new light\n  on these issues and may lead to further research progress on these important\n  questions -- I note that the discussion of future directions in the\n  conclusion is almost all concerning possible uses for the diffeomorphism\n  ensemble, rather than implications of Figure 5.\n- I am not sure about the broad significance of the discussion and experiments\n  surrounding the training set size $P$. Although the stated conclusion that\n  large training sets are necessary to get a small $R_f$ seems reasonable and\n  useful, this is somewhat confounded by the fact that *diffeomorphism\n  stability* is not as well correlated with large $P$ (section 5). It is\n  therefore important to have a sense of the broad significance of a small\n  $R_f$, as in the previous bullet. Also, the comparisons to the theoretical\n  model of section 6 and the predicted $P^{-1}$ slope (Figure 3) might be\n  spurious, as the fit is only 'good' in a compact region of the plot, whereas\n  what really matters for such a claim to be accurate is the tail rate (and the\n  datasets being experimented with obviously do not allow one to simulate out\n  this far without relying on augmentations, which would not really serve the\n  purpose).\n- It would be nice to know if the authors' findings persist at larger dataset\n  scales: although experiments on ImageNet are shown in Figure 3, the\n  centerpiece Figure 5 (and corresponding plots in the appendices) are only for\n  CIFAR10 and not ImageNet.\n- It would be helpful throughout the paper if certain technical claims and\n  implementation details were specified precisely, either in the body or in the\n  appendices. On the technical side,  I found it difficult in\n  certain cases to interpret which claims in section 2 are rigorous\n  mathematical claims, which are heuristic (asymptotic; etc.) claims, and which\n  are empirical claims: for example it would be nice if the appendix had a\n  section summarizing all the calculations that are omitted in section 2, like\n  \"For large $n$, the norm then reads...\" at line 108; the definition of\n  $\\delta$ and \"It is straightforward to obtain...\" at line 126; \"that add\n  pre-asymptotic terms to Eq.3\" at line 128 (the fit seems not particularly\n  good in fig 8a?). These details seem quite important to specify, because \n  the introduction of the maximum entropy distribution on diffeomorphisms is\n  claimed as a contribution here, and the implementation of this distribution\n  seems from the discussion in section 2 and the appendix to have many finicky\n  aspects that may be challenging to navigate if applying these methods to a\n  new dataset (e.g. different image size or resolution or the like).\n  On the implementation side, although it seems to be made more or less clear clear\n  implicitly via Fig 2, the later discussions, and section D, it would be\n  helpful in section 3 if it was stated exactly what $\\eta$ is (I think it is\n  additive independent gaussian noise on the image pixels, but there are\n  certain vague/confusing statements around this term (line 138 \"all random\n  directions $\\eta$\"; figure 2 caption \"For large $n$, it is equivalent to\n  adding an i.i.d.\\ Gaussian noise to all the pixel values of $x$\" (one would\n  like to know **what is being implemented** in the experiments, rather than\n  how this fits into a theoretical formalism here)) that confound this\n  interpretation.\n\n## Questions, etc.\n\n- In many specific vision applications, the types of diffeomorphism one is\n  interested in obtaining invariance to with a deep net are far more structured\n  than the generic near-identity transformations considered here: for example,\n  in a registration application with a piecewise planar object undergoing a\n  rigid body motion the transformations are homographies (e.g. [1]) and\n  actually carry a low-dimensional structure. I understand the focus here is on\n  generic transformation semi-invariance and how this relates to deep learning\n  performance on standard benchmark datasets, but I am interested in whether\n  the authors have considered measuring invariances to these types of\n  structured transformations as well, and/or whether the maximum entropy\n  formalism can be extended to this kind of setting. Such an extension could be\n  of significant interest in practice, beyond generic 'small' diffeomorphism\n  invariance.\n\n## Minor issues (polishing, organization, etc.)\n\n- Throughout section 3, following the definition of $R_f$, the authors\n  repeatedly conflate having a small $R_f$ value with \"stability to\n  diffeomorphisms\". It would seem to be more appropriate to add the qualifier\n  \"relative\" in these cases, especially given the later claim (section 5) that\n  stability to diffeomorphisms is \"...not the good observable to characterize\n  how deep nets learn invariance toward diffeomorphisms...\"! In a similar vein,\n  I found it somewhat jarring that $R_f$ was introduced at the start of section\n  3 without a discussion of the natural complaint that this relative measure\n  can be lowered significantly just by increasing the denominator -- given the\n  prominence of $R_f$ in the results that will be interpreted in section 3\n  immediately after this definition, I would think it appropriate to quickly\n  mention at the end of the \"Relative stability to diffeomorphisms\" paragraph\n  in section 3 that this issue will be discussed in detail later, in section 5\n  (and perhaps foreshadow the conclusion, so the reader can trust that this is\n  an interesting measure and process the figures in section 3).\n- In lines 142-150, the choice $\\delta = 1$ seems to me to be incompletely\n  justified -- isn't the interpolation independence only significant for\n  Resnet, and not EfficientNet? I might have read the figure wrong.\n- The paragraph in lines 228-238 is somewhat difficult to parse; superficially\n  it might be good to use a word other than \"gradient\" in this context since it\n  is easily confused with the gradients one is using in training? (What seems\n  to be meant here is large changes in the output of the predictor as a\n  function of input; of course these can happen without a gradient existing.)\n\n[1] http://arxiv.org/abs/1012.3216\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "5"}, "extra_text": {}}]

Author rebuttal:

Meta-review: All four knowledgeable reviewers recommend accepting this submission. I agree. This submission makes a valuable contribution by demonstrating the invariance of image classifiers to diffeomorphism.
Decision: Accept (Poster)