How deep convolutional neural networks lose spatial information with training
Authors: Umberto Maria Tomasini
Leonardo Petrini
Francesco Cagnetta
Matthieu Wyart
Venue: Submitted to ICLR 2023 (2023)
Keywords: Deep Learning Theory
Convolutional Neural Networks
Curse of Dimensionality
Representation Learning
Feature Learning
Computer Vision
Pooling
Stability
Diffeomorphisms
Gaussian noise
Image Classification
Learning Invariants

Abstract: A central question of machine learning is how deep nets  manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building a representation of the data where information  irrelevant to the task is lost. For image data sets, this view is supported by the observation that after (and not before) training,  the neural representation becomes less and less sensitive to diffeomorphisms acting on images as the signal propagates through the net.  
This loss of sensitivity correlates with performance, and surprisingly  also correlates  with a gain of sensitivity to white noise acquired during training. These facts are unexplained, and as we demonstrate still hold when white noise is added to the images of the training set. Here, we (i) show empirically for various architectures that stability to image diffeomorphisms is achieved by spatial pooling in the first half of the net, and by channel pooling in the second half, (ii) introduce a scale-detection task for a simple model of data where pooling is learnt during training, which captures all empirical observations above and (iii) compute in this model how stability to diffeomorphisms and noise scale with depth. The scalings are found to depend on the presence of strides in the net architecture. We find that the increased sensitivity to noise is due to the perturbing noise piling up during pooling, after a ReLU non-linearity is applied to the noise in the internal layers.

Under review as a conference paper at ICLR 2023
HOW DEEP CONVOLUTIONAL NEURAL NETWORKS LOSE SPATIAL INFORMATION WITH TRAINING
Anonymous authors Paper under double-blind review
ABSTRACT
A central question of machine learning is how deep nets manage to learn tasks in high dimensions. An appealing hypothesis is that they achieve this feat by building representation of the data where information irrelevant to the task is lost. For image datasets, this view is supported by the observation that after (and not before) training, the neural representation becomes less and less sensitive to diffeomorphisms acting on images as the signal propagates through the net. This loss of sensitivity correlates with performance, and surprisingly correlates with a gain of sensitivity to white noise acquired during training. These facts are unexplained, and as we demonstrate still hold when white noise is added to the images of the training set. Here, we fi) show empirically for various architectures that stability image diffeomorphisms is achieved by both spatial and channel pooling, (i)) introduce a model scale-detection task which reproduces our empirical observations spatial pooling and (iii) compute analitically how the sensitivity diffeomorphisms and noise scales with depth due to spatial pooling, The scalings are found to depend on the presence of strides in the net architecture. We find that the increased sensitivity to noise is due to the perturbing noise piling up during pooling, after being rectified by ReLU units.
INTRODUCTION
Deep learning algorithms can be successfully trained solve large variety of tasks (Amodei et al., 2016; Huval et al., 2015; Mnih et al., 2013; Shi et al., 2016; Silver et al., 2017), often revolving around classifying data in high-dimensional spaces. If there was little structure in the data, the learning procedure would be cursed by the dimension of these spaces: achieving good performances would require an astronomical number of training data (Luxburg Bousquet, 2004). Consequently, real datasets must have a specific internal structure that can be learned with fewer examples. It has been then hypothesized that the effectiveness of deep learning lies its ability of building 'good' representations of this internal structure, which are insensitive to aspects of the data not related to the task (Ansuini et al., 2019; Shwartz-Ziv & Tishby, 2017; Recanatesi et al., 2019), thus effectively reducing the dimensionality of the problem.
In the context of image classification, Bruna & Mallat (2013); Mallat (2016) proposed that neural networks lose irrelevant information by learning representations that are insensitive to small defor- mations of the input, also called diffeomorphisms. This idea was tested in modern deep networks by Petrini et al. (2021), who introduced the following measures
Ee,- ([(())) f(x)112 Dj = E,, (l) - f(x2))
Ey.n 1))
Dj Gy
G = E E [lf((1)) - (x2))
(1)
to probe the sensitivity function fithe the output or an internal representation of a trained network-to to random diffeommrphisms of x (see example in Fig. 1, left), to large white noise perturbations y of magnitude [|-(x) - ær|, and in relative terms, respectively. Here the input images r, I and 22 sampled uniformly from the test set. In particular, the test error of trained networks is correlated with D, when f is the network output. Less intuitively, the test error is anti-correlated with the sensitivity to white noise Gy. Overall, it is the relative sensitivity R, which correlates best with the error (Fig. 1, middle). This correlation is learned over training- as it is not seen at initialization-ann and built up layer by layer (Petrini et al., 2021). These phenomena are not simply due

---

Under review as a conference paper at ICLR 2023
teas error E
MacNel
pic,
10
Dj
1/Gr
R2
10-1
10
10-1
nnise magnitude f
10
r7
Figure Left: example a random diffeomorphism т applied to an image. Center: test er- vs relative sensitivity diffeomorphisms of the predictor for set of networks trained CIFARIO, adapted from Petrini et al. (2021). Right: Correlation coefficient between test er- and Dy. Gy and R when training different architectures noisy CIFARIO, X(. Cov c, log X)/ VVr(llo c) Var(log X). Increasing noise magnitudes are shown on the æ-axis and 17" E4, (xx) zII² the one used for the computation of Gy. Samples of a noisy CIFARIO datum are shown on top. Notice that D, and particularly R are positively correlated with whilst
Gy negatively correlated with c. The corresponding scatter plots are in Fig. 10 (appendix).
(a) Spatial pooling
(b) Channel pooling
1.1
1
input
1.3
8
7
5
0.2
avg. pooling
7
6.5
4
9
5
size 2x2
6
6.5
2
9
3
stride
0.2
rotated
input x
1.1
Figure 2: Spatial vs. channel pooling, (a) Spatial average pooling (size 2x2, stride 1) computed on representation of size 3x3. One can notice that nearby pixel variations are smaller after pooling, (b) If the filters of different channels are identical up to e.g. rotation of angle 6, then, averaging the output of the application of such filters makes the result invariant input rotations of 6. This averaging is an example of channel pooling.
benchmark data being noiseless, as they persist when input images are corrupted by some small noise (Fig. 1, right).
Operations that grant insensitivity to diffcomorphisms in deep network have been identified previously (e.g. Goodfellow et al. (2016), section 9.3, sketched in Fig. 2). The first, spatial pooling, integrates local patches within the image, thus losing the exact location of its features. The second, channel pooling, requires the interaction of different channels, which allows the network to become invariant to any local transformation by properly learning filters that are transformed versions of one another. However, it is not clear whether these operations are actually learned by deep networks and how they conspire in building good representations. Here we tackle this question by unveiling empirically the emergence of spatial and channel pooling, and disentangling their role. Below is detailed list our contributions.
1.1 OUR CONTRIBUTIONS
We disentangle the role of spatial and channel pooling within deep networks trained CIFARIO (Section 2). More specifically, our experiments reveal the significant contribution of spatial pooling in decreasing the sensitivity to diffcomorphisms.
2

---

Under review as a conference paper at ICLR 2023
In order to isolate the contribution of spatial pooling and quantify its relation with the sensitivities to diffeomorphism and noise, we introduce idealized scale-detection tasks (Section 3). In these tasks, data are made of two active pixels and classified according to their distance. We find the same correlations between test error and sensitivities of trained networks as found in Petrini al. (2021). ). In addition, the neural networks which perform the best on real data tend to be the best on these tasks.
We theoretically analyze how simple CNNs, made by stacking convolutional layers with filter size F and stride learn these tasks (Section 4). We find that the trained networks perform spatial pooling for most of its layers. We show and verify empirically that the sensitivities D2 and Ga of the k-tth hidden layer follow Gy As and Dş Ag " where Ag is the effective receptive field size and 0g 2 if there is no stride, 02 = 1 otherwise.
The code and details for reproducing experiments are available online at sinyurl.com/gee githhub experiments.
1.2 RELATED WORK
In the neuroscience literature, the understanding of the relevance of pooling in building invariant representations dates back to the pioneering work of Hubel & Wiesel (1962). By studying the cat visual cortex, they identified two different kinds of neurons: simple cells responding e.g. edges at specific angles and complex cells that pool the response of simple cells and detect edges regardless of their position orientation in the receptive field. More recent accounts of the importance of learning invariant representations in the visual cortex can be found in Niyogi et al. (1998); Anselmi et al. (2016); Poggio & Anselmi (2016).
In the context of artificial neural networks, layers jointly performing spatial pooling and strides have been introduced with the early CNNs of Lecun et al. (1998), following the intuition that local averaging and subsampling would reduce the sensitivity to small input shifts. Ruderman et al. (2018) 8) investigated the role of spatial pooling and showed empirically that networks with and without pooling layers converge to similar deformation stability, suggesting that spatial pooling can be learned in deep networks. In our work, we further expand in this direction by jointly studying diffeomorphisms and noise stability and proposing theory of spatial pooling for simple task.
The depth-wise loss of irrelevant information deep networks has been investigated by means of the information bottleneck framework (Shwartz-Ziv & Tishby, 2017; Saxe et al., 2019) and the intrinsic dimension the networks internal representations (Ansuini et al., 2019; Recanatesi et al., 201 9). However, these works do not specify what is the irrelevant information to be disregarded, nor the mechanisms involved in such a process.
The stability of trained networks to noise is extensively studied in the context of adversarial robust- ness (Fawzi Frossard, 2015; Kanbak et al., 2018; Alcorn et al., 2019; Alaifari et al., 2018; Athalye al., 2018; Xiao et al., 2018 Engstrom et al., 2019). Notice that our work differs from this literature by the fact that we consider typical perturbations instead of worst-case ones.
2 EMPIRICAL OBSERVATIONS ON REAL DATA
In this section we analyze the parameters of deep CNNs trained on CIFARIO and ImageNet, so to understand how they build representations insensitive to diffeomorphisms (details of the experiments in App. B). The analysis builds on two premises, the first being the assumption that insensitivity is built layer by layer in the network, as shown in Fig. 3. Hence, we focus on how each of the layers in deep network contribute towards creating an insensitive representation, More specifically, let us denote with fa the internal representation of an input at the k-th layer of the network. The entries of fú have three indices, one for the channel e and two for the spatial location (i,j). The relation between fa and fx-1 is the following.
[fa( = (
Hy-1
E
wae Pis (Lfk ((F))
Ve= I,.., Ha.
(2)
3

---

Under review as a conference paper at ICLR 2023
where: H3 denotes the number of channels at the k-th layer; by and w the biases and filters of the k-tth layer; each filter w c,c' a F x F matrix with F the filter size; Pi, ([fe-1())),)) denotes a F x F-dimensional patch of [ centered at (i. @ the activation function. The second premise is that general diffeomorphism can be represented as a displacement field over the image, which indicates how each pixel moves in the transformation. Locally, this displacement field can be decomposed into a constant term and a linear part: the former corresponds to local translations, the latter to stretchings, rotations and shears.
Invariance to translations via spatial pooling. Due to weight sharing, i.e. the fact that the same filter w²,e is applied to all the local patches (i.j) of the representation, the output of a convolutional layer is equivariant to translations by construction: a shift of the input is equivalent to a shift of the output. To achieve an invariant representation it suffices to sum up the spatial entries of fa-an an operation called pooling in CNNs, we refer to it as spatial pooling to stress that the sum runs over the spatial indices of the representation. Even if there are no pooling layers at initialization, they can be realized by having homogeneous filters, i.e. all the F x F entries of wh+ are the same. Therefore, the closer the filters are to the homogeneous filter, the more they decrease the sensitivity of the representation to local translations.
k+1
Invariance to other transformations via channel pooling, The example of translations shows that building invariance can be performed by constructing an equivariant representation, and then pooling it. Invariance can also be built by pooling across channels. A two-channel example is shown Fig. 2. panel (b), where the filter of the second channel is built so as to produce the same output as the first channel when applied to the other components of diffeomorphisms-Sa such as local stretchings and shears. Below, we refer generically to any operation that build invariance to diffeomorphisms by assembling distinct channels as channel pooling.
rotated input. The same idea can be applied more generally, e.g.
Disentangling spatial and channel pooling. The relative sensitivity to diffeomorphisms Rg of the k-th layer representation f& decreases after each layer, as shown in Fig. 3. This implies that spatial or channel pooling are carried out along the whole network. To disentangle their contribution we perform the following experiment: shuffle at random the connections between channels of successive convolutional layers, while keeping the weights unaltered. Channel shuffling amounts to randomly permuting the values of c, c' in Eq. 2, therefore it breaks any channel pooling while not affecting single filters. The values of R3 for deep networks after channel shuffling are reported in Fig. 3 as dashed lines and compared with the original values of Rg in full lines. If only spatial pooling was present in the network, then the two curves would overlap. Conversely, if the decrease in Rg was all due to the interactions between channels, then the shuffled curves should be constant. Given that neither of these scenarios arises, we conclude that both kinds of pooling are being performed.
Emergence of spatial pooling after training. To bolster the evidence for the presence of spatial pooling, we analyze the filters of trained networks. Since spatial pooling can be built by having homogeneous filters, we test for its presence by looking at the frequency content of learned filters wf,,, In particular, we consider the average squared projection of filters onto "Fourier modes" {Vi)t= taken the eigenvectors of the discrete Laplace operator on the F x F filter grid. The square projections averaged over chamnels read
Ha H
The1 H
M
w
c.e' 12
and are shown in Fig. 4, and 2dd row. When training a deep network such VGG11 (with and without batch-norm) (Simonyan & Zisserman, 2015) on CIFARIO, filters of layers 2 to 6 become low-frequency with training, while layers 1, 7, 8 do not. Accordingly, larger gaps between dashed and full lines in Fig. 3 (right) open at layer 1, 7, 8: reduction in sensitivity is not due to spatial pooling in these layers. Moreover, the fact that the two dashed curves overlap is consistent with the frequency
'The displacement field around a pixel (wo, vo) is approximated as r(u, v) ~ T(uo, vo) + Juo, up) No. ro) where 7(u0, no) corresponds to translations and J is the Jacobian matrix of 7 whose trace, antisymmetric and symmetric traceless parts correspond stretchings, rotations and shears, respectively.
4

---

Under review as a conference paper at ICLR 2023
content of filters being the same for the two architectures after training. In the case of ImageNet, filters at all layers become low-frequency, except for 1.
10
VGG11
original
vgg11 vggl lbn vgg13
vgg16 original shuffie
shuffl e ch.
10a
Rx
CIFAR10
10 1
ImageNet
7
8
0.0
0.2
0.4
0.6
0.8
1.0
convolutional layer index
relative depth &
Figure 3: Relative sensitivity Rs as function of depth for VGG architectures trained CIFARIO (left) and ImageNet (right). Full lines refer to the original networks, dashed lines to the ones with shuffled channels. K is the total depth of the networks. Experiments with different architectures reported in App. C.
k=1 1
k ==
k=3
k
k=5
k=6
k k=7
k=
CIFARIO ImageNet
10-1
Vk.1
10-1
VGG11 VG61 1bn
iiit
10-1
trained
2x
10-1
eigenvalue index,
Figure 4: Projections of the network filters for VGG11 and VGG1 1bn onto the 9 eigenvectors of the (3 x 3)-grid Laplacian when training on CIFARIO (1 row), ImageNet, (2nd row) and the scale-detection task (3ad row): dotted and full lines correspond initialization and trained networks, respectively. The x-axis reports low to high frequencies from left to right. Deeper layers are reported in rightmost panels. Low-frequency modes are the dominant components in layers 2-6 when training on CIFAR10, in layers 2-8 for ImageNet. The first (constant) mode has most of the power throughout the network for scale-detection task 1. An aggregate measure of the spatial frequency content of filters is reported in App. C, Fig. 12.
SIMPLE SCALE-DETECTIO TASKS CAPTURE REAL-DATA OBSERVATIONS
To sum up. the empirical evidence presented in Section 2 indicates that (i) the generalization performance of deep CNNs correlates with their insensitivity to diffeomorphisms and sensitivity to Gaussian noise (Fig. 1); (ii) deep CNNs build their sensitivities layer by layer via spatial and channel pooling. We introduce now two idcalized scale-detection tasks where the phenomena (i) and (ii) emerge again, and these tasks with respect to real data, we can understand quantitatively how spatial pooling builds up insensitivity to diffeomorphisms and sensitivity Gaussian noise, as we show in Section 4.
isolate the contribution of spatial pooling. Given the simpler structure of
Definition of scale-detection tasks. empty background.
Consider input images x consisting of two active pixels on an
Task 1: Inputs are classified by comparing the euclidean distance d between the two active pixels and some characteristic scale &, in Fig. 5, left. Namely, the label is y sign (5 d).
5

---

Under review as a conference paper at ICLR 2023
y
sigu(
sign(g d) +1
Different patches,
Same patch, y
(b)
Figure 5: Example inputs for the scale-detection tasks. Task 1 (a): the label depends on whether the euclidean distance d is larger (left) smaller (right) than the characteristic scale Task (b): the label depends on whether the active pixels belong to the same patch of size & (right) or not patche- shown different colors.
Notice that a small diffeomorphism of such images corresponds to a small displacement of the active pixels. Specifically, each of the active pixels is moved to either of its neighboring pixels or left in its original position with equal probability." 2 By introducing a gap g such that d E - E + 9/2], task becomes invariant to displacements of size smaller than g. Therefore, we expect that neural network trained on task will lose any information on the exact location of the active pixels within the image, thus becoming insensitive to diffeomorphisms. scale is the most direct mean achieve such insensitivity. The result of the integration depends on whether none, one or both the active pixels lie within the pooling window, thus it is still informative of the task. We will show empirically that this is indeed the solution reached by trained CNNs.
9/2,
Intuitively, spatial pooling up to the
Task 2: Inputs are partitioned into nonoverlapping patches of size &, as in Fig. 5, right. The label y is +1 if the active pixels fall within the same patch, 1 otherwise.
In task 2, the irrelevant information is the location of the pixels within each of the non-overlapping patches. The simplest means to lose such information requires to couple spatial pooling with a stride of the size of the pooling window itself.
Same phenomenology as in real image datasets. Although these scale-detection tasks are much simpler than standard benchmark datasets, deep networks trained on task 1 display the same phenomenology highlighted in Section for networks trained on CIFAR1O and ImageNet. First, the test error is positively correlated with the sensitivity to diffeomorphisms of the network predictor (Fig. left panel, in App. C) and negatively correlated with its sensitivity to Gaussian noise (middle panel) for whole range of architectures. As a result, the error correlates well with the relative sensitivity Ry (right panel). Secondly, the internal representations of trained networks fa become progressively insensitive to diffeomorphisms and sensitive to Gaussian noise through the layers, as shown in Fig. 9 of App. C. Importantly, the curves relating sensitivities to the relative depth remain essentially unaltered if the channels of the networks are shuffled (shown as dashed lines in Fig. 9). We conclude that, on the one hand channel pooling is negligible, and, on the other hand, all channels approximately equal the mean channel. Finally, direct inspection of the filters (Fig. 4, bottom row) shows that the O-frequency component grows much larger than the others over training for layers 1-7, which are the layers where Rg decreases the most in Fig. Filters are thus becoming nearly homogeneous, which means that the convolutional layers become effectively pooling layers.
4 THEORETICAL ANALYSIS OF SENSITIVITIES SCALE-DETECTION TASKS
We now provide a scaling analysis of the sensitivities to diffeomorphisms and noise in the internal representations of simple CNNs trained on the scale-detection tasks of Section It allows to quantitatively understand how spatial pooling makes the internal representations of the network progressively more insensitive to diffeomorphisms and sensitive to Gaussian noise.
We fix the length of these displacements to pixel hecause (t) is the smallest value that prevents the use of pixel interpolation, which would make one active pixel an extended object ((i) allows for the analysis of Section 4.

---

Under review as a conference paper at ICLR 2023
(a)
tileer sizo
5, strido
(b) filter size 2, stride
ingut
layer
layes
layer
layer
layer
layer
2layer
Figure 6: Hidden layers representations of simple CNNs for a scale-detection input for stride s 1 and filter size F ="=5 5 (left) and s == F= 2 (right) when having homogeneous filters at every layer. The effective receptive field size of the last layer in the two different cases is shown in red. (Left) every active pixel in the input becomes Gaussian profile whose width increases throughout the network. (Right) every neuron in layer & has activity equal to the number of active pixels which are present in its receptive field of width 2*, The dark blue in the last layer indicates that there are 2 active pixels in its receptive field, while the lighter blue of the precedent layers indicates that there is just 1.
Setup. We consider simple CNNs made by stacking K identical convolutional layers with generic filter size F, stride s s=1 or F and ReLU activation function ø(x) max (0, .z). In particular, we train CNNs with stride 1 on task and CNNs with stride F on task 2. For the sake of simplicity, we consider the dieeeinnnio version of the scale-detection tasks, but our analysis carries unaltered to the two-dimensional case. Thus, input images are sequences
I, 0 for all pixels except two. For the active pixels 2; = VL/2, (zi))= of L pixels, where that all input images have - We will also consider single dingle-pixe data 8, (6,,a)i=1....... If the active pixels in x are the i-th and the j-th, then x == LL/2 (5, + 5;). For each layer k, the internal representation (ar) of the trained network is defined as in Eq. 2. The receptive field of the k-th layer is the number of input pixels contributing to each component of (z)).. We define the effective receptive field Ag as the typical size of the representation of single-pixel input, fa (6;), as illustrated in red in Fig, 6. We denote the sensitivities of the k-th layer representation with a subscript (D₂ for diffeomorphisms, G3 for noise, R3 for relative).
Assumptions. All our results are based on the assumption that the first few layers of the trained network behave effectively as a single channel with a homogeneous positive filter and no bias. The equivalence of all the channels with their mean is supported by Fig. 9, which shows how shuffling channels does not affect the internal representations of VGGs. In addition, Fig. (bottom row) shows that the mean filters of the first few layers nearly homogeneous. We set the homogeneous value of each filter so as to keep the norm of representations constant over layers. Moreover, we implement a deformation of the input æ of our scale-detection tasks random displacement of each active pixel at either left or wight with probability 1/2.
4.1 TASK STRIDE
For a CNN with stride 1, under the homogeneous filter assumption, the size of the effective receptive field Aş grows as VT. A detailed proof is presented in App. A and Fig. 6, left panel, shows an illustration of the process. Intuitively, applying homogeneous filter to a representation equivalent to making each pixel diffuse, i.c. distributing its intensity uniformly over neighborhood of size F. With a single-pixel input 6, the effective receptive field of the k-th layer (6. is equivalent to k-step diffuusinn of the pixel, thus it approaches a Gaussian distribution of standard deviation Vk centered at i. The size A& is the standard deviation, thus A Vk. The proof we present in App. A requires large depth K >> and large image width L > FK1/2 and the empirical studies of Section 3 satisfy these contraints (F ~ 3, L 32 and K 10).
We remark that at initialization, fx( behave, in the limit of large number of channels and width (and small bias), as Gaussian random fields with correlation matrix E [2) zs 5(x ((x=y). with 5 the Dirac delta (Schoenholz et al., 2017; Xiao et al., 2018b). This spiky correlation matrix implics that for any perturbation y + e, the representation fy) changes with respect fx(w; independently
7

---

Under review as a conference paper at ICLR 2023
on This behavior is remarkably different to the smooth case achieved by the diffusion, after training. Consequently, both D2 and Gj: are constant with respect to & at initialization. This consistent with the observations reported in Fig. 7.
Sensitivity to diffeomorphisms. Let and denote the active pixels locations, so that x x ò, 85- Since both the elements of the inputs and those of the filters are non-negative, the presence of ReL.U nonlinearities is irrelevant and the first few hidden layers are effectively linear layers. Hence the representations are linear in the input, so that fx(x) = fx + +,)) = fá( + f6,)). In addition. since the effect of a diffeomorphism is just a 1-pixel translation of the representation irrespective of the original positions of the pixels, the normalized sensitivity D; can be approximated as follows
D2 [[x(6++1) fc(
(4)
(6.x
The denominator Eq. is the squared norm of Gaussian distribution of width vk, (lff k-1/2, The numerator compares f& with small translation of itself, thus it can be approximated by the squared norm of the derivative of the Gaussian distribution, (6++1)) - (6.) Consequently, we have
k k-3/2
DE k- AF
(5)
Sensitivity to Gaussian noise. To analyze G3 one must take into account the rectifying action of ReLU, which sets all the negative elements of its input to zero. The first ReLU is applied after the first homogeneous filters, thus the zero-mean noise is superimposed on patch of F active pixels. Outside such a patch, only positive noise terms survive. Within the patch, being summed to a positive background, also negative terms can survive the rectification of ReLU. Nevertheless, if the size of the image is much larger than the filter size, the contribution from active pixels to G. is negligible and we can approximate the difference between noisy and original representations fi + +1) fi (xx) with the rectified noise Φ(n). After the first layer, the representations consist of non-negative numbers, thus we can forget again the ReLU and write
E, (fx()())) GR
(6)
(o.
Repeated applications of homogeneous filters to the rectified noise ((1) result again in diffusion of the signal. Since σ(n) has different independent and identically distributed non-zero entries for different realizations of 1,, averaging over 17 is equivalent to considering a homogeneous profile for
fo()))). As a result, the numerator in Eq. 6 is constant independent of k. The denominator is the same as in Eq. 4, (5 ~ k 1/2,
11/2 hence
Gy ~ 22/2 ~ Av:
iee the sensitivity to Gaussian noise grows as the size of the effective receptive fields. From the ratio of Eq. 5 and Eq. 7, we get Rş= ~ A,
4.2 TASK 2, STRIDE EQUAL FILTER SIZE
When the stride s equals to the filter size F the number of pixels of the internal representations is reduced by factor F at each layer, thus fa consists of L/ FK pixels. Meanwhile, the effective size of the receptive fields grows exponentially at the same rate: A₃ = Fk (see Fig. 6, left for an illustration).
Sensitivity to diffeomorphisms. For given layer k consider partition of the input image into L/F* patches. Each pixel of fa only looks at one such patch and its intensity coincides with the number of active pixels within the patch. As a result, the only diffeomorphisms that change fs are those which move one the active pixels from one patch another. Since active pixels move by 1, this can only occur if one of the active pixels originally located at the border of patch, which in turn occurs with probability ~ 1/F*. addition, the norm [lf (6.) Il2 at the denominator does not scale with k so that
Dg F-k A
(8)
8

---

Under review as a conference paper at ICLR 2023
10
TASK 1,
10-1
10-3
10
2 10P
5 100
10P
2 10"
5 10"
10
2 100
3xx 10
10
16a
100
10
TASK 2,
Enit
3.0
10"
Train
10"-3
Meon channel
167
A
Figure 7: Sensitivities of internal representations fx of simple CNNs against the k-tth layer receptive field size Aş, for trained networks (solid blue) and at initialization (solid gray). The top refers to task I with s = and F = 3; the bottom row to task 2 with F = s = 2. For a first large part of the network, the sensitivities obtained by replacing each layer with the mean channel (blue dotted) overlap with the original sensitivities. Predictions Eq. 5, Eq. for task and Eq. 8, Eq. 9 for task 2 are shown as black dashed lines.
Sensitivity to Gaussian noise. Each pixel of f' looks patch of the input of size F thus fx is affected by the sum of all the noises acting on such patch. Since these noises have been rectified by ReLU, by the Central Limit Theorem the sum scales as the number of summands Fk. Thus, the contribution of each pixel of f& to the numerator of G2 scales as (2 As there are L/F pixels in fk. one has
G ~ GF~ (L/F*) ph Ak.
Without rectification, the sum of F* independent noises would scale as the square root of the number of summands F* yielding a constant Gg. We conclude that the rectifying action of ReLU is crucial in building up sensitivity to noise. R3 A follows from the ratio of Eq. and Eq. 9.
4.3 COMPARING PREDICTIONS WITH EXPERIMENTS
We test our scaling predictions (Eq. 5 to Eq. 9) in Fig. 7, for stride 1 CNNs trained on task I and stride F CNNs trained on task 2 in the top and bottom panels, respectively. Notice that if all the filters at given layer are replaced with their average, the behavior of the sensitivities as a function of depth does not change (compare solid and dotted blue curves in the figure). This confirms our assumption that all channels behave like the mean channel. In addition, Tables and 2 show that the mean filters are approximately homogeneous, Further details on the experiments are provided in App. B.
5 CONCLUSION
The meaning of an image often depends on sparse regions of the data, as evidenced by the fact that artists only need a small number of strokes to represent visual scene. The exact locations of the features determining the image class are flexible, and indeed diffcomorphisms of limited magnitude leave the class unchanged. Here, we have shown that such an invariance is learned in deep networks by performing spatial pooling and channel pooling. Modern architectures learn these suggeetttre-sgggtinin that is best let the
pooling operations- as they are not imposed by the pooling adapt to the specific task considered. Interestingly, spatial pooling comes together with an increased sensitivity random noise in the image, as captured in simple artificial models of data.
It is commonly believed that the best architectures are those that extract the features of the data most relevant for the task. The pooling operations studied here, which allow the network to forget the exact locations of these features, are probably more effective when features are better extracted. This point may be responsible for the observed strong correlations between the network performance and its stability to diffeommrphisms. Designing synthetic models of data whose features combinatorial and stable to smooth transformations very much needed to clarify this relationship. and ultimately understand how deep networks learn high-dimensional tasks with limited data.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Rima Alaifari, Giovanni S. Alberti, and Tandri Gauksson. ADef: an Iterative Algorithm to Construct Adversarial Deformations. September 2018. URL https::/openzeett net forum?
id=Hk4dF JR5K7.
Michael A. Alcorn, Qi Li. Zhitao Gong, Chengfei Wang, Long Mai. Wei-Shinn Ku, and Anh Nguyen. Strike (With) a Pose: Neural Networks Are Easily Fooled by Strange Poses of Familiar Objects. In 2019 IEEE/CVF Conference
Computer Vision and Pattern Recognition (CVPR), pp. 4840-4849, Long Beach, CA, USA, June 2019. IEEE. ISBN 978-1-72813-2994 doi: 10.1 10.1109/CPP.2209 00498. URL https: htteeeeeeeeeeeeeeeeeeeeeeeeeeeee weee. org. httpo/////oooommee ott...og///ccmmme///////////
Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. Deep speech 2: End-to- end speech recognition in english and mandarin. International conference on machine learning. pp. 173-182, 2016.
Fabio Anselmi, Joel Z. Leibo, Lorenzo Rosasco, Jim Mutch, Andrea Tacchetti, and Tomaso Poggio. Unsupervised learning of invariant representations. Theoretical Computer Science, 633:112- 121, June 2016. ISSN 0304-3975. doi: 20115..0....22055...0444.4 URL https:/ /www. sciencedirects /ttp:///cc.a.ctt...a.aa/////////
Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. In Advances in Neural Information Processing Systems, pp. 6111-6122, 2019.
Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. Synthesizing Robust Adversarial Examples. In International Conference on Machine Learning, pp. 284-293. PMLR, July 2018. URL http: http///poceeeengn.eeeee/ htthtt///pprooeeiiiim
m.l..press v http://///aaaaalaeaeeeaeeeeea html. ISSN: 2640-3498.
Joan Bruna and Stéphane Mallat, Invariant scattering convolution networks. IEEE transactions on pattern analysis and machine intelligence, 35(8):1872-1888) 2013.
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the Landscape of Spatial Robustness. In International Conference on Machine Learning, pp. 1802- 1811. PMLR, May 2019. URL httLtttpp////cceeeeee html. ISSN: 2640-3498.
ml prwssressssee / http://////////////////s /engsttossttttmooooooooooommmm 9a.
Alhussein Fawzi and Pascal Frossard. Manitest: Are classifiers really invariant? In Procedings of the British Machine Vision Conference 2015. pp. 106. 1-106.16.11. 13, Swansea, 2015. British Machine Vision Association. ISBN 978-1-901125-555 bmva. org/ Derg/bbvc//2110 http:///c////0////p/es////p//pp/
doi: 10.5244/C.29.10 200....... URL http:/ www.
index..... html.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning The MIT Press, Cambridge, Massachusetts, November 2016. ISBN 978-0-262-03561-3.
D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction and functional architecture in the cat's 's visual cortex. The Journal of Physiology, 10000))):166154222 January 1962. ISSN 0022-3751. URL https::/p//.
/wwww www...cbi...o mwm ni http://.ww....i..g.////////////a
Brody Huval, Tao Wang, Sameep Tandon, Jeff Kiske, Will Song, Joel Pazhayampallil, Mykhaylo Andriluka, Pranav Rajpurkar, Toki Migimatsu, Royce Cheng- Yue, et al. An empirical evaluation of deep learning on highway driving. arXiv preprint arXiv: 1504.01116 2015.
Can Kanbak, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. Geometric Robustness of Deep Networks: Analysis and Improvement. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4441-4449, Salt Lake City, UT, June 2018. IEEE. ISBN 978-1-
5386-6420-9. doi: 10. 109/CVP 2012.0008000 document/ 8878855/
URL http::///e/eeeep i ieee.ieeerooooo org/
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 96(11):227-223) 2324, November 1998. ISSN 1558-2256. doi: 10.1 10.1109/5.72679. Conference Name: Proceedings of the IEEE.
10

---

Under review as a conference paper at ICLR 2023
Ulrike von Luxburg and Olivier Bousquet. Distance- classification with lipschitz functions. Journal of Machine Learning Research, 5(Jum):669- 0910999--5,22 2004.
Stéphane Mallat. Understanding deep convolutional networks. Philosophical Transactions of the
Royal Society A: Mathematical, Physical and Engineering Sciences, 374(2065):0015020,3 2016.
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, loannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning. arXiv preprint arXiv:13/2.5602, 2013.
P. Niyogi, F. Girosi, and T. Poggio. Incorporating prior information in machine learning by creating virtual examples. Proceedings of the IEEE, 86(11):219-22200 November 1998. ISSN 1558-2256. doi: 10.1109/5.726777 Conference Name: Proceedings of the IEEE.
Leonardo Petrini, Alessandro Favero, Mario Geiger, and Matthieu Wyart. Relative stability toward diffcomorphisms indicates performance in deep nets. In Advances in Neural Information Processing Systems, volume 34, pp. 8727-8739. Curran Asso- ciates, Inc., 2021. URL p/ttceed//p newrips ceeuuips.ccppppeeeeeeeee 497476fe6188 6251 10weee9eeaaaaae5a252222 Abstract html.
Tomaso A. Poggio and Fabio Anselmi. Visual Cortex and Deep Networks: Learning Invariant Representations. September 2016. doi: 10.7551/mitpress// 10177.001.0001 www---orrrer----------eeett--ee-
URL https://direct: www..eedbbookk//oottt///// //8k///8//////////////// Invariant.
Stefano Recanatesi, Matthew Farrell, Madhu Advani, Timothy Moore, Guillaume Lajoie, and Eric Shea-Brown. Dimensionality compression and expansion in deep neural networks. arXiv preprint arXiv:1906.00443 2019.
Hannes Risken. The Pokker-Plande Equation Springer Series in Synergetics. 1996.
Avraham Ruderman, Neil C. Rabinowitz, Ari S. Morcos, and Daniel Zoran. Pooling is neither necessary nor sufficient for appropriate deformation stability in CNNs. arXiv: 1804, 04438 [cs, stat], May 2018. URL http:htt//// /ttp///aiivv../..e//ae/e..e.aaia /ttaaaiiv.rgg/abaaa//a/ org/ 1804. 4383.. arXiv: 1804.04438.
Laurent Saloff-Coste and Pierre Bremaud. Markov chains: Gibbs fields, monte carlo simulation, and queues. Journal of the American Statistical Association, 95, 2000. ISSN 01621459, doi: 10.2307/2669802.
Andrew M Saxe, Yamini Bansal, Joel Dapello, Madhu Advani, Artemy Kolchinsky, Brendan D Tracey, and David D Cox. On the information bottleneck theory of deep learning, Journal of Statistical Mechanics: Theory and Experiment, 2019(12): 2019.
Samuel S. Schoenholz, Justin Gilmer, Surya Ganguli, and Jascha Sohl- Sohl-Dickstein. Deep information propagation. 2017.
Baoguang Shi, Xiang Bai, and Cong Yao. An end-to-end trainable neural network for image- based sequence recognition and its application to scene text recognition. IEEE transactions on pattern analysis and machine intelligence, 39(11):2298 2304, 2016.
Ravid Shwartz-Ziv and Naftali Tishby. Opening the black box of deep neural networks via information. arXiv preprint T00i::170300081111 2017.
David Silver, Julian Schrittwieser, Karen Simonyan, loannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):5443503 2017.
K. Simonyan and Andrew Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. ICLR, 2015.
Chaowei Xiao, Jun- Yan Zhu, Bo Li, Warren He, Mingyan Liu, and Dawn Song. Spatially Transformed Adversarial Examples. February 2018a. URL https: http:////ppeeeeeeee//e net forrum? id= HyydRMZC-.
11

---

Under review as a conference paper at ICLR 2023
Lechao Xiao, Yasaman Bahri, Jascha Sohl-Dickstein, Samuel S. Schoenholz, and Jeffrey Penning- ton. Dynamical isometry and a mean field theory of cnns: How to train 10,000-layer vanilla convolutional neural networks, volume 12, 2018b.
12

---

Under review as a conference paper at ICLR 2023
APPENDIX
A TASK 1, STRIDE 1: PROOFS
In Section 4.1 we consider simple CNN with stride s and filter size F' trained on scale-detection task 1. We fix the total depth of these networks be K. We postulated in Sec. 4 that this network displays one-channel solution with homogeneous filter [1/F. 1/F] and no bias. We can understand the representation f& (z) layer & of an input datum by using single-pixel inputs 8,. Let us recall that these inputs have all components to 0 except the i-th, set to 1. Then, we have that general datum x is given by x x (6; + dj), where i and j are the locations of the active pixel in æ. We have argued in the main text that the representation fu (5)) is Gaussian distribution with width Vk. In this Appendix we prove this statement.
First, we observe that in this solution, since both the elements the filters and those of the inputs non-negative, the networks behaves effectively as linear operator. In particular, each layer corresponds to the application a L x L circulant matrix M, which is obtained by stacking all the L shifts of the following row vector,
1. 1,.00, 10.0.0....0
(10)
L-F
with periodic boundary conditions. The first row of such matrix is fixed follows. If F is odd the patch of size F is centered on the first entry of the first row, while if F is even we choose to have (F/2) ones left of the first entry and (F/2) 1 its right. The output fx of the layer k is then the following: (6.) = &r
Proposition A.1 Let's 's consider the L x L matrix M and a given L vector 8, as defined above. For odd 2 3, in the limit of large depth K >>> and large width L >>>> FV K, we have that
1
Iu 82 aDitVE
D 1
(M*) VVVDDV
(F - -1)²
(11)
12F
while for even F::
(AM*)abd; (VTVD2VV
D
(F3 - -3F2 +6F - -4) (12)
12F
with v(2) (1-FF)/(F))
Proof: The matrix M can be seen as the stochastic matrix of Markov process, where each step the random walker has uniform probability 1/F to move a patch of width F around itself. We write the following recursion relation for odd F.
(k+1) 1 (k) F (P-(F-1))22 +...
(K) + Pa. + + (K) Pof(F-11/22)
Pas
(13)
and even F.
Pa.i =
(k+1)
F. (po- +... Pa.d +... + (Ff(F/2-1).)
1 (p)))
(14)
In any of these two cases, this is the so-called master equation of the random walk (Risken, 1996).
In the limit of large image width I, and large depth K, we can write the related equation for the continuous process (a, k), which is called Fokker-Planck equation in physics and chemistry (Risken, 1996) or forward Kolmogorov equation in mathematics (Saloff-Coste & Bremaud, 2000),
Ō.Pa. v0.p.. (k)
(k)
Do2 (R)
(15)
where the drift coefficient v and the diffusion coefficient D are defined in terms of the probability distribution Wz(x) of having a jump x starting from the location
v | =ddaWit)
D do):
(16)
13

---

Under review as a conference paper at ICLR 2023
In case have (z(z) = 1/F for z E - 11)2-1)),-- (F 1)/2] for odd F and x E [ - F/2, F/2 1] for even F, yielding the solutions for the Fokker-Plane Planck equations for and odd F' reported in Eq. 11 and Eq. 12.
We can better characterize the limits of large image width L and large network depth K follows. The proof relies on the fact that random walk, after a large number of steps, converges to a diffusion process. Here the number of steps is given by the depth K of the network. Consequently, we need K > 1. Moreover, we want that the diffusion process not influenced by the boundaries of the image, of width L. The average path walked by the random walker after K steps is given by FVK. Then, we require FVK- < L.
B EXPERIMENTAL SETUP
All experiments are performed PyTorch. The code with the instructions on how to reproduce experiments are found here: pawu./com/giuubbbeeernneet
B. DEEP NETWORKS TRAINING
In this section, we describe the experimental setup for the training of the deep networks deployed in Sections 1, 2 and 3.
For CIFARIO, fully connected networks trained with the ADAM optimizer and learning rate 0.1 while for CNNs SGD, learning rate = 0.1 and momentum = 0.9. In the latter case, the learning rate follows cosine annealing scheduling. In all cases, the networks with batch size of 128 and for 250 epochs. Early stopping at the best validation error is performed for selecting the networks study. During training, we employ standard data augmentation consisting of random translations and horizontal flips of the input images. On the scale-detection task, we perform SGD on the hinge loss and halve the learning rate 0.05. All results are averaged when training on 5 or more different networks initializations.
trained on the cross-entropy loss,
For ImageNet, we used pretrained models from Pytorch, torchvis wwwiiiiiionnn sion. models.
B.2 SIMPLE CNNS TRAINING
In this section we present the experimental setup for the training of simple CNNs introduced in Section 4, whose sensitivities to diffeomorphisms and Gaussian noise shown in Fig. 7.
To learn task we use CNNs with stride s = 1 and filter size = 3. The width of the CNN is fixed to 1000 channels, while the depth to 12 layers. We use the Scale Detee-Detettion task in the version of Fig. 5 (b), with g = 11 and gap g and image size L = 32. For the training, we use training points and Stochastic Gradient Descent (SGD) with learning rate 0.01 and batch size 8. We use weight decay for the L2 norm of the filters weights with ridge 0.01. We stop the training after 500 times the interpolation time, which is the time required by the network to reach zero interpolation eтror of the training set. The goal of this procedure is to reach the solution with minimal norm. The generalization eтгог of the trained CNNs is exactly zero: they learn spatial pooling perfectly. We show the sensitivities of the trained CNNs, averaged over 4 seeds, in the top panels of Fig. 7, where we also successfully test the predictions (Eq. 5, Eq. 7). We remark that to compute G2 we inserted Gaussian noise with already the ReLU applied on, since we observe that without it would see pre-asymptotic behaviour for G3 with respect to A+-
48
Task 2 is learned using CNNs with stride equal to filter size F 2. For the dataset, we use the block-wir wise version the Scale-Dest Detection task shown in Fig. 5 (c). fixing { = 2 and L 27. We use 7 layers and 1000 channels for the CNNs. The training is performed using SGD and weight decay with the same parameters as in task 1, with P = 20 training points. In the bottom panels of Fig. show that the predictions (Eq. 8, Eq. 9) capture the experimental results, averaged over 10 seeds.
14

---

Under review as a conference paper at ICLR 2023
To support the assumption done in Section 4 that the trained CNNs are effectively behaving one channel with homogencous positive filters, we report the numerical values of the average filter over channels per layer in Table 1 for Task and Table 2 for Task 2. They positive in the first 9 hidden layers, where channel pooling is most pronounced.
Init. [0.0132. 0.0023, 0.0068
After training
k 1
[0.2928, 0.2605, 0 0.2928
k 2
0.0014,-0.0000 -0.0009
[0.0039, 0.0035, 0.0039
k 3
0.0000. 0.00000,0000000
[0.0043, 0.0038, 0.0043)
k 4
33.4610e
05, 6.56876 04,19.1634e 04]
0.00399 0.0033, 0.0038
k = 5
0.00006, 0.0002,
-0.0009 -0.0009
[0.0038, 0.0032, 0.0038
k 6
00.0038, 0.0031, 0.0038
k = 7
-0.0000,0.0004, 0.0003
0..0041, 0.0032, 0.0040
K 8
0.0005, -0.0012, 0.0010
0.0036, 0.0024, 0.0035
K = 9
-0.0012,
0 1.0021,000016,
K 10
-..0025, 0.0015,
[ 0.0013, -0000,,,00000000..10
K 11
1-.0006, 0,0005,,0000000000000 0.0009]
0.0002, 0.0002,0.0005 [0.0009, 0.0008, 0.0008,0.0009 0.0009]
12
[3.3418e 0.,3.3521e
05, 1.3936e 03]
Table 1: Average over channels of filters in layer k, before and after training, for simple CNNs with 8 = 1 and = 3 trained on task 1. The network learns filters which are much more homogeneous than initialization.
Init.
After training
k 1
0.0559, 0.0291
0.3828, 0.3737]
k = 2
-0.0022, 0.0010
0.0060, 0.0059
k 3
0.0006,
0.0064, 0.0065
k = 4
00.0020, 0.0009
0.00559, 0.0060]
k 5
[0.0002, 0.0008]
[9.9935e
05, 2.1380e 04]
k = 6
[- 0.0003, -0.0010]
0.0028 -0.009
k = 7
[-7.4610e 04, 8.4595e
05)
0.0009, -0.0009
Table Average over channels of filters in layer k before and after training, for simple CNNs with 8 = F = 2 trained on task 2. The network learns filters which are much more homogencous than initialization.
15

---

Under review as a conference paper at ICLR 2023
C ADDITIONAL HIGURES AND TABLES
coed. 0.70
coef: -0.72
coef.: 0184
10
2001
10
10
10
10
10'
10a
10
109
Dj
G₁
R2
Figure 8: Generalization emror e versus sensitivity to diffeomorphisms Dy (left), noise Gj (center) and relative sensitivity Ry (right) for wide range of architectures trained scale-detection task (train set size: 1024, image size: 32, & = 14, = 2). As in real data, is positively correlated with D and negatively correlated with Gy. The correlation is the strongest for the relative measure Ry.
109
Mooee
Shafff ON
10
Aloke:
10
Rx.
10
10
10a
VOSLI
10a
VO011
1003
109
VOSEL
VO
101
10
10
10a
10:2
10-3
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
relative depth $
relative depth ?
relative depth &
Figure 9: Sensitivities (Dx left, G₁. middle and RS right) of the internal representations relative depth for AlexNet (1st row) and VGG networks (2nd row) trained on scale-detection task 1. Dot- dashed lines show the sensitivities of networks with shuffled channels.
16

---

Under review as a conference paper at ICLR 2023
10
10
10
10
10
10
17
10
10
Dj
Gy
Ry
Figure 10: Test enror sensitivities (columns) when training noisy CIFARIO. The different rows correspond to increasing noise magnitude Different points correspond to networks architectures, gray labels. The content of this figure is also represented in compact form in Fig. 1, right.
AlexNet LoNet
original shuftle
alexnet original
shuffle ch.
10a
10"
6 10
4 10
3 10
CIFAR10
ImageNet
2 x 10-1
0.0
0.2
0.6
0.8
1.0
0.0
0.2
0.8
1.0
relative depth?
relative depth
Analogous of Fig. 3 for different network architectures: relative sensitivity R& as function of depth for LeNet and AlexNet architectures trained on CIFARIO (left) and ImageNet (right), Full lines indicate experiments done on the original networks, dashed lines the ones after shuffling channels. K indicates the networks total depth.
Figure 11:
17

---

Under review as a conference paper at ICLR 2023
CIFAR10
ImageNet
scale-detection
spatial frequency content
1.0
1.0
VGG11
Init:
traiteed ed
VGG11
0.6
VGGL1bn
0.4
0.7
2
5
8
1
3
6 7
1
2
4
5
8
layer
layer
layer
Figure 12: Spatial frequency content of filters for CIFARIO (left), ImageNet (center) and the scale-detection task (right). The y-axis reports an aggregate measure among spatial frequencies: Σi (r w²) where v the 3 Laplacian eigenvectors
N(E <|]=> and Å the corresponding eigenvalues, w the e-th filter of layer k and (-) denotes the average This is an aggregate measure over frequencies, the frequencies distribution is reported in the main
text, Fig. 4.
layer
layer
layer
layer
layer
layer
layer
layer
0.5
vop
veg13 vop29
0.0
layer 9
layer 10
layer 11
layer 12
layer 13
layer 14
layer 15
layer 16
0.5
0.0
eigenvalue index,
Figure 13: Analogous of Fig. 4 for deep VGGs trained on ImageNet. Dotted and full lines respectively correspond to initialization and trained networks. The ø-axis reports low to high frequencies from left to right. Deeper layers reported in rightmost panels.
k 1
k = 2
k 3
k =
k
0.2
CIFAR10
0.5
alexnet
trained
ImageNet SCIVE dieeddddeiooo
Yel
inil
0.0
ATEXTTL
tratteu
init
0.10
sinenvalue Index.
Figure 14: Analogous of Fig. 4 for AlexNet trained CIFARIO (1 row), ImageNet row) and the scale detection task (3 row). Dotted and full lines respectively correspond initialization and trained networks. The --axis reports low to high frequencies from left to right. Deeper layers are reported in rightmost panels.
18


Reviews:
[{"review_id": "WdrxxYwPtP", "reviewer": "Reviewer_eGsy", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript analyzes how convolutional neural networks (CNNs) process spatial information present in the images. The analysis builds on the observation in Petrini 2021 that performance of CNNs is correlated with their invariance towards diffeomorphisms, namely “smooth” transformations of the images,  and anti-correlated with their sensitivity to noise added to the images. The specific goal of this paper is disentangling the role of spatial pooling and channel pooling, operations which are at the basis of the working principle of CNNs. The analysis is performed using real data (CIFAR10) in Sec 2 and synthetic tasks in Sec 3-4. ", "summary_of_the_review": "The manuscript provides some interesting and novel insight on the manner in which CNNs process the information. The analysis of a very simple synthetic dataset presented in section 3 and 4 is clear, while the analysis of CIFAR10 (section 2) is in my opinion not fully convincing. I think the contribution is valuable, but some points need clarification. ", "strength_and_weaknesses": "Strengths:\n1.\tInvestigating the working principle of deep CNNs remains a topic of interest, and the manuscript sheds some light on the different roles of special and channel pooling\n2.\tThe results on the artificial task, section 3 and 4, are neat. In particular, the analytical derivation in section 4 is enlightening with respect to the manner in which noise is processed in a CNN\nWeaknesses:\n1.\tThe empirical results presented in support to the first “contribution”, disentangling spacial and channel pooling, are not fully convincing. Fig 3 shows the changes in the sensitivity to shuffling the channel connections. The scale is linear in the left panel (AlexNet and LeNet) and log in the right panel (VGG) creating some confusion.  The change is claimed to be small in the first layers, but this change looks approximately 30 % also for very deep layers in VGG. Only in LeNet the change is clearly small (~ 5 %). Is 30 % small or large? Maybe a baseline should be defined. Moreover, the qualitative differences between the different architectures are not explained. Do they derive from the depth?\n2.\tFig. 4 illustrates the frequency content of the filters of the layers of a CNN at different depth. I do not understand why the curves do not all start from 1. Moreover, I do not understand what we learn from the time dependence of these coefficient.  The main message conveyed by this figure should be that deep layers (1-5) “become low-frequency” (while, I assume, late layers become high frequency). A better observable would possibly be the average frequency of the filter\\ sum_{kl} gamma_kl omega. I also have a problem with the interpretation of the frequency content as defined by eq. 3. The receptive field becomes larger and large with depth. Therefore, the same frequency in different layers corresponds to different physical lengths in the input image. Shouldn’t the frequency be mapped in common units across the layers? I also do not fully understand why the observation that early kernels are \"low-frequency\" supports the claim that channel pooling happens late in the network. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "SmUBHfk926n", "reviewer": "Reviewer_Zxqq", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper investigates the mechanism of spatial information loss in image classification networks. This loss in sensitivity to small perturbations in the input is important in learning robust invariant representations of features relevant to the task (image classification in this work). Authors propose that deep convolutional networks learn to first spatially and then channel-wise pool information as we go deeper in the network layer-by-layer. They support their claims with numerical experiments on the CIFAR-10 dataset on multiple architectures. Then, they construct some simple vision tasks, where the simplest solution according to human intuition is pooling, and they demonstrate that the above phenomena from image classification still hold. Finally, they quantify in this simple toy case how sensitivity to input diffeomorphism and additive white noise scales with network depth.", "summary_of_the_review": "Overall, I would lean towards rejecting the paper in its current state. I think the empirical evidence provided is not strong and convincing enough due to the reasons I have detailed above under 'Weaknesses'. Furthermore, the motivation for introducing the scale-detection tasks and the theoretical analysis is not clear to me.", "strength_and_weaknesses": "Strengths:\n\n- The paper is organized and easy to follow. The structure is logical and the paper is mostly well-written.\n  \n- The problem the paper investigates is very interesting and is important in better understanding the success of modern deep learning. Overall, the motivation of the paper is clear.\n  \n\nWeaknesses:\n\n- In my opinion, the main claims of the paper are not well supported by the experiments. Based on a single small dataset (CIFAR-10) authors conclude that \"spatial and channel pooling are carried out along the whole network\", however it is not clear to me how authors arrive at this conclusion. The decrease in sensitivity to diffeomorphisms and increase to white Gaussian noise may be caused by some other factors not investigated in this paper. The proposed underlying reasons (different types of pooling at different depths) are a possible and logical explanation, but I don't see how this simple experiment proves this. I am also not convinced by the experiments disentangling the effect of the two types of pooling.\n  \n  - First, in Figure 3 right, in case of VGG11bn the gap between original and channel-shuffled models is already significant in early layers, and in case of VGG11 the gap is almost constant in the first 80% of the network. Overall, the presented evidence is not sufficient to conclude (especially based on a single small dataset) that channel pooling is not happening early in the network.\n    \n  - Second, the plots showing the frequency distribution of learned filters with respect to spatial pooling are difficult to make out. Authors claim that \"layers 2 to 5 become low-frequency with training\", however the lines are so close to each other that it is impossible to verify this. Probably presenting the distribution at t=0 and at end of training would be much easier to interpret, the exact values during training seem to be irrelevant to the paper. Moreover, even if the claim is true, why do we not see pooling at layer 1?\n    \n- It is not clear to me what is the goal of defining the scale-detection tasks. Authors concluded based on standard image classification that the phenomenon holds (early spatial pooling, late channel pooling). Then, they design a problem where our intuition suggests that pooling is the simplest solution, and we observe the phenomena again. I don't see what the scale-detection tasks add that we did not see in the image classification experiments.\n  \n- With respect to task 1, authors claim that \"Spatial pooling is the most direct mean to achieve such insensitivity\" with respect to sensitivity to diffeomorphism. This might be true based on our human intuition, but the learning dynamics of neural networks is not very well understood and this claim sounds too strong.\n  \n- The results on task 1 are not very convincing in Figure 7. In particular, it looks like VGG11's sensitivity to Gaussian noise is more or less constant over layers. The phenomenon seems to hold for networks with batch normalization only.\n  \n- The theoretical analysis is a bit confusing to me. What diffeomorphisms have been used? Why is the translation only 1 pixel? Why are filters necessarily non-negative?", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NrAprHft4H7", "reviewer": "Reviewer_x5ja", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper studies the question of why trained deep convolutional networks are insensitive to image perturbations such as small translation and rotation. Through a set of studies on CIFAR-10, the paper argues that this is mostly due to spatial pooling and channel pooling, and the two components can be further decoupled at different layers. The paper then propose a simple synthetic task on which neural networks demonstrate similar behavior, and use this synthetic task to theoretically study the arguments made earlier. ", "summary_of_the_review": "This paper studies the question of why deep convolutional networks are insensitive to image perturbations such as translation. I find the empirical studies weak and the theoretical studies need some explanations (see above) to both clarify the results and to make connection to the empirical studies. \n\n-------\nAfter rebuttal: thanks to the authors for additional experiments on ImageNet. This addition to the paper require another round of careful review. And my other concerns (e.g. regarding the parameter regime of the proof and the justification for the non-negativity) are not well addressed by the response. Therefore I'm keeping my original rating.", "strength_and_weaknesses": "**Strength**\n1. This paper identifies a clean question to study\n2. This paper includes code to reproduce the experiments\n\n**Weakness**\n1. The empirical studies are weak\n    1. The neural networks tested in this paper are very simple. Including some more modern networks would make the study more useful. At the minimum a deep resnet should be included, because skip connections are omnipresent in modern neural networks and it would be interesting to see how the arguments about the decoupling between filter and channel pooling behave with skip connections.\n    2. What happen to other networks if you do the same kind of analysis as in Figure 4? I could not find such results in the Appendix either.\n    3. It would be useful to add error bars to the plots (e.g. Fig. 3).\n    4. Please include more than one dataset so that we get an idea of how generalizable the observations are.\n    5. For many convnet architectures, the number of channels tend to increase at higher layers. In the experiments showing that lower layers are less sensitive to channel permutation, how does the number of channel impact the results? \n\n2. The synthetic tasks are specifically designed to encourage network insensitivity to translations, and in task 2 the neural network architecture (strides equals to filter size) is also tailored to match the task bias. I'm having a hard time to see how this analysis here could help understand what happens in the real world scenarios.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject