In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?
Authors: Henry Kvinge
Tegan Emerson
Grayson Jorgenson
Scott Vasquez
Timothy Doster
Jesse Lew
Venue: NeurIPS 2022 Accept (2022)
Keywords: Invariance and equivariance
augmentation training
out-of-distribution generalization

Abstract: It is often said that a deep learning model is ``invariant'' to some specific type of transformation. However, what is meant by this statement strongly depends on the context in which it is made. In this paper we explore the nature of invariance and equivariance of deep learning models with the goal of better understanding the ways that they actually capture these concepts on a formal level. We introduce a family of invariance and equivariance metrics that allow us to quantify these properties in a way that disentangles them from other metrics such as loss or accuracy. We use our metrics to better understand the two most popular methods used to build invariance into networks, data augmentation and equivariant layers. We draw a range of conclusions about invariance and equivariance in deep learning models, ranging from whether initializing a model with pretrained weights has an effect on a trained model's invariance, to the extent to which invariance learned via training can generalize to out-of-distribution data.

In What Ways Are Deep Neural Networks Invariant and How Should We Measure This?
Henry Kvinge Tegan H. Emerson 13.4, 1.3,4 Grayson Jorgenson', Scott Vasquez!, Timothy Doster" Jesse D. Lew
Pacific Northwest National Laboratory DDeparmentt of Mathematics, University of Washington DDeparmentt of Mathematics, Colorado State University
Department of Mathematical Sciences at the University of Texas, El Paso National Geospatial Intellggenen Agency henry.kvingeee.. wwi.ke@pnnn.n gov
Abstract
It is often said that deep learning model is "invariant" to specific type of transformation. However, what meant by this statement strongly depends on the context which it is made. In this paper explore the nature of invariance and equivariance of deep learning models with the goal better understanding the ways in which they actually capture these concepts introduce family of invariance and equivariance metrics that allows us to quantify these properties in way that disentangles them from other metries such loss accuracy. We use our metrics to better understand the two most popular methods used to build invariance into networks: data augmentation and equivariant layers. We draw range of conclusions about invariance and equivariance in deep learning models, ranging from whether initializing a model with pretrained weights has an effect on trained model's 's invariance, to the extent to which invariance learned via training can generalize -iitt--diittitttion data.
formal level. We
Introduction
The notions of invariance and equivariance have been guiding concepts across diverse range of scientific domains, from physics to psychology. In machine learning (ML) the concept of invariance frequently invoked to describe models whose output does not change when input is transformed in ways that are irrelevant to the task the model was designed for. For dog or cat image classifier, for example, it is desirable for the model to reflection invariant so that the same prediction made whether or not the input image is reflected across its vertical axis. This type of invariance is useful in this model because although an image generally changes when reflected, whether it contains a dog or cat does not. Equivariance is used to describe models whose output changes in manner that is aligned with the way that input transformed. For example, as an image is rotated, the position of bounding boxes predicted by an object detector should also rotate (thus, the object detector should be rotation equivariant).
While mathematicians have developed rigorous theory that can describe invariance and equivariance, the amount of this that actually used in the context of ML varies dramatically among different works. Within computer vision, research relatively simple image transformations (i.e., rotations or translations) is often presented within a solid group-theoretic framework K 52, y Other work dealing with more complex types of transformations such as changes in image background have (by necessity) needed to be more informal [5]. Furthermore, while model invariance and equivariance are frequently central component in broad range of ML research, limited effort has been put
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

into trying to measure them directly with the purpose of understanding the general invariance and equivariance properties of deep learning models. Rather, most works measure them indirectly through other MI. metrics that align with the ultimate purpose of the model (e.g., loss or accuracy). While such a strategy makes sense when optimizing model performance is the primary objective, we miss an opportunity to better understand how and why deep neural networks work (or fail). The purpose of this paper to propose a group-theoretie family of metrics, associated to an arbitrary group of symmetries G, which we call G-empirical equivariance deviation (G-EED), that quantify the G-equivariance (and G-invariance) of a model.
Informally, the G-EED of model measures the extent to which it fails to be G-equivariant on specific data distribution and with respect aligns with needs in ML where the user of model may care only that their model is equivariant on data that the model will actually encounter in practice and not on any possible input. Further, since invariance is a special case of equivariance, G-EED also measures the extent to which model fails to be invariant to the action of G. To show the breadth of the G-EED concept, we give number of different ways it can be applied to measure different aspects of equivariance in a model. For example, we show how G-EED can be applied to a model's latent space representations to measure the extent to which model extracts G-invariant features.
specific notion of distance in output space. This
Finally, we use G-EED to answer a range of questions about invariance and equivariance in neural networks, with focus on the two most popular ways of inducing invariance in these models: data augmentation and equivariant architectures. Some of the conclusions we draw from our experiments include the following. (1) Training with augmentation does not tend to induce invariance through learned equivariant layers; rather, invariance arises through some other mechanism.
(2) Invariance learned through augmentation does generalize mildly to aut-of-distribution data (e.g., common image corruptions), but apparent invariance seen for far out-of-distribution data (e.g., images from a completely different domain) could be the result of model insensitivity. (3) Invariance should not be assumed to correlate with model performance: models with random weights can be more invariant (in the usual mathematical sense) than models with learned or hard-coded invariance. (4) Models initialized with pretrained weights tend to have different invariance or equivariance properties than models initialized with random weights though whether these models are judged to be more invariant depends on the specific notion of distance one chooses to use. (5) Self-supervised models do not seem to be more invariant than supervised models except when they are trained with contrastive loss and augmentations from the relevant symmetry group G.
In summary, the contributions of this paper include:
The introduction of G-empirical equivariance deviation, the first family of metrics that can rigorously and directly measure range of notions of invariance and equivariance in deep learning models.
A demonstration of the flexibility of G-EED, showing that it can be applied easily to a range of different components of a deep learning model to measure different forms of invariance and equivariance.
Use of G-EED to answer a range of questions, shedding light on the extent to which neural networks are not invariant and equivariant.
2 Related Work
The literature on invariance and equivariance in machine learning can roughly be partitioned into two groups: those works that focus on how to build invariance and equivariance into a model or its and those works that focus on the theory behind 2
components [27 g 52 45
IC
46
19
12
37
41
invariance and equivariance
3
Three common approaches are used to build invariance into deep learning models: data augmentation, feature averaging, and equivariant architectures. this paper we focus on the first and third of these. Outside of few ubiquitous layer types (such as standard translation equivariant convolutional layers), data augmentation is by far the most commonly used method, being a standard component of many training routines, particularly in computer vision. Since it has become common procedure when training deep learning models, data augmentation research has expanded in many directions
12 37 41 2
2

---

The idea that G-invariance can be hardcoded into a model by combining multiple G-equivariant layers with data reduction layers (e.g.. pooling) has a long history deep learning. The most famous example of this idea is the conventional convolutional neural network (CNN) R2 Since then, a multitude of other group equivariant layers have been designed including two-dimensional rotation 39] tiree-dimensional rotational equivariant layers HG 11 [6].
equivariant layers 47 50 50
58
layers equivariant to the Euclidean group and its subgroups 45 (which we test against in this paper),
and layers that are equivariant with respect to the symmetric group [3
Our work is not the first to analyze various aspects of invariance in neural network models. Lyle et al. [4]] analyzed invariance with respect to the benefits and limitations of data augmentation and feature averaging, presenting both theoretical and empirical arguments for using feature averaging over data augmentation. More recently. Chen et al. [ presented a useful group-theorettc framework with which understand data augmentation. Relevant to the present work, Chen al. (5) introduced a notion of approximate invariance. Unlike that work, however, which focused on theoretical results related to data augmentation, this paper aims to introduce metrics that can be applied to modern deep learning architectures and answers questions about invariance from an empirical perspective. There are a number of existing works that proposed metrics aimed at measuring the extent to which a model is not equivariant (e.g. B
43 49 ). Our work differs from these in two ways: (1) we build general metrics based on basic group theory that are designed to work across different groups and
datatypes and (2) unlike other works that use their metric to evaluate the equivariance of a specific model, we use our metrics to explore how models learn (or do not learn) to be equivariant generally.
Finally, range of recent works have shown that even beyond the standard evaluation statistics (e.g.. accuracy), invariance is an important concept to consider when studying deep learning models. For example, Kaur et al. inputs. further series of works investigated whether excessive invariance can reduce adversarial robustness P 24 40 All of this work reinforces one of the primary messages of this paper, that it is important to be able to measure invariance and equivariance directly in a model.
2 showed that lack of invariance can be used to identify out-of distrr-iisttiittion
3 Quantifying Invariance and Equivariance
We begin this section by recalling the mathematical definitions of equivariance and invariance. We present these definitions in terms of the mathematical concept of a group, which formally captures the notion of symmetry 15
Assume that is a group. We say that G acts on sets X and Y if there are maps ox G x X - X and Øy : G x Y + Y that respect the composition operation of G. That is, for 91, E G and E X,
dx (x2, Φx (01, x)) = dx (#291.
with an analogous condition for Φy. Whenever the meaning is clear, we simplify notation by writing Φx (g. .)) = gx (with an analogous convention for Φv). A map f X + Y is said to be G-equivariant if for all x E X and g E G.
=f
(I)
In the case where the map Øy is trivial that gy y for all g E G and y E Y, say that f is G-invariant. Thus, invariance is a special case of equivariance.
Assume that ʃ X - Y is neural network where X is the ambient space of input data and Y is the target space. In many cases there is a natural way to factorize I into a composition f = f2 o Si where f1 X + Z is known as the feature extractor, 2 is the latent space of f, and f2 7 - Y is the classifier. For example, if of is a ResNet50 CNN 200 then fi may consist of all residual blocks while fa would consist of the final affine classification and softmax layers. We say that machine learning model f extracts G G-equivariant features if fi is a G-equivariant function. This is an especially meaningful distinction in the context of transfer learning where invariance equivariance can be transferred to a new task via the invariance or equivariance of f1. Note that the definition of G-equivariant feature extraction requires well-defined action of G on Z, which may not be obvious in many cases. Because the trivial action is defined for any G and Z, we can always ask whether f extracts G-invariant features.
The following proposition provides some insight into how the invariance (or lack of invariance) of fi relates to the invariance (or lack of invariance) of f.
3

---

Proposition 3.1. Let f: : X - Y be function that decomposes into f f2 fi where fi :X - Z and f2 : Z + Y. Suppose that G acts on X, Z, and Y.
f can be G-invariant even if f1 and f2 are not.
2. If fi is G-invariant, then f is G-invariant.
A proof of Proposition] 3.1 can be found in Section A.5 earlier layers of network achieve invariance with respect to some group, then this invariance will persist into later layers. This may be seen as part of the justification of the common practice within the equivariant architectures community of building invariance through successive combinations of G-equivariant layers and pooling layers. Of course, the statement holds only when exact invariance is achieved. We see below that this is not generally the case.
Note that Proposition 3.1 implies that if
3.1 Measuring equivariance
In this section we assume that both X and Y are vector spaces and the action of G' on both X and Y linear. In all of our experiments we assume that the action G on and Y known. In the case that f is also linear map, the equivariance of f can be checked directly by checking equivariance on a basis for X. By extension, the equivariance of many common types of neural network layers can be checked when these can be framed linear maps (e.g., convolutional layers). However, there is no systematic procedure for checking the equivariance of a nonlinear function f. If is the composition of sequence of functions fi; ... Jn and we can check that each is equivariant, then know that f is equivariant, but we cannot prove that a function is not equivariant just by proving that each layer is not equivariant (this follows from Proposition B. We are thus motivated introduce family of metrics that can be used to empirically measure the extent to which a function deviates from being equivariant on data distribution D X.
Since we assume in this section that the action of G on Y is linear, we can define the kernel of this action, ker(óy), ), which is subgroup of G. ker(øy) consists of all those g G such that g acts as the identity on Y. For E X, we define f (x) be the expected value of f(gx) over ker(dy),
[
f(x) == (y bv)
fgxx)d
(2)
gEker((r)
where y is the usual normalized Haar measure on subgroup kcr(G). Note that when f is G-equivariant, then for each g E ker( f(gx) = gf(x) f(x) and hence f f-
Let (D, v) be a probability distribution on X and let m Y x Y Rzo be a distance function on Y. We can use m measure the extent to which deviates from being G-equivariant by computing
So
(f)))))g()))ppp
(3)
where this time is the normalized Haar measure on G. Note that the argument mff(gz),g)) (x)) measures the extent which ① fails hold across distribution D, except that gx(x) is replaced by gz). We use f(x) because it averages over all values in the orbit of x (under G) that should map to f(x). If f were genuinely G-equivariant, all these values f(zz) would yield f(r). This choice is supported by the fact that naturally interpolates between the two extreme cases: acts trivially on Y (invariance) and G acts faithfully on Y. In the former case f(x) is the average value of ʃ (gx) over all of g E G and in the latter case f(x) f(x).
The proposition below proves that when the action of G faithful X, Y. Bb being 0 is equivalent to f satisfying I on a set of measure 1.
Proposition 3.2. Let f: X + Y be continuous function, G group that acts linearly and faithfully on both X and Y, and m Y > 1 a metric. Let be a distribution on X. Then 3 zero and only if f is G-equivariant almost surely, i.e., on set of measure 1.
We provide proof of Proposition Appendix A.6

---

To approximate ① for real models and data where we always work with finite groups and finite samples of D, we define the G-EED of with respect to m be
8(.G) D
ΣE ml m(f(ge) gED
(4)
Note that since G is a finite group (and hence discrete), the Haar measure turns into the usual counting measure.
In the remainder of this section we describe some specific types of G-EED that may be relevant to computer vision tasks. By convention, when using a distance function m for which larger values of m(x1,23) 22) indicates that 21 and Z2 are "closer" (the opposite of a proper metric), we attach negative sign to m. For example, rather than using cosine similarity, we use negative cosine similarity (e.g.. 5 This way, larger values of G-EED consisently indicate less invariance, regardless of the m used.
Latent space CDD calculations
Compare
Channehwise Cy- EED calculations
Unoor lover
Compare to sentroid
Conelutional block
Convol Jtonal Mook
H
S
S
Figure 1: diagram illustrating the different types of empirical equivariance deviation (EED) that we investigate in this paper for the rotation action of cyclic group C4 on MNIST images.
The channelwise G-equivariance of convolutional layer activations: Throughout most layers of CNN an individual image is represented as 3-tensor. Let fe X - RC H2 W. be the composition of the first l layers of CNN such that for input image X, fe(x) is C₁ x He We tensor where the first dimension corresponds to the channels of the representation and the second and third dimensions correspond the two spatial dimensions.
Suppose that G is a finite group that acts on images and other 2-tensors (e.g., rotations, translations, and reflections). To understand the extent to which the first & layers of the network G-equivariant, we can measure the G-EED of fe. Although there are numerous choices of m for (3) that could be used measure the channelwise difference between fe(gx) and g choose the following: let
S : H, xWe x HR > [-1, 1] be the cosine similarity on individual channels treated as vectors H,W, Write [fe(z)], for the ith channel of fz)). Then we
c
m(felgz))
1
Σ
S((F(lr)),
C
i=1
That is, let m be the negative of the average cosine similarity between individual channels of fe(gz) and gz)( This gives:
Echannel (f, = DIG
ΣΣ (fff((oz).gf(())). DED
(5)
Note that since rotation, translation, and reflection groups all act faithfully on HexW; for these specific G, f - f. We call this version of G-EED channehvise G-EED. This metric assumes that G acts on each channel in a 3-tensor independently. does not account for the more complicated setting where the action of G either permutes mixes channels of C& X XW in non-trivial way. In this work we always assume that the action of on channels of C H.x Wt is identical to the action of G' on X (up to differences in spatial scale and number of channels) except for the models

---

with equivariant architectures, where the action of G' is hardcoded to permute channels in a specific way. In these latter cases we match channels in accordances to the specifics of the architecture. We consider the case where the action of G on IR CRx may permute channels in non-equivariant architectures in Appendix A.4
Invariance of latent space representations: Consider a model f consisting of feature extractor fi fi:x- + _ and classifier f2: I* + IR.". If is some group that acts on X, we can ask whether fi extracts G-invariant features. Define M to be either the average lz-distance or cosine similarity between distinct points in the set (D).. By setting m to be the fz-distance (respectively cosine similarity), we calculate the latent space G-EED as
Etatem (f.)) = M DI|
ΣΣ gED
m f(f((o))
(6)
Note that because in this setting G acts trivially on the latent space TRAt (because of the invariance assumption). (x) = লল| Egea Si (gz). We describe why the normalization term M is necessary in Section A.7 of the Appendix. See also Figure 14 As show in Section 4.1 it is often useful to consider latent space G-EED with respect to both Euclidean distance and negative cosine similarity.
Invariance at the softmax layer of classifiers:
While the two examples above focused on calculating invariance and equivariance at different intermediate layers of computer vision models, invariance of classification models is most commonly considered with respect the final softmax output. In this paper we choose use KL-divergence for m because it is a distance function designed to handle probability distributions, which softmax outputs approximate. In particular,
Evoltmax (f, D|GG|
(kLgg))
gED
We call this metric the softmax G-EED.
4 Understanding invariance and equivariance in deep learning
To prototype G-EED experiment with the finite cyclic group Cg that acts images by 45 rotations. This symmetry is relevant to datasets where imagery does not have a preferred orientation. The datasets we use include Rotated MNIST which consists of randomly rotated MNIST digits [4] from classes 0-8 and xView Maritime, a chipped version of the x Xiew object detector overhead dataset [31] composed of the nine maritime classes. We chose xView Maritime because it is more real-world dataset that has an obvious symmetry (rotation) that is not hardcoded into standard CNNs. We describe these and other datasets we use in detail in Section
To reduce the computational burden, when calculating 4 we only used 50 randomly chosen examples from the corresponding dataset. We found that evaluating more points did not substantially change the results. When not otherwise stated all invariance and equivariance measurements were calculated using points from the test set, not the training set. The normalization constant for latent space G-EED is calculated over 200 randomly chosen points.
4.1 Do networks trained with augmentation learn equivariant layers?
In this section we apply our metrics to standard CNNs and Cs-rotation equivariant, steerable CNNs (Cs-CNN) 45 to compare the invariance or cquivariance of a model trained with augmentation with the invariance cquivariance of a model that is explicitly designed to Icarn invariance through progressive equivariant layers and pooling. We train 10 copies of each model type for 2,000 iterations on rotated MNIST. We apply random rotation augmentation to the models during training and all achieve an accuracy of over 98% on the test set. By design, for any < k S 6, the first h-blocks of one of the Cs-CNN models will be approximately rotation equivariant (up to hardcoded permutation of channels which we take into consideration when applying our metrics). The standard wisdom is that, with pooling, this will lead an invariant representation in the latent space.
Despite the fact that both families of models achieve similar performance, we note that their invariance and equivariance properties differ significantly. The channelwise CgD of the composition of

---

the first k blocks (where 1 v A: < 6) of the Cg-CNN is significantly higher than that of the CNN, indicating that (as claimed) the Cc-CNN is more equivariant with respect individual channel rotations of 45 in both input and hidden representations (see Figure 7 Beyond that we see that equivariance decays for both models as input travels through additional blocks. This particularly true for the Cg-CNN. We conjecture that this is likely caused by the accumulation of interpolation artifacts associated with rotation. Perhaps even more interesting is the fact that for no layer of either model do the channelwise Cg-EED values increase beyond a small jump at the beginning of training. This suggests that layerwise equivariance may not always be aligned with the learning objective and that learned invariance does not arise through the naive form of learned equivariance.
Both in terms of the Euclidean distance version of latent space Cg- Cg-EED and softmax Cg-EED, the Cs-CNNs show more invariance than the conventional CNNs (see Figure 2 but the difference is less dramatic than is for channelwise Cg-EED. This suggests that the CNNs likely catch up in terms of Cg-invariance using mechanisms distinct from layerwise Cg-equivariance. Notable also the fact that unlike channelwise Cg-EED, which did not indicate increased Cg- C'-eevvvare over the course of training, both latent space and softmax Cg-EED decreased with training (suggesting learned invariance at these layers).
Most surprisingly, in Figure S (left) we see that the Cg-CNNS are less invariant than the conventional CNNs with respect to the cosine similarity version of latent space Cg-EED. We conjecture two possible reasons for this. The first is based on the fact that Cg- Cg-CNNs are designed to be equivariant on the nose, not up to scaling. The structural constraint of exact equivariance may actually be misaligned with cosine similarity where vector magnitude no longer matters. From this perspective, projective representation theory, where symmetries are only taken up to scaling, might be a profitable direction of future research in equivariant architectures. The second possible explanation arises from Figure g (right) which shows that in terms of absolute cosine similarity (that is, without the normalization term M in (, the Cg-CNN is more invariant. It may be that while Cg- Cg-CNNs indeed cluster orbits of points under the C3 action closer together, the conventional CNNs are better at
spreading other points further away in the latent space.
0.40
Model
0.35
CB CNN
0.010
CB CB:CWN
CNN
0.30
Late2 apice C8-15
0.000
0.25
Sofmmax CSFF3
0.008
0.20
0.1
0.032
0.05
0.000
0.00
250
300
1502 1750
2009
230
750
1000
1250
1500
1330
2009
Iteration
Figure The latent space Cs- EED (left) and softmax Cg- (right) for conventional and Cs- equivariant CNNs (Cs-CNNs) 45
with respect to the rotated MNIST dataset. Both plots include 95% confidence intervals. Lower values for both plots indicate more -x-invariance. 95% confidence intervals are over 10 randomly initialized models.
Summary: Equivariant architectures and nonequivariant architectures capture invariance in distinct ways. Nonequivariant networks do not learn layerwise equivariance (at least with respect to the naive Cs action given by rotating individual channels in hidden representation). Rather, their invariance seems to be captured by the final layers. The level of invariance measured in the latent space depends on the underlying metric which is used.
4.2 Does learned invariance hold for out-of-distribution data?
was noted in Lyle et al. [54 that while augmentation may be satisfactory for many tasks, there is risk that learned invariance will not extend to out-of-distributtin (OOD) data. We investigate this question using G-EED metrics, studying the conventional CNN and Cs-equivariant CNN from Section and a conventional CNN with untrained weights that can be used for comparison. Note
7

---

that because rotation invariance is at some level hardcoded into the architecture of the Cs-CNN, our default assumption that the Cg-CNN models will tend be more invariant on OOD data than models with learned augmentation. Images of example OOD data as well as dataset descriptions can be found in Appendix A.2
The left plot in Figure B shows the latent space C-EEED for the three model families that we tested. We can see that random weights extract significantly less invariant latent space representations compared to both the augmentation-trainen surprising for the CN-CNNs, for the standard CNNs it indicates that augmentation does learn some rotation- notation-invariant invariant features that generalize to OOD data. Furthermore, while Cg-CNNs do exhibit more latent space invariance than the augmentation trained CNNs, the difference is insignificant compared to the difference between augmentation- trainned CNNs and CNNs with random weights. Note that in these OOD experiments, the normalization term M in 6 is calculated using the OOD) data (rather than the training or test set). This choice was made when considering situations in which
CNNs and Cs-CNNs. Although this difference is not
a frozen network is used in combination with metric learning methods, as in Snell et al. 42 to solve a task with data drawn from a shifted distribution.
0.8
Model type
0.0175
Modee type
0.7
random
CMN random
Latent space CB
0.6
CNN - trained
0.0150
CNN trained
CB-CMN
Sotmax C8-EED
0.0125
CB:CNV
0.5
0.0100
0.4
0.3
0.2
0.0050
0.1
0.0025
0.0
0,0000
NWIST train
CIFAR1O
MHIST (noioy)
MNIST (roisy) 2
SNIST (nsioy)
UNST (noloy)
SHIET Insioy!
MNST (noisy) 6
MNITT (blurr)
Cintggitt
Inverted SINIST
MNIST train
GPARIO
MHIST (telly) 1
ANIST (naloy) 2
MisT (roloy)
UNIST (naloy)
MNNIST (raiop)
MAIT moaoy?? 6
MNST (bburr)
Graniggat
Inverted MIKES
GoC type
Goo Syae
Figure 3: The latent space Cg-EED (left) and softmax Cg-EED (right) for conventional untrained CNNs, conventional CNNs trained on MNIST, and Cg-equivariant CNNs (C&- (Cg-CNNs) 45 trained on MNIST with respect to a range of in- and oft-t--distriuttio datasets. Both plots include 95% confidence intervals over 10 randomly initialized models.
As a showcase of how single model can display different types of invariance, the CNNs with random weights are far more invariant in terms of sofimax Cg-EED than the trained Cs-CNN, which is more invariant than the trained CNN (right plot, Figure 3p. We suspect that invariance of the CNNs with random weights arises from the fact that an untrained network tends to be insensitive to input. This observation is useful to keep in mind, especially as the concept of invariance is frequently treated as net benefit any model. Invariance is useful when it is aligned with other metrics more directly related to the task the model was designed for.
Our experiments suggest that augmentation trained networks do learn invariance that mildly generalizes to OOD data. A single model can display differing kinds of invariance (an untrained network does not extract invariant features but through insensitivity may make invariant predictions). In cases where invariance is critical and OOD data is expected, equivariant architectures may be safer choice than training with augmentation.
Summary:
4.3 How does the use of pretrained weights affect invariance and equivariance?
It has become standard practice in computer vision to initialize model using weights trained on a large, diverse image dataset such as ImageNet [[3] reasonable to ask how such strategy affects the invariance and equivariance of the resulting models. We choose four different image classifier architectures: ResNet50 [20], AlexNet 29) DenseNet121 21.. and ViT [.. For each of these architectures, we initialized five models with random weights and five models with weights from either the Torchvision package +5 (ResNet50, AlexNet, and DenseNet121) or Timm 48 (LeViT192). We then trained all of these models the x View Maritime training set. Next,
8

---

we evaluated the latent space Cg-EED and softmax Cg-EED on the x View Maritime test set. It is important to note that pretrained weights, which were generated by training on ImageNet, might not be expected to have learned to extract fully rotationally invariant features. Indeed, the objects found ImageNet mostly have a preferred orientation at which they generally are seen (there are very few instances of upside-down dogs or vehicles in ImageNet).
Figure shows the latent space Cs-EED for all eight model types recorded every 200 iterations. We see that in this experiment and for this specific metric, models trained from scratch extract more invariant features in their latent space. Moreover, while the invariance of models trained from scratch tends to decrease slightly over the course of training (latent space Cg-EED increases), the invariance of models trained using pretrained weights tends to increase slightly. This may relate ImageNet's lack of object classes that appear at wide range of orientations. The models with pretrained weights may need to learn additional rotation invariance. Contrast that with the weights trained from scratch that may learn non-robust rotation invariant features early which are refined to less rotationally invariant features that better optimize the cross-entropy loss function later. This, along with the fact that models using pretrained weights tend to have slightly higher accuracy, points to a complicated dynamics between model robustness, model invariance, and model performance which likely requires further exploration. We also calculated the latent space D3-EED (where Dg is the order 16 dihedral group) for these same models and dataset. The results are recorded in Figure 10] We find the patterns to be fairly similar.
On the other hand, Figure 16 in the Appendix shows that, in terms softmax CE-E the invariance properties of pretrained networks vs. networks trained from scratch is somewhat more ambiguous.
Model
0.6
ResNet50
AlexNet
0.5
DenseNeti21
LeViT192
Latent space C8-EED
Training
From scratch
Pretrained
0.3
0.2
0.1
2000
4000
6000
8000
10000
Iteration
Figure 4: The latent space Cs-EED measured every 200 training iterations (for total of 10,000 iterations) for a range of model architectures that were initialized with either random or ImageNet pretrained weights. Confidence intervals are over randomly initialized models for 'from scratch' and 5 independent trainings for 'pretrained'.
Summary: Our experiments make it clear that the way in which invariances are learned differs between networks using pretrained weights and networks trained from scratch.
4.4 The invariance of supervised and self-supervised models
In additional experiment, we compared the invariance of supervised models the invariance of self-supervised models. To do this we evaluated four different convolutional neural networks in one of the simplest possible settings where
the order 2 group of reflections across the vertical axis. The models we chose to test were a standard ResNet50 (we used the Torchvision pretrained weights)
20 335 a Big Transfer ResNet50V2_10 101 26 (with weights from 'PyTorch Image Models' [4S] ). ResNet50 trained using the self- supervede method DINO (we used the author's pretrained weights) and a ResNet50 trained using the self-supervised method SimCLR 7 (we used weights from
Note that the first two models utilized supervised training (with different size training sets)

---

and the second two utilized self-supervised training techniques. Further, SimCLR is trained with contrastive loss that directly optimizes for invariance of input specific transformations (including reflection). On the other hand, DINO uses 'knowledge distillation', without contrastive loss. We choose concentrate on latent space G-EED in this experiment.
Our results can be found in Figure 5 Note that since all of these models have a 2048-dimensional latent space, we can directly compare them. This something could not do in Section 4.3 We see that in terms of both versions of latent space G-EED (that which uses Euclidean distance and that which uses cosine similarity), the model trained using SimCLR has significantly more invariance. This is not surprising given that the contrastive loss that SimCLR uses explicitly optimizes for similarity between an image and its reflection. DINO, which also trained in self-supervised manner, but which does not explicitly use the contrastive loss is less invariant (how it compares to the two supervised models depends on which metric is used).
Summary: Use of contrastive loss can improve invariance but otherwise there is not strong evidence that supervipervi increases model invariance.
0.0
reflection LED (cosine)
Latent space reflectoo-EED
-0.5
-1.0
-1.5
-2.5
BIT
DINO
SimCLR
RosNet50
BIT
DINO
SimCLR
RosNet50
Model
Figure 5: Latent space G-EED for two models trained using supervised methods (BiT and ResNet50) and two models trained using self-supervised methods (DINO and SimCLR), where G is the order group of reflections across the vertical axis. SimCLR is the only model trained with contrastive loss. All models use some version of the ResNet architecture. (Left) The standard latent space G-EED metric. (Right) G-EED where cosine similarity is used instead of Euclidean distance. Error bars represent 95% confidence intervals over 100 random inputs from the ImageNet test
5 Limitations and Conclusion
Although the G-EED family of metrics that we introduce is flexible in many ways, it is not capable of capturing notions of invariance that do not have known underlying symmetry group (e.g., changes in image background), do not come from the action of a group (e.g., scaling corresponding to semigroup action). or (for the purpose of measuring equivariance of internal representations) do not have known action on the hidden activations of model (e.g., changes in color). An important next step would to bring our metrics to these broader notions of invariance and equivariance. would be particularly valuable if our metrics could be modified so that they can measure equivariance without explicitly specifying a group action in a hidden layer. This would enable us to measure G-EED for emergent types of learned equivariance where the action of G on the output space is not specified by the user.
In this paper we described novel family of metrics meant to measure equivariance in deep learning models. We use these metrics to suggest answers questions related the extent which neural networks are are not invariant or equivariant to different types of transformations. Surprisingly, while invariance and equivariance are fundamental topics within machine learning, this one the first works (to our knowledge) that tries to broadly measure these properties empirically in modern neural networks. hope that this work will open a conversation on this important topic that will ultimately lead to better understanding of why deep learning models behave the way that they do.
10

---

References
[1] LeViT: a vision transformer in convnet's clothing for faster inference. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12259-12269, 2021.
[2] A. Antoniou, A. Storkey, and H. Edwards. Data augmentation generative adversarial networks.
arXiv preprint arXiv: (1077711.444 1.04340, 2017.
[3] B. Bloem- Rioem-Reddy and Y. W. Teh. Probabilistic symmetries and invariant neural networks. J. Mach. Learn. Res., 21:90-1, 2020.
[4] M. Caron, H. Touvron, I. Misra, H. Jégou, J. Mairal, Bojanowski, and A. Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the International Conference on Computer Vision (ICCV), 2021.
[5] Chen, Dobriban, and H. Lee. Invariance reduces variance: Understanding data augmen- tation in deep learning and beyond. 2019.
[6] S. Chen, E. Dobriban, J.d J. Lee. A group-theoretic framework for data augmentation. Advances in Neural Information Processing Systems, 33:21321-21333 2020.
[7] T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597-1607, PMLR, 2020.
[8] O. J. Cobb, C. G. Wallis, A. N. Mavor-Parker, A. Marignier, M. A. Price, M. d' Avezac, and D. McEwen. Efficient generalized spherical CNNs. arXiv preprint arXiv:2010. 11661, 2020.
[9] T. Cohen and M. Welling. Group equivariant convolutional networks. In International confer- ence machine learning, pages 2990-2999. PMLR, 2016.
[10] S. Cohen and M. Welling. Steerable CNNs. arXiv preprint arXiv: 2016.
[11] S. Cohen, M. Geiger, Köhler, and M. Welling. Spherical CNNs. arXiv preprint 1rXiv:1801.10133 2018.
[12] E. D. Cubuk, B. Zoph, D. Mane, V. Vasudevan, and Q. V. Le. Autoaugment: Learning augmentation policies from data, arXiv preprint arXiv: 1805. 305.09501, 2018
[13] J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. In CVPRO9, 2009.
[14] L. Deng- The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, 2916):141-142, 2012.
[15] D. S. Dummit and R. M. Foote. Abstract algebra, volume 3. Wiley Hoboken, 2004.
[16] C. Esteves, C. Allen--Blanhhen Blanchette, A. Makadia, and K. Daniliddis. Learning SO(3) equivariant representations with spherical CNNs. In Proceedings of the European Conference on Computer Vision (ECCV), pages 52-68, 2018.
[17] G. B. Folland. Real analysis: modern techniques and their applications, volume 40. John Wiley Sons, 1999.
[18] Fuchs, D. Worrall, V. Fischer, and M. Welling. SE(3)-transformers: 3D roto-translation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970- 1981, 2020.
[19] S. Hauberg, O. Freifeld, A. B. L. Larsen, J. Fisher, and L. Hansen. Dreaming more data: Class-dependent distributions over diffcomorphisms for learned data augmentation. In Artificial Intelligence and Statistics, pages 342-350. PMLR, 2016.
[20] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770- 778, 2016.
11

---

[21] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger. Densely connected convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition. pages 4700-4708, 2017.
[22] M. J. Hutchinson, C. Le Lan, S. Zaidi, E. Dupont, Y. W. Teh, and H. Kim. Lietransformer: Equivariant self-attention for Lie groups. In International Conference on Machine Learning, pages 3333-4543. PMLR, 2021.
[23] J.-H. Jacobsen, J. Behrmann, R. Zemel, and M. Bethge. Excessive invariance causes adversarial vulnerability. arXiv preprint arXiv:1811.0040/ 2018.
[24] Kamath, A. Deshpande, and K. Subrahmanyam. Invariance vs robustness of neural networks. 2019.
[25] R. Kaur, S. Jha, A. Roy, S. Park, E. Dobriban, O. Sokolsky, and I. Lee. idecode: In-distribution equivariance for conformal cut-of-distributioo detection, 2022.
[26] A. Kolesnikov, L. Beyer, X. Zhai, J. Puigcerver, J. Yung. S. Gelly, and N. Houlsby. Big transfer (BiT): General visual representation learning. In European conference on computer vision, pages 491-507. Springer, 2020.
[27] R. Kondor and S. Trivedi. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In International Conference Machine Learning, pages
PMLR, 2018.
[28] A. Krizhevsky, G. Hinton, et al. Learning multiple layers of features from tiny images. 2009.
[29] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems- Volume 1, NIPS' 12, page 1097-1105, Red Hook, NY, USA, 2012. Curran Associates Inc.
[30] B. M. Lake, R. Salakhutdinov, and J. B. Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266): 333-1338, 2015.
[31] D. Lam, R. Kuzma, K. McGee, S. Dooley, M. Laielli, M. Klaric, Y. Bulatov, and B. McCord.
x XView: Objects in context in overhead imagery. arXiv preprint arXiv:/802.078556 2018
[32] Y. LeCun, P. Haffner, L. Bottou, and Y. Bengio. Object recognition with gradient- learning. In Shape, contour and grouping computer vision, pages 319-345, Springer, 1999.
[33] J. Lee, Y. Lee, Kim, A. Kosiorek, S. Choi, and Y. W. Teh. Set transformer: A framework for attention-based permutation-innariint neural networks. In K. Chaudhuri and R. Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 37533. PMLR, 09-15 Jun 2019. URL https://proceediigg
mlr wtwss//97/eeeeeeeeeeeeee html
[34] C. Lyle, M. van der Wilk, M. Kwiatkowska, Y. Gal, and B. Bloem- Bloem-Reddy, On the benefits of invariance in neural networks. arXiv preprint arXiv:2005.00177, 2020.
[35] S. Marcel and Y. Rodriguez. Torchvision the machine-vision package of torch. In Proceedings of the 18th ACM international conference on Multimedia, pages 1485-1488, 2010.
[36] D. Marcos, M. Volpi, N. Komodakis, and D. Tuia. Rotation equivariant vector field networks. In Proceedings of the IEEE International Conference on Computer Vision, pages 5048-5057, 2017.
[37] A. J. Ratner, H. Ehrenberg, Z. Hussain, J. Dunnmon, and C. Ré. Learning to compose domain- specific transformations for data augmentation. Advances in neural information processing systems, 30, 2017.
[38] I.. Sifre and S. Mallat. Combined scattering for rotation invariant texture analysis. In ESANN, volume 44, pages 68-81. Citeseer, 2012.
12

---

[39] L. Sifre and S. Mallat. Rotation, scaling and deformation invariant scattering for texture dis- crimination. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1233-1240) 2013.
[40] V. Singla, S. Ge, B. Ronen, and D. Jacobs. Shift invariance can reduce adversarial robustness. Advances in Neural Information Processing Systems, 34, 2021.
[41] L. Sixt, B. Wild, and T. Landgraf. Rendergan: Generating realistic labeled data. Frontiers in Robotics and AI, 5:66, 2018.
[42] Snell, K. Swersky, and R. Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017.
[43] I. Sosnovik, M. Szmaja, and A. Smeulders, Scale-equivariant steerable networks. arXiv preprint arXiv: 1093, 2019.
[44] J. Spijkervet. Simelr. httpss//githhu..oooo//llaaavrrrs
2020.
[45] M. Weiler and G. Cesa. General E(2)-equivariant steerable CNNs. Advances in Neural Information Processing Systems, 32: 14334-14345, 2019.
[46] M. Weiler, M. Geiger, M. Welling, W. Boomsma, and T. Cohen. 3D steerable CNNs: Learning rotationally equivariant features in volumetric data.
[47] M. Weiler, F. Hamprecht, and M. Storath. Learning steerable filters for rotation equivariant CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 849-853 858, 2018.
[48] R. Wightman. pytorch- image-maeeeee models 2019.
Pytorch image models.
htttp:///gituub..oo/riiggg
[49] D. Worrall and M. Welling. Deep scale-spaces: Equivariance over scale. Advances in Neural Information Processing Systems, 32, 2019.
[50] D. E. Worrall, S. J. Garbin, D. Turmukhambetov, and G. J. Brostow. Harmonic networks: Deep translation and rotation equivariance. In Proceedings of the IEEE Conference on Computer Vision and Puttern Recognition, pages 5028-5037, 2017.
[51] K. Xiao, L. Engstrom, A. Ilyas, and A. Madry. Noise signal: The role of image backgrounds in object recognition. arXiv preprint (tt:::2000.0999999 2020.
[52] R. Zhang. Making convolutional networks shift-invariant again. In International conference on machine learning. pages 7324-7334. PMLR, 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See Section
(c) Did you discuss any potential negative societal impacts of your work? [No] As the work investigates how deep learning algorithms work in a non-application specific setting, we did not see this as relevant.
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes] In both Proposition 3. and Proposition 3.2 we give rigorous and specific of assumptions.
(b) Did you include complete proofs of all theoretical results? [Yes] See Appendix A.5
and A.6
13

---

3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material explicit definitions for all metrics in Section 3.1 and include details that would allow
URL)? [Yes] We include
the experiments to be recreated in Section 4 and Appendix A.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [No] We specify all training and dataset details in Appendix A. and except for the training/test splits.
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes] We include confidence intervals in all of our figures.
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] See Appendix A.1
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite creators? [Yes] We cited all datasets and pretrained models that we used in our experiments.
(b) Did you mention the license of the assets? [Yes] We cite licenses for MNIST, x View, CIFAR1O, and Omniglot.
(c) Did you include any new assets either in the supplemental material or as URL? [No] The only asset our version of the public xViww dataset, VVew Maritime. We do describe how we created our version of this dataset.
(d) Did you discuss whether and how consent obtained from people whose data you're using/curating? [N/A]
(c) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid participants and the total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "iszFPpSTyZv", "reviewer": "Reviewer_Lyvv", "summary": "This paper proposes a measure of equivariance (and invariance) in 3 different contexts for multi-class classifiers: (1) at channel level in a CNN, (2) at representation level and (2) at softmax level.\n\nThe proposed measure, generically called *G*-EED, is derived from the equivariance requirements of a mapping $f$ made up of the compositions of several mappings $f_1, \\ldots, f_n$. The derivation is clear and insightful, I appreciated the that the mathematical derivation is well written and can be followed nicely.\n\nMore precisely, this work uses the generic formulation of *G*-EED to assess how **equivariant** intermediate layers of a CNN are, and how **invariant** the representation and softmax outputs are. The reason being that input transformation acts faithfully on intermediate spatial feature maps (thus allowing to measure equivariance), but are harder to transfer to representation and softmax (thus choosing to measure invariance).\n\nOne interesting aspect is that results are obtained using few test samples (50), which adds value to the method.\n\nThe results focus on using *G*-EED to assess how equivariance (and invariance) evolves during training, comparing a regular CNN and a group equivariant CNN for the C8 group (C8-CNN). Several OOD datasets are tested, yielding interesting findings related to how equivariance is preserved across layers and during training, as well as invariance properties of different DNNs and different initialization strategies.\n\nOverall, the paper is well written, with clear language and is enjoyable to read.", "questions": "* What is the justification for the use of xView Maritime? Is it because it has features in all orientations? If so, I'd specifically point this out in the paper.\n\n* The $Ker(\\phi_Y)$ definition in L153-154 is interesting. However,  I wonder if it is strictly necessary, since later on such  $Ker(\\phi_Y)$ is assumed to be the full $G$: all groups considered in the paper either act faithfully (for channelwise) or the $g$ elements are not required (for latent and softmax). \n  * I believe the paper would benefit from showcasing a usecase where the use of $Ker(\\phi_Y)$ is required. It would also reinforce the derivation of *G*-EED using the $Ker(\\phi_Y)$ definition.\n\nSome questions/suggestions related to the analysis of pre-trained vs. scratch models:\n* By eye-balling the results, it seems that Resnet converges to almost the same level of invariance with both initialization strategies. Could this be related to the presence of skip connections in that architecture? A similar (milder) result is observed for DenseNet, which also has several skip connections. I'd suggest doing a \"channel\"-wise analysis of the models shown in Figure 5, which could lead to some better understanding.\n* Resnet pre-trained achieves better invariance than LeViT trained from scratch. Is the architecture playing a big role in these results, even more than the pre-training strategy itself? \n* I would suggest adding a pre-trained model using some modern Self-Supervised Learning method (eg. SimCLR [1], BYOL [2]) and comparing those features with the ones pre-trained supervised.\n\nRelated to the different metrics used in the different *G*-EED variants:\n  * What is the justification for the the use of cosine similarity, $\\ell_2$ and $D_{KL}$ respectively. See the previous comment in the strengths/weaknesses section.\n  * Minor comment: There is a definition of  $\\mathcal{E_{latent}}$ and $\\mathcal{E_{softmax}}$, for completeness I suggest to add $\\mathcal{E_{channel}}$ in the equation after L188.\n\n*References:*\n* [1] Chen, Ting, et al. \"A simple framework for contrastive learning of visual representations.\" International conference on machine learning. PMLR, 2020.\n* [2] Grill, Jean-Bastien, et al. \"Bootstrap your own latent-a new approach to self-supervised learning.\" Advances in neural information processing systems 33 (2020): 21271-21284.\n", "limitations": "\nThe authors mention the limitations in Section 6, pointing out that the method is applied to classification but could be extended to other ML tasks. \nThe fact that exact symmetry groups are required is also mentioned as a limitation of the method.\n\n\nI do not see other societal issues related to this work.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n\n* The theoretical aspects of the paper are solid and the derivation of *G*-EED is reasonable.\n\n* The results show interesting findings: (1) Equivariance diminishes as training progresses for all CNNs, (2) Equivariance diminishes as we go deeper in the CNN, (3) C8-CNN yields higher invariance for all OOD datasets tested, (4) random networks can be perfectly invariant, raising a valid point about the assessment of invariance only and (5) pre-trained models show different invariance a properties than models trained from scratch.\n\n* I would like to emphasize the easy implementation of the proposed measure, as well as the low computational requirements (at least with the datasets shown in the paper), only requiring 50 data points.\n\n* The paper is well written, easy to follow and seems a priori reproducible from the data given.\n\n**Weaknesses**:\n\nListing hereafter some weaknesses I found while reading the paper, and also some suggestions aiming at making the paper even stronger.\n\n\n* The proposed approach is limited to exact symmetry groups such as rotation or reflection, which act faithfully on the spatial feature maps. While these groups are of great interest to the community, I still believe this is a limitation of the method. The community is actively researching how to tackle equivariance properties for non exact groups such as color augmentations, and any step in that direction would be highly appreciated.\n\n* Although theoretically sound, the metrics proposed eventually get simplified into basic equivariance/invariance losses. \n\n* Only one group (C8) is analyzed. I believe the paper would improve if the analysis was performed for some other exact group.\n\n* I found the use of 3 different metrics $m$ (one per each version of *G*-EED proposed) not as elegant as the rest of the paper. Using the same metric would be preferred. Here some comments about the metrics used in the paper:\n  * To compare (flattened) feature maps it seems more appropriate to use $\\ell_2$, since the magnitude is important in intermediate layers (their output will be consumed by later layers). By using cosine similarity, magnitude is not considered.\n  * Similarly, to compare representations ($Z$), it seems more appropriate to use cosine distance. In this case, the high dimensional representation is not expected to be clustered in terms of absolute distance but in terms of angle. In such high dimensionality, being close-by in terms of angle is already sufficient, since any pair of randomly sampled vectors is (mostly) orthogonal. Therefore, cosine distance is a common choice.\n  * The choice of $D_{KL}$ for softmax can also be misleading. Let $f(x)$ be classifier that performs poorly (eg. yields $f(x)=constant+\\epsilon$, with $\\epsilon \\sim \\mathcal{N}(0, \\sigma)$ and $\\sigma$ very small). Then $\\mathcal{E_{softmax}}$ will be very low, indicating invariance, since the softmax distribution of $f(x)$ and $f(gx)$ are the same.  However, the classifier is actually poorly invariant ($f(x)$ changes randomly). Summarising, the choice of $D_{KL}$ implicitly assumes a well-trained classifier. This could also partly explain why *G*-EED yields 0 (fully invariant) for the randomly initialised model.\n  * I believe the choice of metric is critical in this method and should be evaluated in an ablations study (and theoretically if possible).\n\n* The comparison of pre-trained and scratch models is not as conclusive as the other results in the paper. I believe some additional analysis could be done in that sense. \n\n* Minor comment: I found the schema in Figure 1 hard to interpret. I'd add some extra information to it that more accurately shows what is done at each step in the DNN chain.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "_obrShA8pQU", "reviewer": "Reviewer_YrEP", "summary": "The paper aims to formally define group-theoretic family of metrics to evaluate empirical equivariance of a model. The contributions can be summarized as\n- the introduction of a G-empirical equivariance deviation (G-EED)\n- the introduction of possible instantiations of such deviation to measure specific invariances and equivariances \n- the use of G-EED to obtain insights on equivariance and invariance in neural networks\n", "questions": "- G-EED goodness strictly depends on the choice of the finite group. \nFor example, if we were to test against any possible rotation, the cyclic C_8 group the authors selected might not be sufficient in fully evaluating invariance to rotation. \nSimilarly in the context of permutation invariance (which would be measured at the latent space and softmax level) one would ideally try all the possible permutations over the input set. \nHowever, to experimentally evaluate G-EED we need to provide a finite group. Could the authors comment on how to select the most suitable group in practice?  How does the choice of the group impact the reliability of the deviation? A discussion about regular spacing vs random sampling would be interesting as well in selecting the finite rotation group. What is the best spacing? And/or what is the best number of sampled rotations? Are uniform samples always the best options or are other possible sampling strategies? \nA possible experiment could be to show how G-EED changes by adding more rotation angles to C_8. \n- The authors write in line 277-278 “Invariance is useful when it is aligned with other metrics more directly related to the task the model was designed for”, yet in their motivation they state that understanding invariance allows them to detect model failures. While it seems that G-EED allows for interesting comparison over different models designed for similar tasks, wouldn't the authors agree that to actually evaluate failure and success G-EED provides less of a tool than standard metrics? \n\n- It could be good to have an intuition of the deviation range to be able to use G-EED on a new model. Ideally, one would like to know that  below (or above) a certain value the G-equivariance of the network is good. Do the authors have a way to estimate such value? If not, would the authors say that the proposed deviation has to always be used comparatively? \nTo make a more practical example, the right panel of Figure 3 shows a high variability in the softmax for CNN. The plot intuitively seems to suggest that CNN has  worse performance, yet the classification results are good >98%. To my understanding, this means that the relative distance of the two models compared to the overall possible range of the deviation is not that significant.  \n- Figure 2 shows that as iterations go by the equivariance properties of CNN decreases, the authors propose as an explanation the fact that inductive bias on equivariance might not be aligned with learning objective, yet, in a later experiment they show that random weights pertain higher equivariance than transfer weights, could the decay then simply due to initialization? \n- The discrepancies between latent space and softmax suggest that the decoder has a high importance in learning invariances. If the network is $f = f2 \\circ f1$, the results seem to suggest that even if $f1$ is not G-equivariant $f$ might be through the mean of the decoder $f2$. However, generally, in the construction of architectures more importance is given in encoding equivariance in early layers. If I'm interpreting the results correctly, this could actually mean that such efforts are misplaced. Do the authors agree? Could they comment on this? \n\nPlease improve the figures, for example by making the text bigger and improving definition as well as aligning the ticks in Figure 4 to improve readability. \n\nTypos:\n\t\n- Line 134 (then instead of than)\n\t\n- line 162 where \n", "limitations": "The authors address some limitations in the paper. I would have liked a broader discussion on the usability of the proposed approach given the many prior choices that G-EED depends on. I'm happy to increase my score based on authors’ response and if other more expert reviewers have better feedback. \n", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well written and easy to follow. I found particularly pleasant the formal introduction of G-EED and its possible instantiations to measure equivariance and invariance at different parts of the network. The intuitions provided by the authors about different equivariant and invariant mechanisms are interesting. However,  the experimental validation of the proposed approach is somewhat lacking and the paper could have higher impact with a more well-thought experiment and result section. The authors test their deviation just in the context of 45 degrees rotation. It would make a more compelling argument of the usefulness of the proposed deviation to analyze other invariances, e.g. translation, permutation. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4mB1SwgzvUm", "reviewer": "Reviewer_1AHW", "summary": "This paper proposes a method for empirically measuring the degree of equivariance of a CNN to a given dataset.  In experiments on real datasets, the paper finds that equivariant architectures (but not data augmentation) does lead to equivariant layers, that equivariant architectures do not always lead to invariant final layers (perhaps due to accumulation of artifacts), and that use of pretrained weights may prevent the learning of equivariant layers.", "questions": "1) How can one ensure that the group action is defined in a way that could capture any hidden group structure in a non-equivariant architecture?  Wouldn't it be necessary to search over possible group structures within a layer?\n2) How did you define the C8-group actions for the standard CNN and for the steerable CNN?\n3) How would you define the appropriate group actions for a more complicated example, such as 3d rotations for volumetric data?\n4) Line 538: How does it follow from the assumptions of Proposition 3.2 that the action of G on Y is trivial?\n5) Appendix A.4: How do the (if) and (only if) directions in Proposition 3.2 follow from measurability of h, non-negativity of h, m, and from (6)?  Could you write out these steps of the proof explicitly?\n", "limitations": "The paper addressed a number of important limitations, but does not address the serious problem of how to correctly define the group actions, especially when there could exist hidden approximate group structure that was not specified by the architecture but rather emerged during training.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "## Originality\nWhile many methods have been developed for CNNs to be built in a manner to be exactly or approximately equivariant, it is a challenging problem to assess the degree of equivariance in neural net architectures that are not a priori equivariant.  To my knowledge, this paper is the first to attempt to do so.\n## Quality\nThe question of how to define an approximately equivariant layer, even in a network not explicitly designed to be equivariant to a particular group, is a very interesting question.  It has implications for the entire subfield of equivariant neural networks, and so I think it is important to investigate this question thoroughly.  Therefore, before I present any criticisms I might have about the paper, I would like to personally encourage the authors to not be discouraged but rather to deepen their investigation, as it could have impact on how much the community invests into researching equivariant CNNs in the future.\n\nGiven the claim of the paper that \"data augmentation does not lead to equivariant layers\", it is crucial that we have a clear definition of \"equivariant layer.\"  The notion used by the paper, roughly stated, is that an (approximately) equivariant layer is one where for any transformation of the the input taken from the group G, there exists a corresponding group action on the layer which renders the result to be approximately equivalent to the output of the layer on the untransformed input.  However, within the definition there is some ambiguity to what latitude one has to define the group action.  Different approaches to defining the group action may either reveal or fail to reveal an equivariant structure.  Therefore, in order for this paper to succeed at answering the question \"how to measure degree of equivariance,\" I think it is crucial to tighten the definition of approximate equivariance developed by this paper.  In particular, it needs to be decided:\n* Is an equivariant network one where there exists *some* group action that achieves close to the optimal value of Equation 3?\n* Or is equivariance a property of a *pair* (network, group action), so that we cannot say that \"this network is equivariant\" or \"this network is not equivariant\", but only that \"this network is (not) equivariant *with respect to* this particular group action\"?\n\nMy guess (given lack of details) is that for the standard CNN, the group action employed was one that produces a rotation of the output of the layer given a rotation of the input.  However, this is not the only group action that might make sense for the CNN!  You see, I am concerned that the proposed approach is failing to detect more complex equivariant structures that might emerge during training.  I do not know these hypothesized equivariant structures actually occur in practice.  However, I think it is a crucial question to check whether they exist.  I will now proceed to describe the possible hidden equivariant structures that I think the proposed approach might be missing.  In short, I think a structure similar to what equivariant CNNs are designed to learn might emerge naturally during the training of a non-equivariant CNN.\n\nI call these hypothetical structures *emergent equivariant stacks*.  In fact, Cohen and Welling (2016) first broach the possibility of such an equivariant structure, as they state on page 5 \"if an ordinary CNN learns rotated copies of the same filter, the stack of feature maps is equivariant, although individual feature maps are not.\"  To illustrate my understanding of what an equivariant stack of feature maps might look like, consider the following.  Let $\\{e, r, r^2, r^3\\}$ be the elements of the C4 group (90-degree rotations).  An equivariant stack would consist of four feature maps, labeled $\\phi_0, \\phi_1, \\phi_2, \\phi_3$, where $\\phi_i(X) \\approx r \\phi(r^{-i} X)$.  Without loss of generality, suppose the layer $f$ consists of these four neurons, and the output of $f$ is\n\n$$f(X) = (\\phi_0(X), \\phi_1(X), \\phi_2(X), \\phi_3(X))$$\n\nThen for the proposed approach to capture the approximate equivariance of the network, the group action would have to be defined such that\n\n$$r f(X) = (\\phi_1(r X), \\phi_2(r^{-1} X), \\phi_3(r^{-1} X), \\phi_0(r^{-1} X))$$\n$$r^2 f(X) = (\\phi_2(r^{-2} X), \\phi_3(r^{-2} X), \\phi_0(r^{-2} X), \\phi_1(r^{-2} X))$$\n$$r^3 f(X) = (\\phi_3(r^{-3} X), \\phi_0(r^{-3} X), \\phi_1(r^{-3} X), \\phi_2(r^{-3} X)).$$\n\nSince the group structure of $\\{\\phi_0, \\phi_1, \\phi_2, \\phi_3\\}$ would be unknown a priori, to identify such a hidden equivariant structure, one approach might be to search over permutations of the neurons as part of the process of defining the group action on the layer.  Therefore, perhaps the proposed approach could be augmented by including such a group structure, leading to a stronger notion of equivariance (rather than equivariance with respect to a particular pre-defined group action), and which I would consider a far more comprehensive way to answer the question \"does data augmentation lead to equivariant layers?\"  Note also that the framework of the paper (equations 2 and 3) would be more or less kept intact: the search for group structure would be a valuable add-on to the existing material in the paper.\n\nI personally think that it is essential for the paper to look into the possibility of emergent equivariant stacks.  That is why I have recommended borderline rejection for the current submission.  However, it is possible that reviewers with more experience with equivariant networks may disagree with my assessment of the plausibility or the importance of accounting for emergent equivariant stacks.  In that case, I could easily see them recommending a higher score.\n\nNow, it could also very well be the case that equivariant stacks of more than one feature are *not* likely to emerge for non-equivariant CNNs.  I think it is important to check.  But if the problem I outlined does not actually matter in practice, then the approach adopted by the paper is fine.  Conditional on hidden equivariance not being an issue, there are many positive aspects of the paper.  The measure of equivariance (eqs 2 and 3) appears to be sound, and the evaluation is reasonable and produces interesting results.  The quality would be higher if more types of symmetry groups and data types were evaluated, such as a simple p4 G-CNN, where the method should show exact equivariance, and a more complicated example such as 3d rotations for point clouds.\n## Clarity\nThe paper is clearly written, but I found one of the proofs to be lacking in details.  It would also greatly improve the clarity of the paper if the example presented (C8) could be explained in detail.  The difficulty in understanding exactly how the method operates in a non-elementary concrete example, such as C8, lowers the clarity of the paper.\n## Significance\nAs I stated in \"Quality\", I think the question being investigated is very important, as it could have impact on how much the community invests into researching equivariant CNNs in the future.  While it is unclear how well the method can be applied to non-equivariant CNN architectures, the method seems to be reasonable for equivariant architectures, and is immediately useful for evaluating the success of equivariant architectures. The finding that C8 has lower equivariance for later layers is potentially novel and interesting, although details to reproduce the finding are currently missing.\n## Suggestions\nI would ideally like to see an approach similar to the paper that could include a strategy to look for emergent equivariant stacks.  However, I acknowledge that the authors are not likely to have enough time to implement this during the response period.  I would be willing to increase my score to 'borderline accept' or 'weak accept' if the authors can discuss the limitation of the current method in being insensitive to stacks of approximately equivariant features, and also include missing details that would increase the reproducibility of the experiments.  In that case, publication of the paper could serve to stimulate further research in measuring equivariance in networks. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "2k4hHLbNoz", "reviewer": "Reviewer_9Wpf", "summary": "The paper introduces a metric for measuring invariance/equivariance of a model wrt a certain symmetry group and data distribution, called G-empirical equivariance deviation. This metric is used to compare to what extent equivariant models and models trained with data augmentation are invariant/have equivariant feature maps, and also to assess to what extent invariance/equivariance holds for OOD data.The paper also compares the extent of invariance/equivariance for using pretrained weights & fine tuning vs training from randomly initialized weights.", "questions": "Questions & Suggestions:\n- It's confusing that the channelwise G-EED is different from softmax & latent space EED, in that a higher value means the model is more equivariant. Perhaps it's more consistent to use negative cosine similarity to be consistent with other metrics.\n- Why is L2 distance used for latent space EED whereas cosine similarity is used for channelwise EED? If cosine similarity is also used for the former, it might not be necessary to normalizie by M, the average L2 distance between points.\n- Figure 2 is used as evidence to claim \"learned invariance does not arise through learned equivariance\". It would be more convincing to show the plot for each value of k between 1 and 6 as opposed to just 2 & 5. As you mentioned in Prop 3.1, it could be that the first 5 layers aren't equivariant but all 6 conv layers are equivariant.\n- In line 277-278, you mention that \"invariance is useful when it is aligned with other metrics more directly related to the task the model was designed for\". This makes me curious as to what would happen if you use each EED metric as a regulariser for a CNN - would it help of harm generalization?\n- (minor) Figure 4 - x axis shifted by one.\n", "limitations": "n/a", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- the paper is very clearly written, and does a good job of explaining concepts and motivation.\n- it formalizes much needed metrics for measuring equivariance and invariance.\n- the summaries at the end of each experimental section was useful to recap the conclusions of each experiment.\n- Although the empirical results of this paper are not necessarily surprising and quite aligned with my intuition, it is useful to have such intuitions validated by empirical evidence (e.g. nonequivariant networks not learning layerwise equivariance, models trained from scratch are better at extracting invariant features)\n\nWeaknesses\n- the equivariance error metric in the paper is very much related to those used in various previous works on group equivariance, so not entirely novel. Search for 'equivariance error' in e.g. [1-3]. It would be good to cite these papers and mention that similar metrics have been used in the past, but all with slight differences. I still think there is value in formalizing such a metric so that researchers can agree to all use a common metric.\n- would be more practically relevant if the experiments were beyond just C_8 rotations on mnist, but closer to SOTA classifiers on imagenet with practically used augmentations such as rotations with arbitrary angles, cropping, flipping, colour distortions, shears etc.\n- related point to above - I don't see what is preventing you from going beyond finite groups to generalize the EED metric e.g. to non-group transformations that need not be finite such as those transformations mentioned above. In Eq (3), you could approximate the integral wrt the set of transformations G with samples (i.e. MC approximation) as you are already doing for approximating the integral wrt the data distribution.\n\n[1] SE(3)-Transformers - Fuchs et al., 2020\n[2] LieTransformer: Equivariant self-attention for Lie Groups - Hutchinson et al, 2020.\n[3] Efficient Generalized Spherical CNNs - Cobb et al., 2021", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept