Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a measure of equivariance (and invariance) in 3 different contexts for multi-class classifiers: (1) at channel level in a CNN, (2) at representation level and (2) at softmax level.

The proposed measure, generically called *G*-EED, is derived from the equivariance requirements of a mapping $f$ made up of the compositions of several mappings $f_1, \ldots, f_n$. The derivation is clear and insightful, I appreciated the that the mathematical derivation is well written and can be followed nicely.

More precisely, this work uses the generic formulation of *G*-EED to assess how **equivariant** intermediate layers of a CNN are, and how **invariant** the representation and softmax outputs are. The reason being that input transformation acts faithfully on intermediate spatial feature maps (thus allowing to measure equivariance), but are harder to transfer to representation and softmax (thus choosing to measure invariance).

One interesting aspect is that results are obtained using few test samples (50), which adds value to the method.

The results focus on using *G*-EED to assess how equivariance (and invariance) evolves during training, comparing a regular CNN and a group equivariant CNN for the C8 group (C8-CNN). Several OOD datasets are tested, yielding interesting findings related to how equivariance is preserved across layers and during training, as well as invariance properties of different DNNs and different initialization strategies.

Overall, the paper is well written, with clear language and is enjoyable to read.
strengths_and_weaknesses: **Strengths**:

* The theoretical aspects of the paper are solid and the derivation of *G*-EED is reasonable.

* The results show interesting findings: (1) Equivariance diminishes as training progresses for all CNNs, (2) Equivariance diminishes as we go deeper in the CNN, (3) C8-CNN yields higher invariance for all OOD datasets tested, (4) random networks can be perfectly invariant, raising a valid point about the assessment of invariance only and (5) pre-trained models show different invariance a properties than models trained from scratch.

* I would like to emphasize the easy implementation of the proposed measure, as well as the low computational requirements (at least with the datasets shown in the paper), only requiring 50 data points.

* The paper is well written, easy to follow and seems a priori reproducible from the data given.

**Weaknesses**:

Listing hereafter some weaknesses I found while reading the paper, and also some suggestions aiming at making the paper even stronger.


* The proposed approach is limited to exact symmetry groups such as rotation or reflection, which act faithfully on the spatial feature maps. While these groups are of great interest to the community, I still believe this is a limitation of the method. The community is actively researching how to tackle equivariance properties for non exact groups such as color augmentations, and any step in that direction would be highly appreciated.

* Although theoretically sound, the metrics proposed eventually get simplified into basic equivariance/invariance losses. 

* Only one group (C8) is analyzed. I believe the paper would improve if the analysis was performed for some other exact group.

* I found the use of 3 different metrics $m$ (one per each version of *G*-EED proposed) not as elegant as the rest of the paper. Using the same metric would be preferred. Here some comments about the metrics used in the paper:
  * To compare (flattened) feature maps it seems more appropriate to use $\ell_2$, since the magnitude is important in intermediate layers (their output will be consumed by later layers). By using cosine similarity, magnitude is not considered.
  * Similarly, to compare representations ($Z$), it seems more appropriate to use cosine distance. In this case, the high dimensional representation is not expected to be clustered in terms of absolute distance but in terms of angle. In such high dimensionality, being close-by in terms of angle is already sufficient, since any pair of randomly sampled vectors is (mostly) orthogonal. Therefore, cosine distance is a common choice.
  * The choice of $D_{KL}$ for softmax can also be misleading. Let $f(x)$ be classifier that performs poorly (eg. yields $f(x)=constant+\epsilon$, with $\epsilon \sim \mathcal{N}(0, \sigma)$ and $\sigma$ very small). Then $\mathcal{E_{softmax}}$ will be very low, indicating invariance, since the softmax distribution of $f(x)$ and $f(gx)$ are the same.  However, the classifier is actually poorly invariant ($f(x)$ changes randomly). Summarising, the choice of $D_{KL}$ implicitly assumes a well-trained classifier. This could also partly explain why *G*-EED yields 0 (fully invariant) for the randomly initialised model.
  * I believe the choice of metric is critical in this method and should be evaluated in an ablations study (and theoretically if possible).

* The comparison of pre-trained and scratch models is not as conclusive as the other results in the paper. I believe some additional analysis could be done in that sense. 

* Minor comment: I found the schema in Figure 1 hard to interpret. I'd add some extra information to it that more accurately shows what is done at each step in the DNN chain.
questions: * What is the justification for the use of xView Maritime? Is it because it has features in all orientations? If so, I'd specifically point this out in the paper.

* The $Ker(\phi_Y)$ definition in L153-154 is interesting. However,  I wonder if it is strictly necessary, since later on such  $Ker(\phi_Y)$ is assumed to be the full $G$: all groups considered in the paper either act faithfully (for channelwise) or the $g$ elements are not required (for latent and softmax). 
  * I believe the paper would benefit from showcasing a usecase where the use of $Ker(\phi_Y)$ is required. It would also reinforce the derivation of *G*-EED using the $Ker(\phi_Y)$ definition.

Some questions/suggestions related to the analysis of pre-trained vs. scratch models:
* By eye-balling the results, it seems that Resnet converges to almost the same level of invariance with both initialization strategies. Could this be related to the presence of skip connections in that architecture? A similar (milder) result is observed for DenseNet, which also has several skip connections. I'd suggest doing a "channel"-wise analysis of the models shown in Figure 5, which could lead to some better understanding.
* Resnet pre-trained achieves better invariance than LeViT trained from scratch. Is the architecture playing a big role in these results, even more than the pre-training strategy itself? 
* I would suggest adding a pre-trained model using some modern Self-Supervised Learning method (eg. SimCLR [1], BYOL [2]) and comparing those features with the ones pre-trained supervised.

Related to the different metrics used in the different *G*-EED variants:
  * What is the justification for the the use of cosine similarity, $\ell_2$ and $D_{KL}$ respectively. See the previous comment in the strengths/weaknesses section.
  * Minor comment: There is a definition of  $\mathcal{E_{latent}}$ and $\mathcal{E_{softmax}}$, for completeness I suggest to add $\mathcal{E_{channel}}$ in the equation after L188.

*References:*
* [1] Chen, Ting, et al. "A simple framework for contrastive learning of visual representations." International conference on machine learning. PMLR, 2020.
* [2] Grill, Jean-Bastien, et al. "Bootstrap your own latent-a new approach to self-supervised learning." Advances in neural information processing systems 33 (2020): 21271-21284.

limitations: 
The authors mention the limitations in Section 6, pointing out that the method is applied to classification but could be extended to other ML tasks. 
The fact that exact symmetry groups are required is also mentioned as a limitation of the method.


I do not see other societal issues related to this work.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper aims to formally define group-theoretic family of metrics to evaluate empirical equivariance of a model. The contributions can be summarized as
- the introduction of a G-empirical equivariance deviation (G-EED)
- the introduction of possible instantiations of such deviation to measure specific invariances and equivariances 
- the use of G-EED to obtain insights on equivariance and invariance in neural networks

strengths_and_weaknesses: The paper is well written and easy to follow. I found particularly pleasant the formal introduction of G-EED and its possible instantiations to measure equivariance and invariance at different parts of the network. The intuitions provided by the authors about different equivariant and invariant mechanisms are interesting. However,  the experimental validation of the proposed approach is somewhat lacking and the paper could have higher impact with a more well-thought experiment and result section. The authors test their deviation just in the context of 45 degrees rotation. It would make a more compelling argument of the usefulness of the proposed deviation to analyze other invariances, e.g. translation, permutation. 
questions: - G-EED goodness strictly depends on the choice of the finite group. 
For example, if we were to test against any possible rotation, the cyclic C_8 group the authors selected might not be sufficient in fully evaluating invariance to rotation. 
Similarly in the context of permutation invariance (which would be measured at the latent space and softmax level) one would ideally try all the possible permutations over the input set. 
However, to experimentally evaluate G-EED we need to provide a finite group. Could the authors comment on how to select the most suitable group in practice?  How does the choice of the group impact the reliability of the deviation? A discussion about regular spacing vs random sampling would be interesting as well in selecting the finite rotation group. What is the best spacing? And/or what is the best number of sampled rotations? Are uniform samples always the best options or are other possible sampling strategies? 
A possible experiment could be to show how G-EED changes by adding more rotation angles to C_8. 
- The authors write in line 277-278 “Invariance is useful when it is aligned with other metrics more directly related to the task the model was designed for”, yet in their motivation they state that understanding invariance allows them to detect model failures. While it seems that G-EED allows for interesting comparison over different models designed for similar tasks, wouldn't the authors agree that to actually evaluate failure and success G-EED provides less of a tool than standard metrics? 

- It could be good to have an intuition of the deviation range to be able to use G-EED on a new model. Ideally, one would like to know that  below (or above) a certain value the G-equivariance of the network is good. Do the authors have a way to estimate such value? If not, would the authors say that the proposed deviation has to always be used comparatively? 
To make a more practical example, the right panel of Figure 3 shows a high variability in the softmax for CNN. The plot intuitively seems to suggest that CNN has  worse performance, yet the classification results are good >98%. To my understanding, this means that the relative distance of the two models compared to the overall possible range of the deviation is not that significant.  
- Figure 2 shows that as iterations go by the equivariance properties of CNN decreases, the authors propose as an explanation the fact that inductive bias on equivariance might not be aligned with learning objective, yet, in a later experiment they show that random weights pertain higher equivariance than transfer weights, could the decay then simply due to initialization? 
- The discrepancies between latent space and softmax suggest that the decoder has a high importance in learning invariances. If the network is $f = f2 \circ f1$, the results seem to suggest that even if $f1$ is not G-equivariant $f$ might be through the mean of the decoder $f2$. However, generally, in the construction of architectures more importance is given in encoding equivariance in early layers. If I'm interpreting the results correctly, this could actually mean that such efforts are misplaced. Do the authors agree? Could they comment on this? 

Please improve the figures, for example by making the text bigger and improving definition as well as aligning the ticks in Figure 4 to improve readability. 

Typos:
	
- Line 134 (then instead of than)
	
- line 162 where 

limitations: The authors address some limitations in the paper. I would have liked a broader discussion on the usability of the proposed approach given the many prior choices that G-EED depends on. I'm happy to increase my score based on authors’ response and if other more expert reviewers have better feedback.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a method for empirically measuring the degree of equivariance of a CNN to a given dataset.  In experiments on real datasets, the paper finds that equivariant architectures (but not data augmentation) does lead to equivariant layers, that equivariant architectures do not always lead to invariant final layers (perhaps due to accumulation of artifacts), and that use of pretrained weights may prevent the learning of equivariant layers.
strengths_and_weaknesses: ## Originality
While many methods have been developed for CNNs to be built in a manner to be exactly or approximately equivariant, it is a challenging problem to assess the degree of equivariance in neural net architectures that are not a priori equivariant.  To my knowledge, this paper is the first to attempt to do so.
## Quality
The question of how to define an approximately equivariant layer, even in a network not explicitly designed to be equivariant to a particular group, is a very interesting question.  It has implications for the entire subfield of equivariant neural networks, and so I think it is important to investigate this question thoroughly.  Therefore, before I present any criticisms I might have about the paper, I would like to personally encourage the authors to not be discouraged but rather to deepen their investigation, as it could have impact on how much the community invests into researching equivariant CNNs in the future.

Given the claim of the paper that "data augmentation does not lead to equivariant layers", it is crucial that we have a clear definition of "equivariant layer."  The notion used by the paper, roughly stated, is that an (approximately) equivariant layer is one where for any transformation of the the input taken from the group G, there exists a corresponding group action on the layer which renders the result to be approximately equivalent to the output of the layer on the untransformed input.  However, within the definition there is some ambiguity to what latitude one has to define the group action.  Different approaches to defining the group action may either reveal or fail to reveal an equivariant structure.  Therefore, in order for this paper to succeed at answering the question "how to measure degree of equivariance," I think it is crucial to tighten the definition of approximate equivariance developed by this paper.  In particular, it needs to be decided:
* Is an equivariant network one where there exists *some* group action that achieves close to the optimal value of Equation 3?
* Or is equivariance a property of a *pair* (network, group action), so that we cannot say that "this network is equivariant" or "this network is not equivariant", but only that "this network is (not) equivariant *with respect to* this particular group action"?

My guess (given lack of details) is that for the standard CNN, the group action employed was one that produces a rotation of the output of the layer given a rotation of the input.  However, this is not the only group action that might make sense for the CNN!  You see, I am concerned that the proposed approach is failing to detect more complex equivariant structures that might emerge during training.  I do not know these hypothesized equivariant structures actually occur in practice.  However, I think it is a crucial question to check whether they exist.  I will now proceed to describe the possible hidden equivariant structures that I think the proposed approach might be missing.  In short, I think a structure similar to what equivariant CNNs are designed to learn might emerge naturally during the training of a non-equivariant CNN.

I call these hypothetical structures *emergent equivariant stacks*.  In fact, Cohen and Welling (2016) first broach the possibility of such an equivariant structure, as they state on page 5 "if an ordinary CNN learns rotated copies of the same filter, the stack of feature maps is equivariant, although individual feature maps are not."  To illustrate my understanding of what an equivariant stack of feature maps might look like, consider the following.  Let $\{e, r, r^2, r^3\}$ be the elements of the C4 group (90-degree rotations).  An equivariant stack would consist of four feature maps, labeled $\phi_0, \phi_1, \phi_2, \phi_3$, where $\phi_i(X) \approx r \phi(r^{-i} X)$.  Without loss of generality, suppose the layer $f$ consists of these four neurons, and the output of $f$ is

$$f(X) = (\phi_0(X), \phi_1(X), \phi_2(X), \phi_3(X))$$

Then for the proposed approach to capture the approximate equivariance of the network, the group action would have to be defined such that

$$r f(X) = (\phi_1(r X), \phi_2(r^{-1} X), \phi_3(r^{-1} X), \phi_0(r^{-1} X))$$
$$r^2 f(X) = (\phi_2(r^{-2} X), \phi_3(r^{-2} X), \phi_0(r^{-2} X), \phi_1(r^{-2} X))$$
$$r^3 f(X) = (\phi_3(r^{-3} X), \phi_0(r^{-3} X), \phi_1(r^{-3} X), \phi_2(r^{-3} X)).$$

Since the group structure of $\{\phi_0, \phi_1, \phi_2, \phi_3\}$ would be unknown a priori, to identify such a hidden equivariant structure, one approach might be to search over permutations of the neurons as part of the process of defining the group action on the layer.  Therefore, perhaps the proposed approach could be augmented by including such a group structure, leading to a stronger notion of equivariance (rather than equivariance with respect to a particular pre-defined group action), and which I would consider a far more comprehensive way to answer the question "does data augmentation lead to equivariant layers?"  Note also that the framework of the paper (equations 2 and 3) would be more or less kept intact: the search for group structure would be a valuable add-on to the existing material in the paper.

I personally think that it is essential for the paper to look into the possibility of emergent equivariant stacks.  That is why I have recommended borderline rejection for the current submission.  However, it is possible that reviewers with more experience with equivariant networks may disagree with my assessment of the plausibility or the importance of accounting for emergent equivariant stacks.  In that case, I could easily see them recommending a higher score.

Now, it could also very well be the case that equivariant stacks of more than one feature are *not* likely to emerge for non-equivariant CNNs.  I think it is important to check.  But if the problem I outlined does not actually matter in practice, then the approach adopted by the paper is fine.  Conditional on hidden equivariance not being an issue, there are many positive aspects of the paper.  The measure of equivariance (eqs 2 and 3) appears to be sound, and the evaluation is reasonable and produces interesting results.  The quality would be higher if more types of symmetry groups and data types were evaluated, such as a simple p4 G-CNN, where the method should show exact equivariance, and a more complicated example such as 3d rotations for point clouds.
## Clarity
The paper is clearly written, but I found one of the proofs to be lacking in details.  It would also greatly improve the clarity of the paper if the example presented (C8) could be explained in detail.  The difficulty in understanding exactly how the method operates in a non-elementary concrete example, such as C8, lowers the clarity of the paper.
## Significance
As I stated in "Quality", I think the question being investigated is very important, as it could have impact on how much the community invests into researching equivariant CNNs in the future.  While it is unclear how well the method can be applied to non-equivariant CNN architectures, the method seems to be reasonable for equivariant architectures, and is immediately useful for evaluating the success of equivariant architectures. The finding that C8 has lower equivariance for later layers is potentially novel and interesting, although details to reproduce the finding are currently missing.
## Suggestions
I would ideally like to see an approach similar to the paper that could include a strategy to look for emergent equivariant stacks.  However, I acknowledge that the authors are not likely to have enough time to implement this during the response period.  I would be willing to increase my score to 'borderline accept' or 'weak accept' if the authors can discuss the limitation of the current method in being insensitive to stacks of approximately equivariant features, and also include missing details that would increase the reproducibility of the experiments.  In that case, publication of the paper could serve to stimulate further research in measuring equivariance in networks. 

questions: 1) How can one ensure that the group action is defined in a way that could capture any hidden group structure in a non-equivariant architecture?  Wouldn't it be necessary to search over possible group structures within a layer?
2) How did you define the C8-group actions for the standard CNN and for the steerable CNN?
3) How would you define the appropriate group actions for a more complicated example, such as 3d rotations for volumetric data?
4) Line 538: How does it follow from the assumptions of Proposition 3.2 that the action of G on Y is trivial?
5) Appendix A.4: How do the (if) and (only if) directions in Proposition 3.2 follow from measurability of h, non-negativity of h, m, and from (6)?  Could you write out these steps of the proof explicitly?

limitations: The paper addressed a number of important limitations, but does not address the serious problem of how to correctly define the group actions, especially when there could exist hidden approximate group structure that was not specified by the architecture but rather emerged during training.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The paper introduces a metric for measuring invariance/equivariance of a model wrt a certain symmetry group and data distribution, called G-empirical equivariance deviation. This metric is used to compare to what extent equivariant models and models trained with data augmentation are invariant/have equivariant feature maps, and also to assess to what extent invariance/equivariance holds for OOD data.The paper also compares the extent of invariance/equivariance for using pretrained weights & fine tuning vs training from randomly initialized weights.
strengths_and_weaknesses: Strengths
- the paper is very clearly written, and does a good job of explaining concepts and motivation.
- it formalizes much needed metrics for measuring equivariance and invariance.
- the summaries at the end of each experimental section was useful to recap the conclusions of each experiment.
- Although the empirical results of this paper are not necessarily surprising and quite aligned with my intuition, it is useful to have such intuitions validated by empirical evidence (e.g. nonequivariant networks not learning layerwise equivariance, models trained from scratch are better at extracting invariant features)

Weaknesses
- the equivariance error metric in the paper is very much related to those used in various previous works on group equivariance, so not entirely novel. Search for 'equivariance error' in e.g. [1-3]. It would be good to cite these papers and mention that similar metrics have been used in the past, but all with slight differences. I still think there is value in formalizing such a metric so that researchers can agree to all use a common metric.
- would be more practically relevant if the experiments were beyond just C_8 rotations on mnist, but closer to SOTA classifiers on imagenet with practically used augmentations such as rotations with arbitrary angles, cropping, flipping, colour distortions, shears etc.
- related point to above - I don't see what is preventing you from going beyond finite groups to generalize the EED metric e.g. to non-group transformations that need not be finite such as those transformations mentioned above. In Eq (3), you could approximate the integral wrt the set of transformations G with samples (i.e. MC approximation) as you are already doing for approximating the integral wrt the data distribution.

[1] SE(3)-Transformers - Fuchs et al., 2020
[2] LieTransformer: Equivariant self-attention for Lie Groups - Hutchinson et al, 2020.
[3] Efficient Generalized Spherical CNNs - Cobb et al., 2021
questions: Questions & Suggestions:
- It's confusing that the channelwise G-EED is different from softmax & latent space EED, in that a higher value means the model is more equivariant. Perhaps it's more consistent to use negative cosine similarity to be consistent with other metrics.
- Why is L2 distance used for latent space EED whereas cosine similarity is used for channelwise EED? If cosine similarity is also used for the former, it might not be necessary to normalizie by M, the average L2 distance between points.
- Figure 2 is used as evidence to claim "learned invariance does not arise through learned equivariance". It would be more convincing to show the plot for each value of k between 1 and 6 as opposed to just 2 & 5. As you mentioned in Prop 3.1, it could be that the first 5 layers aren't equivariant but all 6 conv layers are equivariant.
- In line 277-278, you mention that "invariance is useful when it is aligned with other metrics more directly related to the task the model was designed for". This makes me curious as to what would happen if you use each EED metric as a regulariser for a CNN - would it help of harm generalization?
- (minor) Figure 4 - x axis shifted by one.

limitations: n/a
review_title: null

Meta-review: metareview: The paper proposes metrics to empirically explore the nature of invariance and equivariance of deep learning models with the goal of better understanding the ways that they actually capture these concepts on a formal level. They utilize their proposed metrics to shed light on two popular methods used to build invariance into networks, data augmentation and equivariant layers. The reviewers agree on the significance of the contribution and quality of the presentation. Some questions were raised by the reviewers that required some clarification and the the authors did a great job addressing these, leading to  improvements in the paper and as a result in the reviewer’s opinion of the paper.
confidence: Certain
Decision: Accept