Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper introduces a way to learn partial equivariances from data by using a Monte Carlo group convolution where the samples are taken from a 1D uniform distribution with its bounds learned via the reparametrization trick. The method is shown to work on some tasks where equivariant models fail, like when the task is to detect a specific transformation. It also seems to compare favorably against somewhat weak baselines on natural image classification tasks (CIFAR10/100).

strengths_and_weaknesses: # Strengths

The submission addresses an interesting and hard problem, which is how to learn which symmetries are present in the data and how to exploit them. It shows some intriguing results when allowing each layer of the model to be equivariant to a different subset of the euclidean group. 

# Weaknesses 

1) The major weakness I see is that the main ideas of the submission are similar to Augerino [1], which also uses the reparametrization trick to learn the bounds of a uniform distribution to which the model should be invariant. The main difference is that Augerino uses the learned distribution to augment the input, whereas the proposed method applies the idea to the samples of the Monte Carlo group convolution from Finzi et al [2].

2) A second difference with respect to Benton et al [1] is that the submission introduces a way to learn discrete distributions; however, this seem only evaluated on the simplest 2-element group of reflections, and even in that case it seems unstable to train as stated in L373. I don't understand the motivation of the approach for the discrete case in L206-, wouldn't it make more sense to one  probability mass for each point and sample several points from the resulting distribution?

3) The comparison in Table 2 doesn't seem fair to Augerino, as they report an accuracy of 93.81% on CIFAR10, versus <85% reported in the submission. I understand that the architectures and training scheme are different, but the results would be much more convincing if they followed the baseline's hyperparameters and showed improvements over those numbers. Moreover, Benton et al [1] also evaluate on more interesting tasks such as SE(3) invariance in molecular property prediction and color-space invariance on STL-10. The submission would be made stronger by including these and/or other tasks that involve groups other than E(2) subgroups.

4) Some results in tables 2, 3, 4 are very close to one another, so it is unclear if the differences are statistically significant. I recommend running each training job several times and including confidence intervals.

# References

[1] Benton et al, "Learning Invariances in Neural Networks", NeurIPS'20.

[2] Finzi et al, "Generalizing Convolutional Neural Networks for Equivariance to Lie Groups on Arbitrary Continuous Data", ICML'20.
questions: - The patterns shown in Fig 5 are intriguing and are the most interesting result of the paper in my opinion. I wouldn't expect the bounds to narrow down in middle layers and widen up at the final. I suggest investigating this further as it might provide useful insights. 

- There is a mismatch between the accuracy for SE(2), 16, Partial on CIFAR10 in tables 3 and 4.

- L85- states "The group convolution requires integration over a continuous domain. Hence, it cannot be computed in finite time and must be approximated.". This is not true, there are many integrals over continuous domains that can be computed exactly in finite time, including group convolutions under certain conditions (for example, between bandlimited functions on compact groups).

- L331- missing reference: "Tab. ??".

limitations: Adequately addressed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper presents a strategy to learn partial equivariances from the data, in the setting of image classification. This approach allows to learn different degrees of equivariance, from the full E(2) to identity map. The idea is to parametrise the group elements via a probability distribution, which is learned via montecarlo sampling of convolution at different activation layers of a deep network. Results on toy and real data show good results, competitive with standard translation equivariant architectures.  
strengths_and_weaknesses: ### Strengths 

- The paper makes an interesting point, and indeed I think that allowing models to learn partial equivariances to some sub-groups would improve or worst case match performance of standard models. At least, this seem what results are pointing at, even if datasets are maybe not so convincing. 
- The proposed solution, although maybe a bit hidden in the cumbersome formulations and description, is simple yet effective, and seem to be a very interesting solution. I find interesting the fact that the model can learn different levels of equivariance to predefined groups, at different layers. I would be interesting in future work potentially, to constrain specific layers to not have this level of flexibility, and to see if others compensate of if the learning breaks down. In Fig. 5 it seem that the model naturally learns to not be equivariant in layers 3/4 and it would be interesting to see why and what are the limits. 

### Weaknesses
- The theoretical developments in the paper are complex, and as such, they should be followed by clear and exemplifying text, which is not always the case. I found these parts hard to follow, for messages that essentially can be simplified to high-level descriptions and reader sent to references. 
- This would leave some room to describe better the Monte Carlo estimation (sec. 4) and to move some details and results from the supplementary material to main text. I feel for some of that, the supp mat is not the right place (eg App A, with a clear description of what SE(2), T(2), E(2) are, App. D, some of E and F). 
- I feel CIFAR datasets are generally not so easy to use for further generalisation on realistic cases. MNIST even less. I unfortunately deeply miss some more realistic image classification dataset. ALthough is interesting to see results are improving by learning partial equivariances (some tilts, balancing horizontal flips in training set, pose for some classes, etc), I feel that those results alone are not enough to have general claims. 
- Figure 1 is impossible to read (resolution, colour), captions in general are not great. 
- Section 3.3 is unclear overall, maybe needs some reworking
- In the experiments, I miss an illustration of a loss curve, or something that shows how the training converges. It is also underlined in Sec 7, that the training is unstable and tough, so I wonder, how the authors coped with it. If any trick, weird scheduling, or anything, were used, I think it would be fair to mention it. 
questions: - It is unclear to me what is the base model structure used in T(2) settings. Authors mention the use of ResNets, but what are base models? De facto standard architectures are achieving lots better on CIFAR, but the current setting is a bit obscure to me. Not that I want to see numbers better than SOTA, of course, but I cannot understand the comparison. Maybe some details, a number of parameters to compare, could help. 
- In papers [1] numbers in Tab 4 for the CIFAR are competitive and the model is also simple. Models [1] and [2], which are similar, can be both extended to known partial equivariances (but not learned, which is the nugget here). Maybe those references could serve some comparisons or discussions, if time and space allow. 
- L220-229. Could it be that some group elements are not independent? e.g. vertical flips would prevent some specific rotations? Is there a way to cope with it? 
- Sec  5: Is not clear what "mirror CNN" are.
- The number of elements vary always as 8 or 16. I am fine with that, but there does not seem to be a good explanation / motivation / justification. 
- There are some missing references (??) in the latex in the main text (L331) and supp mat (L53).





[1] Zhou et al, oriented response network, CVPR 17
[2] Marcos et al, rotation equivariant vector field networks, ICCV 2017
limitations: Limitations are addressed in Sec 7, which concludes the paper. I think those are enough and relevant. Some points raised above could be also discussed there, as probably not all my comments can be addressed.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper analyzes "the extent of equivariance"  when the input and output domains are the whole groups or the subsets of the group. Based on the analysis, the authors propose the partial G-CNN by learning the probability distribution $p(u)$ over the group in every neural network layer. Their method works for the discrete group, continuous group, and combinations. The experiments show that partial G-CNN can learn the different sets of transformations depending on the task.
strengths_and_weaknesses: Strength:
1.The problem, transformations in the data might be better represented by a subset of the group rather than the whole group, proposed in the paper is novel and is not discussed in previous work in equivariance. This problem deserves more exploration and attention.

2.The analysis of group convolution for different input and output domains is sound, and the method to obtain the output domain by learning the probability distribution over the group is novel.

 3.The paper is well organized, and the images in the paper help audience understand the paper easily.

Weakness:
1.The paper proposes a method of learning the probability distribution of discrete groups, continuous groups, and the combination. Could the authors show more experiments on the group other than SE(2), E(2), or the flipping group?

2.I'm a little confused about the equation (3); what is $v^{-1}u$ since v is in the domain that $G$ acting on and $u$ is in G? How to get $v^{-1}$ and how does it act on the group element? Correct me if I'm wrong.

3.There is a strong baseline that I think the authors should compare with; that is, the equivariant network over the whole group by replacing the max-pooling layer with the flatting operation and the MLP layer.  The conventional equivariant networks are equivariant to the whole group, and if we don't do pooling over the group dimension, then the transformation information is kept; for example, here, the permutation of features for "6" and "9" are different, which also can help the network to classify 6 and 9.

4.There are several typos in the paper, especially the reference to the figure and the table. I suggest the authors could check the misalignment.
questions: 1.Line 237 replaces the [15] with a convolution, but I'm confused about what $v^{-1}u$ is, which is the same question in weakness 2.
 
2.In the experiments, there are some tasks we already know; for example, MNIST6-180 and MNIST6-M, can we derive the fixed out domain but not learn the out domain layerwise to get a better result than conducting convolution over the whole group. I'm a little confused about how to leverage the analysis on partial equivariance quantitively.

limitations: The authors list the reasonable limitation in the paper and the broader social impact in the supplementary materials.
review_title: null

Meta-review: metareview: All three reviewers lean toward acceptance. After the author-reviewer discussion, reviewers find most of their concerns clarified. Concerns about several experimental issues remain, and the authors provided reasonable responses. After careful consideration, AC recommends accepting the paper.
confidence: Certain
Decision: Accept