Title: The Surprising Effectiveness of Equivariant Models in Domains with Latent Symmetry
Authors: Dian Wang
Jung Yeon Park
Neel Sortur
Lawson L.S. Wong
Robin Walters
Robert Platt
Venue: ICLR 2023
Keywords: Equivariant Learning
Reinforcement Learning
Robotics

Abstract: Extensive work has demonstrated that equivariant neural networks can significantly improve sample efficiency and generalization by enforcing an inductive bias in the network architecture. These applications typically assume that the domain symmetry is fully described by explicit transformations of the model inputs and outputs. However, many real-life applications contain only latent or partial symmetries which cannot be easily described by simple transformations of the input. In these cases, it is necessary to learn symmetry in the environment instead of imposing it mathematically on the network architecture. We discover, surprisingly, that imposing equivariance constraints that do not exactly match the domain symmetry is very helpful in learning the true symmetry in the environment. We differentiate between extrinsic and incorrect symmetry constraints and show that while imposing incorrect symmetry can impede the model's performance, imposing extrinsic symmetry can actually improve performance. We demonstrate that an equivariant model can significantly outperform non-equivariant methods on domains with latent symmetries both in supervised learning and in reinforcement learning for robotic manipulation and control problems.

TL;DR: This paper discovers that equivariant models are surprisingly effective in domains with latent or partial symmetries. 

Review discussion:
Strengths:
1. Very important idea that the equivariance may be latent, and not necessarily captured in the input transformation.
2. Useful definitions of correct, incorrect and extrinsic equivariances that capture the intuition of the problem.
3. Really neat proof of Proposition A.1.
Weaknesses:
1. The authors address it in the end of their paper, but it would be nice to consider the method in other tasks / settings. I wonder if the power of the equivariant policy would be even further demonstrated in a realistic scenario, where the states are actual natural images. In that setting the extrinsic equivariances will be "natural." Might be a cool experiment to try.
2. Some minor comments:
* It would be great if you could put Figure 2 on the first page to visualize what you mean in the discussion.
* In Section 4.1. it is not clear what the relationship between $\hat{\rho}_x$ and $\rho_x$ is. Can you give some examples for intuition early in the paper?
* Small typo in the proof of Proposition A.1. "labels and maximally" <- "labels are maximally".
The clarity of the paper is great. I understood the authors' message without much effort, and I do not expect most of the audience to be challenged with that.
The quality of the work is also very good: the experiments are extensive and they prove the point empirically. The theoretical result makes sense: it is simple, elegant and useful.
I think the work is quite original, due to combining simple group theoretical arguments and strong experiments, and deserves its spot in the conference.
NO.
The authors thank the reviewer for their helpful comments. Please see our comments below:
> The authors address it in the end of their paper, but it would be nice to consider the method in other tasks / settings. I wonder if the power of the equivariant policy would be even further demonstrated in a realistic scenario, where the states are actual natural images. In that setting the extrinsic equivariances will be "natural." Might be a cool experiment to try.
That’s a great point! We definitely agree the benefits of extrinsic equivariance are not limited to the cases we consider here. Trying this method on natural images would be a great experiment. We plan to extend the current approach and include more realistic data and a variety of different datasets in future work. We do point out that we mimic some realistic “effects” in the simulator, such as the ‘light effect’ corruption in Figure 3 and tilted observation in Figure 5.
> It would be great if you could put Figure 2 on the first page to visualize what you mean in the discussion.
Thanks for the suggestion, we have moved Figure 2 to the first page (now Figure 1 in the revision) and edited the image to better illustrate our core idea.
> In Section 4.1. it is not clear what the relationship between $\hat{\rho}(x)$ and $\rho(x)$ is. Can you give some examples for intuition early in the paper?
This is a good point, thanks for pointing this out. We added an example in Section 4.1.
> Small typo in the proof of Proposition A.1. "labels and maximally" <- "labels are maximally".
Thanks for pointing this out, we fixed this typo in the revision.
Strengths
+ The experimental evaluation is extensive and significantly improves upon established results for equivariant SAC/RAD (Fig 7, 9, 11) and DrQv2 (Fig 12).
+ The appendix includes a significant amount of ablations and details, for example the tasks in Table 3 and Figure 27 on evaluating the DMC tasks with corruptions to the symmetry/camera pose
Weaknesses
+ While the paper is motivated by unknown latent symmetries, I thought it involve learning them.
However, as reading the paper, I realized that the extrinsic equivariances still need to be manually specified as auxiliary equivariances that are simply other known transformations.
The paper is clear and well-written. For one minor note, definition 4.3 seems incomplete and is missing the equivariance definition.
NO.
None
The authors thank the reviewer for their thoughtful review. Please see our comments below:
> While the paper is motivated by unknown latent symmetries, I thought it involve learning them.
However, as reading the paper, I realized that the extrinsic equivariances still need to be manually specified as auxiliary equivariances that are simply other known transformations.
You are correct, the extrinsic equivariance needs to be manually specified. As we point out in the introduction, the problem we are trying to address is when ‘the designer knows that a latent symmetry is present in the problem but cannot easily express how that symmetry acts in the input space’. This may be the case in many scenarios.
For example, if a pair of images represent a 2D rotation of an object, the group action may be difficult to characterize if the input image is not a top-down view and has a skewed perspective. In this scenario, one can easily identify the symmetry group (SO(2)) but the group action on the input space is not straightforward to compute. We have added some sentences in the related work section to clarify the difference between our work and other symmetry learning works (There is a difference between the latent symmetry group and the group action on input space).
> definition 4.3 seems incomplete and is missing the equivariance definition.
In extrinsic equivariance (definition 4.3), for $x\in D$, $\hat{\rho}_x(g)x$ will be out of the input space, thus $f(\hat{\rho}_x(g)x)$ (i.e., the ground truth label) is undefined, so that the equivariance definition with respect to the ground truth function does not apply here. As a side note, the equivariance with respect to the model function $f_\phi(\hat{\rho}_x(g)x) = \rho_y(g)f_\phi(x)$ does apply to all three definitions, as stated in Section 4.1.
Overall, I view the paper as a useful contribution to the literature
as it provides greater clarity for where equivariant models can really
shine.
While the paper does introduce an equivariant version of DrQv2,
I feel the contribution is introducing the distinction between "incorrect
equivariance" and "extrinsic equivariance".
The main weakness of the paper is it doesn't introduce new models. All
the theory and most of the experiments are done using existing equivariant
models in the literature. Much of the benefits of equivariant models are
already known as the related work highlights and there are already many
methods for learning latent symmetries.
The paper is very well-written and of high quality. I found it easy to follow and the experiments were well-chosen for demonstrating the different symmetries. The Appendix was detailed. I browsed the including software for the experiments but did not run them. I am reasonably confident that the work is reproducible. The work is novel, but it can feel a bit incremental as it's main contribution isn't a new model.
NO.
The authors thank the reviewer for their careful review. Please see our comments below:
> The main weakness of the paper is it doesn't introduce new methods. All the theory and most of the experiments are done using existing equivariant methods in the literature.
The main contribution of our work is not a new **model** (though we do propose a novel equivariant DrQv2 model, as the reviewer points out), however, we believe our proposal to repurpose fully symmetric models for non-fully symmetric problems does constitute a new **method**.
> Much of the benefits of equivariant models are already known as the related work highlights
It is true that many prior works have shown the benefits of equivariant models, while they use equivariant models under the ‘correct equivariance’ scenario. We believe that our work opens the gate for the equivariant models to be useful for a wider area of applications.
> and there are already many methods for learning latent symmetries.
Our work can also be considered as learning latent symmetries, but compared to prior works, our approach does not require a carefully designed network architecture. We also compare with the Symmetric Embedding Network (Park et al., 2022), one very new work in learning latent symmetries, in Figure 7 and show that our work outperforms it.
Thanks for the clarifications. I'll adjust my review to be more precise about the contributions of the paper.
Strength
* This work empirically shows that extrinsic equivariance can still aid learning, which is very well-motivated contribution to model architecture with image transformation augmentation.
* Empirical evaluations are strong and comprehensive, covering many kinds of transformations, supervised learning, and reinforcement learning.
Weaknesses
* The scale of the model and data used in these experiments are still considered small. Given the results in slightly larger data-regime, it's not clear how much it could be useful with large model and large data.
* When the term correct, incorrect, and extrinsic equivariance/symmetry appear the first time in the paper, they are not well-explained. Even in the following, it's still unclear what the definition is and examples are somewhat unclear. I wonder if it would be useful to at least have section in appendix to clarify is the main paper space is too constrained.
I consider the work novel.
However, as pointed out in the weaknesses part, there are still rooms to improve clarity.
NO.
The authors thank the reviewer for their insightful review. Please see our comments below:
> The scale of the model and data used in these experiments are still considered small. Given the results in slightly larger data-regime, it's not clear how much it could be useful with large model and large data.
A significant benefit of equivariance is to improve sample efficiency by enabling the model to learn faster with less data [A]. This is particularly important in robotics where acquiring samples is expensive.
Thus our manipulation experiments focus on the low-data regime to highlight the benefits of equivariance, even when it is extrinsic.
However, we do also consider larger-data regimes as in the DMC experiments in Figure 12 which use larger models and data where it uses 1e6~3e6 samples (at least 2 orders of magnitude more data than robotic manipulation). Figure 12 shows that extrinsic equivariance still noticeably learns faster. This suggests that extrinsic equivariance would still be useful with larger models and more data and we will investigate this further in future work.
[A] Cohen, T.S., “Equivariant convolutional networks”, 2021.
> When the term correct, incorrect, and extrinsic equivariance/symmetry appear the first time in the paper, they are not well-explained. Even in the following, it's still unclear what the definition is and examples are somewhat unclear. I wonder if it would be useful to at least have section in appendix to clarify is the main paper space is too constrained.
Thanks for pointing this out, we added a brief explanation of correct, incorrect, and extrinsic equivariance in the introduction. We also added a new section in the Appendix (Appendix B in the revision) to extend Section 4.2 with detailed examples to better illustrate those terms.
Thanks for the clarification. I still consider all the experiments small-scale in today's standard. That includes DMC, which is single-task and with limited data distribution. I still suspect the benefit of the proposed approach will potentially decay with much more data and larger models.
But I recognize the benefits of the proposed approach in the regime of little in-domain data. I will increase my rating.
The authors thank all reviewers for their careful and helpful review. We appreciate that all reviewers like our idea/definition of extrinsic equivariance and acknowledge our experimental results. We have edited the paper to incorporate reviewer feedback and believe it has improved as a result (edits labeled in blue in the manuscript). Here is a summary of the revisions:
1. Moved Figure 2 in the initial submission to the first page (now Figure 1) and edited it to better illustrate the main idea of the paper.
2. Added a high-level explanation of correct, incorrect, and extrinsic equivariance in the introduction where they first appear.
3. Edited the related work section on ‘Symmetric Representation Learning’ to better illustrate the difference between our work and the prior works.
4. Added an example to illustrate the relationship between $\hat{\rho}(x)$ and $\rho(x)$ in Section 4.1.
5. Fixed a typo in Appendix A.
6. Added examples to better explain correct, incorrect, and extrinsic equivariance in Appendix B.
Dear reviewers,
We have uploaded a supplementary video to help explain our idea. You can find the video through this link https://www.youtube.com/watch?v=V_448CfUOwE (uploaded using an anonymous account) or in the 'Supplementary Material' zip file.

Meta-review:
role: PC
data: metareview:_summary,_strengths_and_weaknesses: This paper introduces the concepts of correct/incorrect/extrinsic equivariance and shows that while incorrect equivariance limits performance, extrinsic equivariance can be of benefit. The reviewers unanimously rated the paper 8, and found it to be well written. The reviewers appreciated the new concepts introduced in the paper, as well as the comprehensive experiments involving different kinds of transformations in supervised and reinforcement learning. The main limitations that were discussed are that the experiments involved relatively small models and datasets, and that the paper does not introduce a new equivariant model (although it could be argued that it introduces a new method). Overall this is a solid paper and I recommend that it be accepted to ICLR.
justification_for_why_not_higher_score: Paper only includes relatively small experiments.
justification_for_why_not_lower_score: The paper introduces some interesting new concepts that will be of interest to the equivariant networks community.

Decision: Accept: notable-top-25%