Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The goal of this paper is to learn world models - where the network must predict actions from images - in an efficient manner.  This is done by exploiting symmetries in the data.   Results are shown for a number of world model benchmarks, showing improvement over standard methods.

main_review: I found this to be a very difficult paper to read and understand.  In part it is a new area for me, but also I feel the authors are writing to others who live in their own world rather than a broader community of researchers.

Figure 2 is helpful, but I could not understand what is the purpose of the symmetric embedding network S as opposed to the encoder E?  Why two networks instead of one?

Also, as far as I understand S, E and T are just standard neural networks, there is nothing about SO(3) per se or the mathematics of transformations engineered into them.  It appears the group theory mainly comes into play in how the networks are trained, as opposed to the computations they perform per se.  If so, then to me this seems a shame because I think where group theory could help us the most is in moving away from standard MLP models to more powerful and interesting computational frameworks.

The tables 2-6 showing model performance are also very hard to understand - extremely cryptic.   A bit more information about what these abbreviations mean and what is being measured would be helpful.

summary_of_the_review: An interesting idea but a difficult paper to read and understand.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes to learn an equivariant world model without access to group representation on input space $\rho_S$. To this end, it learns a symmetric abstract state mapping from states $s$ to abstract states $z$ in a space $Z$ with an explicit action $\rho_Z$ of the symmetry group G. It then learns a transition model in latent space that can enforce symmetry using an equivariant neural network. 

The problem of learning the equivariant model without assuming any group representation on input space is important and motivated well in the paper. The paper evaluates the model and compares it extensively on various benchmarks. 
main_review: The presentation can be improved in several ways. I agree that the paper studies various models, symmetry assumptions, and different controlled environments so it is a challenge to make it self-contained. Still, there are several things that I found hard to understand:

- 'We consider the case of MDPs with symmetry as in van der Pol et al. (2020)'. Maybe I missed something but this model is not motivated anywhere.  is it obvious for computational purposes? it is not obvious to me why study/build on this model? The background section gives some motivation behind Kipf et al. and structured contrastive models but Van der Pol et. al 2020 is mentioned here for the first time.
How central is the MDP assumption in validation of the main idea? Is it reasonable to ask to demonstrate the main idea on other structured spaces?

-'We require that ρA(G) · A′ = A. That is, every MDP action is a G-transformed version of one in A′ .' this should be mentioned at the beginning of Model overview, Section 4.1.

The experiments are not completely convincing. In particular, fully equivariant networks that assume the correct transformation $\rho_S$  is a simple transformation of the pixels performs very well in all tests and thus, somewhat contradicts the main motivation of the paper.

The paper is riddled with typos, some mentioned below: 
-Future work will include applying *out* method  - The MDP action *is* input directly to T. - 'each element $g \in G$ *must* invertible with respect to composition' .. We demonstrate that this meta-architecture can be *use* to learn world models with a variety. We then learn a transition model in latent space where *can* enforce symmetry
summary_of_the_review: I give marginally above as the experimental setup seems rigorous. The authors are quite transparent about the lack of theoretical novelty when they state they are not proposing a new equivariant neural network design and instead, they build upon previous work to demonstrate working in new domains with unspecified group actions. I will revisit the score depending on other reviews. I can not comment on the utility of the idea only in MDP structured space but in general, the problem is relevent.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes to learn latent representation and transition models that are equivariant to symmetry transformations of states and actions. The proposed “meta-architecture” consists of an encoder followed by an equivariant encoder and an equivariant transition model. The model is trained using (state, action, state) triples, where contrastive loss is used to prevent the collapse. The paper evaluates the proposed model in four different settings. In each case, a different choice of symmetry group and equivariant architecture is used. The quality of the learned latent representation is evaluated by comparing the relative distance of the predicted next state embedding to the true next state embedding, in comparison to negative samples.

main_review: 
The goal of the paper in using equivariance in latent space models in RL is interesting and motivated. Another positive point about the paper is that it evaluates its proposal on diverse problems. However, I have questions and concerns about the claims of contribution, the basic setup, theoretical presentation, and experimental results of this paper.

Contributions
------------------
The paper claims that “the idea of learning symmetric embedding networks has, to our knowledge, not previously been proposed or demonstrated.” and this is discussed as one of the main contributions. There have been several works in this area in recent years, including using symmetries in latent space for dynamical environments; e.g., 

- Higgins, Irina, et al. "Towards a definition of disentangled representations." arXiv preprint arXiv:1812.02230 (2018).
- Quessard, Robin, Thomas D. Barrett, and William R. Clements. "Learning group structure and disentangled representations of dynamical environments." arXiv preprint arXiv:2002.06991 (2020).


The proposed model
---------------------------
In the general case, the symmetry transformation of the action should be state-dependent. This is noted in prior work such as the one cited below. The theoretical setup used in the paper is therefore inadequate for symmetric environments. Generally, there is no discussion on prior works on symmetric MDPs and the way this work fits in.

- van der Pol, E., Worrall, D., van Hoof, H., Oliehoek, F., & Welling, M. (2020). MDP homomorphic networks: Group symmetries in reinforcement learning. Advances in Neural Information Processing Systems, 33.

Also, the motivation for having an equivariant encoder "after" a non-equivariant one is not clear, since the embedding will not be equivariant. This is especially so given that the equivariance of the representation is not enforced in any other way. My understanding is that the loss function used here only ensures that the transition in the latent space is consistent with the state space - i.e., the loss is exactly the same as that of Kipf et al (2019), am I correct?

Experiments
---------------
While the diversity in the choice of environments is great and the evaluation metric is reasonable, it is difficult to read much from the results. There are also some baselines that are difficult to justify: why use an equivariant network with a particular assumption of linear action on the input space where we know the group action is non-linear? This is further complicated by the fact that such a baseline is producing good results. With the exception of visualizations in the appendix (which are supportive), the results are mixed, borderline, or difficult to interpret. 


summary_of_the_review: see above
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This article assumes knowledge of an underlying group to learn symmetric latent representations, by utilizing equivariant transition models. The paper claims that the basic setup can be applied to a variety of existing equivariant networks (or non-equivariant ones) to improve performance as well as data efficiency. Experimental results are shown on 3 tasks, which are somewhat hand-crafted, in that for each of these tasks the allowed groups are assumed to be known a priori. Some of these experimental results are intriguing but the results themselves beg the question of why one needs all the complex machinery, at least in comparison with other recent efforts which admit a more direct mechanism (e.g. with regard to SO(3) and the teapot rotation task - see comments below).
main_review: I found this paper incredibly dense in parts, and the presentation quite confusing. The paper mixes a variety of interesting but sometimes disparate topics - equivariance as it applies to networks, the concept of symmetry as it applies to group actions, and equivariant transitions. In a sense the authors might be trying to sell too much here, and as a result the actual contributions are quite hard to tease out. The quality of writing is also a concern. To the extent that I follow the arguments, the main strength is the richness of the problems considered here - equivariance is indeed a hot topic, and the general goal of trying to recover representations that capture aspects of symmetries of the underlying phenomena (e.g. in the latent space) when from observations such as visual images, these symmetries are not easy to tease out (e.g. the rotating car example in Figure 1.). The experimental results do appear to be promising, although the use of some of the metrics (e.g. DIE) are only relevant for the restricted case of the Euclidean groups or its sub-groups.

Weaknesses of the article include the quality of presentation which could be vastly improved, e.g, the related work section is rather disorganized - there doesn't seem to be a common thread. Simplifying the claims and accordingly the presentation, would add focus to the paper. There is also a rather a strong limitation of their assumption that for the task at hand the relevant groups are known. For particular cases there could be much more straightforward solutions. For example, Quessard et al. in the article "Learning Group Structure and Disentangled Representations of Dynamical Environments" (NeurIPS 2020) offer a much cleaner solution where a group G acts on pixel space, in such a way that the operations can be captured by matrix multiplication. The basic formulation in the Quessard et al. paper could certainly be applied to the teapot task it seems, one doesn't need the complexity of the equivariant transitions in Section 4.2 of the present paper to do this.

summary_of_the_review: A lot is packed in to the paper, making the message a bit confusing, and the main contribution not so easy to tease out. Further, it isn't clear to me what the theoretical guarantees are and it seems that a fair amount has to be known to apply the methods and to effect the equivariant transitions. To me it seems that simpler solutions do exist, at least for some cases, but perhaps I am missing something here. I also found the symmetric embedding steps in the Meta-Architecture to be somewhat engineered. The authors are indeed grappling with some interesting and very relevant questions, but separating the theory from what is engineered is not so easy (for this reviewer at least).
review_title: null

Meta-review: This paper proposes a new method for learning symmetric representations for equivariant world models. All reviewers recognized the interesting results in the paper. The reviewers have raised some concerns, which were not addressed well yet after the rebuttal. For example, Reviewers LG1G and Uu3z mentioned about the limitation of using the group for a task and the generality of the approach. Reviewer 5ro3 mentioned about the lack of novelty. Though they gave 6, they were quite neutral about the paper acceptance. Eventually, after a second round of deiscussions, we had to make this difficult decision: The current form of this paper is not ready for publications.
Decision: Reject