From: NeurIPS.cc/2022/Workshop/NeurReps/Program_Chairs
title: Paper Decision
decision: Accept (Oral)

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper8/Reviewer_Vx4g
title: NeurIPS 2022 Workshop NeurReps Paper8 Reviewer gpb2
confidence: 3: The reviewer is fairly confident that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: This paper introduces a method to learn latent representations on world models. Transitions between states are modeled as linear group actions on the latent space. Section 4 shows some interesting experiments on a toy dataset generated by translation on an image of an ellipse at a fixed scale and orientation. 
strengths_and_weaknesses: Originality: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work. In particular, many of the ideas can also be found in Quessard 2020(Learning Disentangled Representations and Group Structure of Dynamical Environments) and Tao 2022(Towards Building a Group-Based Unsupervised Representation Disentanglement Framework). 

Quality: The submission is technically sound. The stated propositions are true and the proofs seem correct.

Clarity: The paper is generally clear and the claims are easy to follow.

Significance: The results are interesting and somewhat significant. This can be important for learning world models and learning disentangled representations.


questions: Why does the latent prediction loss require predicting the subsequent latent state up to j timesteps? Simply predicting the latent state for the subsequent timestep would impose the same constraint.
limitations: It seems that the assumption in this paper is that the transition between states is deterministic. This would be a significant limitation because in most cases transition between world states cannot be assumed to be deterministic. Also the action effect in each state is the same (multiplication by the matrix /rho(g_t)), since this has no dependency on the state in which the action is performed. In most cases the action effect depends on the state from which the action is performed and ideally it should be modeled as \rho(g_t, o_t). Environments with partial observability can also pose a challenge for the proposed method.

soundness: 3: Good
presentation: 4: Excellent
contribution: 3: Good
overall_rating: 6: Technically solid, moderate impact paper, with no major concerns with respect to evaluation and reproducibility.
relevance: 4: Highly relevant
recommended_decision: 3: Accept

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper8/Reviewer_7SY9
title: Interesting work with the rich appendix
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: The authors propose the Homomorphism AutoEncoder (HAE) in this extended abstract. The HAE can learn the group-structured representations of transitions. The authors give math proofs to motivate the method and conduct numerical results to validate the effectiveness of HAE.
strengths_and_weaknesses: First of all, I like the research direction posed in this paper, i.e., learning group-structured representations. The proposed method is guided by math (group theory), and some proofs are provided, making the paper look more solid. Lastly, I also appreciate that the authors can provide a rich appendix (almost 18 pages), including the necessary background of the group theory, numerical experiments, and so on. 

The major weaknesses come from the writing. I feel the paper is not easy to follow and understand due to (1) some very high-level (even philosophical) discussions. (2) Some sentences and math symbols are not well illustrated. (3) Limited examples to help readers to build intuitive understanding. 

Overall, I think this paper is good, especially for the “abstract track.”

questions: One suggestion comes from the writing part:

(1)	Make sure you will give enough intuitive examples/sentences/figures to help the reader establish a sense of what you are doing, especially when you submit the paper to the main conference research track in the future. 

(2)	In the main body, there are some math symbols not well illustrated (such as $GL(Z)$ on the top of page 3). Please note that people are not required to read the appendix. 

(3)	In the appendix, the proposition number is confused. For example, the proposition number directly jumps to 10 on page 12. 

(4) Please double check all the citation command is used correctly. For example, “See Appendix ?? for associated results” on page 14.

limitations: I appreciate that the author honestly pointed out their limitations on page 23. 

I also encourage authors to give complexity and convergence analysis in future work (either theoretically or in numerical experiments). More state-of-art methods are also needed for comparison.

soundness: 4: Excellent
presentation: 3: Good
contribution: 4: Excellent
overall_rating: 8: Technically strong paper with novel ideas and high-to-excellent impact, and excellent evaluation and reproducibility.
relevance: 4: Highly relevant
recommended_decision: 3: Accept

From: NeurIPS.cc/2022/Workshop/NeurReps/Paper8/Reviewer_9zqH
title: Elegant method - some concerns on whether the stated assumptions are sufficient
confidence: 4: The reviewer is confident but not absolutely certain that the evaluation is correct
submission_track: Extended Abstract (4 Page)
summary: The authors assume that a dataset exists of triples of observations, with a known pair of group elements that transforms between the observations with an unknown representation. The authors propose a method of learning an auto-encoder in which the group acts linearly on the latent space. The learning is done by a reconstruction loss plus a loss that enforces that subsequent observations transform by a group action. The method is evaluated on a toy dataset of black and white sprites.
strengths_and_weaknesses: ## Strengths
Neat and elegant form of structured representation learning. The algorithm seems practical. The toroidal example illustrates the method nicely.

## Concerns
I have some concerns regarding the injectivity/surjectivity/bijectivity assumptions.
- I don't see anywhere that $b$ is supposed to be in/surj/bijective, but the proof uses its inverse $b^{-1}$. Should this be read as a left-inverse and is $b$ assumed surjective or is $b$ bijective?
- On page 2, it says that the commutativity of the right diagram in fig 1 implies that the lower morphism is a group action on Z. This is not true in general without additional assumptions on $f$. If $f$ is not surjective, then the lower map can do anything outsize the image of $f$, e.g. be a fixed (non-identity) bijective transformation for all group elements - making it not a group action. So I suspect that additionally it needs to be assumed that $f$ is surjective. Perhaps with the assumption that $\rho(g)$ acts linearly, a for a weaker assumption is possible, such as that the span of the image should be all of $Z$. Alternatively, you could say that $\rho$ is not necessarily a group representation, but that its restriction to the image of $f$ is a group representation.
- A similar argument holds for proposition 1. If $h$ is not surjective then $\rho(g)$ can do anything outside the image. Thus, a similar surjectivity criterion should be assumed for $h$. Or nuance the conclusion that $\rho$ is a representation as above.

questions: Could the authors clarify why the exponential map is necessary? Shouldn't the loss already ensure that when restricted to the image of $h$, $\rho$ is a group representation, thus invertible? I could imagine that the exponential map makes training more difficult.
limitations: It would be great if the authors could add some discussion when the required data regime - triples of observations with known group elements that transform between them - is realistic.
soundness: 3: Good
presentation: 4: Excellent
contribution: 3: Good
overall_rating: 7: Technically solid paper, with moderate-to-high impact, and good-to-excellent evaluation and reproducibility.
relevance: 4: Highly relevant
recommended_decision: 3: Accept