Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper proposes to apply the formalism of Lie groups to capture continuous transformationsto improve models robustness to distributional shifts. Specifically, it structures the representation of corresponding vector space of the assumed Lie group by learning some basis metrics and then constructs their Lie operator by using this vector space structure. The general idea is impressive, and the experimental results are promising.
strength_and_weaknesses: Strength: This paper shows great potantial of learning the transformation in laten space induced by changes in pose to enable self-supervised models to generalize variation across objects. The perspective is novel and ablation study shows promising results.

Weaknesses: In Section 2.1, there is no description about how to decide the dimention d of the Lie algebra. In my understanding, there should be a way to specify the dimention, or it is the degree of freedom of the transformation matrix in Li group. Is my understanding correct?
clarity,_quality,_novelty_and_reproducibility: This paper is well-written and presents a novel perspective to enable self-supervised models to generalize variation across objects.
summary_of_the_review: In general, the presented method is novel and effective, which is also well-supported by the experimental results.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: Though deep learning has achieved many advances, many existing methods suffer from poor generalization performance due to the domain-shift impact. To this end, this paper focuses on improving the robustness of self-supervised methods. Specifically, the authors choose matrix Lie groups to model continuous transformations and employ contrastive learning to enhance the generalization. In the experiments, the proposed method is evaluated on multiple datasets.



strength_and_weaknesses: 
Using group theories to improve OOD robustness is an interesting idea.


1. The motivation of this paper is not clear. In Introduction Section, the authors directly indicate that self-supervised models own limited OOD robustness. However, I am not clear why self-supervised models are with limited OOD robustness. The authors should give more interpretations.

2. Some indications in this paper are not accurate. For example, the authors indicate that for image data, augmentations can only be applied to the pixels themselves. However, to the best of my knowledge, there exist some feature-level augmentation methods. Besides, the authors should further interpret why augmentation fails to generalize to novel objects.

3. The writing of the method section is very chaotic. The proposed method contains multiple operations. However, the authors do not introduce the motivation of these operations clearly. Besides, it is better to give some theory analyses for the proposed method.

4. In Table 1 and 2, this paper only compares MAE and VICReg. The authors should evaluate the effectiveness on more methods and tasks. The proposed method involves multiple operations. The authors should make sufficient ablation experiments. Finally, the authors should give more training details, e.g., the training curves, and feature-level visualization analyses, which is beneficial for further understanding the proposed method.


clarity,_quality,_novelty_and_reproducibility: The writing of this paper is not clear. The proposed method only uses the idea of Lie group and contrastive learning. The novelty of this paper is limited. Besides, the proposed method contains multiple operations and is complex. It is better to give the code to reproduce the results.


summary_of_the_review: The writing of this paper is not clear. The proposed method only uses the idea of Lie group and contrastive learning. The novelty of this paper is limited. Besides, the proposed method contains multiple operations and is complex.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper addresses the problem of OOD robustness with respect to pose changes. To this end, a Lie group transformation of latent representations is learned. This groups is parametrized via its Lie algebra in order to properly integrate it into a (vector space-driven) deep learning framework.

To train the network, pairs of images are used that show the same object at different poses.
Using these pairs, an SSL is performed by using a combination of four loss functions:
1. Standard SSL training loss
2. InfoNCE to measure the similarity of the latent representation (w.r.t. the algebra estimation)
3. Euclidean similarity between the latent representation
4. Task-dependent algebra-regularization

The proposed method is trained on a dataset of about 500K generated images based on models owned by Trimble Inc.
strength_and_weaknesses: Strengths:
+ The used mathematical concept of Lie groups/algebras is nicely explained
+ The four different loss components are explained in detail and ablation studies are performed to emphasize their usefulness
+ Newest models (like CLIP) are used to demonstrate the usefulness of this new approach

Weakness:
- It is not clear why 3D rotations should translate to group operations in the latent space. This two operations seem to be logically disconnected
- The paper is not clear about the (manifold) dimension of the Lie group. Depending on the latent space, this dimension might be quite high
- The proposed method uses different hyper-parameters for the four different loss components. Since these are not disclosed, it is difficult to reproduce the results.
- In Section 2, it is not clear why the scalar `t` is necessary. After all, the Lie algebra is a vector space and is closed under scalar multiplication
clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and introduces all relevant concepts in the main paper.

The concept of Lie groups has been used before in machine learning, but to my knowledge not for OOD robustness.

According to the "General Model License" of Trimble Inc., it is not allowed to "[...] use or incorporate such Model in any application, product, service, database or repository". This raises a reproducibility issue of the paper, since the used 500K images for training cannot be used by other researchers to reproduce the presented results.

summary_of_the_review: Overall, I like the idea presented in this paper. There are minor concerns like the Lie group's dimension.

My main concern is with respect to the reproducibility with stems from the license of the used data.
Therefore, I see the paper marginally below the acceptance threshold.

If the reproducibility issue can be resolved, I am happy to raise my vote.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes a method for improving the robustness to distribution shifts of self-supervised methods by modeling continuous transformations via Lie groups. A majority of reviewers are negative toward acceptance of this paper. There is only one positive reviewer, but it is hard to say the positive review is strong enough to support the acceptance and did not show strong support in the AC-reviewer discussion phase upon other reviewers' concerns. Although AC somewhat understands the authors' complaint about the reproducibility issue concerning dataset license but does not think the authors succeed in resolving other concerns of negative reviewers. In particular, AC agrees that "the authors should evaluate the effectiveness on more methods and tasks", "the authors should make sufficient ablation experiments", and "the proposed method only uses the idea of Lie group and contrastive learning," pointed out by reviewers. In addition, the authors claim that the proposed method can model continuous data transformations such as pose changes, but AC thinks that it should be further justified and investigated whether it actually does so. Moreover, AC thinks the difference between Lie and contrastive loss also should be further discussed to clarify where the benefits come from. 
justification_for_why_not_higher_score: Although the paper performs valuable explorations on the topic, AC thinks the reasons to reject slightly outweigh the reasons to accept at the current form.
Decision: Reject