Venue: CoRL 2023 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The authors propose a continual reinforcement learning algorithm for robotic manipulation tasks that utilizes symmetries in the task space to improve sample efficiency and generalization. An equivariant feature extractor is utilized to facilitate training under symmetries within a task group, and subsequently an invariant latent representation of the initial rollout is used to determine if the current episode is from a task group that is new or already encountered. Wasserstein distance is used to determine whether the current task is significantly different from the previously encountered ones, in which case a new random policy is instantiated.
review: The paper is well-written and well-structured, it contains sufficient experiments on hardware and explains the limitations correctly. The authors compare the proposed approach with other methods and variants of their method as baselines and show that it outperforms the other models. However, the experiment scenario feels quite simplified, there are several questions related to this below. 

In the reviewer’s opinion there is a limited novelty as the methodology combines several existing approaches, however it combines them in a new and convincing way by addressing the concept of continual learning.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: The topic addressed by the paper is very relevant for current robot learning research as its focus is on important challenges in the field such as sample efficiency, representation learning and continual learning. The paper will be beneficial to the community as it provides unified methodology that to some degree addresses all these challenges.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: This paper proposes a novel continual reinforcement learning (CRL) algorithm that effectively recognizes a new group of tasks and generalizes to SO(2) variations of the observation. Specifically, the method utilizes 1-Wasserstein distance on the extracted latent features clustering tasks. Moreover, leveraging equivariant models, the method generalizes to SO(2) perturbed tasks.
review: Evaluation of the quality, clarity, originality, and significance of this work

The method address task clustering and SO(2) perturbations challenges in the CRL. The presentation is clear and the method is novel. This work could be applied to various reinforcement learning applications and real-world learning setups. 

strengths
Novel task clustering method that also generalizes to SO(2) rotated task.

weaknesses
1-Wasserstein distance is the key to detecting new groups of tasks and assigning a new policy. However, there is a very limited introduction and intuition of this distance.

Figures 6 and 7 could be improved. Fig. 6, it can be breakdown into two figures, one for baseline comparison and the other for ablation. For Fig. 6. 7, the X- axis seems to be (task1, task2, task3, task4, task1, ...), this could be added to the x-labels.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper addresses the erroneous clustering of geometrically similar tasks by using SO(2) equivariant networks and effectively recognizing new tasks. Experiments in simulation and real robots verified the effectiveness of the method.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: this paper proposes to learn a single policy to a class of symmetry equivalent tasks. It is achieved by equivariant DNNs with 1-Wasserstein distances between distributions. The results are convincingly remonstrated both on simulated and real robot. 
review: good paper.  I have a few comment in "Questions For Rebuttal".
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: good paper with demonstration on both simulated and real robot
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary_of_paper: The authors consider the problem of continual reinforcement learning, i.e. learning from a changing stream of tasks, from the perspective of equivariance. The authors argue that identifying symmetries in observed tasks is crucial for generalization. Following this insight, they propose a continual reinforcement learning approach that continually grows the library of equivariant skills. For this purpose, the authors propose an architecture based on an equivariant MLP and an equivariant CNN. By comparing the 1-Wasserstein distance of the states in the invariant feature space, they decide whether to apply an existing policy to the current task or to add a new one. Finally, the authors test the proposed method in simulation and in real-life experiments.
review: Strengths:
- An equivariant approach to growing new tasks is an interesting perspective on continual reinforcement learning problems. The idea to adaptively group together different tasks with symmetries is intuitive and quite elegant.
- The evaluation protocol is mostly satisfactory. The empirical results show that the proposed approach works quite well. The authors also conduct ablation studies on the role of equivariant components
- The real-world experiments are good to see, even if rather brief.
- The paper is nicely written and easy to understand.


Weaknesses:
- Although the empirical evaluation shows sufficiently that the proposed method works in the desired setup, I think the paper would benefit from additional experiments that examine particular assumptions and implementation details. In particular, I would be interested in the following ablation studies:
    - For example, the $d_\epsilon$ hyperparameter used to decide whether to assign the current task to an existing group or create a new group seems to be quite crucial, so it would be great to see how sensitive it is to set in practice and if there are any hints on how to set it.
    - How big do the buffers have to be in order for the method to work? How small can we make it before the model will break down?
    - The top-down assumption is quite strong here. I would be interested to see how it behaves when the camera is at least slightly disturbed (e.g. not centered perfectly).
    - Related to the previous point, seeing how the model behaves when the equivariance assumption is broken would also be nice. E.g. imagine a task where you have to do execute different actions depending on whether the object is on the left or on the right. It seems like this model would completely break down. Can we deal with it somehow, e.g. by using a non-equivariant MLP on top of the equivariant features?
- The method introduces some operations that seem to be expensive both in terms of computation as well as memory. I would appreciate a more thorough discussion of the complexity this approach introduces.
- The authors describe the limitations of their work in the "Conclusion" section, but I think it would be good to create a separate section for that purpose.
- This is more "nice to have" than a weakness, since it extends beyond the paper's scope, but I would be interested in additional experiments in a multi-task setting, i.e. we see all the tasks at once. In general this setup is easier for CL methods, but in this case it might actually be more tricky since we build the representations for each policy group at the same time.

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Although the paper has certain drawbacks, I find the idea of using equivariance in continual reinforcement learning compelling. The experiments are also quite convincing, although additional ablation studies would make the picture clearer. As such, I am going with "weak accept" for now.
review_title: null

Meta-review: Thank you for your submission to CoRL 2023. The reviewers have spent considerable time working with this paper and the current version is recommended for acceptance.

Please follow up with promised improvements, experiments and additional suggestions from the discussion phase. These include in particular improved clarity of the method section and further experiments and ablations.
Decision: Accept (Oral)