Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper applies the recent progress of equivariant policy learning to top-down grasping using trained on real robot arms. The experiments show that the effectiveness of high sample efficiency of the equivariant policy learning. 

strengths_and_weaknesses: ## Strengths

The on-robot results are pretty impessive in terms of the sample-efficiency of learning process. The results are compared to one of the state-of-the-art methods FERM (Framework for Efficient Robotic Manipulation) in top-down grasping leanring, and show superior performance. In addition to purely on-robot learning bootstrapped from 20 human demonstrations, sim2real results are shown and analyzed. It's also worth-noting that this paper discussed the limitations of using equiviance in policy learning very well in the limitation section. 

## Weakness

1. The delta of this paper's methodology from the prior work, Equivariant SAC (SO(2)-equivariant reinforcement learning), that this paper is largely based on, is small. Indeed, authors discovered that using discrete rotation and reflection groups for equivariant sac training can lead to better performance than the continuous counterpart, and found some interesting phenomena in sim2real transfer. However, the change in equivariant groups doesn't seems like a crucial part of the success of fast on-robot learning, since Figure 3 (a) only demonstrates minor performance boost by not using SO(2). 

2. Similar to the prior work Equivariant SAC, depth images are used as input to the policy. One limitation of using depth images is that they are not robust to many objects (e.g. plastic bottle, reflective silverwares) that are common in daily life. One solution to this is to directly use RGB images like what FERM did. It will be interesting to see a discussion about why not using RGB for equivariant policy learning. 
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: Although the methodology of is this work is largely based on the prior work (Equivariant SAC), I would still recommend for a weak accept. I appreciate the large amount of work done by authors to make on-robot learning work really well, which aligns with this conference by creating learning methods that actually work on real robots and real problems. 
issues: 1. More details about what D_n and C_n groups mean in the context of depth images
2. Discussion about why using depth and why not using RGB images. Is it simply because depth images give better sample efficiency, or there is some difficulties to prevent RGB images from being using by Equivariant SAC?
3. Clearly state this work is based on Equivariant SAC (SO(2)-equivariant reinforcement learning) and E2CNN (General E(2)-Equivariant Steerable CNNs).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: The paper presents a study on using equivariant models for deep reinforcement learning of robotic manipulation policies. It studies a variant of a well-known deep RL algorithm, soft actor-critic, when using rotation and reflection equivariant neural architectures for the learned components, as in prior work on equivariant SAC. The authors experiment to determine which symmetry groups lead to improved performance, and also show that interesting, further data augmentation can improve sample efficiency on top of discrete equivariances. They use real-world robot learning experiments to demonstrate the sample efficiency afforded by equivariant models.
strengths_and_weaknesses: Strengths:
 - The motivation is very strong – sample efficiency is a significant limiting factor for current deep RL algorithms when trying to learn robot control policies from scratch.
- The authors are able to successfully demonstrate real world on-robot learning with impressive sample efficiency, which is quite a nice result. 
- The experimental contributions of the paper are quite useful for practitioners and I think will be of interest to the community. 

Weaknesses:
- The methods evaluated in the paper have limited novelty, and are mostly drawn from prior work. However, the experiments and analysis of these methods still provide significant value.
- The first claim is that $D_4$ is empirically the best symmetry group is hard to conclude given the results in Figure 3, since the error bars are quite wide. Also, the statement that “In all four environments, $D_4$ shows the best performance over the four different groups” seems unclear to me – which metric is this based on? For example, $D_8$ does not seem to be worse than $D_4$ for the “block in bowl” task. 
- As with all equivariant methods, much care must be taken to ensure that the action and state space satisfy the conditions necessary for invariance and equivariance. This limits the types of tasks that such methods can be applied to. In spite of this, I believe that many important robotic manipulation tasks can be framed in such a way.
- The reset mechanism for the block picking task requires the robot to already be able to pick up the block which somewhat obviates the task, but this could have been done using a number of other reset mechanisms (which aren’t the focus of this work).

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: This paper presents a nice empirical study on using methods like equivariant SAC for real robot learning. It is a good fit for the CoRL venue and I think it is well motivated and interesting. Some of the claims made in the paper do not seem to be backed by experimental results with significant effect size, which is my main concern. However, I think the demonstration of highly sample efficient on-robot deep model-free RL is quite impressive. I currently vote for weak acceptance.
issues: - I think it would be helpful to see the performance of non-equivariant SAC with the expert demonstrations in the buffer as well as non-equivariant SACfD, both in the simulated and real experiments. The claim in the paper is made that the sample efficiency of the equivariant method is significantly improved, but it’s never shown in any experiments. I would agree that in general on-robot learning using RL from scratch can be extremely time-consuming, but it would depend on the action space, reward, expert demonstrations provided, etc. so it would be good to be able to see that comparison here as well.
- Why were 20 expert demonstrations chosen for the real-robot experiments? It seems from the experimental results in Figure 13 that 10 demonstrations doesn’t seem significantly worse. 
- Is the input to the networks in the real robot experiments a depth image, or is it a two-channel image like in the simulated experiments which also includes the binary channel indicating whether the gripper is holding the object? 
- It would be helpful for clarity to include that the plots in Figure 3(a) are generated using buffer augmentation as the additional data augmentation strategy (if I am not mistaken).
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper proposes an on-robot equivariant model to handle manipulation tasks. The work leverages the task structures such as symmetry explicitly for efficient learning of a control policy.  Moreover, the paper finds that data augmentation is still a powerful tool together with equivariant models. The experiments demonstrate the effectiveness of the work as well as compare different types of equivariant models.
strengths_and_weaknesses: Strengths:
The paper is clearly written.
The paper demonstrates the effectiveness of equivariant models for on-robot learning. While the idea itself is not new, the overall system is valuable. 
The authors carefully study the variants of different equivariant models. This shed light on future research in this direction.

Weaknesses:
The limitation of this work stands out. The requirement of the task structure prevent the method from scaling up to a general method. 
The tasks are relatively on the easy end of the family. But this is a minor issue.
The robustness of the proposed method is not proved. For example, would this method work with arbitrary camera angle？

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: This work aligns well with the scope of CORL. It demonstrates the effectiveness of equivariant models on real robots. The authors test with various implementations and provide useful information for future works.
issues: The key issue is that the method might not be scalable to different tasks or camera settings. 
The lack of in-depth analysis of data augmentation in equivariant models. The reason why it works is not clear.
review_title: null

Meta-review: metareview: Strengths:

+ Extensive set of experiments demonstrate impressive performance on real robots, something which is generally lacking and thus highly valued in the related literature.
+ The paper is clear and well written.

Weaknesses:

- The paper does not present any algorithmic and technical advances compared to previous work on equivariant learning, but rather empirical findings from an extensive set of experiments on real robots. Nevertheless, this has real value for the community.
- Some of the conclusions are insufficiently supported by data, since some of the stated differences in performance between some of the experimental conditions are in fact marginal and inside the noise margins of a typical learning curve. This downside is somewhat alleviated by the fact that experiments show averages over multiple seeds. Still, conclusions can be drawn on only a subset of the presented tasks.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)