Venue: CoRL 2022 Conference

Reviews:

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper is motivated by the fact that different coordinate frames have a significant impact for point cloud-based reinforcement learning for manipulation tasks. In a set of manipualtion tasks involving mobile and dual-arm manipulators, the paper compares with four coordinate frames: World frame, robot-base frame, end-effector frame and target-part frame and show that not a single frame can consistenly outperform the reset. The paper then proposes multiple ways of combining different coordinate frames by incoporating point clouds in different frames into the input and use the RL loss to find a combination of different frames. The proposed frame combination method achieves significant performance gain over single-frame baselines.
strengths_and_weaknesses: ### Strengths
1. The paper is well motivated and can bring the community's attention to the importance of choosing the right coordinate frames, which has been so far less explored.
2. The paper is well structured and easy to follow. The tables, figures, their captions as well as the video in the supplement are very informative.
3. The arguments in the paper are well supported by the quantitative and qualitative resutls.
4. Finally, the proposed method is reasonable and well motivated. The supporting experiments in Figure 8 show interesting insights into how the method works.

### Weakness
The weaknesses I listed below are all minor and serve mostly as suggestions to improve on the paper.
1. The real world experiment is underwhelming. 
* Due to the sample complexity of reinforcement learning, there is no training in the real world and only the trained policy is transferred. In this case, the sample complexity in the simulation becomes less of a concern. As such, I wonder if with more training samples, do different choices of coordinate frames yield similar asymptotic performance?
* Only the pick object is done in the real world. For this case, the end-effector frame is already achieving the best performance. It would be very helpful to demonstrate one more task in the real world where frameminer outperforms single-frame baselines.
* From the visualization in the paper, it seems that the full point cloud is used in simulation. How does this transfer to the real world?
2. In the current method, actions from different coordiante frames are averaged. This seems to be a strange choice to me. That means all coordinate frames always contribute to the final actions. Why not sample action from one of the frames based on the weight?
3. Only RL setting is demonstrated in this case. What about imitation learning? It seems that the same principle should apply as well. This could further enlarge the impact of this work. A related work in this area is Bowen et al. 2022 [1]. 

[1] Wen, Bowen, et al. "You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration." RSS 2022.
robotics_focus: Highly relevant to robotics but no hardware experiments
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Overall, the paper presents interesting finds with sufficient supporting experiments that deserve publication at the conference. I believe these findings will be beneficial for other researchers in the community. 
issues: Please see my weakness in the above sections. 

For the robotics focus, I chooose "no hardware experiments" as the current hardware experiments are not sufficient to demonstrate the core idea of the proposed method.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper presents an approach for fusing different coordinate frames for point clouds in robot manipulation tasks. In robot manipulation, you can use different coordinate frames to express the given point cloud. The paper considers five different coordinate frames: the world frame, which is a fixed frame, robot base frame, which is a frame attached to the robot's base, an end-effector frame, which is attached to the robot's end-effector, and target-part frame, which is attached to the part of the target object the robot wishes to manipulate. The paper demonstrates the impact that each coordinate frame has on learning in five different manipulation tasks, and propose an architecture that can fuse different frames to produce an action.
strengths_and_weaknesses: Strengths:
- Delves into the important-yet-often-overlooked problem of choosing a frame for sensory data in robot manipulation
- Performs thorough empirical analysis of their hypotheses (ex. different frames lead to different performances; a fusion of point clouds expressed in different frames will lead to better performance)

Weaknesses:
- The paper lacks a detailed analysis of why they have chosen a particular manipulation task to validate their hypotheses. Currently, it sounds like they have been selected just because they were available.
- A more thorough analysis of empirical results is needed.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think this is a very helpful paper in general, but it could use additional analyses on the empirical results. More concretely, why did you choose the five tasks? Was it because they were available? It seems that you have chosen them based on their merits - ex. some require movement of the base, some don't; some require moving a part of an object and some don't, etc. It would be beneficial for the reader if you could provide why these particular tasks have been chosen so that they can clearly understand what you are trying to test in each task.

Second, I think you need a more elaborated and thorough description of your insights and experimental results. For instance, I found lines 170-175, which outline your insights as to why certain frames would help to learn, to be very interesting and helpful, and would have liked to see more validation of these insights than the ones you provided in lines 201-202. Also, I would like to know why Table 1 and Figure 9 are the way they are. Is it just the fusion of the point cloud expressed in different frames that result in this? Or was there something else? If it is the former, why is that so? If it was the latter, what were the other factors?

Lastly, for frames such as target-object-part frame, it is important where on the object you choose your frame (ex. you chose the handle). So, it seems that human judgment is not only needed in choosing the type of frame but also the where on the object. The paper needs to be more upfront about this.
issues: Please see the summary of the recommendations above.
review_title: null

final_score: rating: Weak Accept: I recommend accepting the paper, but will not argue for my recommendation if the majority of other reviewers have a different opinion.
review_content: summary: This paper studies the effect of different coordinate frames of point clouds on learning robotic manipulation skills with RL. 4 coordinate frames are investigated on 5 simulated manipulation tasks with 2 different 3D backbones. The experimental results demonstrate different coordinate frames indeed lead to different performances, and some coordinate frame, e.g., object-part frame, seems consistently better than other. Three schemes of fusing different coordinate frames are proposed and demonstrated to outperform single-frame baselines. The method is also demonstrated to work on a real-world object picking task.
strengths_and_weaknesses: Strength:
- According to my best knowledge there has not been prior work investigating frame choice for 3D RL on robotic manipulation tasks, which seems to be an essential choice when doing RL with point clouds, so the problem itself is worth studying and thus the paper is well-motivated.
- The results are also interesting which demonstrates different performances with different coordinate frames. This suggests the tested 3D backbone networks are not se3 equivirant. 
- The proposed FrameMiners method provides a free lunch to use for RL with point cloud inputs.

Weakness:
- The paper is fully empirical, with some limited discussion on why different frames lead to different performances. It would be nice if the authors can provide more insights into why different frames work so differently.
- If the only difference between different frames is just a se3 transformation, is it possible to learn a transformation layer to preprocess the input pointcloud before passing it to the policy? The transformation layer can be jointly trained using the RL loss. I would like to hear the author's thoughts on this.
- What frame is the robot action in? Is it always in a fixed frame, or does it change accoordingly when the input point cloud frame changes?
- There has been some recent work on transformation-invariant or transformation-equivirant point-cloud backbones / object representations [1][2]. The authors are encouraged to discuss if such works can be used to address the frame choice problem for robotic manipulation.
- The real-world experiments only demonstrate the proposed FM-MA policy can work. It would be good to know the baseline's performance on the real world as well to see if the difference also exists in the real world.

[1] Deng et al, Vector Neurons: A General Framework for SO(3)-Equivariant Networks
[2] Simeonov*, Du*, et al, Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation 

robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Limitations are addressed clearly
summary_of_recommendation: Please see the strenght/weakness sections. 
issues: Please see the strenght/weakness sections.
review_title: null

final_score: rating: Strong Accept: I recommend accepting the paper and will argue for my recommendation even if other reviewers hold a different opinion.
review_content: summary: The paper identifies the importance of using particular coordinate reference frames for reinforcement learning with point-cloud inputs, demonstrates this experimentally, and proposes a novel technique to learn policies from a fusion of reference frames. Commonly used frames are investigated: world, robot base, end-effector and object frames (world and robot are meaningful to look at separately since a mobile robot is considered). Three different "frame mining" techniques are proposed (i.e. using information from different frames to select the action to execute), though it is shown in the experiments that while they do not differ significantly amongst themselves, they all provide improvements over using a single frame for the entire task. Experiments are performed on a variety of simulated platforms including a single fixed arm, a bimanual mobile robot, and a demonstration of the method on a real robot.
strengths_and_weaknesses: The paper is very clearly written and in a more colloquial manner without sacrificing technical detail which I appreciate. I think the paper conclusions are intuitive and insightful. I believe the claims being made regarding the benefits of particular frames benefitting different tasks is known in robotics, but I have not seen a rigorous demonstration of this. This to me is the main strength of the paper, it is validating knowledge that is informally accepted in robotics in a systematic way.

The paper goes a step further though, and additionally proposes a novel method to fuse useful information from different frames to generate actions. It's very interesting (surprising, even) that a weighted combination of the proposed actions from each module in the network ends up producing a meaningful action that improves over using any one frame individually.

The other two FrameMiner methods beyond FM-MA are nice to show, but there is one more I think would be interesting to look at: take only the max-weighted action from the FM-MA architecture, i.e. don't do a linear combination but take the one with the highest weight. I suspect there may be diminishing contributions and that one frame ends up dominating in certain phases of different tasks, or that frames contribute equally enough that it won't matter which one is selected. I would gain more insight about the importance of the linear combination in FM-MA seeing this comparison.

The experiments are thorough and I appreciate the variety of tasks and platforms (mobile, bimanual, single-arm).

The limitations section is underwhelming. A big limitation that received no discussion is only PPO is looked at for behavior generation. It would be informative to look at other RL methods, but beyond RL, I think it would be much more informative to look also at imitation learning and model-based planning. As it stands the paper can primarily support the benefit of frame mining for RL, whether those benefits extend equally into other domains of policy learning and planning is unclear.

A final factor that I think is overlooked is the camera views. It is emphasized that acquiring new camera views is not necessary for the method and it only takes in a fused point cloud - however, the camera views still matter. For example, what if the camera is mounted on the robot's mobile base, or on the end-effector? Then the perception is situated to one of the frames identified as relevant for action. Does that change things w.r.t. to the conclusions the paper makes? I think discussing the aspect of a moving camera, if not showing additional experiments on it, is a worthwhile point to elaborate on.
robotics_focus: Sufficient demonstration on hardware
quality_of_the_limitations_section: Additional details required
summary_of_recommendation: I think this is a great paper, it had me excited while I was reading it, and I will advocate strongly for acceptance even if other reviewers feel differently. The results are simple, intuitive, and thorough. I believe the conclusions will have a strong impact on the manipulation learning community, since the reference frame one learns in is often an overlooked matter. If one can apply a simple method to fuse the best information from many reference frames without having to change the experimental setup at all (i.e. the cameras can stay wherever you want them), then it's an all around win. I do think there are more limitations to the method than discussed, which I describe in more detail in the strengths/weaknesses section. These are matters that can likely be addressed though in the rebuttal and in minor paper revisions.
issues: Please see my comments in the strengths/weaknesses section. I would particularly appreciate the authors' input on the alternative to FM-MA I proposed (select max-weighted action), and the matter of a possibly moving camera that may be situated in an action-relevant frame (e.g. on the mobile base or on the end-effector, both common camera placements).

There are some smaller issues I identify here to improve the paper:
    - Figure 1, the 4th image caption should be "PushChair".
    - Pg. 3, last paragraph, second sentence, "... to extract visual features..." (pluralize features)
    - Figure 3, it would be easier to see with a notably different color for the door, yellow and orange are too similar.
    - Figure 8, the axes should be the same so that we can compare the plots on the same scale.
review_title: null

Meta-review: metareview: There is consistent agreement from all 4 reviewers that this is a well structured paper with contributions that can potentially provide broad impact in robotics and are well supported by results and evidence.
confidence: 5: The area chair is absolutely certain
Decision: Accept (Poster)