Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds
Authors: Minghua Liu
Xuanlin Li
Zhan Ling
Yangyan Li
Hao Su
Venue: CoRL 2022 Poster (2022)
Keywords: point cloud
coordinate frame
robot manipulation
3D
RL

Abstract:  We study how choices of input point cloud coordinate frames impact learning of manipulation skills from 3D point clouds. There exist a variety of coordinate frame choices to normalize captured robot-object-interaction point clouds. We find that different frames have a profound effect on agent learning performance, and the trend is similar across 3D backbone networks. In particular, the end-effector frame and the target-part frame achieve higher training efficiency than the commonly used world frame and robot-base frame in many tasks, intuitively because they provide helpful alignments among point clouds across time steps and thus can simplify visual module learning. Moreover, the well-performing frames vary across tasks, and some tasks may benefit from multiple frame candidates. We thus propose FrameMiners to adaptively select candidate frames and fuse their merits in a task-agnostic manner. Experimentally, FrameMiners achieves on-par or significantly higher performance than the best single-frame version on five fully physical manipulation tasks adapted from ManiSkill and OCRTOC. Without changing existing camera placements or adding extra cameras, point cloud frame mining can serve as a free lunch to improve 3D manipulation learning.

Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds
Minghua Liu"", Xuanlin Li"", Zhan Ling" Yangyan Li², Hao Su!
UC San Diego AAlibbba https://colin97. .it.ub..g.t........ io/FrameWining/
Abstract: We study how choices of input point cloud coordinate frames impact learning manipulation skills from 3D point clouds. There exist a variety of coor- dinate frame choices to normalize captured robot-ohject-interaction point clouds. We find that different frames have a profound effect on agent learning perfor- mance, and the trend is similar across 3D backbone networks. In particular, the end-effector frame and the target-part frame achieve higher training efficiency than the commonly used world frame and robot-base frame in many tasks, intuitively because they provide helpful alignments among point clouds across time steps and thus simplify visual module learning. Moreover, the well-performing frames vary across tasks, and some tasks may benefit from multiple frame candidates. We thus propose FrameMiners to adaptively select candidate frames and fuse their merits in a task-agnostic manner. Experimentally, Frame.Miners achieves on- par significantly higher performance than the best single-frame version five fully physical manipulation tasks adapted from ManiSkill and OCRTOC. Without changing existing camera placements adding extra cameras, point cloud frame mining can serve as a free lunch to improve 3D manipulation learning.
Keywords: point cloud, coordinate frame, robot manipulation, 3D, RL
Introduction
With the rapid development and proliferation of low- cost 3D sensors, point clouds have become more cessible and affordable in robotics tasks [1]. Also, the tremendous progress in building neural networks with 3D point clouds [2, 3, 4, 5, 6, 7] has enabled powerful and flexible frameworks for 3D visual un- derstanding tasks such as 3D object detection [5, 8,
Enekctlecor
Target Part Frame
Framet
9]. 6D pose estimation [10, 11], and instance seg- mentation [12, 13J. Very recently, point cloud started be used as the input to deep reinforcement learn- ing (RL) for object manipulation [14, 15, 16], which aims at learning mappings directly from raw 3D sensor observations of unstructured environments to robot action commands. These end-to-end learning methods avoid highly structured pipelines and labo- rious human engineering required by conventional robot manipulation systems.
Input Point Cloud
Robos-oooe Frame
Candidicto
PC In Frame
Frame !
Candidate Frame 2
PC In Frame
Robot Action
Candidale
PC
Figure 1: A 3D point cloud a dual-arm robot pushing chair, which can be repre- sented in various coordinate frames without changing camera placements requiring tra camera views. Our FrameMiner takes as input point cloud represented in multiple candidate frames and adaptively fuses their merits, resulting in better performance.
When building agent with point cloud input, existing works [14, 15, 16] typically incorporate off-the-shelf point cloud backbone networks (e.g., PointNet (2]) into the pipeline as feature extractor of the scene. However, some facets in constructing point cloud representations have been overlooked. For example, in the literature of 3D deep learning, the choice of coordinate frame significantly affects
*equal contribution
6th Conference on Robot Learning (CoRI. 2022), Auckland, New Zealand.

---

task performance [2, 17, 18, 19, 20, 21]. On 3D instance segmentation benchmarks for autonomous driving, previous work such as [5] showed a pipeline to process input point clouds in the camera frame, frustum frame, and object frame subsequently, leading to a large performance boost in com- parison to using the camera frame alone. For our goal of manipulation skill learning, point clouds describe dynamic interactions between rohots and objects, including frequent contacts and occlu- sions. This is a novel and more complex setting that differs from well-explored scenarios in 3D supervised learning (e.g., single objects, outdoor scenes for autonomous driving). Under this set- ting, choices of coordinate frames are more flexible and diverse as multiple entities ge.g., robot and manipulated object) and dynamic movements are involved.
In this work, we first examine whether and how different coordinate frames may impact the per- formance and sample efficiency of point cloud-based RL for object manipulation tasks. We study four candidate coordinate frames: world frame, robot-base frame, end-effector frame, and target- part frame. These frames differ in positions of origin and orientations of axes, and canonicalize inputs in different manners (e.g., a fixed third-view, ego-centric, hand-centrie, object-centric). The comparison and analysis are performed on five distinct physical manipulation tasks adapted from ManiSkill [22] and OCRTOC [23], covering various numbers of arms, robot mobilities, and camera settings. Results show that the choice of frames has profound effects. In particular, the end-effector frame and the target-part frames, rarely considered in previous works, lead to significantly better sample efficiency and final convergence than the widely used world frame and robot-base frame on many tasks. Visualization and analysis indicate that, by using different coordinate frames to rep- resent input point clouds, we are actually performing various alignments of input scenes through SE(3) transformations, which may simplify the learning of visual modules.
However, the well-performing single coordinate frame may vary from task to task, and in many cases, we may need coordination between decisions made according to multiple coordinate frames. For example, tasks equipped with dual-arm robots may benefit from both left-hand and right-hand frames. For mobile manipulation tasks involving both navigation and manipulation, different frames could favor different skills (e.g.. robot-base frame for navigation skills, end-effector frame for ma- nipulation skills). We thus propose three task-agnostic strategies to adaptively select from multiple candidate coordinate frames and fuse their merits, leading more efficient and effective object manipulation policy learning. Because we do not need to capture additional camera views or rely on task-specific frame selections, prove existing methods on point cloud-based policy learning. We call these fusion approaches as FrameMiners. Experimentally, we find that it matters to fuse information from multiple frames, but the specific FrameMiner to choose does not create much performance difference. In particular, we use one of the FrameMiners, MixAction, interpret the importance of different frames in the policy execution process, and the interpretation agrees with our intuitions.
frame mining strategies can be used as a free lunch im-
In summary, the main contributions of this work are follows:
We find that the choice of coordinate frame has a profound impact on point cloud-based object manipulation learning. In particular, the end-effector frame and the target-part frame lead to much better sample efficiency than the widely-used world frame and robot-base frame on many tasks;
We find that well performeform frames differ task by task, necessitating task aggostic ways to select and fuse frames. This observation is consistent across 3D backbone networks;
We propose FrameMiners, a collection of methods to fuse information from multiple candiate frames. FrameMiners provide a free lunch to improve existing point cloud-based manipulation learning methods without changing camera placements or requiring additional camera views.
2 Related Work
Manipulation Learning with Point Clouds Visual representation learning for object manipula- tion has been extensively studied [24, 25, 26, 27, 28, 29, 30, 31]. With the flourishment of 3D deep learning [2, 3. 4, 5, 6, 71. a major line of work learns representations from 3D point clouds for object manipulation [32, 33, 34, 35, 36, 37, 38]. Recently, people have also started incorporate point clouds into deep reinforcement learning (RI.) pipelines for manipulation learning [14, 15, 16). However, existing point doudd manipulation learning methods have not paid enough attention to coordinate frame selections of input point clouds, which is fundamental in 3D visual learning- Some very recent work [39, 40J explored placement and selection of camera views and fusion of multi-view images. We differ from them in that we focus on the preprocessing of captured input point clouds without modifying existing camera configurations or adding additional cameras.
2

---

Task Robot Arm
PickObjoct Single Arm
OpenCabinetDoor Single Arm Mobile Panoramic on Head
OpenCabinetDrawer Single Arm Mobile
PushChair
MoveBucket Dual Anm
Dual Arm
Mobility
Fixed Base
Mobile
Mobile
Cameras
3rd- View
Panoramic on Hcad
Panoramic Head
Panoramic Head
Figure 2: We study coordinate frame mining manipulation tasks adapted from OCRTOC [23] and ManiSkill [22] covering various setups (e.g., #arms, mobility, camera). Simulation is fully physical.
Normalization and View Fusion in Point Cloud Learning Normalizing input point clouds is common practice in 3D deep learning literature. For example, in single object analysis (e.g., classification and part segmentation), people often normalize input point clouds into categorical canonical pose with unit scale [2, 3], simplifying network training. Prior works find that existing
point cloud networks [2. 3, 7, 41, 42] are very sensitive to input normalization [21, 43, 44), and many recent attempts explore rotation invariant [45, 46, 47J and equivariant methods [21, 48, 49] for 3D deep learning. Compared to well-studied scenarios (e.g., single object and autonomous driving), normalizations of point clouds under robot-object interactions are under-explored.
In LiDAR point cloud learning for autonomous driving, many work focuses on the fusion of multiple views [17, 18, 19, 20J. Unlike fusing multiple camera scans, there is only one point cloud. They propose to process the point cloud from different views (e.g., perspective view and birds-eye view) to combine their merits, which has proven to be helpful. Our work shares a similar idea. However, we focus on robotic object manipulation settings, and the choice of coordinate systems is more diverse.
3 Point Cloud Coordinate Frame Selection Matters
3.1 Problem Setup
We aim to learn agents with point cloud input for object manipula- tion tasks via Reinforcement / Im- itation Learning (RL/IL). A task is formally defined as Partially- Observable Markov Decision Process (POMDP), which is represented by tuple M (S, A, μr, T, R2 22, Here S and A are the environment
3D (Fused) Point Cloud
3D Backbone (e.g,, PointNert
Policy MLP
Robot Action
Proprioceptive Robot State
Value MLP
Value
Architecture of 3D point cloud-based agent, which is optimized by actor-critic RL algorithms. We study coordinate frame selection of input (fused) point cloud.
Figure 3:
state space and the action space. μus). T(s'|s, a), R(s, and are the initial state distribution, state transition probability, reward function, and discount factor, respectively. O(s) S - Ω the observation function that maps environment states to the observation space Ω. Our agent is repre- sented by policy π Ω > A, which aims maximize the expected accumulated return given by J(x O) = Ex.r.x (E ""r(st, 4.). Note that и does not have access to the environment state and only has access the observation (s)). this work, O(s) consists of two parts: (1) 3D point cloud captured by depth cameras; (2) proprioceptive states for the robot, such joint positions and joint velocities. For the first part, there are multiple cameras, we fuse all point clouds into a single one by transforming them into the same coordinate frame and concatenating the points together.
Fig. 3 shows the architecture of 3D point cloud-based agent, which we use to discuss in this section. It first exploits a 3D backbone (c.g.. PointNet [2]) to extract visual features from 3D (fused) point cloud. The extracted features are then concatenated with proprioceptive robot states and fed into separate multi-layer perceptrons (MLP) for action and value prediction. The input (fused) point cloud can be represented in different coordinate frames before being fed into the 3D backbone network, and the choice of coordinate frame is independent of camera views. For example, a point cloud captured by camera mounted on the robot's head can be transformed into the end-effector frame. In this work, we study how point cloud coordinate frames affect sample efficiency and final convergence of object manipulation learning. Unlike prior works [39, 40], we do not change robot camera configurations (e.g., camera placement, inclusion of additional cameras).

---

World Frame
Robot-Base Frame
End-Effector Frame
Target-Part Frame
Time Step
Figure 4: Illustration of four coordinate frames, which provide different alignments across time steps. We visualize three point clouds (three time steps) of an OpenCabinetlDoor trajectory. Each row shows the same point cloud represented in different coordinate frames. Please zoom in for details. Robot arm, cabinet door handle, cabinet door, and cabinet body are colored in blue, red, yellow, and brown, respectively. RGB arrows indicate the corresponding origin and axes for each frame. Since the point clouds used for policy learning can be rather sparse, we show dense point clouds here for better visualization.
As shown in Fig. 2, we exemplify the frame selection problem five fully-physical manipulation tasks, covering various numbers of robot arms, mobilities, and camera settings. Among them, Pick- Object is adapted from OCRTOC [23], and the other four tasks are adapted from ManiSkill [22]. On PickObject, a fixed-base single-arm robot learns to physically grasp an object from the table, lift it up to target height, and keep it static for while. Point clouds are captured from 3rd-view camera. On ManiSkill tasks, agents learn generalizable physical manipulation skills (i.e., opening cabinet doors drawers, pushing chairs/ / moving buckets target positions) across objects with diverse topology, geometry, and appearance. We utilize mobile robots with one or two arms. Point clouds come from a panoramic camera mounted the robot's head. Action space includes joint velocities of the arm(s) and the mobile robot base, along with joint positions of the gripper(s). More details are presented in the supplementary material.
3.2 Choices of Point Cloud Coordinate Frame
For 3D supervised learning tasks such object classification and detection, it's common prac- tice to normalize input point clouds, and the choice of coordinate frames significantly affects task performance [2, 21, 17, 18, 19, 20]. In point cloud-based manipulation learning, are faced with an underexplored, yet more challenging. setting. First, point clouds describe more complex robot- object interactions, possibly including frequent contacts and occlusions. Furthermore, compared to supervised learning, 3D visual modules receive weaker supervision signals during RL training. Therefore, it may become properly normalizing input point clouds. Unlike previous well-studied point cloud learning scenar- ios (e.g., single-object point clouds, LiDAR point clouds for autonomous driving), there exist more diverse choices of coordinate frames. In this paper, we compare and analyze four candidates:
more important lessen the burden of visual module learning by
A world frame attached a fixed point the world (c.g., the start point of trajectory).
A robot-base frame is attached to the robot base, offering an egocentric perspective on a mobile robot. For fixed-base robot, world frame and robot-base frame could be equivalent.
In many object manipulation tasks, movements robot end-effector(s) play important roles, and can attach an end-effector frame to each of them. Note that for dual-arm robots, there are two end-effectors and thus two end-effector frames.
A target-part frame is attached to the object part the robot intends to interact with (e.g., target door handle for the OpenCabinetDoor task).
When we transform captured point clouds into the world frame, the robot-base frame, and the end- effector frame, we may need proprioceptive robot states and potential robot movement tracking, which is typically accessible in modern robots. When we transform point clouds into the target-part

---

OpenCabinetDoo
OpenCabbineDrrsver
PushCChair
MoveBucket
PickOhjjct
390
900
-193
300
-700
-00
190
130
-300
503
-90
100
-100
10
-1900
10
20
-1293
10
Envionment Steps (xtel)
Envirnmennt Stops 0C1e6)
Emvronment Steps (xte6)
Envirommm Stops (xtef)
Emvronmennt Steps Ster octe6)
* World Franee
Robrot Base Franme
End- Elfoctor Frame
Target Part Fraie
Comparison of four coordinate frames on five fully-physical manipulation tasks. The (fused) point cloud is transformed to single coordinate frame hefore being fed to the visual back- bone network. For dual-arm tasks (i.e., PushChair and MoveBucket), as the end-effector frame. For PickObject, which has a fixed base, the world frame is the same as the robot-base frame. Mean and standard deviation over 5 seeds are shown.
Figure 5:
the right-hand frame
frame, we may need to leverage off-the-shelf 3D object detection and pose estimation techniques. However, in this paper, we mainly focus on the choices of coordinate frames themselves. In our simulated experiments, we use ground truth object poses for the target-part frame.
In Fig. 4, we visualize an example trajectory under four coordinate frames. As shown in the figure, different coordinate frames canonicalize inputs in different manners (e.g.. fixed third-view, ego- centric, hand-centric, and object-centric), which essentially performing various alignments among point clouds across multiple time steps. For example, in the end-effector frame, the end-effector is always aligned at the origin throughout a trajectory. Such alignments may simplify the learning of visual modules in distinct ways. With the end-effector frame, the network does not need to locate the end-effector in point clouds (always at the origin). Similarly, with the target-part frame, it can be easier to determine the relative position between the target part and the robot end-effector. The robot-base frame naturally aligns its frame axes with the moving directions of the robot's 's base.
3.3 Single-Frame Comparison on Manipulation Tasks
We compare the four coordinate frames on the five manipulation tasks by training PPO [50] agents using PointNet [2] as the 3D visual backbone. In this section, the (fused) point cloud is transformed into a single coordinate frame. For PushChair and MoveBucket tasks that use a dual-arm robot, we use the right hand frame as the end-effector frame (we observe almost identical performance using the left hand frame). For the target-part frame, we choose the handle frame for OpenCabinetDoor and Drennabine tasks, chair seat frame for the PushChair task, bucket for the MoveBucket task, and the target object for the PickObject task. Further details are presented in the supplementary.
Fig. 5 shows the results. We observe that distinct coordinate frames lead to very different agent train- ing performances. Overall, the world frame is the least effective, especially in PushChair and Move- Bucket that involve more pronounced movement of the robot base. This suggests that the alignment of a static point in the world-frame is less helpful for the tasks. Compared to the commonly-used world-frame and robot-base frame, the end-effector frame has much higher sample efficiency on all and PickObject), demonstrating the
single-an arm tasks (i.e., OpenCabinetDoor, OpenCabinetDrawee benefits of end-effector alignment. However, it shows similar or worse performance on PushChair and MoveBucket, intuitively because these tasks rely on dual-arm coordination, but our point cloud normalized to a single end-effector frame (i.e., right hand frame). In addition, the target-part frame achieves the best sample efficiency on most tasks, suggesting that the target-part alignment across time could be of great help for point cloud-based visual manipulation learning.
3.4 Further Analysis
In robot manipulation tasks, agents often need to infer binary relations between subjects (e.g., rela- tive pose between the end-effector and the cabinet handle). By aligning point clouds under certain frames (c.g., end-effector frame), these tasks may be reduced to single-subject location tasks (c.g., simply copying the handle pose), which become much easier to solve. To confirm this hypothesis, we perform a diagnosis experiment on OpenCabinetDoorr, where we intentionally remove all robot points (i.e., blue points in Fig. 4) and see its effect different coordinate frames. As shown in Fig. 6, after the robot points are removed, the end-effector frame performs the same, while the robot-base frame performs worse (the task is still solvable since the end-effector position is also provided in the proprioceptive robot state). This suggests that the end-effector frame allows an agent to focus on the target object, along with its interaction with the robot hand, which verifies our hypothesis.
5

---

OpenCabinetDoor
OpenCabinelDrawer
PushChair
MoveBucket
-709
200
Return
200
-000
-130
-100
10
-1100
10
16
10
16
Enviro mont Stups (xfe6)
Environment Stups (K1e6)
Enfronmemt Steps (x1e6)
Environment Steps (cte6)
Wortdl Frame
Robot-Base Frame
End-Erector Frame
Target-Part Frame
Figure 7:
Using SparseConVNet [51] as the 3D visual backbone, we observe similar trends as Fig 5. Mean and standard deviation over 5 seeds are shown.
OpenCabinetDoor
We utilized PointNet [2] as our 3D visual backbone for its fast speed and general good performance. However, it's unclear whether point cloud frame selection is also crucial for other 3D backbones, especially those more complex and powerful. Therefore, we conduct the same experiments as Sec 3.3 using SparseConvNet [51], a heavier 3D backbone network, on the four ManiSkill tasks (further details in the supplementary). As shown in Fig. 7, we observe similar relative performance between frames as before (e.g., the world frame performs poorly; the end-effector frame outperforms the world frame and the robot-base frame on OpenCabinettoow and Open- CabinetDrawwrr). Interestingly, using SparseConvNet doesn't improve the overall performance over PointNet.
0.8
Basc Rts
Rate Rate
0.6
EE-Fut,
EE-No, Rubot Pts
Success
0.2
15
Environment Steps (x1e6)
Figure 6: Removing robot points does not harm the performance of the EE frame OpenCabinetDoor.
Mining Multiple Coordinate Frames
We have shown that different point cloud coordinate frames lead to distinct sample efficiencies and final per- formances in manipulation learning. However, a sin- gle frame can perform well on some tasks but poorly on others, and we wish to find good frames in task- agnostic manner. Moreover, for complex manipulation tasks, a single frame could be insufficient, and synergis- tic coordination between multiple frames could provide unparalleled advantages. For example, when robots are equipped with multiple arms, each arm may have its pre- ferred coordinate frame (c.g., left-hand frame and right- hand frame). In addition, in tasks that involve simultanc- ous navigation and manipulation (c.g., on PushChair and MoveBucket, an agent needs to move towards the target while manipulating chairs or buckets), different frames could benefit different skills (e.g.. robot- frame for navigation skills, and end-effector frame for manipula- tion skills). Therefore, it is of great help propose a prior-agnostic method that can automatically select the best frame from multiple candidates combine the mer- its of them. Again, we are not talking about fusing mul- tiple camera views. Point clouds from multiple camera views are first fused together into single point cloud, before being transformed to each coordinate frame.
Finil Robot Action
Softmax Linear Combination
Action
Action
Weight
(m)
Robaa state
Proprooeptiee
MLP1
MLP
MLP
Concat
MLP
Pointhict
PointNet 2
PointNet n
PC in
in Frame 2
PC Framen
Framee
(Fosedi
Transtormation
Frame
Frame 2
Frame
Figure 8: The pipeline of FrameMiner- MixAction (FM-MA). Each frame out- puts an action proposal. Actions are then fused through input-dependent and joint- specific weights.
In this section, we will present a collection of three strategies to adaptively select and fuse multiple candidate coordinate frames, and call them FrameMiners. In particular, will first introduce FrameMiner-MixAl
in Section 4.1 in detail to interpret the importance of different frames in the policy execution process. We will then briefly introduce the other two FrameMiners and compare different approaches with single-frame baselines.
4.1 FrameMiner-MixxAc
Inspired by the idea of mixture of experts [52], we propose a general and interpretable framework, FrameMiier-MixAction (FM-MA). As shown in Fig. 8, FM-MA takes (fused) point cloud and

---

Base Joints
Left Hand Joints
Right Hand Joints
weight
25
50
25 Timestap
50
25 Timestap
Timestop
Roboot Base Frame
Left-Hand Frame
Right-taadd Franne
Figure 9: Left: learned weights of TrameMMiner-MxAAttin over a MoveBucket trajectory, where three coordinate frames are fused. We divide robot joints into three groups and show the average weights of each group in each coordinate frame. Right: three stages of the trajectory. (a) Approach- ing the bucket. (b) Moving the bucket to the platform. (c) Placing the bucket on the platform.
candidate coordinate frames as input, and first transforms the point cloud into coordinate frames. For each transformed point cloud, FM-MA employs an expert network, consisting of a 3D visual backbone (e.g., PointNet [2]) and MIP, propose full robot actions (e.g. target velocity of m joints). Since different experts use different coordinate frames, they are encouraged to specialize different skills and controls of different joints. Finally,
combine actions from the experts through input-dependent weights. Specifically, we concatenate extracted visual features from all n frames with the proprioceptive robot state, feed it into an MLP, and predict weight for each pair of expert and joint (there are n X m weights in total). For each joint, we normalize the weights over n experts via softmax and fuse the actions through weighted linear combination.
FM-MA fuses actions by predicting joint-specific weights, since we believe that, for different joints, need to extract information from different coordinate frames. Furthermore, the weights input-dependent, potentially allowing the model to capture dynamic joint-frame relations at differ- ent stages of a task. Fig. 9 confirms these hypotheses. On MoveBucket trajectories, observe distinct frame preferences between different robot joints. The left and right-hand frames contribute significantly to their respective joint actions. In addition, the weight distribution changes greatly over different trajectory stages. Initially, when the robot is moving towards the bucket but not in- teracting with it, the base frame contributes more. However, when the hands start manipulate the bucket, the hand frames' weights increase. In particular, when the robot places the bucket onto the platform, we need careful coordination among all joints, and thus similar weights from each frame.
4.2 FrameMiners Single Coordinate Frame
To study how network architectures influence coordinate frame fusion, we also propose other two strategies: FrameMiner-
FrameMMiner-FeatturCCCCCa FentureConcat (FM-FC) and FrameMiner- CrameMiner-TTrnnffrmmerr TransformerGrrop (FM-TG). For each transformed point cloud, FM-FC uses an individual PointNet to extract visual feature. All visual features are then concatenated and fed into an MLP to predict robot action. FM-TG decomposes our robot action into three groups: base joint actions, left-hand joint actions, and right- hand joint actions (only two groups for single-arm tasks). After visual features are extracted from PointNets, they are fused through Transformer [53] to produce feature for each action group, which passes through an MLP predict its respective joint actions (see the supplementary material for details).
We compare our three FrameMiners with single-frame frame base- lines. Specifically, in this section, we focus on frame mining among the robot-base frame and end-effector frume(s). For the dual-arm tasks (i.e., PushChair and MoveBucket), the end- effector frames include both the left-hand frame and the right- hand frame. We will discuss the inclusion of target-part frame in Section 4.3. Fig. 10 and Tab. show the comparison re- sults. On single-arm tasks (i.c., DpenCabbneeDoor/rraweerr our FrameMiners perform on par with the end-effector frame, which suggests that FrameMiners can automatically select the best single frame. On dual-arm tasks, our FrameMiners significantly outperform single-frame baselines, demonstrating the advantage of coordination between multiple coordinate frames. We also demonstrate that our FrameMiners outperform alternative designs and provide robust advantages (more details in the supplementary material). While it matters to fuse information from multiple frames, the specific FrameMiner to choose does not create much perfor- mance difference. Empirically, we find FM-MA less sensitive to training parameters, and FM-TG more computationally expensive.
Basc
EE
FM-FC
FM-MA
FM-TG
Door 54177
80
791:3
84:12
70-6
Drawer 88:22 93=1
94=1
93-1
93=2
74:3
32::
36:14
3426
Bucket 23:6 19=4
77:55
81+3
90=2
Table 1: Success rates (%) four ManiSkill tasks.
7

---

OpenCabinet
OpenCabineiDrawer
PushChair
MoveBucket
4ce
-662
400
Retum
100
288
660
1902
1530
10
15
10
10
Environment Steps (61
Environment Steps (x1e6)
Environment Stups (6)
Enviorment Staps (xfe8)
Robio Baso Fromm
.......
End Elfector Framz:
FM-FC
FM MA
FM-TG
Figure 10: Comparison of different frame mining approaches on the four ManiSkill tasks, where the robot-base frame and end-effector frame(s) are fused. Black lines indicate single-frame baselines. Mean and standard deviation over 5 seeds are shown.
Our previous experiments were conducted using online RL. To investigate whether our previous findings can generalize to other algorithm domains, we perform experiments on imitation learning, and more details are presented in the supplementary material. The results corroborate our previous findings, L.., different coordinate frames have profound effect on point houudbaased manipulation skill learning, and FrnmeMiners are capable automatically selecting the best coordinate frame or combining the merits from multiple frames and outperforming single-frame results.
4.3 Target-Part Frame
PushChair
PickObject
In Section 4.2, we focus on the fusion of robot-base frame and end-effector frames, since the target-part frame relies on pose estimation of target objects, which requires ex- tra efforts in real-world settings. As shown in Fig. 11, if object poses estimated, sometimes observe further performance boost of FrameMiners from the target-par frame. For example, PushChair, by incorporating the target-part frame, the success rate of FM-MA increases from 36=4% 53+33% On PickObject, FM-MA al- ready achieves good performance without the target-part frame (94:22%); ); incorporating it slightly improves the success rate to 1 %..
Ratum
Envronment Stops (rtes)
Environent Stopo (xtel)
Robot- Rese Frams
Torget-Port Frams
w/o TP Frome
Figure 11: Fusion of target-part frame could further boost the performance.
5 Real World Experiments
To further verify that our Iearned policies can be deployed on real- world robots without introducing extra domain gaps, test Pick- Object with a Kinova Jaco2 Spherical 7-DoF robot, an Intel Re- alSense [54] D435 camera for uncolored point cloud capture, and Rubik's cube from YCB objects [55, 56] (see Fig. 12). We use a 3-DoF end-effector position controller and 1-DoF gripper position controller. We train the FM-MA policy (Section 4.1) by fusing the robot-base frame and the end-effector frame. At test time, we build digital twin in the simulator over 25 sampled initializations of the real environment with vision-based pipeline like Jiang et al. [57]. By following trajectories from policy rollout, we obtain a 84% success rate with FM-MA, compared an 80% success rate with the end- effector frame. The robot-base frame is unable to achieve successful picks under our training budget. Note that the performance differences in the real world are very similar to the simulation environment, indicating that point cloud frame selection or mining does not affect original domain gap. More details are presented in the supplementary material.
Figure 12: Real robot setup.
Conclusion and Limitations
We find that choices of point cloud coordinate frames have profound impact on learning manipu- lation skills. Our proposed FrameMiners can adaptively select and fuse multiple candidate frames, serving as free lunch for 3D point cloud-based manipulation learning. Currently, our FrameM- iners need to process each frame separately. leading to more network computation. In the future, we would like to explore more advanced fusion strategies to further improve network efficiency as well as performance. In addition, the target-part frame requires human judgment to determine the target part candidates and 6D pose estimation of the target parts, although we have shown our method can also achieve great improvements without the target-part frame (Section 4.2).

---

Acknowledgments
This work is supported in part by gifts from Qualcomm. We would like to thank Jiayuan Gu and Zhiao Huang for their helpful discussion and manuscript proofreading.
References
[1] Kinova,
com, 2022. Accessed: 2022-06-14.
[2] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 652-660, 2017.
[3] C. R. Qi, L. Yi, H. Su, and L. J. Guibas. Pointnet++: Deep hierarchical feature learning on point sets metric space. Advances neural information processing systems, 30, 2017.
[4] A. X. Chang, T. Funkhouser, I.. Guibas, P. Hanrahan, Q. Huang, Z. Li, S. Savarese, M. Savva,
S. Song, H. Su, et al. Shapenet: An information-rich 3d model repository. arXiv preprini VrXiv:1512.030
2015.
[5] C. R. Qi W. Liu, C. Wu, H. Su, and L.. J. Guibas. Frustum pointnets for 3d object detec- tion from rgb-d data. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 918-927, 2018.
[6] H. Fan, H. Su, and L. Guibas. A point generation network for 3d object reconstruction from single image. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 605-613 2017.
[7] Y. Wang. Y. Sun, Z. Liu, S. Sarma, M. M. Bronstein, and J. M. Solomon. Dynamic graph enn for learning point clouds. Acm Transactions On Graphics (tog), 38(5):1-12, 2019.
[8] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom. Pointpillars: Fast en- coders for object detection from point clouds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12697-12705, 2019.
[9] C. R. Qi, O. Litany, K. He, and L. J. Guibas. Deep hough voting for object detection in point clouds. In proceedings of the IEEE/CVF International Conference Computer Vision, pages 9277-9286, 2019.
[10] Y. He, W. Sun, H. Huang, J. Liu, H. Fan, and J. Sun. Pvn3d: A deep point-wise 3d keypoints voting network for 6dof pose estimation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11632-1 11641, 2020.
[11] C. Wang. D. Xu, Y. Zhu, R. Martin-Martin, C. Lu, L. Fei-Fei, and S. Savarese. Densefusion: 6d object pose estimation by iterative dense fusion. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3343-3352, 2019.
[12] B. Yang, J. Wang. Clark, Q. Hu, S. Wang, A. Markham, and N. Trigoni. Learning object bounding boxes for 3d instance segmentation on point clouds. Advances in neural information processing systems, 32, 2019.
[13] L. Yi, W. Zhao, H. Wang, M. Sung, and J. Guibas. Gspn: Generative shape proposal network for 3d instance segmentation in point cloud. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3947-3956, 2019.
[14] W. Huang, I. Mordatch, P. Abbeel, and D. Pathak. Generalization in dexterous manipulation via geometry-aware multi-task learning. arXiv preprint arXiv:2111.03002 2021.
[15] Chen, J. Xu, and P. Agrawal. A system for general in-hand object re-orientation. In Confer- ence on Robot Learning, pages 297-307, PMLR, 2022.
[16] Y.-H. Wu, J. Wang, and X. Wang. Learning generalizable dexterous manipulation from human grasp affordance. arXiv preprint arXiv:2204.02.300 2022.

---

[17] Y. Zhou, P. Sun. Y. Zhang, D. Anguelov, J. Gao, T. Ouyang, J. Guo, J. Ngiam, and V. Vasude- van. End-to-end multi view fusion for 3d object detection lidar point clouds. In Conference on Robot Learning, pages 923-932. PMLR, 2020.
[18] Y. A. Alnaggar, M. Afifi, K. Amer, and M. ElHelw. Multi projection fusion for real-time semantic segmentation of 3d lidar point clouds. In Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision, pages 1800-1809, 2021.
[19] M. Gerdzhev, R. Razani, E. Taghavi, and L. Bingbing. Tomado-net: multiview total variation semantic segmentation with diamond inception module. In 2021 JEEE International Confer- ence on Robotics and Automation (ICRA), pages 9543-9549. IEEE, 2021.
[20] V. E. Liong. T. N. T. Nguyen, S. Widjaja, D. Sharma, and Z. Chong. Amvnet: Assertion- based multi-view fusion network for lidar semantic segmentation. arXiv preprint arXiv:2012.049:: 2020.
[21] C. Deng, O. Litany, Y. Duan, A. Poulenard, A. Tagliasacchi, and L. J. Guibas. Vector neu- rons: A general framework for (3)-equivariant networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12200-12209, 2021.
[22] T. Mu, Z. Ling, F. Xiang, D. Yang, X. Li, S. Tao, Z. Huang, Z. Jia, and H. Su. Maniskill: Generalizable manipulation skill benchmark with large-scale demonstrations. arXiv preprint 1rXiv:2107.14483, 2021.
[23] Z. Liu, W. Liu, Y. Qin, F. Xiang, M. Gou, S. Xin, M. A. Roa, B. Calli, H. Su, Y. Sun, et al. Ocrtoe: A cloud-based competition and benchmark for robotic grasping and manipulation. IEEE Robotics and Automation Letters, 7(1):486-49) 490,6-493,220 2021.
[24] Y. Zhu, Z. Wang, J. Merel, A. Rusu, T. Erez, S. Cabi, S. Tunyasuvunakool,, J. Kramár, R. Had- sell, N. de Freitas, et al. Reinforcement and imitation learning for diverse visuomotor skills. arXiv preprint arXhv:1802.09560 2018.
[25] B. Chen, P. Abbeel, and D. Pathak. Unsupervised learning of visual 3d keypoints for control. In International Conference on Machine Learning, pages 1539-1549, PMLR, 2021.
[26] M. Vecerik, J.-B. Regli, O. Sushkov, D. Barker, R. Pevceviciute, T. Rothörl, C. Schuster, R. Hadsell, Agapito, and J. Scholz. S3k: supervupe semantic keypoints for robotic manipulation via multi-view consistency. arXiv preprint arXiv:2009.14770 14711, 2020.
[27] R. Cheng, A. Agarwal, and K. Fragkiadaki. Reinforcement learning of active vision for manip- ulating objects under occlusions. In Conference on Robot Learning, pages 422-431. PMLR, 2018.
(28) Y. Zaky, G. Paruthi, B. Tripp, and J. Bergstra. Active perception and representation for robotic manipulation. arXiv preprint arXiv:2003.06734 2020.
[29] R. Shah and V. Kumar. Rrl: Resnet as representation for reinforcement learning. arXiv preprint arXiv:2107.03388. 2021.
[30] Pari, N. Muhammad, S. Arunachalam, Pinto, et al. The surprising effectiveness of 2021.
representation learning for visual imitation. arXiv preprint
[31] S. Nair, A. Rajeswaran, V. Kumar, C. Finn, and A. Gupta. R3m: A universal visual represen- tation for robot manipulation. arXiv preprint arXiv:2203.12601 2022.
[32] X. Lin, Y. Wang, Z. Huang, and D. Held. Learning visible connectivity dynamics for cloth smoothing. In Conference Robot Learning, pages 256-266. PMLR, 2021.
[33] A. Alliegro, M. Rudorfer, F. Frattin, Leonardis, and T. Tommasi. End-to-end learning to grasp from object point clouds. arXiv preprint arXiv:2203.05588 2022.
[34] D. Yarats, I. Kostrikov, and R. Fergus. Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. In International Conference on Learning Representations, 2020.
10

---

[35] J. Lv. Q. Yu, L. Shao, W. Liu, W. Xu, and C. Lu. systim:: Towards sample-efficient, generalizable, compositional, and incremental robot learning. In 2022 IEEE International Conference Robotics and Automation (ICRA). IEEE, 2022.
[36] S. James and A. Davison. Q-attention: Enabling efficient learning for vision-based robotic manipulation. IEEE Robotics and Automation Letters, 2022.
[37] M. Sundermeyer, A. Mousavian, R. Triebel, and D. Fox. Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages 1 13438-13444. IEEE, 2021.
[38] B. Eisner, H. Zhang, and D. Held. Flowbol3d: Learning 3d articulation flow to manipulate articulated objects. Robotics Science and Systems (RSS), 2022.
[39] K. Hsu, M. J. Kim, R. Rafailov, J. Wu, and Finn. Vision-based manipulators need to also from their hands. International Conference Learning Representations, 2022.
[40] R. Jangir, N. Hansen, S. Ghosal, M. Jain, and X. Wang. Look closer: Bridging egocentric and third-person person views with transformers for robotic manipulation. IEEE Robotics and Automation Letters, 2022.
[41] M. Atzmon, H. Maron, and Y. Lipman. Point convolutional neural networks by extension operators. arXiv preprint arXiv: 10091, 2018.
[42] Y. Li, R. Bu, M. Sun, W. Wa, X. Di, and B. Chen. Pointenn: Convolution on x- x-transformed points. Advances neural information processing systems, 31, 2018.
[43] Sun, Q. Zhang, B. Kailkhura, Z. Yu, C. Xiao, and Z. M. Mao. Benchmarking robustness of 3d point cloud recognition against common corruptions. arXiv preprint S.ii:2200.112996 2022.
[44] T. Lorenz, A. Ruoss, M. Balunović, G. Singh, and M. Vechev. Robustness certification for point cloud models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7608-7618, 2021.
[45] C. Chen, G. Li, R. Xu, T. Chen, M. Wang, and L. Lin. Clusternet: Deep hicrarchical cluster net- work with rigorously rotation-invarian representation for point cloud analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4994 5002, 2019.
[46] X. Li, R. Li, G. Chen, C.-W. Fu, D. Cohen-Or, and P.-.. Heng. A for deep point cloud analysis. IEEE Transactions Visualization and Computer Graphics, 2021.
framework
[47] Z. Zhang, B.-S. Hua, D. W. Rosen, and S.-K. Yeung. Rotation invariant convolutions for 3d point clouds deep learning. In 2019 International Conference on 3D Vision (3DV), pages 204-213, IEEE, 2019.
[48] F. Fuchs, D. Worrall, V. Fischer, and M. Welling, Se (3)-transfosmmrs: 3d roto- otoottranslation equivariant attention networks. Advances in Neural Information Processing Systems, 33:1970- 1981, 2020.
[49] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff, and P. Riley. Tensor field networks: Rotation-ant translation rranslatio-eeuiivariant neural networks for 3d point clouds. arXiv preprint 1802.08002.0221 2018.
[50] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 2017.
[51] B. Graham and L. van der Maaten. Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.00 01 2017.
[52] S. Masoudnia and R. Ebrahimpour. Mixture of experts: a literature survey. Artificial Intelli- gence Review, 42(2):275-292, 2014.
11

---

[53] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, E. Kaiser, and I. Polo- sukhin. Attention is all you need. Advances neural information processing systems, 30, 2017.
[54] L. Keselman, J. Iselin Woodfill, A. Grunnet-Jepsen, and A. Bhowmik. Intel realsense stereo- scopie depth cameras. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pages 1-10, 2017.
[55] B. Calli, A. Walsman, A. Singh, S. Srinivasa, P. Abbeel, and A. M. Dollar. Benchmarking in manipulation research: The ycb object and model set and benchmarking protocols. arXiv preprint arXiv: /50.. 03143, 2015.
[56] B. Calli, A. Singh, Walsman, Srinivasa, P. Abbeel, and A. M. Dollar. The ycb object and model set: Towards common benchmarks for manipulation research. In 2015 international conference on advanced robotics (ICAR), pages 510-517. IEEE, 2015.
[57] Z. Jiang. C.-C. Hsu, and Y. Zhu. Ditto: Building digital twins of articulated objects from interaction. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5616-5626, 2022.
12


Reviews:
[{"review_id": "nhTk35kVfW", "reviewer": "Reviewer_LU2t", "summary": "This paper studies the effect of different coordinate frames of point clouds on learning robotic manipulation skills with RL. 4 coordinate frames are investigated on 5 simulated manipulation tasks with 2 different 3D backbones. The experimental results demonstrate different coordinate frames indeed lead to different performances, and some coordinate frame, e.g., object-part frame, seems consistently better than other. Three schemes of fusing different coordinate frames are proposed and demonstrated to outperform single-frame baselines. The method is also demonstrated to work on a real-world object picking task.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- According to my best knowledge there has not been prior work investigating frame choice for 3D RL on robotic manipulation tasks, which seems to be an essential choice when doing RL with point clouds, so the problem itself is worth studying and thus the paper is well-motivated.\n- The results are also interesting which demonstrates different performances with different coordinate frames. This suggests the tested 3D backbone networks are not se3 equivirant. \n- The proposed FrameMiners method provides a free lunch to use for RL with point cloud inputs.\n\nWeakness:\n- The paper is fully empirical, with some limited discussion on why different frames lead to different performances. It would be nice if the authors can provide more insights into why different frames work so differently.\n- If the only difference between different frames is just a se3 transformation, is it possible to learn a transformation layer to preprocess the input pointcloud before passing it to the policy? The transformation layer can be jointly trained using the RL loss. I would like to hear the author's thoughts on this.\n- What frame is the robot action in? Is it always in a fixed frame, or does it change accoordingly when the input point cloud frame changes?\n- There has been some recent work on transformation-invariant or transformation-equivirant point-cloud backbones / object representations [1][2]. The authors are encouraged to discuss if such works can be used to address the frame choice problem for robotic manipulation.\n- The real-world experiments only demonstrate the proposed FM-MA policy can work. It would be good to know the baseline's performance on the real world as well to see if the difference also exists in the real world.\n\n[1] Deng et al, Vector Neurons: A General Framework for SO(3)-Equivariant Networks\n[2] Simeonov*, Du*, et al, Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Please see the strenght/weakness sections. ", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "3: The work is incremental and unlikely to have much impact, even though it may be technically correct and well executed.", "issues": "Please see the strenght/weakness sections.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "dpRUzTf7EU", "reviewer": "Reviewer_1Hcu", "summary": "This paper presents an approach for fusing different coordinate frames for point clouds in robot manipulation tasks. In robot manipulation, you can use different coordinate frames to express the given point cloud. The paper considers five different coordinate frames: the world frame, which is a fixed frame, robot base frame, which is a frame attached to the robot's base, an end-effector frame, which is attached to the robot's end-effector, and target-part frame, which is attached to the part of the target object the robot wishes to manipulate. The paper demonstrates the impact that each coordinate frame has on learning in five different manipulation tasks, and propose an architecture that can fuse different frames to produce an action.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Delves into the important-yet-often-overlooked problem of choosing a frame for sensory data in robot manipulation\n- Performs thorough empirical analysis of their hypotheses (ex. different frames lead to different performances; a fusion of point clouds expressed in different frames will lead to better performance)\n\nWeaknesses:\n- The paper lacks a detailed analysis of why they have chosen a particular manipulation task to validate their hypotheses. Currently, it sounds like they have been selected just because they were available.\n- A more thorough analysis of empirical results is needed.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I think this is a very helpful paper in general, but it could use additional analyses on the empirical results. More concretely, why did you choose the five tasks? Was it because they were available? It seems that you have chosen them based on their merits - ex. some require movement of the base, some don't; some require moving a part of an object and some don't, etc. It would be beneficial for the reader if you could provide why these particular tasks have been chosen so that they can clearly understand what you are trying to test in each task.\n\nSecond, I think you need a more elaborated and thorough description of your insights and experimental results. For instance, I found lines 170-175, which outline your insights as to why certain frames would help to learn, to be very interesting and helpful, and would have liked to see more validation of these insights than the ones you provided in lines 201-202. Also, I would like to know why Table 1 and Figure 9 are the way they are. Is it just the fusion of the point cloud expressed in different frames that result in this? Or was there something else? If it is the former, why is that so? If it was the latter, what were the other factors?\n\nLastly, for frames such as target-object-part frame, it is important where on the object you choose your frame (ex. you chose the handle). So, it seems that human judgment is not only needed in choosing the type of frame but also the where on the object. The paper needs to be more upfront about this.", "technical_quality": "Good: Generally convincing, but some additional experiments or analyses needed.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "issues": "Please see the summary of the recommendations above.", "reviewer_expertise": "4: The reviewer is confident but not absolutely certain that the evaluation is correct"}, "extra_text": {}}, {"review_id": "5mF5swT33qp", "reviewer": "Reviewer_aJn6", "summary": "The paper is motivated by the fact that different coordinate frames have a significant impact for point cloud-based reinforcement learning for manipulation tasks. In a set of manipualtion tasks involving mobile and dual-arm manipulators, the paper compares with four coordinate frames: World frame, robot-base frame, end-effector frame and target-part frame and show that not a single frame can consistenly outperform the reset. The paper then proposes multiple ways of combining different coordinate frames by incoporating point clouds in different frames into the input and use the RL loss to find a combination of different frames. The proposed frame combination method achieves significant performance gain over single-frame baselines.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n1. The paper is well motivated and can bring the community's attention to the importance of choosing the right coordinate frames, which has been so far less explored.\n2. The paper is well structured and easy to follow. The tables, figures, their captions as well as the video in the supplement are very informative.\n3. The arguments in the paper are well supported by the quantitative and qualitative resutls.\n4. Finally, the proposed method is reasonable and well motivated. The supporting experiments in Figure 8 show interesting insights into how the method works.\n\n### Weakness\nThe weaknesses I listed below are all minor and serve mostly as suggestions to improve on the paper.\n1. The real world experiment is underwhelming. \n* Due to the sample complexity of reinforcement learning, there is no training in the real world and only the trained policy is transferred. In this case, the sample complexity in the simulation becomes less of a concern. As such, I wonder if with more training samples, do different choices of coordinate frames yield similar asymptotic performance?\n* Only the pick object is done in the real world. For this case, the end-effector frame is already achieving the best performance. It would be very helpful to demonstrate one more task in the real world where frameminer outperforms single-frame baselines.\n* From the visualization in the paper, it seems that the full point cloud is used in simulation. How does this transfer to the real world?\n2. In the current method, actions from different coordiante frames are averaged. This seems to be a strange choice to me. That means all coordinate frames always contribute to the final actions. Why not sample action from one of the frames based on the weight?\n3. Only RL setting is demonstrated in this case. What about imitation learning? It seems that the same principle should apply as well. This could further enlarge the impact of this work. A related work in this area is Bowen et al. 2022 [1]. \n\n[1] Wen, Bowen, et al. \"You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration.\" RSS 2022.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "Overall, the paper presents interesting finds with sufficient supporting experiments that deserve publication at the conference. I believe these findings will be beneficial for other researchers in the community. ", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Very Good:  Well organized and clear.", "potential_impact": "", "robotics_focus": "Highly relevant to robotics but no hardware experiments", "quality_of_limitations": "Limitations are addressed clearly", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "Please see my weakness in the above sections. \n\nFor the robotics focus, I chooose \"no hardware experiments\" as the current hardware experiments are not sufficient to demonstrate the core idea of the proposed method. \n\n"}}, {"review_id": "4thrRGWRFVU", "reviewer": "Reviewer_vXBE", "summary": "The paper identifies the importance of using particular coordinate reference frames for reinforcement learning with point-cloud inputs, demonstrates this experimentally, and proposes a novel technique to learn policies from a fusion of reference frames. Commonly used frames are investigated: world, robot base, end-effector and object frames (world and robot are meaningful to look at separately since a mobile robot is considered). Three different \"frame mining\" techniques are proposed (i.e. using information from different frames to select the action to execute), though it is shown in the experiments that while they do not differ significantly amongst themselves, they all provide improvements over using a single frame for the entire task. Experiments are performed on a variety of simulated platforms including a single fixed arm, a bimanual mobile robot, and a demonstration of the method on a real robot.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is very clearly written and in a more colloquial manner without sacrificing technical detail which I appreciate. I think the paper conclusions are intuitive and insightful. I believe the claims being made regarding the benefits of particular frames benefitting different tasks is known in robotics, but I have not seen a rigorous demonstration of this. This to me is the main strength of the paper, it is validating knowledge that is informally accepted in robotics in a systematic way.\n\nThe paper goes a step further though, and additionally proposes a novel method to fuse useful information from different frames to generate actions. It's very interesting (surprising, even) that a weighted combination of the proposed actions from each module in the network ends up producing a meaningful action that improves over using any one frame individually.\n\nThe other two FrameMiner methods beyond FM-MA are nice to show, but there is one more I think would be interesting to look at: take only the max-weighted action from the FM-MA architecture, i.e. don't do a linear combination but take the one with the highest weight. I suspect there may be diminishing contributions and that one frame ends up dominating in certain phases of different tasks, or that frames contribute equally enough that it won't matter which one is selected. I would gain more insight about the importance of the linear combination in FM-MA seeing this comparison.\n\nThe experiments are thorough and I appreciate the variety of tasks and platforms (mobile, bimanual, single-arm).\n\nThe limitations section is underwhelming. A big limitation that received no discussion is only PPO is looked at for behavior generation. It would be informative to look at other RL methods, but beyond RL, I think it would be much more informative to look also at imitation learning and model-based planning. As it stands the paper can primarily support the benefit of frame mining for RL, whether those benefits extend equally into other domains of policy learning and planning is unclear.\n\nA final factor that I think is overlooked is the camera views. It is emphasized that acquiring new camera views is not necessary for the method and it only takes in a fused point cloud - however, the camera views still matter. For example, what if the camera is mounted on the robot's mobile base, or on the end-effector? Then the perception is situated to one of the frames identified as relevant for action. Does that change things w.r.t. to the conclusions the paper makes? I think discussing the aspect of a moving camera, if not showing additional experiments on it, is a worthwhile point to elaborate on.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "I think this is a great paper, it had me excited while I was reading it, and I will advocate strongly for acceptance even if other reviewers feel differently. The results are simple, intuitive, and thorough. I believe the conclusions will have a strong impact on the manipulation learning community, since the reference frame one learns in is often an overlooked matter. If one can apply a simple method to fuse the best information from many reference frames without having to change the experimental setup at all (i.e. the cameras can stay wherever you want them), then it's an all around win. I do think there are more limitations to the method than discussed, which I describe in more detail in the strengths/weaknesses section. These are matters that can likely be addressed though in the rebuttal and in minor paper revisions.", "technical_quality": "Very Good:  Convincing.", "clarity_of_presentation": "Excellent: Enjoyable to read.", "potential_impact": "", "robotics_focus": "Sufficient demonstration on hardware", "quality_of_limitations": "Additional details required", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"impact": "4: The work contains interesting new ideas that will potentially have major impact in robotics or machine learning.", "reviewer_expertise": "5: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature"}, "extra_text": {"issues": "Please see my comments in the strengths/weaknesses section. I would particularly appreciate the authors' input on the alternative to FM-MA I proposed (select max-weighted action), and the matter of a possibly moving camera that may be situated in an action-relevant frame (e.g. on the mobile base or on the end-effector, both common camera placements).\n\nThere are some smaller issues I identify here to improve the paper:\n    - Figure 1, the 4th image caption should be \"PushChair\".\n    - Pg. 3, last paragraph, second sentence, \"... to extract visual features...\" (pluralize features)\n    - Figure 3, it would be easier to see with a notably different color for the door, yellow and orange are too similar.\n    - Figure 8, the axes should be the same so that we can compare the plots on the same scale."}}]

Author rebuttal:

Decision: Accept (poster)