Visual Whole-Body Control for Legged Loco-Manipulation

Authors: minghuan  liu
zixuan  chen
xuxin  cheng
yandong  ji
rizhao  qiu
ruihan  yang
xiaolong  wang
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-14 08:33:35

Abstract: We study the problem of mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by conducting whole-body control. That is, the robot can control the legs and the arm at the same time to extend its workspace. We propose a framework that can conduct the whole-body control autonomously with visual observations. Our approach, namely Visual Whole-Body Control (VBC), is composed of a low-level policy using all degrees of freedom to track the body velocities along with the end-effector position, and a high-level policy proposing the velocities and end-effector position based on visual inputs. We train both levels of policies in simulation and perform Sim2Real transfer for real robot deployment. We perform extensive experiments and show significant improvements over baselines in picking up diverse objects in different configurations (heights, locations, orientations) and environments.

Main paper: 
 introduction: 
 The study of mobile manipulation has achieved large advancements with the progress of better manipulation controllers and navigation systems. While installing wheels for a manipulator can help solve most household tasks [1, 2] , it is very challenging to adopt these robots outdoors with challenging terrains. Imagine going camping, having a robot picking up trash and wood for us would be very helpful. To achieve such flexibility and applications in the wild, we study mobile manipulation with legged robots equipped with an arm, i.e., legged loco-manipulation. Specifically, we are interested in how the robot can conduct tasks using visual observation autonomously.An important aspect of loco-manipulation is the opportunity to use all degrees of freedom (e.g., 19 DoF in this paper) of the robot for whole-body control. While the legs are typically utilized for mobility, there is a large potential for coordinating legs and arms together to amplify manipulation capabilities and workspace. We humans do this, too: we stand up to reach books placed on a shelf; we bend to tie our shoes. Fig. 1 shows tasks our robot can do by such a joint control. For example, our robot can go to the side of the road and pick up the trash on the grass. This will be unachievable without coordinating the legs of the robot, as the arm alone can hardly reach the ground.While appealing, it is a very challenging control problem. The robot will need to exploit the contact with its surroundings and the objects and maintain stability and robustness to external disturbances all at the same time. Besides, acquiring a mobile platform to do precise manipulation jobs requires a steady robot body, which is much harder for legged robots compared to wheeled ones. An even larger challenge comes from achieving all these in diverse environments autonomously, given only the observation from an egocentric camera. Recent visual-learning-based approaches have shown promising results of locomotion on legged robots, like avoiding obstacles, climbing stairs, and jumping over stages [3, 4, 5, 6] . However, loco-manipulation requires more precise control, and the extra complexity can make direct end-to-end learning infeasible.In this paper, we introduce a two-level framework for loco-manipulation, where a high-level policy proposes end-effector pose and robot body velocity commands based on visual observations and a low-level policy tracks these commands. Such a low-level controller is general for different tasks and the high-level policy is task-relevant. We named our framework Visual Whole-Body Control (VBC). Specifically, VBC contains three stages of training: first, we train a universal low-level policy using reinforcement learning (RL) to track any given goals and achieve whole-body behaviors; then, we train a privileged teacher policy by RL to provide immediate goals and guide the low-level policy to accomplish a task (e.g., pick-up); finally, we distill the teacher policy into a depth-imagebased visuomotor student policy via online imitation learning. All training is done in simulation, and we perform Sim2Real transfer to deploy policies on real robots with zero human-collected data.Our hardware platform is built on a Unitree B1 quadruped robot equipped with a Unitree Z1 robotic arm. We validate the effectiveness of each designed module of VBC in simulation on various objects and heights. As for the real-world experiments, we conducted the pickup task with 14 different objects, including regular shapes, daily uses, and irregular objects, under three height configurations, i.e., on the ground, on a box, and on a table, achieving a high success rate with emergent retrying behaviors and generalization ability in all scenarios. 
related work: 
 A set of works has exploited legged loco-manipulation to take advantage of legged robots. Bellicoso et al. [7] designs an online motion planning framework, together with a whole-body controller based on a hierarchical optimization algorithm to achieve loco-manipulation. Fu et al. [8] proposed an effective method to train a whole-body controller end-to-end through RL. Ma et al. [9] combined model-based manipulator control and learning-based locomotion, but only showed the response to external push instead of achieving manipulation tasks. Notably, these works all require human involvement and teleoperation to execute and achieve tasks, rather than operating autonomously. Zhang et al. [10] , Yokoyama et al. [11] , Arcari et al. [12] all performed an autonomous mobile manipulation system on quadruped robots while they utilized the default model-based controller provided through the robot APIs (only control robot locomotion in a static height) and only focused on the manipulation part without any whole-body behaviors to adapt to different heights of objects. Ferrolho et al. [13] proposed a trajectory optimization framework based on a robustness metric for solving complex loco-manipulation tasks, but the task is assigned and the target positions are calculated via a motion capture system, thus only works in limit. Sleiman et al. [14] realized a bilevel search strategy for motion planning, by incorporating domain-specific rules, combing trajectory optimization, informed graph search, and sampling-based planning, enabling robots to perform complex tasks. They require the user to provide a dense description of the scene to initialize the solver. Zimmermann et al. [15] also utilized trajectory optimization to enable a legged robot to walk and pick up objects. However, they only show limited surroundings and simple objects. Besides these works with arm-based configuration, Wolfslag et al. [16] use the legs for pushing and box lifting tasks by introducing additional support legs named prongs to enhance the robustness and manipulation capabilities of quadrupedal robots. Jeon et al. [17] adopted a similar hierarchical framework to us, but does not involve any visual inputs, and only learned a quadruped for pushing large objects without an additional arm. The system we design highly differs from the aforementioned works, as our work is a autonomous and vision-based locomanipulation system on quadruped robots, with very limited human annotation before start. The robot is controlled and planned by end-to-end policies integrated with perception and control, able to adapt to objects with varying heights through its wholebody behavior, and emerges retrying behaviors when fails. Due to our design of visual inputs, our system can work both indoors and outdoors without external limits, as illustrated in Fig. 1 , compared with almost all of the above-mentioned works that are only tested in limited scenarios. Fig. 2 shows why VBC is highly advantaged of static-height methods [10, 11, 12] , for the body behavior allowing much more flexible poses (e.g., bending) to reach objects in random heights, like the one on the ground. 
visual whole-body control: 
 The proposed VBC framework (Fig. 3 ) consists of a low-level goal-reaching policy and a high-level task-planning policy. The low-level policy tracks a root velocity command and a target endeffector pose. The high-level policy provides these commands to the low-level policy, based on segmented depth images and proprioception states. When combined, our robot can achieve locomanipulation tasks autonomously. This section draws details of the proposed framework.The low-level control policy π low that takes over the whole-body control as described in the blue part in Fig. 3 , is trained to track any given end-effector pose and body velocity across various terrains. We train such a policy with RL to realize a universal and robust behavior that can become the foundation of any high-level tasks.Commands. The command b t for our low-level policy is defined as b t = [p cmd , o cmd , v cmd lin , ω cmd yaw ] where p cmd ∈ R 3 , o cmd ∈ R 3 are end-effector position and orientation1 command, defined under a height-invariant robot frame (details in Appendix C.1); v cmd lin ∈ R, ω cmd yaw ∈ R are the desired forward linear (x-axis) and yaw velocity under robot base frame, respectively. During the low-level policy training, we uniformly sample the linear velocity from the range [-0.6m/s, 0.6m/s] and robot yaw velocity from [-1.0rad/s, 1.0rad/s]. When solving downstream tasks, the commands are specified by the high-level task-planning policy and can be further clipped into smaller ranges as needed.Observations. The observation of our policy is formalized as a 90- and foot contact patterns), a t-1 ∈ R 12 is the last output of policy network, z t ∈ R 20 is environment extrinsic vector, which encodes environment physics [8] . Following the work of Agarwal et al. [4] , we aim to learn a steady walking behavior, such as trotting. To achieve this, we incorporate additional timing reference variables, denoted as t t , into the observation. These variables are computed from the offset timings of each foot by repeating the gait cycle of the desired symmetric quadrupedal contact pattern.Actions. Our low-level goal-reaching policy mainly controls the quadruped robot. For the quadruped robot, our low-level policy outputs the target joint angles for all 12 leg joints, which is converted to torque command by a PD controller.Arm control. Previous work [8] has revealed that whole-body RL policy struggles in controlling accurate end-effector position and orientation by controlling arm joint positions, which is critical for our manipulation tasks. Hence, we predict the end-effector pose command and convert it into target joint angles by solving the inverse kinematic (IK) with the pseudoinverse Jacobian method 2 . Formally, IK computes the increment of the current joint position ∆θ ∈ R 6 : ∆θ = J T JJ T -1 e, where J is the Jacobian matrix, and e ∈ R 6 is the difference between the current end-effector pose and the next end-effector pose represented as the position and the Euler angle.End-effector goal poses sampling. To train a robust low-level controller, we sample and track various gripper poses. To make sure we can sample smooth trajectories of gripper target poses for tracking, we employ a height-invariant coordinate system originating at the robot's base and randomly sample goals in this frame at every fixed timestep. This helps to learn a whole-body behavior and enables an expanded workspace. Details can be referred to Fig. 9 in Appendix C.1.Due to the page limit, we leave more details of low-level policy training (rewards, policy architecture, goal sampling, randomization, etc.) in Appendix C.1.When deploying our policy into the real world, the robot only perceives the object through depth images to achieve high-frequency control. However, RL with visual observation is quite challenging and requires particular techniques to be efficient [18, 19, 20, 21, 22] . To achieve simple and stable training, we train a privileged state-based policy that can access the shape and pose information of the object (the orange part of Fig. 3 ) and distill a visuomotor student (the green part of Fig. 3 ).Like the low-level policy, the privileged teacher policy is trained through RL using task-specific reward functions and domain randomization, with a fixed pre-trained low-level policy underlying. Privileged observations. The privileged observations include the latent shape features and object poses. We formally define it as a 1094-dim vector:o t = [z shape , s obj t , s proprio t , v base vel t , a t-1 ], where z shape ∈ R 1024 is the latent shape feature vector encoded from the object point clouds using a pretrained PointNet++ [23] , which is fixed and z shape remains invariant during training. s obj t ∈ R 6 is the object pose in local observation w.r.t the robot arm base. It contains the local pose of the object. The use of local coordination inherently incorporates the information regarding the distance to the object. s proprio t ∈ R 53 consists of joint positions including gripper joint position q t ∈ R 19 , joint velocity not including gripper joint velocity qt ∈ R 18 , and end-effector position and orientation s ee t ∈ R 6 . v base vel t ∈ R 3 is the base velocity of robot and a t-1 ∈ R 9 is the last high-level action. High-level Actions. The high-level policy π high outputs actions that determine the velocities and the target end-effector position command for the low-level policy: a t = [p cmd t , v cmd t , p gripper t ] ∈ RWhen training our high-level policy, we randomized the friction between the robot and the terrain, the mass, and the center of mass of the robot. Although the low-level policy already handles some of these gaps, these randomizations help the high-level policy adjust well to different low-level behaviors. Moreover, due to the uncertainty of the high-low signal transmission in the real world, we also randomize the high-level frequency by involving random low-level policy calls during one high-level call. We also randomize the Kp and Kd parameters for the arm controller. Regarding the pick-up task, we randomize table heights, the initial position and pose of the robot, along with the position and pose of objects.We find robust and adaptive picking-up behaviors emerge from the privileged teacher policy training, with inaccessible object poses and shape features in reality. To deploy our policy in the real world, we must use easily accessible real-world observations like images. Observations and actions. Although camera images are easily accessible both in simulation and the real world, there is a huge gap for colored images between them. As the main difference and noise come from the surrounding environment in simulated and real images, we turn to segmented depth images to train the visuomotor policy. Formally, the observation of our vision policy iso t = [o image t , s proprio t , a t-1 ],where o image t consists of object segmentation masks and segmented depth images. s proprio and a t-1 ] are proprioceptive observation and last action defined above. The output actions are the same as those of the teacher policy. Imitation learning for student distillation. To deploy our policy in the real world, we distill our state-based high-level planning policy into a vision-based policy using DAgger, which warms up the training by teacher sampling the initial transition data. Then we sample transitions using the student policy and request correction from the teacher. We find this is much more efficient and effective than directly learning a visuomotor student policy using RL. Randomization and augmentation. To bridge the sim-and-real gap, we slightly randomize the position and rotation of both cameras to handle slight offsets in real. Besides, we clip the depth image to a minimum of 0.2m to avoid low-quality images when the distance is too close to a real camera, after which we normalize the depth value. We adopt augmentations like RandomErasing, Gaussian-Blur, GaussianNoise, and RandomRotation for the depth images. Additional details regarding the high-level policy training including rewards can be found in Appendix C.2.Robot Platform The robot platform used in this work comprises a Unitree B1 quadruped robot with a Unitree Z1 robot arm mounted on its back. Our robot equips two cameras, one on the head and another close to the gripper. We use the onboard computer of B1 to control the low-level policy, and an additional onboard computer to receive visual inputs and control the high-level policy. The Unitree B1 quadruped has 12 actuatable DoFs, the arm has 6 DoFs, and the gripper has 1 DoF, making 19 DoFs in total. An overview of the hardware setup is shown in Fig. 4 .Vision masks. Note that our system works with little human intervention, i.e., we need to annotate and segment objects from environment backgrounds. To this end, we take use of TrackingSAM [24] (details in Appendix J), a tool that combines a video object tracking model [25] with interactive segmentation interface [26] . The implementation details and an example for illustration are given in Sec. J and Fig. 14 . During real-world experiments, we annotated the objects from one of the RGBD cameras at the beginning of each reset trial. TrackingSAM will keep tracking the segmented object and providing the mask in real time (∼10 fps on the robot). The depth images obtained from the RGBD cameras are pre-processed by hole-filling and clipping. Empirically, as long as the object is visible to one camera (either the head or the wrist camera), the annotation mask can be propagated to both views to initialize an automated pickup. 
experiments: 
 Real World Simulation We conduct a set of experiments on pick-up tasks to compare VBC with baselines both in simulation and the real world, showing the effectiveness of training a loco-manipulation policy with VBC, and the generalizability over varying objects with different heights. Experiment setups. We utilize Isaac-Gym [27] for parallel training. In the real-world experiment, we initialize objects in a random location and orientation in front of the robot, making sure it is visible in at least one camera (around 0.5∼1.5m). We test three heights by putting the objects on the ground, on a box, and on a table, corresponding to the heights of 0.0m, ∼0.3m, ∼0.5m. Objects. We use 33 YCB objects [28] The dots/crosses represent the mean performance, and the upper/lower bound is the max/min success rate over 7 categories, with the category name noted. The non-hierarchical baseline is not included for not reaching meaningful results.simulation, we consider a successful grasp if the object is picked up and a failure when the object falls down the table or is not successful in 150 high-level steps (∼ 18 -24s). We compare both the privileged teacher policies of VBC against baselines and their visuomotor student policies to provide an analysis of each module of VBC: 1) VBC w.o. shape feature: a visuomotor policy that is trained the same as VBC, but does not access the pre-trained object shape features during the teacher training stage. 2) Floating base: a floating base policy with a perfect low-level navigation ability but without arm-leg coordination; in other words, the robot can always follow the given velocity commands and body height commands ranging from 0.4 to 0.55 meters (we set 0.4 meters as the lower bound as the default controller in the real-world only allows 0.47 meters in minimum). We compare both the teacher and its corresponding student policy using the same policy network structure. 3) Non-hierarchical: a uni- (c) Objects on a table (∼0.5m) Figure 7 : Average success times w.r.t. the attempts of picking up 14 objects at different heights, tested in the real world. We allow 5 continuing attempts (including 4 continuous retries, x-axis) during one configuration, i.e., one reset of picking up one specific object at a specific height. A failure is recorded if the robot attempts more than 5 attempts in one reset or if the object falls from its surface. We test 5 times for each object and each height, and calculate the average success time (y-axis) across these trials. The emergent retrying behavior improves the performance.fied policy trained with low-level policy and visuomotor high-level policy jointly in an end-to-end style. This policy takes the observation of our low-level and high-level policies together and outputs all the target positions of 12 robot joint angles and the target pose of the gripper. We failed in training such a policy, indicating the effectiveness and necessity of VBC's training pipeline. Picking up different objects. We test the picking-up success rate on each distinct object for more than 300 episodes and collect the results over every category. As is shown in Tab. 1, VBC achieves the best performance on 4/7 categories of objects. It is also interesting that with 3D features, VBC becomes worse than VBC w.o. Shape Feature at regular objects, but is much better on irregular objects with complicated shapes, i.e., long box, cup, bowl, and drill. One possible reason is that, without pre-trained features, the policy targets the object's center for grasping, which is effective enough on simpler objects yet inapplicable for objects with more complicated shapes. One may notice that the student policy is much worse than the teacher policy, since when the robot touches but fails to grasp on the first try, the object may be randomly laid so that its pose will be much harder to grasp within only 150 steps using only visual inputs. In addition, the performance of the student policy of VBC behaves the worst on long boxes. This is because without shape features, it is much harder to determine the pose of the object, and some initial poses of long boxes are extremely hard to grasp for the gripper we used at some height (also as shown in Fig. 6 ). Picking up over various heights. We test the mean success rate on each object category on a fixed height, where we also list the object with the maximum/minimum success rate on each height. We compare VBC and the floating base baseline on 7 different heights, and illustrate the results in Fig. 6 . From the significant improvement of VBC on almost every height, compared to the floating base policy, we highlight the advantage of VBC in achieving flexible whole-body behaviors.Evaluation principles and baselines. We deploy the trained visuomotor student policy of VBC directly into the real world. Particularly, we test how many times the robot can pick up the assigned object in five attempts without resetting (i.e., the robot is allowed to continuously retry four attempts when it fails to grasp), since we find that the emergent retrying behaviors improve the robustness, reliability, and success rate of finishing a job. We consider it a success when the object is picked more than 0.1m higher than the placed plane and a failure when the robot tries more than five attempts or the object falls from its surface. Each object is evaluated over three different heights and we do five resets for each configuration. Regarding baselines, as it is hard to build a fair autonomous method for comparison, we compared VBC with teleoperating the default Unitree base controller (allowing the heights of the arm base to vary from 0.47m to 0.55m) combined with a similar highlevel policy trained on stationary legs, in which the robot has no arm-leg coordination but also retains the retrying. For this baseline, We teleoperated the robot to be close enough to the object before grasping it. During all tests, we put objects at a distance that one mounted camera can observe, thus they can be easily manually annotated using the TrackingSAM tool. After the annotation, the robot walks towards and picks up the objects in a fully autonomous way. In experiments, we observe that the policy is capable of recovering from short-term tracking loss, which is not uncommon due to the robot turning or tilting.Emergent retrying behaviors. The robot emerges to learn a retrying behavior in simulation training. In other words, when the robot fails to grasp an object, it automatically retries to grasp without human intervention. This shows an advantage and a clear difference compared to model-based methods. In our observation, as long as the visual mask is on-track/not-lost, the success rates benefit from 1) the change of the object pose in the early tries; and 2) the movement of the robot body. Pickup performance. We collect all results and conclude the averaged performance over all objects on each height setting and show in Fig. 7 . It is worth noting that our method allows for autonomous retrying behaviors when one pick-up trial fails. Obviously, VBC surpasses the baseline method on all settings. To highlight, the baseline methods fail at 0.0m and 0.3m, as the default controller keeps a fixed robot height, similar to the floating base baseline. Even on the 0.5m setting, where the teleoperated baseline without whole-body behavior is equivalent to a static robotics arm, VBC is generally better. Besides, VBC shows great generalization ability on unseen shapes, thanks to the training strategy and observation design in the training pipeline. Qualititive analysis. We analyze whether the commands sent by the high-level policy are reasonable to call the low-level policy to achieve the task. To this end, we visualize the velocity commands, the gripper command, along with their corresponding robot behavior during the testing time in Fig. 8 .Our system has small gaps when deploying our policy trained in the simulator into the real world, which is able to provide the correct commands that show clear phases. 
conclusions and limitations: 
 We proposed Visual Whole-Body Control (VBC), a fully autonomous loco-manipulation system on quadruped robots with a hierarchical training pipeline. VBC trains a low-level goal-reaching policy and a high-level task-planning policy through reinforcement learning and imitation learning.Limitations. Although we showed great success in picking various objects, there are several limitations regarding the system and the tasks that can be of interest in future work. A detailed analysis is in Appendix I and we outline some reasons: (1) visual tracking loss; (2) inadequate gripper design;(3) in-precise depth estimation; (4) compounding error. 
 
 Appendix: Legged Locomotion. For decades, legged locomotion has been an essential topic in the field of robotics and has gained much success. Classical methods solve the legged locomotion task through model-based control to define controllers in advance. For example, Miura and Shimoyama [29] , Sreenath et al. [30] utilized traditional methods to enable the walking of biped robots. MIT cheetah 3 [31] , ANYmal-a [32] by ETH, and Atlas [33] by Boston Dynamics are good examples of robots designed with a robust walking controller. Based on these developed robots, researchers have been able to develop advanced controllers for even more agile locomotion tasks, such as jumping or walking over varying obstacles [34, 35, 36, 37, 38, 39] . To make a robust controller that can adapt to varying environments, researchers integrate some sensor data such as elevation maps [40, 41, 42, 43, 44] and state estimators [45, 46, 47, 48, 49] . Although classical methods have greatly succeeded in legged locomotion, generalizability in pre-designed controllers would require careful and tedious engineering efforts. With the fast development of deep neural networks, learning-based end-to-end methods have become a simple way to gain robust controllers with strong generalization abilities.Learning-based methods often train a robust controller in physics simulators such as PyBullet [50] and Isaac Gym [27] and then perform sim-to-real transfer to a real robot. For example, Rudin et al. [51] uses Isaac Gym with GPU to enable a quadruped robot to walk in only several minutes, Kumar et al. [52, 53] proposed the RMA algorithm to make the sim-to-real transfer process easier, and more robust, Lee et [67] combine RL with imitation learning to produce naturalistic and robust controllers for legged robots. Kim et al. [68] map human motions to a quadruped robot to complete a wide range of locomotion and manipulation tasks. Mobile Manipulation. Previously mentioned works mainly focus only on the mobility part; adding manipulation to the mobile base is yet to be well-solved. Autonomous mobile manipulation on wheeled robot platforms has made significant progress over the past few years. Wong et al. [69] collected teleoperated mobile manipulation data and learned imitation-learning-based policies in simulation. Lew et al. [70] built an efficient framework for table cleaning with RL and trajectory optimization. Garrett et al. [71] and Srivastava et al. [72] integrate task-specific planner for various mobile manipulation tasks. Gu et al. [73] and Xia et al. [74] developed modularized mobile manipulators with RL, Yokoyama et al. [75] introduced a high-level controller for coordinating different low-level skills. Sun et al. [76] designed a modularized policy with components for manipulation and navigation, trained by RL to learn navigation and grasping indoors. Yang et al. [77] proposed simultaneously controlling the robot arm and base for harmonic mobile manipulation using RL training with large-scale environments. Ahn et al. [78] pre-trained a set of single mobile or manipulation tasks and relied on large language models to decompose and call the skills to achieve long-horizon tasks. Brohan et al. [79] train a robotics transformer to perform real-world tasks given human instructions, trained on large-scale and real-world demonstration. Shafiullah et al. [80] learned to solve household tasks from 5-minute demonstrations based on pre-trained home representations. While encouraging, most of these mobile manipulators with wheels cannot operate with challenging terrains in the wild.Reinforcement learning (RL) is usually formulated as a γ-discounted infinite horizon Markov decision process (MDP) M = ⟨S, A, T , ρ 0 , r, γ⟩, where S is the state space, A is the action space, T : S × A → S is the environment dynamics function, ρ 0 : S → [0, 1] is the initial state distribution, r(s, a) : S × A → R is the reward function, and γ ∈ [0, 1] is the discount factor. The goal of RL is to train an agent with policy π(a|s) : S → A to maximize the accumulated reward R = t γ t r(s t , a t ). 1 -C cmd foot (θ cmd , t) 1 -exp -∥(f foot ) 2 /σcf ∥ -0.2 Stance Phase Tracking (Velocity) foot C cmd foot (θ cmd , t) 1 -exp -∥(v foot z ) 2 /σcv∥ -0.2Proximal policy optimization (PPO) [81] is one of the popular algorithms that solve RL problems. The basic idea behind PPO is to maximize a surrogate objective that constrains the size of the policy update. In particular, PPO optimizes the following objective:EQUATIONwhere r = π(a|s) π old (a|s) defines the probability ratio of the current policy and the old policy at the last optimization step, A(s, a) is the advantage function, and θ π is the parameter of the policy π.In general, imitation learning (IL) [82, 83] studies the task of learning from expert demonstrations. In this work, instead of learning from offline expert data, we refer to an online IL method, dataset aggregation (DAgger) [82] such that we request an online expert to provide the demonstrated action. Formally, the goal is to train a student policy π minimizing the action distance between the expert policy π E under its encountered states:EQUATIONIn this paper, ℓ is the mean square error in practice.Whole-body control in robotics refers to controlling a mobile manipulator with the function of locomotion and manipulation by one unified framework [9, 8] . In this work, we consider a system that contains a quadruped robot combined with a 6-DoF robot arm, each of which has an underlying PD controller, and we use a learned policy to generate control signals and feed them to the controller given 1) the robot states and 2) commands.In detail, the policy used for controlling the quadruped robot is driven by external commands, such as the base velocity command; and the arm is controlled by a PD controller driven by the SE(3) end-effector position-orientation command and inverse kinematics.Reward functions. Our reward for training the low-level policy for whole-body control mainly consists of four parts: command-following reward, energy reward, alive reward, and phase reward. The command following reward encourages the robot to track and follow the provided commands and explore various whole-body behaviors; the energy reward penalizes energy consumption to enable smooth and reasonable motion; the alive reward encourages the robot not to fail; the phase reward encourages the robot to walk in a steady behavior as mentioned above. The detailed definitions of the low-level reward functions are shown in Tab. 2, where we define ϕ = exp(-∥x∥ 2 0.25 ) for tracking the desired velocity of the robot. It is worth noting that we introduce the swing phase tracking rewards to help the robot learn trotting, and C cmd foot (θ cmd , t) computes the desired contact state of each foot from the phase and timing variable, as described in Margolis and Agrawal [59] . Low-level commands sampling. To train a robust low-level controller for goal-reaching, we should sample and track various goals (i.e., base velocity commands and end-effector commands) as possible. As for the base velocity commands, we uniformly sample the linear velocity from a range of [-0.6m/s, 0.6m/s], and the angular velocity from a range of [1.0rad/s, 1.0rad/s]. Regarding the end-effector, same to Fu et al. [8] , we employ a height-roll-pitch-invariant coordinate system originating at the robot's base, where the sampled goal poses are only affected by the robot's horizontal movement instead of changes in height. In detail, the origin (cyan point in Fig. 9 ) is set on a fixed plane, which is on the same line of Z axis as the arm base. Therefore, the sampled goals are also not affected by the robot's height, and orientation of the roll and pitch axis, but only by the orientation in the yaw axis. Such a design ensures the sampled trajectories are always on one local sphere at any time, guaranteeing the consistency of these samples. Besides, it helps to learn a whole-body behavior to control the legs and the body height to enable the arm to reach an expanded space of goals. If not, the robot may fail to reach a goal out of the workspace of the arm without bending or rising. Therefore, the body and the arm are not decoupled, although we use IK to control the arm. We parameterize the end-effector position command p cmd in spherical coordinate (l, p, y), where l, p, y lie in ranges of [0.4, 0.95], [-1 * π / 2.5, 1 * π / 3], [-1.2, 1.2], respectively. To make a smooth end-effector movement, we set p cmd by interpolation between the current end-effector position p and a randomly sampled end-effector position p end , at intervals of T traj seconds:p cmd t = t T traj p + 1 - t T traj p end , t ∈ [0, T traj ] .p end is resampled in a fixed timestep or if any p cmd t leads to self-collision or collision with the ground. Fig. 9 shows screenshots of the low-level training, where the robot is required to track the given velocity commands and the end-effector pose. The orientation command o cmd is uniformly sampled from SO(3) space. Regularized online adaptation for sim-to-real transfer. We take the training paradigm of Regularized Online Adaptation (ROA) [8, 84, 17] in the RL training, to help decrease the realizability gap when deploying our system into the real world. As illustrated in Fig. 10 , in the first training phase, we train an encoder µ that takes the privileged information e (such as friction and mass...etc.) as inputs, and predicts an environment extrinsic latent vector z µ , which will be used for policy inference. Note that the privileged information is not available in real robot deployment. Thus we will need another way to estimate the latent vector z µ . To this end, we train the adaptation module ϕ in a Table 3 : Details of high-level rewards, where q denotes arm joint velocities; v *x denotes base velocity command; d obj , d ee and d base are the direction from robot body to the object, the direction from end-effector to the object and the direction of the robot body orientation, separately; x obj and x base are the position coordinates in world frame of the object and the robot body. r approach , r progress , and r completion are defined at Section 3. in Tab. 3. Except for r progress , r completion and r completion that is determined by the task stage, we define a set of assistant rewards to help the algorithm converge faster. Among them, r acc limits the arm joints velocities' change rate; r cmd is designed to encourage the robot to slow its velocity when it is close to the object to be picked., r action helps smooth the output of the high-level policy to prevent it from changing quickly; r ee orn and r base orn encourages the arm and the robot body to be aimed at the object; r base approach encourages the robot to be close to the object when the robot is far away from the object.Teacher policy network architecture. Our privileged teacher high-level policy network includes a pre-trained PointNet encoder for modeling object pointclouds, followed by a two-layer MLP that gives the high-level actions.Student policy network architecture. Our vision-based high-level policy network comprises a CNN and a two-layer MLP that outputs the high-level actions. We concatenate the images and other states as observations. We stack 4-step images which are passed through the two-layer CNN network with kernel size being 5 and 3, respectively, and then encoded as a 64-dim latent vector. Then, we concatenate this latent vector with other states as a flattened vector. This vector is then passed to a two-layer MLP with a hidden size of 64. We use ELU as the activation.Simulation We use the GPU-based Isaac Gym simulator [27] . As for the teacher, we train 10240 environments in parallel; regarding the student, we only utilize 240 parallel environments as there is a huge GPU memory cost for rendering visual observations. Experiments are trained on a single GPU (we use GTX4090 and GTX3090). On a GTX4090, The teacher policy takes around 36 hours for training, and the student policy consumes around 48 hours.Additional training techniques. We design several training techniques with our prior knowledge to help the high-level policy to be well-performed and stable, not only in the simulation, but can also be transferred into the real world. We briefly introduce them in the following:1. Action delay. We add a one-step action delay to cover the inference and execution in the real world.2. Command clip curriculum. We gradually clip the linear velocity command during training, as we find that a large velocity helps accelerate the learning convergence but a slow velocity ensures a safe, stable, and smooth behavior in the real world.3. Reward curriculum. The command penalty reward r cmd is added after the robot learns well to pick up objects.4. Randomly changing the object position. During training, we take a small probability (10%) to randomly change the object position and pose. This helps the robot to learn the retrying behaviors when it fails during one trial.5. Forcing stop when closing gripper. We forcibly set the velocity command to 0 when the gripper command is closing. This contributes a lot to a stop-then-pick behavior and thus improves the performance.These training techniques are adopted for both teacher policy and student policy training. When training the student policy, we additionally randomize the camera latency as the policy will never obtain the immediate images at the time it infers, but some pictures that are a few milliseconds before, similar to the real world. We randomize the value by measuring the camera latency of the device we used in our real-world experiments.We summarize and list all domain randomization here during the simulated training. Low-level policy. The randomization of the low-level policy is to train a robot with robust walking skills that adapt to different surroundings, while also knowing how to adjust body poses to reach different end-effector goals.• Terrains, including flat planes and rough ones.• Friction between robots and terrains.• Mass of the quadruped and the arm.• Center of mass of the robot.• Motor strength.High-level teacher policy. The randomization of the high-level part mainly focuses on learning the manipulation task with generalization. The high-level teacher policy follows the same randomization as the low-level policy, along with the following additional terms:• High-level frequency by involving random low-level policy calls during one high-level call. In particular, we randomize 6-8 calls considering the high-level inference time and signal transmission latency, corresponding to 0.12-0.16s (low-level policy run at 50 Hz), i.e., 6.25-8.33Hz.• Kp and Kd parameters for the arm controller.• Table heights, 0-0.5m.• Initial position and pose of the robot.• Position and pose of objects.High-level student policy. The high-level student policy follows the same randomization as the teacher policy, along with the following additional term:• Camera latency.In this work, we control the arm by solving the target joints with inverse kinematics, given the predicted target end-effector 6-DoF poses, rather than directly predicting the arm joints. Such a design is motivated by lessons learned from the previous work, DeepWBC [8] , which struggles to control the accurate end-effector position and orientation. To verify the advantage of our IK-based control predicting end-effector poses compared to predicting the target joints, we conduct a goalreaching experiment, in which we randomly sample end-effector goals as in training, and end the episode once the goal is reached (the distance of end-effector and the goal is less than 2cm) or the episode time is out. We quantitatively evaluate the average error of position and orientation in Tab. 4. The results demonstrated a clearly better precision compared to the target joint prediction solution in Fu et al. [8] , which contributes a lot to training an effective high-level policy to complete complicated manipulation tasks.We compare the trajectories and segmented depth images in simulation and real-world sampled by VBC, shown in Fig. 11 . of more than 800Hz. Since these policies work on different computers, along with the quadruped and the arm should work under different frequencies, we design a client-server multi-processing system for this work, as shown in Fig. 12 . We set up three processes at the low-level policy side: a quadruped robot control process, an arm control process, and a goal receiver process behaving as a client. The high-level policy combines the vision inputs from the depth cameras and the information from the low-level request as the observation works as a server. Every 0.1 seconds, the goal receiver sends a request with the necessary information to the server; the high-level policy makes the latest inference and sends the high-level action back to the robot; at the same time, the quadruped and the arm control processes gather the latest high-level commands to execute, while keep updating the necessary information for the goal receiver process. Component runtime analysis. We record the runtime of all our components running in real-world experiments, shown in Tab. 5. Before an autonomous working pipeline, users are required to click on images to annotate the object to be grasped with SAM, which is further processed by AOT to initialize the first reference frame to track. After initialization, the high-level process reads in a new frame from two cameras, and pre-process them by resizing, hole-filling, and clipping; the images will be further processed by TrackingSAM to track and provide segmentation masks; the masks and segmented images are concatenated as observations along with the proprioception states sent by the low-level process, and call the high-level model to predict the command. The high-level process finally sends the command back to the low-level process, which calls the low-level model to predict the leg joints. Note that we did not record the time cost of process communication between the highlevel and the low-level, each of which runs on different devices and communicates through HTTP.During training, we gave a slightly larger randomization range than the recorded total time (also see in Appendix D). Overall, the high-level runtime limits the control frequency of the high-level policy to 6-8Hz. We also fix the low-level control frequency as 50Hz, even if the policy inference is rather fast.For the simulation, we included 34 objects adapted from the YCB dataset [28] . According to the object shapes, we roughly divided them into 7 categories: ball, long box, square box, bottle, cup, bowl and drill. In the real-world experiments, we choose 14 objects, including four irregular objects, six common objects, and four regular shapes. Objects are illustrated in Fig. 13 I Failure Analysis and Future WorksAlthough we showed great success in picking various objects, we found several failure cases due to the limitations of the system: 1) Compounding error. Our pipeline is hierarchical and requires each module to be coherent and precise to make the robot work well. 2) In-precise depth estimation. The RGBD camera does not reflect a perfect depth estimation, especially on reflective objects. 3) Bad gripper. The gripper of the Unitree Z1 Arm is a beak-like gripper instead of a parallel one, which tends to push objects away and is hard for accurate manipulation. 4) Unstable vision inputs. The tracking SAM that is used to provide the mask of annotated objects, may lose track due to camera deviation and occlusion, or be confused when the selected objects have a similar color to the other things. This is the most common cause leading to failure in our real-world experiments. In our current pipeline, before the robot autonomously picks up objects, we must manually specify the target object in both cameras mounted on the robot. Thanks to the generalizability of trackingSAM, we find that as long as the object is visible to one camera (either the head or the wrist camera), the annotation mask can be propagated to another view for automated picking, and free the robot from being specified in the between of the range of the two cameras, which improves the usability of our system. Since the requirements of annotation and object visibility are totally decoupled from our sim-to-real framework, in the future we can further improve and replace these modules with automated annotation algorithms and a search/exploration module to further improve the automation of the whole system. Our choice of visual inputs-specifically masks and segmented depth images-deliberately excludes RGB images that provide color information. This limitation restricts the system from performing tasks that rely on color differentiation, such as distinguishing between Fanta and Sprite cans, sorting garbage by category, or pressing a green button. During the training of our high-level policy, we utilized 33 distinct objects. While this approach demonstrates strong generalization capabilities on unseen objects in real-world scenarios, future work could enhance this by introducing randomized object sizes to further improve generalization. Future works should also consider upgrading the hardware, like replacing a better gripper and RGBD cameras that provide better depth estimation; and simplify the working pipeline.TrackingSAM [24] provides a very simple and intuitive way to provide segmentation masks which only requires minimal manual intervention. It invokes SAM [26] to generate an initial segmentation mask of objects before the tracking begins, where the user only needs to make a few clicks (as shown in Fig. 14 ). Given the initial segmentation, we use it to populate two instances of trackers for images from the wrist camera and the head camera, respectively. The tracker used is AOT (Associating Objects with Transformers) [25] , using the most lightweight AOT-Tiny variant, which is implemented based on MobileNet and can achieve nearly 10 fps on the onboard computer. 8th Conference on Robot Learning (CoRL 2024), Munich, Germany. * Equal Contribution.We use Euler angle parameterization by default in this paper.Other advanced IK solvers can be exploited for better performance under certain circumstances, but the pseudoinverse Jacobian method works well enough for the problem studied in this paper. 


Decision: The submission presents a new sim2real method for controlling locomotion and manipulation in parallel, leading to strong results and a detailed evaluation. 

The submission effectively breaks down loco-manipulation problems into sub-problems, uses visual observations (segmented depth images), demonstrates successful deployment of the trained policy on a physical robot without requiring real-world data, robust retrying grasping attempts when initial attempts fail, enhancing overall success rates, is rigorously tested through broad simulation experiments and real-world tests on unseen objects.

While effectively addressing a pick-up task, applicability to more intricate manipulation tasks remains unexplored. The reliance on a beak-like gripper restricts the range of objects that can be manipulated, necessitating pushing instead of grasping. Performance heavily relies on accurate depth estimation. More details are needed about the model used for object annotation and segmentation mask generation, computation time of each module. The paper's readability is limited by long paragraphs.

The paper has improved during the rebuttal via detailed communication between reviewers and authors. Based on the above the paper is recommended for acceptance. Please follow up on the remaining open points for a potential camera ready version.

Reviews: review: title: title: This paper proposes Visual Whole-Body Control (VBC) that does whole body control with visual observations. VBC utilizes three phases of training: 1) training of a universal low-level policy using RL to track goals and demonstrate whole whole-body behaviors, 2) training a privileged teacher policy using RL to complete a task, and 3) distilling teacher policy in to depth-image based student policy via online imitation learning. All training has been completed simulation and zero-shot deployment on real word with zero human-collected data.  Extensive simulations demonstrate its effectiveness in picking up diverse objects across various heights. Real-world experiments showcase successful pick-up tasks on unseen objects with emergent retrying behaviors.
paper_summary: paper_summary: The authors propose VBC, a three-phase cycle that maximally utilizes sim-to-real to zero-shot transfer policies to hardware. The main idea revolves around learning a generalized low-level controller that takes high-level goals, and a high-level controller that provides goals to complete a task. The novelty of the approach is twofold: 1) distilling the high-level controller to a visual observation, emerging recovery behaviors, and improving results compared to a static height, and 2) training all tasks in sim and zero-shot transferring them to reality. The methodology does not constrain itself with real-world constraints to define tasks and collect data, therefore, theoretically scaling for many tasks. The downside of the methodology is sim training and hardware deployment, as the objects of interest need to be in the view of the robot. There are no ablations if that assumption is relaxed and the methodology would work. The authors have addressed these limitations. Though Figure 1 shows the methodology worked on different environments, it is not experimentally validated.
main_review: main_review: The paper is well-structured and easy to follow. The authors have included a thorough appendix with additional details and a comprehensive review of related work.

Pros:

+  VBC tackles the complex loco-manipulation problem by dividing it into manageable sub-problems, allowing for effective training and generalization.
+ The focus on using visual observations (segmented depth images) makes the system more autonomous and adaptable to real-world environments.
+ Successful deployment of the trained policy on a real robot with zero real-world data collection signifies the effectiveness of the proposed training pipeline.
+ The system demonstrates an emergent capability to retry grasping attempts when initial attempts fail, increasing robustness and success rates.
+ Extensive simulation experiments across various object categories and heights, alongside real-world tests on unseen objects, provide strong evidence for the framework's effectiveness.

Cons:

- While the pick-up task is convincingly addressed, the framework's applicability to other, more complex manipulation tasks remains unexplored.
- The reliance on a beak-like gripper limits the range of objects that can be manipulated and necessitates pushing objects instead of grasping.
- The framework's performance is highly dependent on accurate depth estimation, making it sensitive to errors in depth perception.
- The visual tracking system used can be prone to losing track of the object, especially in cluttered environments or with similar-colored objects.
- The hierarchical architecture with asynchronous multi-processing and different control frequencies might lead to computational overhead, potentially impacting real-time performance.

Areas for potential improvements:

* Evaluating VBC on a broader range of manipulation tasks beyond pick-up, such as pushing, placing, and opening, would demonstrate its wider applicability.
* Replacing the beak-like gripper with a parallel one would allow for more versatile grasping and improve manipulation accuracy.
* Investigating and integrating more robust and precise depth estimation methods, potentially combining RGB and depth information, could enhance performance.
* Exploring more robust visual tracking algorithms or incorporating multimodal perception, such as tactile sensing, could address tracking limitations.
* Further optimizing the system's computational efficiency, potentially by streamlining the hierarchical structure, would improve real-time control capabilities.
questions: questions: * Line 110 => rad/s
* Line 110 => “task-planning policy and further clipped into smaller ranges”. What is the reason for the discrepancy in size? To what extent is the size difference? The appendix does not contain the information in question.
* Line 114 => yaw
* Line 117 => While the environment extrinsic vector has been introduced, its application in conjunction with low-level hardware controllers remains unclear. Appendix C.1 fails to provide adequate explanation regarding this aspect. Given the crucial role of the extrinsic vector in sim-to-real transfer, it would be greatly appreciated if you could provide a detailed explanation or direct me to a specific reference that addresses this topic.
* Line 128 => wondering whether the authors have experimented with the damped least squares. Provide a reference, e.g., https://www.cs.cmu.edu/~15464-s13/lectures/lecture6/iksurvey.pdf. 
* Line 165-169: “Moreover,  …”. The sim-to-real transfer strategy mentioned in this document requires further clarification. Kindly provide a more detailed explanation and ensure that any verbose information is moved to an appendix.
* Line 182: Type after a_{t-1}.
* Line 185: In the appendix, there is a reference to DAgger. Kindly provide that reference.


* One of the fundamental presumptions in training and deployment is that the objects of interest must be visible within the viewport. If this assumption is not met, can the methodology still function effectively?
* While the paper employs the term "autonomous," the methodology is technically "semi-autonomous." A human user must supply an augmentation mask and ensure the robot faces the object. To accurately reflect this, consider removing the word "autonomous" or adding the adjective "semi-" before it.
* Fig. 6. Is the labeling of minimum and maximum values correct? The description mentions two baselines, but the results only show one baseline. This inconsistency should be addressed.
* What are the primary factors contributing to the average simulation success rate hovering around 60%? What specific strategies or techniques can be implemented to enhance the simulation success rate beyond the current 60% average?
* Table 1: please provide the standard error. 
* The experimental setup in the real world seems to be confined to a singular environment. Although Fig. 1 depicts activities in various environments, the results do not validate this. Thus, Fig. 1 can be considered misleading, particularly regarding the stringent requirements imposed on TrackingSAM and the intricate nature of task completion. 
* Is the implementation based solely on tabular rasa, or have other libraries been utilized? If other libraries have been incorporated, kindly provide the required attributions for acknowledgment. 
* Appendix B.4: please provide the references.
limitations: limitations: 2
review_summary: review_summary: The paper presents a vision-based control framework for legged loco-manipulation. The hierarchical approach, successful sim-to-real transfer, and emergent retrying behavior are key strengths. VBC has potential for future research and development in whole-body control, but there are limitations in gripper design, depth estimation, and tracking robustness.
title: title: Strong Accept
paper_summary: paper_summary: This paper proposes a bi-level approach for achieving fully-autonomous real world vision-based grasping on a quadruped manipulator platform, trained using RL in real.
main_review: main_review: The work demonstrated transfer of simulation trained picking policies, which can operate fully autonomously in real. While not entirely novel (prior works can also do autonomous pick and place), their system does not assume a Boston Dynamics-level team of engineers to calibrate the model and tune the controller, greatly improving accessibility of quadruped manipulation research to new hardware platforms. Further, their formulation of using RL policies enables (1) more elegant behavior thanks to the lack of separation between a walking and manipulation phase (seen in the supplementary video) and (2) the discovery of robust recovery behavior.

Overall, VBC is an impressive system, with many efforts to handle timing accuracies, latencies, and model mismatches. It includes a thorough experiments and description of evaluation setups in both simulation and real.



Small points:
- "Average Success Times v.s. Trial Times" is a bit confusing in Figure 7. Consider changing it to just "Success Rate v.s. Attempt"?
- There are many typos across the paper. Some examples include "previlleged" (Figure 3), "privleged"  (Table 1).
questions: questions: - 1. The work proposed to train a RL-based high-level policy, and have put efforts into tuning the RL set up for pick up task. Could the authors expand on the scalability of using RL for learning new tasks, versus other high-level policy approaches, such as trajectory planning or behavior cloning?
- 2. Using object masks is a very robust way to transfer to real, since as segmentation and tracking tools become very robust. However, this design choice, while sufficient for object picking, may corrupt information that's crucial for other task. Could the authors expand their limitation section on the applicability of this design choice to other tasks?
- 3. I find it interesting that IK worked well for the author's applications, given that DeepWBC tried and reported much worse numbers for the MPC+IK baseline, where one common failure mode in their case were body arm collisions. Is this sidestepped in the author's approach by a better hardware design (mounting the arm closer to the front), or replacing MPC with RL trained WBC (to avoid collisions in training)?
limitations: limitations: 3
review_summary: review_summary: The paper demonstrates novel fully-autonomous capabilities, contains interesting details across the stack for achieving good manipulation object generalization and Sim2Real, has a thorough evaluation in both simulation and real, and is well written.
title: title: Review of Submission Number 12
paper_summary: paper_summary: This paper tackles the problem of learning a pick-and-place task for a quadrupedal robot with an arm. The task is accomplished with a high-level RL policy trained with a teacher-student method that outputs end-effector commands and body velocities, and a low-level policy that tracks these commands. Training is performed just in simulation.
main_review: main_review: This paper tackles a very relevant problem for a quite interesting application. The problem of whole-body loco-manipulation has been quite extensively explored in the related literature, even for tasks which are more difficult than those shown in this paper, e.g., door opening. However, extending such frameworks with autonomous mobile manipulation capabilities which exploit vision is a very unexplored topic. I also find it impressive that the authors are able to learn policies from simulation without any human demonstrations. In addition, the paper has a very large number of pick-and-place experiments in many different scenarios, which show the generality of the approach with respect to many elements, such as object types and shapes, different terrains, and object heights.
However, I would appreciate if the authors could provide more details about:
- the model which is used to annotate objects and output segmentation masks;
- the computation time of each module.
The readability of the paper can be improved - there are many long paragraphs.
questions: questions: - Section 3.1: "any given end-effector pose and body velocity".
- Section 3.1: It is not clear in which frame v_{lin}^{cmd} and omega_{lin}^{cmd} are expressed. Are they expressed in base frame? Why are they scalar values and not vectors? Please improve the description here.
- Section 3.1 - observations: It is not clear what the variable z_t is. The paper says that it encodes environment physics, but it should be described better and more explicitly.
- Section 3.1 - Paragraph starting with "Besides, following the work of [4], etc...": please improve the readability.
- Do the "encoded shape features" mentioned in 3.2.1 coincide with the latent vector obtained by applying PointNet++ to the objects' pointclouds?
- Paragraph 3.2.1: it seems that the authors do not randomize the shape of the objects to pick up. Why? It seems that this would be an important variable to randomize.
- Section 3.3. Can the authors provide more details about the framework used to obtain segmentation masks in real-time ("TrackingSAM")? Is there a publication regarding this? [24] is a PhD thesis. Also, how reliable is this framework? Does it output segmentation masks which are 100% correct?
- Section 3.3: The authors state that all the computations happen on the robot on-board computers, which is quite impressive. Even though evaluating the networks used for the low-level and high-level policies is probably computationally feasible, I imagine that the model used to compute segmentation masks might be quite large. Can you comment about the computation time of all these modules?
- Section 4.1: a failure is considered if the object is not picked up in "150 high-level steps". What does this corresponds to in seconds?
- The authors state that the objects are manually annotated using trackingSAM. Can the authors specify why is this annotation needed and how is it performed? Does the user need to draw a bounding box around the object?
- Supplementary material: In some videos, the authors show the RGB image, masked depth, and segmentation mask from the two camera views. For the sake of clarity, I would suggest to show only the masked depth and segmentation mask, because the RGB images are not used by the policy.
limitations: limitations: 3
review_summary: review_summary: I gave a high ranking because I think that the authors propose an interesting and novel method with convincing experimental validation.