Visual Whole-Body Control for Legged Loco-Manipulation
Authors: Minghuan Liu
Zixuan Chen
Xuxin Cheng
Yandong Ji
Ri-Zhao Qiu
Ruihan Yang
Xiaolong Wang
Venue: CoRL 2024 (2024)
Keywords: Robot Learning; Reinforcement Learning; Imitation Learning; Mobile Loco-Manipulation

Abstract: We study the problem of mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by conducting whole-body control. That is, the robot can control the legs and the arm at the same time to extend its workspace. We propose a framework that can conduct the whole-body control autonomously with visual observations. Our approach, namely Visual Whole-Body Control (VBC), is composed of a low-level policy using all degrees of freedom to track the body velocities along with the end-effector position, and a high-level policy proposing the velocities and end-effector position based on visual inputs. We train both levels of policies in simulation and perform Sim2Real transfer for real robot deployment. We perform extensive experiments and show significant improvements over baselines in picking up diverse objects in different configurations (heights, locations, orientations) and environments.

Visual Whole-Body Control for Legged Loco-Manipulation
Minghuan Liu", Zixuan Chen", Xuxin Cheng, Yandong Ji
Rizhao Qiu, Ruihan Yang, Xiaolong Wang
UC San Diego
https://wholeboode bl giggguuuhh.... .io/
Wisl
Wib!
Wisl
Wisl
Front
Font
Front
Mosk Mesied
Mosied
Real-Robot input visualization and grasping trajectory. Our framework enables the robot to grasp different objects in varying heights and surroundings. Our policy is trained in simu- lation using visual inputs (masks and masked depth images from two views) and can be deployed in real robots without any real-world data collection or fine-tuning.
Figure 1:
We study mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by coordi- nating arms for whole-body control. We propose framework that can conduct the whole-body control autonomously with visual observations. Our approach. namely Visual Whole-Body Control (VBC), is composed of a low-level control policy using all degrees of freedom track the body velocities along with the end-effector position, and a high-level task-planning policy proposing the veloc- ities and end-effector position based on visual inputs. We train all policies simulation and perform Sim2Real transfer for real robot deployment. Extensive experiments show clear advantages over baselines in picking up diverse objects various configurations (heights, locations, orientations) and environments.
Abstract:
Keywords: Robot Learning, Whole-Body Control, Legged Loco-Manipulation
Introduction
The study of mobile manipulation has achieved large advancements with the progress of better manipulation controllers and navigation systems. While installing wheels for a manipulator can help solve most household tasks [1, 2), it is very challenging to adopt these robots outdoors with challenging terrains. Imagine going camping, having robot picking up trash and wood for us
8th Conference Robot Learning (CoRL. 2024), Munich, Germany. *Equal Contribution.

---

would be very helpful. To achieve such flexibility and applications in the wild, we study mobile manipulation with legged robots equipped with an arm, i.e., legged loco-manipulation. Specifically, we are interested in how the robot can conduct tasks using visual observation autonomously.
An important aspect of aco-mmnnpullation is the opportunity use all degrees of freedom (e.g.. 19 DoF in this paper) of the robot for whole-body control. While the legs are typically utilized for mobility, there is a large potential for coordinating legs and arms together to amplify manipulation capabilities and workspace. We humans do this, too: we stand up to reach books placed on shelf; we bend to tie our shoes. Fig. shows tasks our robot can do by such a joint control. For example, our robot can go to the side of the road and pick up the trash on the grass. This will be unachievable without coordinating the legs of the robot, as the arm alone can hardly reach the ground.
While appealing, it is very challenging control problem. The robot will need to exploit the contact with its surroundings and the objects and maintain stability and robustness to external disturbances all at the same time. Besides, acquiring a mobile platform to do precise manipulation jobs requires a steady robot body, which is much harder for legged robots compared to wheeled ones. An even larger challenge comes from achieving all these in diverse environments autonomously, given only the observation from an egocentric camera. Recent disual-learning -baend approaches have shown promising results of locomotion legged robots, like avoiding obstacles, climbing stairs, and jump- ing over stages [3, 4, 5, 6]. However, duco-manipulation requires more precise control, and the extra complexity can make direct end-to-end learning infeasible.
In this paper, we introduce a two-level framework for loco- troommappulation. where a high-level policy proposes end-effector pose and robot body velocity commands based on visual observations and a low-Ievel policy tracks these commands. Such a low-level controller is general for different tasks and the high-level policy is task-relevant. We named our framework Visual Whole-Body Control (VBC). Specifically, VBC contains three stages of training: first, we train universal low-level pol- icy using reinforcement learning (RL) to track any given goals and achieve whole-bodd body behaviors; then, we train privileged teacher policy by RL to provide immediate goals and guide the low-level policy to accomplish a task (c.g., pick-up); finally, we distill the teacher policy into a based visuomotor student policy via online imitation learning. All training is done in simulation, and we perform Sim2Real transfer to deploy policies on real robots with zero human-collecte data.
Our hardware platform is built on a Unitree B1 quadruped robot equipped with a Unitree 71 robotic arm. We validate the effectiveness of each designed module of VBC in simulation on various objects and heights. As for the real-world experiments, we conducted the pickup task with 14 different objects, including regular shapes, daily uses, and irregular objects, under three height configurations, i.e., on the ground, on a box, and on a table, achieving a high success rate with emergent retrying behaviors and generalization ability in all scenarios.
2 Related Work
A set of works has exploited legged loco- mecemmanipllaiion to take advantage of legged robots. Belli- coso et al. [7] designs an online motion planning framework, together with a whole-body controller based on a hierarchical optimization algorithm to achieve loco-manipulation. Fu et al. [8] proposed an effective method to train a whole-body controller end-to-end through RL. Ma al. [9] combined model-based manipulator control and dearning-basee locomotion, but only showed the response to external push instead of achieving manipulation tasks. Notably, these works all require human involvement and teleoperation to execute and achieve tasks, rather than operating autonomously. Zhang et al. [10], Yokoyama et al. [11]. Arcari et al. [12] all performed an autonomous mobile manipulation system on quadruped robots while they utilized the default model-based controller provided through the robot APIs (only control robot locomotion in a static height) and only fo- cused on the manipulation part without any whole-body behaviors to adapt to different heights of objects. Ferrolho al. [13] proposed a trajectory optimization framework based on a robustness metrie for solving complex loco-manipulation tasks, but the task is assigned and the target positions are calculated via a motion capture system, thus only works in limit. Sleiman et al. [14] realized

---

bilevel search strategy for motion planning, by incorporating domain-specinn rules, combing trajec- tory optimization, informed graph search, and sampling-base baaiingg- planning, enabling robots to perform complex tasks. They require the user to provide a dense description of the scene to initialize the solver. Zimmermann et al. [15] also utilized trajectory optimization to enable a legged robot to walk and pick up objects. However, they only show limited surroundings and simple objects. Be- sides these works with arm-based configuration, Wolfslag al. [16] use the legs for pushing and box lifting tasks by introducing additional support legs named prongs to enhance the robustness and manipulation capabilities of quadrupedal robots. Jeon et al. [17] adopted a similar hierarchical framework to us, but does not involve any visual inputs, and only learned quadruped for pushing large objects without an additional arm.
The system design highly differs from the aforementioned works, as our work is autonomous and vision-based loco- manipulation system quadruped robots, with very limited hu- man annotation before start. The robot is controlled and planned by end-to-end policies integrated with perception and control, able to adapt to objects with varying heights through its whole- body behavior, and emerges retrying behaviors when fails. Due to our design of visual inputs, our system can work both indoors and outdoors without external limits, as illustrated in Fig. 1, compared with almost all of the above-mentioned works that are only tested in limited scenarios. Fig. 2 shows why VBC is highly advantaged of static-height methods [10, 11, 12], for the body behavior allowing much more flexible poses (e.g., bending) to reach objects in random heights, like the one on the ground.
(a) Static- height.
(b) VBC.
Visual Whole-Body Control
Figure 2: Illustration of compar- ing VBC to static-height meth- ods. Static-height methods can- not reach objects on the ground without arm-leg coordination.
The proposed VBC framework (Fig. 3) consists of a low-level goal-reaching policy and high-level task-planning policy. The low-level policy tracks root velocity command and target end- effector pose. The high-level policy provides these commands to the low-level policy, based on segmented depth images and proprioception states. When combined, our robot can achieve loco- manipulation tasks autonomously. This section draws details of the proposed framework.
3.1 Low-Level Policy for Whole-Body Goal- ReachRe
The low-level control policy Row that takes over the whole-body control as described in the blue part in Fig. trained to track any given end-effector pose and body velocity across various terrains. We train such policy with RL to realize universal and robust behavior that can become the foundation of any high-level tasks.
Commands. The command by for our low-level policy is defined as b, = [p o emd gmd where pond R and E R are end-effector position and orientation! command, defined under height-invariant robot frame (details in Appendix C.I); u E R, wond E IR are the desired forward linear (x-axis) and yaw velocity under robot base frame, respectively. During the low-level policy training, we uniformly sample the linear velocity from the range [-0.6m/s, 0.6m/s] and robot yaw velocity from 1-1.Orad/s, 1. 1.Orad/s). When solving downstream tasks, the commands are specified by the high-level task-planning policy and can be further clipped into smaller ranges as needed.
lin yaw
Observations. The observation of our policy is formalized as a 90-dimensional vector o, [shase sa.t st a;-1, Zt, te, bz]. Among them, shase E RR is the current quadruped base state in- cluding row, pitch, and yaw velocities, spt E IR' 12 is arm state (position and velocity of each arm joint except the end-effector), leg e 28 is leg state (joint position and velocity of each leg joint,
leg
We use Euler angle parameterization by default in this paper.
3

---

Privigged Teacher
Porthet
Encoder
Ooject Priee Clnade
Object Pow
MLPs
Latant Bay
MLPs
Soree
Sog neeted
Hal-kosi Cort monde
CNNN
MLPs
Daper Visumater Student
Goal Reeching Polley
Baaky
Figure 3: The propused VBC framework for loco-manipulation. VBC trains a low-level control policy and high-level planning policy using reinforcement learning and imitation learning.
and foot contact patterns), a E 12 is the last output of policy network, 2, IR:20 environment extrinsic vector, which encodes environment physies [8]. Following the work of Agarwal et al. [4], aim learn steady walking behavior, such as trotting. To achieve this, incorporate addi- tional timing reference variables, denoted tt, into the observation. These variables are computed from the offset timings of each foot by repeating the gait cycle of the desired symmetric quadrupedal contact pattern.
Actions. Our low-level goal-reaching policy mainly controls the quadruped robot. For the quadruped robot, our low-level policy outputs the target joint angles for all 12 leg joints, which converted to torque command by PD controller.
Arm control. Previous work [8] has revealed that whole-body RL policy struggles in controlling accurate end-effector position and orientation by controlling arm joint positions, which is critical for our manipulation tasks. Hence, we predict the end-effector pose command and convert into target joint angles by solving the inverse kinematic (IK) with the pseudoinverse Jacobian method?. Formally, IK computes the increment of the current joint position AO E R AO = J (JJz) e, where J is the Jacobian matrix, and e E R is the difference between the current end-effector pose and the next end-effector pose represented as the position and the Euler angle.
End-effector geal poses sampling. To train robust low-level controller, we sample and track various gripper poses. To make sure we can sample smooth trajectories of gripper target poses for tracking, we employ height-invariant coordinate system originating at the robot's base and randomly sample goals in this frame at every fixed timestep. This helps to learn a whole-body behavior and enables an expanded workspace. Details can referred to Fig. 9 in Appendix C.I.
Due to the page limit, leave details of low-level policy training (rewards, policy architec- ture, goal sampling, randomization, etc.) in Appendix C.I.
3.2 High-Level Policy for Task Planning
When deploying our policy into the real world, the robot only perceives the object through depth images to achieve high-freque high-frequency control. However, RL with visual observation is quite challenging and requires particular techniques to be efficient [18, 19, 20, 21, 22]. To achieve simple and stable training. we train privileged state-based policy that can access the shape and pose information of the object (the orange part of Fig. 3) and distill visuomotor student (the green part of Fig. 3).
3.2.1 Privileged Teacher Policy
Like the low-level policy, the privileged teacher policy is trained through RL using task-specific reward functions and domain randomization, with a fixed pre-trained low-level policy underlying. Privileged observations. The privileged observations include the latent shape features and object
poses. We formally define it as a 1094-dim vector: o shape obj proprio hase vel a4-1]; where s
*Other advanced IK solvers can be exploited for better performance under certain circumstances, but the pseudoinverse Jacobian method works well enough for the problem studied in this paper.

---

z chape E IR 1024 is the latent shape feature vector encoded from the object point clouds using pre- trained PointNet++ [23], which is fixed and zhhape remains invariant during training. s, E Re is the object pose in local observation w.r.t the robot arm base. It contains the local pose of the object. The use of local coordination inherently incorporates the information regarding the distance to the object. st si E 1R53 consists of joint positions including gripper joint position qe E R joint velocity not including gripper joint velocity Qe E TR 18 and end-effector position and orientation st E RR vlsse E I is the base velocity of robot and a E RR is the last high-level action.
High-level Actions. The high-level policy nhiigh outputs actions that determine the velocities and the target end-effector position command for the low-level policy:
[p]md vt Pt pripper E 12,
where p. E R is the increment of gripper pose, vimmd E IR is the command of quadruped linear
velocity and yaw velocity and pt o E {0, 1) indicates the gripper status (close or open).
Domain randomization. When training our high-level pol- icy, we randomized the friction between the robot and the terrain, the mass, and the center of mass of the robot. Al- though the low-level policy already handles some of these gaps, these randomizations help the high-level policy adjust well to different low-level behaviors. Moreover, due the uncertainty of the high-low signal transmission in the real world, we also randomize the high-level frequency by in- volving random low-level policy calls during one high-level call. We also randomize the Kp and Kd parameters for the arm controller. Regarding the pick-up task, we randomize table heights, the initial position and pose of the robot, along with the position and pose of objects.
Unitre 21 Am
On Beard
D425
Guathpped
Figure 4: Real robot system setup.
3.2.2 Visuomotor Student Policy
We find robust and adaptive picking-up behaviors emerge from the privileged teacher policy training, with inaccessible object poses and shape features in reality. To deploy our policy in the real world, we must use easily accessible real-world observations like images.
Observations and actions. Although camera images are easily accessible both in simulation and the real world, there is huge gap for colored images between them. As the main difference and noise come from the surrounding environment in simulated and real images, we turn to seg- mented depth images to train the visuomotor policy. Formally, the observation of our vision policy image consists of object segmentation masks and segmented depth images. Sproprio and 1 are proprioceptive observation and last action defined above. The output actions are the same as those of the teacher policy.
is o, = [o imuge paupeio a1. where o,
Imitation learning for student distillation. To deploy our policy in the real world, we distill our state-based high-level planning policy into vision-based policy using DAgger, which warms up the training by teacher sampling the initial transition data. Then we sample transitions using the student policy and request correction from the teacher. We find this is much more efficient and effective than directly learning a visuomotor student policy using RL.
Randomization and augmentation. To bridge the sim-and-real gap, we slightly randomize the po- sition and rotation of both cameras to handle slight offsets in real. Besides, we clip the depth image to minimum of 0.2m to avoid low-quality images when the distance too close to a real camera, after which we normalize the depth value. We adopt augmentations like RandomErasinng Gaussian- Blur, GaussianNoise, and RandomRotation for the depth images. Additional details regarding the high-level policy training including rewards can be found in Appendix C.2.
3.3 Real World Deployment
Robot Platform The robot platform used in this work comprises a Unitree B1 quadruped robot with a Unitree Z1 robot arm mounted on its back. Our robot equips two cameras, one on the head and another close to the gripper. We use the onboard computer of B1 to control the low-level policy, and an additional onboard computer receive visual inputs and control the high-level policy, The Unitree B1 quadruped has 12 actuatable DoFs, the arm has 6 DoFs, and the gripper has 1 DoF, making 19 DoFs in total, An overview of the hardware setup is shown in Fig. 4.
5

---

Vision masks. Note that our system works with little human intervention, i.c., we need to annotate and segment objects from environment backgrounds. To this end, we take use of TrackingSAM [24] (details in Appendix J), tool that combines a video object tracking model [25] with interactive segmentation interface [26]. The implementation details and an example for illustration are given in Sec. and Fig. 14. During real-world experiments, we annotated the objects from one of the RGBD cameras at the beginning of each reset trial. TrackingSAM will keep tracking the segmented object and providing the mask in real time (~10 fps on the robot). The depth images obtained from the RGBD cameras are pre-processed by hole-filling and clipping. Empirically, as long the object is visible to one camera (either the head or the wrist camera), the annotation mask can be propagated to both views to initialize an automated pickup.
4 Experiments
We conduct set of experiments pick-up tasks to compare VBC with baselines both in simulation and the real world, showing the effectiveness of training a loco-manipulation policy with VBC, and the generalizability over varying objects with different heights. Experiment setups. We utilize Isaac- Gym [27] for parallel training. In the real-world experiment, we initialize ob- jects in random location and orientation in front of the robot, making sure it is visible in at least one camera (around 1,5~1155m), We test three heights by putting the objects on the ground, on a box, and on a table, corresponding to the heights of 0.0m, ~0.3m, ~0.5m.
Simulation
Real World
0.5mm
0.5-1 Snt
Farallel Enviropments Training
D.0m
Figure 5: Illustration of simulation / real-world setup.
Objects. We 33 YCB objects [28] in simulation. According to the object shapes, roughly divided them into 7 representative categories: ball, long box, square box, bottle cup, bowl and drill. As for real-world experiments, we choose 14 objects, including 4 irregular objects, common objects, and 4 regular shapes. It is worth noting that almost all objects are unseen in the simulation. See Appendix for illustrated details.
4.1 Simulation Results
Evaluation principles and baselines. In the simulation, we consider a successful grasp if the object is picked up and failure when the object falls down the table or is not success- ful in 150 high-level steps (~ 18 24s). We compare both the privileged teacher policies of VBC against baselines and their visuomotor student policies to provide an analysis of each module of VBC: 1) VBC w.o. shape feature: a visuomotor policy that is trained the same as VBC, but does not access the pre-trained ob- ject shape features during the teacher training stage. Floating base: floating base policy with perfect low-level navigation ability but without arm-leg coordination; in other words, the robot can always follow the given velocity commands and body height commands rang- ing from 0.4 to 0.55 meters (we set 0.4 meters as the lower bound as the default controller in the real-world only allows 0.47 meters in min- imum). We compare both the teacher and its corresponding student policy using the same policy network structure. 3) Non Non-hierarchical: uni-
Palyy
1.0
0.8
Success Rate
0.6
0.4
Lorpfixs
0.2
ceeater
0.0
Labor
0.0
0.1
0.2
0.3
0.4
0.5
0.6
Height (m)
Figure 6: Suecess rates of different student poli- cies at different heights in simulation. We test each distinct object for more than 100 episodes. The dots/crosses represent the mean performance, and the upper/lower bound is the max/min suc- cess rate over 7 categories, with the category name noted. The non-hierarchical baseline is not in- cluded for not reaching meaningful results.

---

Table I: Success rates of VBC compared to baseline method seven categories of objects, tested in the simulator. Each distinct object is tested for more than 300 episodes over random seeds. The robot starts from 1.0 1.0-meter distance from the table. Every episode ends when the robot successfully picks up the object, the object falls from the table, or the robot fails until 150 steps. With the 3D shape feature, the algorithm is better handling irregular objects.
Policy Type
Ball
Long Box
Square Box
Bottle
Cup
Boul
Drill
Flouting Base
0.63 a 0.01
0.20 a 0.00
0.78 = 0,01
0.35 1.02
1.71 ou2
0.71 0.001
0.07 1 0.02
Privleged Teucher
VBC w.n. Shape Festure
0.96 0.01
0.74 0.08
0.90
0.00
0.82 + +00.1
0.84
0.0
0.55++000 +000
0.74 + 0.00
VBC (Durs)
0.89 0.01
0.00 0.93
81.7 = 0.00
0.76 +000
0.89 + 0.02
0.78 + 0.08
0.84 + 0.00
Non-Heerarchical
0.0
0.0
0.0
0.0
0.0
0.0
0.0
Visuomotor Student
Flouting Base VBC (Ours)
0.39 1 0.07
0.06 1 0.00
0.42
0.02
0.15
1.03
0.58 1 0.05
0.54 1:01
0.03 1 0.01
0.07 2 0.07
0.42 1: 0.12
0.74
0.00
0.55 1.0.1
0.74 it 0.06
0.52 a 0.02
0.47 a 0.11
Traqputed leuel
dard
Avcroge Success Times
Aversge Success Times
Attempts
(a) Objects on the ground (0.0m). (b) Objects box (~0.3m). (c) Objects table (~0.5m)
Figure 7: Average success times w.r.t the attempts of picking up 14 objects at different heights, tested in the real world. We allow 5 continuing attempts (including 4 continuous retries, x-axis) during one configuration, i.e. one reset of picking up one specific object at specific height. A failure is recorded if the robot attempts more than 5 attempts in one reset or if the object falls from its surface. We test 5 times for each object and each height, and calculate the average success time (y-axis) across these trials. The emergent retrying behavior improves the performance.
fied policy trained with low-level policy and visuomotor high-level policy jointly in an end-to-end style. This policy takes the observation of our low-level and high-level policies together and out- puts all the target positions of 12 robot joint angles and the target pose of the gripper. We failed in training such policy, indicating the effectiveness and necessity of VBC's training pipeline.
Picking up different objects. We test the picking-up success rate on each distinct object for more than 300 episodes and collect the results over every category. As is shown in Tab. 1, VBC achieves the best performance on 4/7 categories of objects. It is also interesting that with 3D features, VBC becomes worse than VBC w.o. Shape Feature at regular objects, but is much better on irregular objects with complicated shapes, i.e., long box, cup, bowl, and drill. One possible reason is that, without pre-trained features, the policy targets the object's center for grasping, which is effective enough simpler objects yet inapplicable for objects with more complicated shapes. One may notice that the student policy is much worse than the teacher policy, since when the robot touches but fails to grasp on the first try, the object may be randomly laid so that its pose will be much harder to grasp within only 150 steps using only visual inputs. In addition, the performance of the student policy of VBC behaves the worst on long boxes. This is because without shape features, is much harder to determine the pose of the object, and some initial poses of long boxes are extremely hard to grasp for the gripper we used some height (also as shown in Fig. 6).
Picking up over various heights. We test the mean success rate each object category on fixed height, where we also list the object with the maximum/minimum success rate each height. We compare VBC and the floating base baseline on 7 different heights, and illustrate the results in Fig. 6. From the significant improvement of VBC on almost every height, compared to the floating base policy, we highlight the advantage of VBC in achieving flexible wholc-body behaviors. 4.2 Real- World Experiments
Evaluation principles and baselines. We deploy the trained visuomotor student policy of VBC directly into the real world. Particularly, we test how many times the robot can pick up the assigned object in five attempts without resetting (i.e., the robot allowed to continuously retry four attempts when it fails to grasp), since we find that the emergent retrying behaviors improve the robustness, reliability, and success rate of finishing job. We consider it success when the object is picked more than 0. higher than the placed plane and a failure when the robot tries more than five
7

---

Initialization
Approach Fing
Resched Command Valecttie Over Times
Graaped
Ficked Up
Vileeyy Command
Unear Valdit Ve
Brigpar Command Time
Oniperr Conmand
Time
Figure 8: Visualization of robot behavior and corresponding commands during the test time. We plot the velocity commands (including the quadruped linear velocity and yaw velocity); along with the gripper command (close or open).
attempts or the object falls from its surface. Each object is evaluated over three different heights and we do five resets for each configuration. Regarding baselines, as it is hard to build a fair autonomous method for comparison, we compared VBC with teleoperating the default Unitree base controller (allowing the heights of the arm base to vary from 0.47m to 0.55m) combined with a similar high- level policy trained on stationary legs, which the robot has no arm-leg coordination but also retains the retrying. For this baseline, We teleoperated the robot to be close enough to the object before grasping it. During all tests, we put objects distance that one mounted camera observe, thus they can be easily manually annotated using the TrackingSAM tool. After the annotation, the robot walks towards and picks up the objects in fully autonomous way. In experiments, we observe that the policy is capable of recovering from short-term tracking loss, which is not uncommon due to the robot turning or tilting.
Emergent retrying behaviors. The robot emerges to learn a retrying behavior in simulation train- ing. In other words, when the robot fails grasp an object, automatically retries to grasp without human intervention. This shows advantage and clear difference compared to model-based meth- ods. In our observation, as long the visual mask is on-track/not-lostt, the success rates benefit from 1) the change of the object pose in the early tries; and 2) the movement of the robot body.
Pickup performance. We collect all results and conclude the averaged performance over all objects on each height setting and show in Fig. 7. It is worth noting that our method allows for autonomous retrying behaviors when one pick-up trial fails. Obviously, VBC surpasses the baseline method on all settings. To highlight, the baseline methods fail at 0.0m and 0.3m, as the default controller keeps fixed robot height, similar the floating base baseline. Even on the 0.5m setting, where the teleoperated baseline without whole-body behavior is equivalent to a static robotics arm, VBC is generally better. Besides, VBC shows great generalization ability on unseen shapes, thanks to the training strategy and observation design in the training pipeline.
Qualititive analysis. We analyze whether the commands sent by the high-level policy are reasonable to call the low-level policy to achieve the task. To this end, we visualize the velocity commands, the gripper command, along with their corresponding robot behavior during the testing time in Fig. 8. Our system has small gaps when deploying our policy trained in the simulator into the real world, which is able to provide the correct commands that show clear phases.
Conclusions and Limitations
We proposed Visual Whole-Body Control (VBC), fully autonomous loco-manipulation system on quadruped robots with a hierarchical training pipeline. VBC trains a low-level goal-reaching policy and a high-level task-planning policy through reinforcement learning and imitation learning. Limitations. Although we showed great success in picking various objects, there are several limita- tions regarding the system and the tasks that can be of interest in future work. A detailed analysis is in Appendix I and outline some reasons: (1) visual tracking loss; (2) inadequate gripper design; (3) in- in-precise depth estimation; (4) compounding error.
8

---

Acknowledgments
This project was supported, part, by the Amazon Research Award, the Intel Rising Star Faculty Award, and gifts from Qualcomm, Covariant, and Meta. We thank Chengzhe Jia for the great help on 3D modeling. We also thank Haichuan Che, Jun Wang. Shiqi Yang, and Jialong Li for their kind help in recording demos. We thank Xuanbin Peng for helping with the experiments.
References
[1] Yang, Y. Jing, H. Wu, Xu, Sima, G. Chen, Q. Sima, and Kong. Moma-force: Visual-force imitation for real-world mobile manipulation. In 2023 TEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pages 6847-6852. IEEE, 2023.
[2] Z. Fu, Z. Zhao, and C. Finn. Mobile aloha: Learning bimanual mobile manipulation with low-cost whole-hody teleoperation. arXiv preprint arXiv:2401.02/11 2024.
[3] X. Cheng, Shi, A. Agarwal, and D. Pathak. Extreme parkour with legged robots. arXiv preprint 1.XXv:2309.143411 2023.
[4] A. Agarwal, A. Kumar, J. Malik, and D. Pathak. Legged locomotion in challenging terrains using egocentric vision. Conference on Robot Learning, pages 403-415. PMLR, 2023.
[5] Z. Zhuang, Z. Fu, J. Wang, C. Atkeson, S. Schwertfeger, C. Finn, and H. Zhao. Robot parkour learning. arXiv preprint arXiv:2309.0560 2023.
[6] H. Duan, B. Pandit, M. S. Gadde, B. J. van Marum, J. Dao, C. Kim, and A. Fern. Learning vision-based bipedal locomotion for challenging terrain. arXiv preprint arXiv:2309. 10009..4599 14594, 2023.
[7] C. D. Bellicoso, K. Krämer, M. Stäuble, D. Sako, F. Jenelten, M. Bjelonic, and M. Hutter. Alma-articulated locomotion and manipulation for a torque-controllable robot. In 2019 Inter- national conference on robotics and automation (ICRA), pages 8477-8483. IEEE, 2019.
[8] Z. Fu, X. Cheng, and D. Pathak. Deep whole-body control: learning unified policy for manipulation and locomotion. In Conference on Robot Learning, pages 138-149. PMLR, 2023.
[9] Y. Ma, F. Farshidian, T. Miki, J. Lee, and M. Hutter. Combining learning-based locomotion policy with model-based manipulation for legged mobile manipulators. IEEE Robotics and Automation Letters, 7(2):2377-233 2384, 2022.
[10] J. Zhang, N. Gireesh, J. Wang, X. Fang, C. Xu, W. Chen, L. Dai, and H. Wang. Gamma: Graspability-aware mobile manipulation policy learning based on online grasping pose fusion. arXiv preprint arXiv:2309. 2023.
[11] N. Yokoyama. A. W. Clegg, E. Undersander, S. Ha, D. Batra, and A. Rai. Adaptive skill coordination for robotic mobile manipulation. arXiv preprint arXiv:2304.004// 2023.
[12] F. Arcari, M. V. Minniti, A. Scampicchio, A. Carron, F. Farshidian, M. Hutter, and M. N. Zeilinger. Bayesian multi-task learning mpe for robotic mobile manipulation. IEEE Robotics and Automation Letters, 2023.
[13] H. Ferrolho, V. Ivan, W. Merkt, I. Havoutis, and S. Vijayakumar. Roloma: Robust loco- manipulation for quadruped robots with arms. Autonomous Robots, 1633:114331181.1 2023.
[14] J.-P. Sleiman, F. Farshidian, and M. Hutter. Versatile multicontact planning and control for legged loco-manipulation. Science Robotics, 8e8d)saa,, 2023.
[15] S. Zimmermann, R. Poranne, and S. Coros. Go fetchl-dynamic grasps using boston dynam- ics spot with external robotic In 2021 IEEE International Conference Robotics and Automation (ICRA), pages 4488-4494, IEEE, 2021.

---

[16] W. J. Wolfslag, C. McGreavy, G. Xin, C. Tiseo, S. Vijayakumar, and Z. Li. Optimisation of body-ground contact for augmenting the whole-body loco-manipulation of quadruped robots. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 3694-3701. IEEE, 2020.
[17] Jeon, Jung, Choi, B. Kim, and Hwangbo. Learning whole-body manipulation for quadrupedal robot. IEEE Roboties and Automation Letters, (11)6999-706, 2023.
[18] He, Y. Zhang, K. Ren, M. Liu, Wang, W. Zhang, Y. Yang, and D. Li. Reinforcement learning with automated auxiliary loss search. Advances in Neural Information Processing Systems, 13:1820-1834 2022.
[19] D. Yarats, R. Fergus, A. Lazaric, and L. Pinto. Mastering visual continuous control: Improved data-augmented reinforcement learning. In International Conference on Learning Representa- lions, 2022. URL https::/p/// / //wpnrevee/eeeeeeeeeeeee/eeee.ee
fort/formm? d=__0-_yyess__
[20] N. Hansen, H. Su, and X. Wang. Td-mpe2: Scalable, robust world models for continuous control, 2023.
[21] D. Hafner, J. Pasukonis, J. Ba, and T. Lillicrap. Mastering diverse domains through world models, 2023.
[22] G. Xu, R. Zheng, Y. Liang, X. Wang, Z. Yuan, Ji, Y. Luo, X. Liu, J. Yuan, P. Hua, S. Li, Y. Ze, D. au2, Huang, and H. Xu. Drm: Mastering visual reinforcement learning through dormant ratio minimization, 2023.
[23] C. R. Qi, H. Su, K. Mo, and L. J. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 652-660, 2017.
[24] R. Qiu. Towards real-tim time robotics perception with continual adaptation. PhD thesis, Univer- sity of Illinois Urbana-Chanan Sabbna-Champaiin, 2023.
[25] Z. Yang, Wei, and Y. Yang. Associating objects with transformers for video object segmen- tation. Advances in Neural Information Processing Systems, 34:2491-2503 2021.
[26] A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead,
A. C. Berg, W.-Y. Lo, et al. Segment anything. arXiv preprint arXiv:2304.02643, 2023.
[27] V. Makoviychuk, L. Wawrzyniak, Y. Guo, M. Lu, K. Storey, M. Macklin, D. Hoeller, N. Rudin, A. Allshire, A. Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108. 10470, 2021.
[28] B. Calli, A. Singh. J. Bruce, A. Walsman, K. Konolige, S. Srinivasa. P. Abbeel, and A. M. Dol- lar. berkeelle dataset for robotic manipulation research. The International Journal of Robotics Research, 36(3):261-268, 2017.
[29] H. Miura and Shimoyama. Dynamic walk of a biped. The International Journal of Robotics Research, 3(2):60-75 74, 1984.
[30] K. Sreenath, H.-W. Park, Poulakakis, and W. Grizzle. compliant hybrid zero dynamics controller for stable, efficient and fast bipedal walking on mabel. The International Journal of Robotics Research, 30(9) 1700)1991111133 2011.
[31] G. Bledt, M. J. Powell, B. Katz, Di Carlo, P. M. Wensing, and Kim. Mit cheetah 3: Design and control of robust, dynamic quadruped robot. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 2245-2252 IEEE, 2018.
10

---

[32] M. Hutter, C. Gehring, D. Jud, A. Lauber, C. D. Bellicoso, V. Tsounis, J. Hwangbo, K. Bodie, Fankhauser, M. Bloesch, et al. Anymal-a highly mobile and dynamic quadrupedal robot. In 2016 JEEE/RSJ international conference on intelligent robots and systems (IROS), pages 38-44, IEEE, 2016.
[33] S. Feng, E. Whitman, X. Xinjilefu, and C. G. Atkeson. Optimization based full body control for the atlas robot. In 2014 IEEE-RAS International Conference on Humanoid Robots, pages 120-127, IERE, 2014.
[34] Q. Nguyen, M. J. Powell, B. Katz, J. Di Carlo, and S. Kim. Optimized jumping on the mit cheetah robot. In 2019 International Conference on Robotics and Automation (ICRA), pages 7448-7454. IEEE, 2019.
[35] C. Nguyen, I.. Bao, and Q. Nguyen. Continuous jumping for legged robots on stepping stones via trajectory optimization and model predictive control. In 2022 IEEE 61 Conference on Decision and Control (CDC), pages 93-99. IEEE, 2022.
[36] C. Gehring, S. Coros, M. Hutter, C. D. Bellicoso, H. Heijnen, R. Diethelm, M. Bloesch, Fankhauser, J. Hwangbo, M. Hoepflinger, et al. Practice makes perfect: An optimization- based approach to controlling agile motions for quadruped robot. IEEE Robotics AAtoma- tion Magazine, 23(1):34-43, 2016.
[37] Q. Nguyen, A. Agrawal, X. Da, W. C. Martin, H. Geyer, J. W. Grizzle, and K. Sreenath. Dynamic walking on randomly-varying discrete terrain with one-step preview. Robotics; Science and Systems, volume 2. pages 384-99, 2017.
[38] R. Antonova, A. Rai, and C. G. Atkeson. Deep kernels for optimizing locomotion controllers. In Conference on Robot Learning, pages 47-56. PMLR, 2017.
[39] K. N. Kumar, I. Essa, and S. Ha. Cascaded compositional residual learning for complex interactive behaviors. IEEE Robotics and Automation Letters, 888):4001-4688, 2023. doi: 10.
[40] P. Fankhauser, M. Bloesch, and M. Hutter. Probabilistic terrain mapping for mobile robots with uncertain localization. JEEE Robotics and Automation Letters, 3(4):3019-3026, 2018.
[41] I.S.. Kweon and T. Kanade. High-resolution terrain map from multiple sensor data. 1EEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):278-292, 1992.
[42] A. Kleiner and C. Dornhege. Real-time localization and elevation mapping within urban search and rescue scenarios. Journal of Field Robotics, 24(8-9):723-745 2007.
[43] D. Belter, P. Labcki, and P. Skrzypczyáski. Estimating terrain elevation maps from sparse and uncertain multi-sensor data. In 2012 IEEE International Conference Robotics and Biomimetics (ROBIO), pages 715-722. IEEE, 2012.
[44] P. Fankhauser, M. Bloesch, C. Gehring, M. Hutter, and R. Siegwart. Robot-centric elevation mapping with uncertainty estimates. In Mobile Service Robotics, pages 433-440. World Sci- entific, 2014.
[45] M. Bloesch, S. Omari, M. Hutter, and R. Siegwart. Robust visual inertial odometry using direct ekf-based approach. In 2015 IEEE/RSJ international conference intelligent robots and systems (IROS), pages 998-304. IEEE, 2015.
[46] D. Wisth, M. Camurri, and M. Fallon. Vilens: Visual, inertial, lidar, and leg odometry for all-terrain legged robots. IEEE Transactions on Robotics, 39(1):309-326, 2022.
[47] R. Buchanan, M. Camurri, Dellaert, and M. Fallon. Learning inertial odometry for dynamic legged robot state estimation. In Conference on robot learning, pages 1575-1584. PMLR, 2022.
11

---

[48] S. Yang, H. Kumar, Z. Gu, X. Zhang, M. Travers, and H. Choset. State estimation for legged robots using contact-centric leg odometry. arXiv preprint arXiv: .06176, 2019.
[49] D. Wisth, M. Camurri, and M. Fallon. Robust legged robot state estimation using factor graph optimization. IEEE Robotics and Automation Letters, 4(4))4507-4514, 2019.
[50] E. Coumans and Y. Bai. Pybullet, python module for physies simulation for games, robotics and machine learning. http://
http::/pybullett httpppyppllll.eee org, 2016-20211
[51] N. Rudin, D. Hoeller, P. Reist, and M. Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pages 91-100. PMLR, 2022.
[52] A. Kumar, Z. Fu, D. Pathak, and J. Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv: arXiv:210..0400: 107.04034, 2021.
[53] A. Kumar, Z. Li, J. Zeng, D. Pathak, K. Sreenath, and J. Malik. Adapting rapid motor adapta- tion for bipedal robots. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 1161-1168. IEEE, 2022.
[54] J. Lee, J. Hwangbo, I.. Wellhausen, V. Koltun, and M. Hutter. Learning quadrupedal locomo- tion over challenging terrain. Science robotics, 5(47):eabc5986, 2020.
[55] T. Miki, J. Lee, J. Hwangbo, L. Wellhausen, V. Koltun, and M. Hutter. Learning robust per- ceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7(62))eab22822, 2022.
[56] R. Yang, G. Yang, and X. Wang. Neural volumetric memory for visual locomotion control. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1430-1440, 2023.
[57] W. Yu, D. Jain, A. Escontrela, A. Iscen, P. Xu, Coumans, Ha, J. Tan, and T. Zhang. Visual- locomotion: Learning to walk on complex terrains with vision. In Sth Annual Conference on Robot Learning, 2021.
[58] Z. Fu, A. Kumar, A. Agarwal, H. Qi. J. Malik, and D. Pathak. Coupling vision and propriocep- tion for navigation of legged robots. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 17273-17287 2022.
[59] G. B. Margolis and P. Agrawal. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning, pages 22-31. PMLR, 2023.
[60] Z. Li, X. Cheng, X. B. Peng, P. Abbeel, S. Levine, G. Berseth, and K. Sreenath. Reinforce- ment learning for robust parameterized locomotion control of bipedal robots. In 202/ IEEE International Conference on Robotics and Automation (ICRA), pages 281 1-2817. IEEE, 2021.
[61] Y. Fuchioka, Z. Xie, and M. Van de Panne. Opt-mimic: Imitation of optimized trajectories for dynamic quadruped behaviors. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 5092-5098, IEEE, 2023.
[62] A. Escontrela, X. B. Peng, W. Yu, T. Zhang, A. Iscen, K. Goldberg, and P. Abbeel. Adver- sarial motion priors make good substitutes for complex reward functions. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 25-32 IEEE, 2022.
[63] X. B. Peng, E. Coumans, T. Zhang, T.-W. Lee, J. Tan, and S. Levine. Learning agile robotic locomotion skills by imitating animals. arXiv preprint arXiv:2004.00758 2020.
[64] R. Yang, Z. Chen, J. Ma, C. Zheng, Chen, Q Nguyen, and X. Wang. Generalized animal imitator: Agile locomotion with versatile motion prior. arXiv preprint arXi::2310.01008, 2023.
12

---

[65] Y. Wang, Z. Jiang, and J. Chen. Amp in the wild: Learning robust, agile, natural legged locomotion skills. arXiv preprint arXiv:2304. 10888, 2023.
[66] C. Li, M. Vlastelica, S. Blaes, J. Frey, F. Grimminger, and G. Martius. Learning agile skills via adversarial imitation of rough partial demonstrations. In Conference Robot Learning, pages 342-352. PMLR, 2023.
[67] K. N. Kumar, I. Essa, and S. Ha. Words into action: Learning diverse humanoid robot behaviors using language guided iterative motion refinement, 2023.
[68] S. Kim, M. Sorokin, Lee, and S. Ha. Human motion control of quadrupedal robots using deep reinforcement learning. In Proceedings of Robotics: Science and Systems, New York, USA, June 2022.
[69] Wong, A. Tung, A. Kurenkov, A. Mandlekar, L. Fei-Fei, S. Savarese, and R. Martín-Martinn Error-aware imitation learning from teleoperation data for mobile manipulation. Conference on Robot Learning, pages 1367-1378. PMLR, 2022.
[70] Lew, S. Singh, M. Prats, Bingham, J. Weisz, B. Holson, X. Zhang, V. Sindhwani, Y. Lu, Xia, P. Xu, T. Zhang, Tan, and M. Gonzalez. Robotic table wiping via reinforcement learning and whole-body trajectory optimization, 2022.
[71] C. R. Garrett, T. Lozano-Pérez, and L. P. Kaelbling. Pddistream: Integrating symbolic planners and blackbox samplers via optimistic adaptive planning. 2020.
[72] S. Srivastava, E. Fang. L. Riano, R. Chitnis, S. Russell, and P. Abbeel. Combined task and motion planning through an extensible planner-independent interface layer. In 2014 IEEE International Conference on Robotics and Automation (ICRA), pages 639-646, 2014. doi: 10.1109/CRA.20144 26906922...
[73] J. Gu, D. S. Chaplot, H. Su, and J. Malik. Multi-skill mobile manipulation for object re- arrangement. In The Eleventh International Conference on Learning Representations, 2023. URL ht https: opwnreviewwww.ee/////p/pe///./wm net 2311M bzv?.
[74] F. Xia, C. Li, R. Martin-Martin, O. Litany, A. Toshev, and S. Savarese. Relmogen: Leveraging motion generation in reinforcement learning for mobile manipulation, 2021.
[75] N. Yokoyama, A. Clegg, J. Truong, E. Undersander, T.-Y. Yang, S. Arnad, S. Ha, D. Batra, and A. Rai. Ase: Adaptive skill coordination for robotic mobile manipulation, 2023.
[76] C. Sun, J. Orbik, C. M. Devin, H. Yang. A. Gupta, Berseth, and Levine. Fully autonomous real-world reinforcement learning with applications to mobile manipulation. In Conference Robot Learning, pages 308-311 -39. PMLR, 2022.
[77] R. Yang, Kim, A. Kembhavi, X. Wang, and K. Ehsani. Harmonic mobile manipulation. arXiv preprint arXiv:2312.06639, 2023.
[78] M. Ahn, A. Brohan, N. Brown, Y. Chebotar, O. Cortes, B. David, C. Finn, C. Fu, K. Gopalakr- ishman, K. Hausman, et al. Do as i can, not as say: Grounding language in robotic affordances. arXiv preprint 201611, 2022.
[79] A. Brohan, N. Brown, J. Carbajal, Y. Chebotar, J. Dabis, C. Finn, K. Gopalakrishnan, K. Haus- man, A. Herzog, J. Hsu, et al. Rt-1: Robotics transformer for real-world control at seale. arXiv preprint arXiv:2212.068877 2022.
[80] N. M. M. Shafiullah, A. Rai, H. Etukuru, Y. Liu, I. Misra, S. Chintala, and L. Pinto. On bringing robots home. arXiv preprint 1rXiv:2311.16098 2023.
[81] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint 1rXiv:1707.0660 2017.
13

---

[82] S. Ross, G. Gordon, and D. Bagnell. A reduction of imitation learning and structured predic- tion to no-regret online learning. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pages 627-635. JMLR Workshop and Conference Pro- ceedings, 2011.
[83] M. Liu, T. He, M. Xu, and W. Zhang. Energy-based imitation learning, WwXiv:2004.0339. 2020.
arXiv preprint
[84] X. Cheng, A. Kumar, and D. Pathak. Legs as manipulator: Pushing quadrupedal agility beyond locomotion. In 2023 IEEE International Conference Robotics and Automation (ICRA), 2023.
14

---

A Extended Related Works
Legged Locomotion. For decades, legged locomotion has been an essential topic in the field of robotics and has gained much success. Classical methods solve the legged locomotion task through model-based control to define controllers in advance. For example, Miura and Shimoyama [29], Sreenath et al. [30] utilized traditional methods to enable the walking of biped robots. MIT cheetah 3 [31], mala [32] by ETH, and Atlas [33] by Boston Dynamies are good examples of robots designed with a robust walking controller. Based these developed robots, researchers have been able to develop advanced controllers for even more agile locomotion tasks, such as jumping or walking over varying obstacles [34, 35, 36, 37, 38, 39]. To make robust controller that can adapt to varying environments, researchers integrate some sensor data such as elevation maps [40, 41, 42, 43, 44] and state estimators [45, 46, 47, 48, 49]. Although classical methods have greatly succeeded in legged locomotion, generalizability in pre-designed controllers would require careful and tedious engineering efforts. With the fast development of deep neural networks, learning-basee based end-to-end methods have become simple way to gain robust controllers with strong generalization abilities. Learning-based methods often train robust controller in physics simulators such PyBullet [50] and Isaac Gym [27] and then perform sim-to-real transfer to real robot. For example, Rudin et al. [51] uses Isaac Gym with GPU to enable quadruped robot to walk in only several minutes, Kumar et al. [52, 53] proposed the RMA algorithm to make the sim-to-real transfer process easier, and more robust, Lee et al. [54], Miki et al. [55], Agarwal et al. [4), Yang et al. [56], et al. [57], Cheng al. [3]. Zhuang al. [5], Fu al. [58], Margolis and Agrawal [59]. Li al. [60] enable the legged robots walk or on varying terrains, and Fuchioka al. [61], Escontrela et al. [62], Peng al. [63]. Yang et al. [64], Wang et al. [65], Li et al. [66], Kumar et al. [67] combine RL with imitation learning to produce naturalistic and robust controllers for legged robots. Kim et al. [68] map human motions to a quadruped robot to complete a wide range of locomotion and manipulation tasks.
Mobile Manipulation. Previously mentioned works mainly focus only on the mobility part; adding manipulation to the mobile base is yet to be well-solved. Autonomous mobile manipulation on wheeled robot platforms has made significant progress over the past few years. Wong al. [69] collected teleoperated mobile manipulation data and learned imitation-leaming-bbseee policies in simulation. Lew et al. [70] built efficient framework for table cleaning with RL and trajectory optimization. Garrett et al. [71] and Srivastava et al. [72] integrate task-specific planner for various mobile manipulation tasks. Gu et al. [73] and Xia et al. [74] developed modularized mobile manip- ulators with RL, Yokoyama et al. [75] introduced high-level controller for coordinating different low-level skills. Sun et al. [76] designed a modularized policy with components for manipulation and navigation, trained by RL to learn navigation and grasping indoors. Yang et al. [77] proposed simultaneously controlling the robot arm and base for harmonic mobile manipulation using RL train- ing with large-scale environments. Ahn et al. [78] pre-trained set of single mobile manipulation tasks and relied on large language models to decompose and call the skills to achieve long-horizon tasks. Brohan et al. [79] train a robotics transformer to perform real-world tasks given human in- structions, trained on large-scale and real-world demonstration. Shafiullah et al. [80] learned to solve houschold tasks from 5-minute demonstrations based on pre-trained home representations. While encouraging, most of these mobile manipulators with wheels cannot operate with challenging terrains in the wild.
B Notations and Background
B.1 Reinforcement Learning
Reinforcement learning (RL) is usually formulated as a y-discounted infinite horizon Markov de- cision process (MDP) M = AS T. r, 7), where S is the state space, .A the action space, T: S x A -> S is the environment dynamics function, Pn : S + [0, 1] is the initial state dis- tribution, r(s, a)
A + the reward function, and 7 E [0, is the discount factor. The goal of RL is to train agent with policy iTaas) S + A to maximize the accumulated reward R = at)

---

Table 2: Details of low-level policy.
Definition of Symbols
Reward Function Details for the Low-Level Policy
Name
Symhol
Leg joint positions
Name
Definition
Weight
Leg joint velocilies
Leg joint accelerations
q
Linear velocity tracking
oft Playy
Target leg joint positions
Log joint lonques
q
Yaw velocity tracking
Angalar velocity penalty
0.05
Base lincar velocity
Joint toques
0,00002
angular velocity
Action
1q'll
0.25
Rase linear velocity command
Collisions
-Meattiiion
0.001
Base angular velocity commandd
Fest time
-0.5]
Number of collisions
Foet contact force
Plo
ffect
Rase Height
Lincar Velocity
cxp
11,2
-0:05 Qichak
-5.0
-1.5
Foet velocity in Z axis
vi
Swing Phase Tracking (Force)
(1 (god ()) exp [1g (o,l))
-0.2
Feet air time
Stanoe Pluse Tracking (Velocity)
(C50l (1em) (() (1 imp
-0.2
Timing offsets
gend
Stepping frequency
good
B.2 Proximal Policy Optimization
Proximal policy optimization (PPO) [81] is one of the popular algorithms that solve RL problems. The basic idea behind PPO is to maximize a surrogate objective that constrains the size of the policy update. In particular, PPO optimizes the following objective:
PPO (PO()=)) [m][mi((rA, clip(r. - c
(1)
where r = (ata(a)s) defines the probability ratio the current policy and the old policy at the last optimization step, A(s, a) is the advantage function, and 0₂ is the parameter of the policy
<alss
B.3 Imitation Learning
In general, imitation learning (II.) [82, 83] studies the task of learning from expert demonstrations. In this work, instead of learning from offline expert data, we refer to an online method, dataset aggregation (DAgger) [82] such that request online expert provide the demonstrated action. Formally, the goal is train student policy â minimizing the action distance between the expert policy ge under its encountered states:
arg min Egwd, [e(8, m)]- xell
In this paper, é is the mean square error in practice.
B.4 Whole-Body Control
Whole-body control in robotics refers controlling mobile manipulator with the function of loco- motion and manipulation by one unified framework [9, 8]. In this work, we consider a system that contains a quadruped robot combined with a 6-DoF robot arm, each of which has an underlying PD controller, and we use a learned policy to generate control signals and feed them to the controller given 1) the robot states and 2) commands.
In detail, the policy used for controlling the quadruped robot is driven by external commands, such as the base velocity command; and the arm is controlled by PD controller driven by the SE((3) end-effector position-oriennation command and inverse kinematies.
C More Training Details
C.1 Low-Level Training
Reward functions. Our reward for training the low-level policy for whole-body control mainly con- sists of four parts: command- command-following reward, energy reward, alive reward, and phase reward. The command following reward encourages the robot to track and follow the provided commands and explore various whole-body behaviors; the energy reward penalizes energy consumption enable smooth and reasonable motion; the alive reward encourages the robot not to fail; the phase reward encourages the robot to walk in steady behavior mentioned above. The detailed definitions of

---

Figure 9: Screenshots of low-level training, where the robot is required to track the given velocity commands (linear and yaw) and the end-effector pose. We employ height-invarian coordinate system originating at the robot's base, such that the sampled goal poses are only affected by the robot's horizontal movement (translation in X, axes and orientation in the yaw axis). The cyan point represents the origin of the resampled goals; the blue point and the yellow point are the current end-effector position and the goal position, respectively. The red dots are the interpolated goals. The red lines represent the space for determining self-collision when resampling new goals.
the low-level reward functions are shown in Tab. 2, where we define ø = exp( 0,25 for tracking
the desired velocity of the robot. is worth noting that we introduce the swing phase tracking re- wards to help the robot learn trotting, and Crod (gend, t) computes the desired contact state of each foot from the phase and timing variable, as described in Margolis and Agrawal [59].
Low-level commands sampling. To train robust low-level controller for goal-reaching, should sample and track various goals (i.e., base velocity commands and end-effector commands) pos- sible. As for the base velocity commands, we uniformly sample the linear velocity from a range of 1-0.6m/s s. 0. 0.6m/ss, and the angular velocity from range of [1.0rad/s, s, 1.rrd/s)..
Regarding the end-effector, same to Fu et al. [8], we employ a height-roll--ae roll-pitch-in-e invariant coordinate system originating at the robot's base, where the sampled goal poses are only affected by the robot's horizontal movement instead of changes in height. In detail, the origin (cyan point in Fig. 9) is set on fixed plane, which is on the same line of Z axis the arm base. Therefore, the sampled goals are also not affected by the robot's height, and orientation of the roll and pitch axis, but only by the orientation in the yaw axis. Such a design ensures the sampled trajectories are always on one local sphere at any time, guaranteeing the consistency of these samples. Besides, it helps to learn a whole-body behavior to control the legs and the body height to enable the arm to reach an expanded space of goals. If not, the robot may fail to reach a goal out of the workspace of the arm without bending or rising. Therefore, the body and the arm are not decoupled, although we use IK control the arm.
We parameterize the end-effector position command pond in spherical coordinate (I,p,y). where 1, p. y lie in ranges of [0.4, 0.95], [-1 π 2.5, 1 π 31, (-1.2, 1 1.2], respectively. To make smooth end-effector movement, we set pemdd by interpolation between the current end-effector position and a randomly sampled end-effector position padd at intervals of Tiny seconds:
emd
P + (1-1(T
p and t E Timj).
Pr
Timg
Timy
end is resampled in fixed timestep or if any pind leads self-collision or collision with the ground. Fig. shows screenshots of the low-level training, where the robot is required to track the given velocity commands and the end-effector pose. The orientation command amd is uniformly sampled from S3(3) space.
Regularized online adaptation for sim-to-real transfer. We take the training paradigm of Regu- larized Online Adaptation (ROA) [8, 84, 17] in the RL training, to help decrease the realizability gap when deploying our system into the real world. As illustrated in Fig. 10, in the first training phase, we train an encoder j that takes the privileged information e (such as friction and mass...etc.) as inputs, and predicts an environment extrinsic latent vector 2", which will be used for policy infer- ence. Note that the privileged information is not available in real robot deployment. Thus will need another way to estimate the latent vector 2 To this end, train the adaptation module o in

---

Latend Env Extrinsica
Previleged Encoder
Proprioceptive States
Privieged Info
supervise-
Traiiing Time
Low-level Palicy
rogularie
Test Time
Adaptation Module
Leg Control
History Observations
High-level Commands
Am Control
Figure 10: Regularized Online Adaptation (ROA) training of the low-level policy.
second training phase, to imitate/distill the output the encoder μs, and estimate this 2" only based on recent history low-level observations. The loss function of the low-level policy w.r.t. policy's pa- rameters f privileged information encoder's parameters O,s and adaptation module's parameters 4g can be then formulated as:
L 10 0p) = - PPO
L (0,m0 ,,) + + +1|12 sg [3 + |s [3a]]
1
(3)
where sg [:] is the gradient stop operator, and is the Laguagrian multiplier. is worth noticing that RMA [52] is a special case of ROA, where the Laguagrian multiplier À is set to be constant zero. The adaptation module e starts training only after convergence of the policy nlww and the encoder js.
Domain randomization. When training, we randomize the terrains, including flat planes and rough ones. We also randomized the friction between robots and terrains. Besides, we randomized the quadruped's mass and center of mass. These randomizations, together with our ROA module, con- tribute to robust low-level policy.
Policy architecture. The low-level control policy network is three-layer MLP, where each layer has 128 dimensions and ELU activations. It receives observations, including the extrinsic latent environment, which is encoded by the privileged encoder during training, and the adaptation module during testing. The privileged information encoder that models environment physics parameters is two-layer MLP with a hidden size 64; the adaptation module encodes history rollouts, which is a two-layer convolutional network.
C.2 Higb-Level Training
Reward functions. The task-specific reward function for training our state-based policy comprises stage rewards and assistant rewards. 1) We design three stages for a task. The first stage is ap- proaching, and the corresponding reward Tapprouch encourages the gripper and quadruped to get close the target object: Taapronch min(delosess d, 0) where delosest is the current minimum distance between the gripper and the target object. The second stage is task progress, and the ward Pprogress leads the robot to the desired final state, e.g., lifting the object in the pickup task: pickup min(d dnighea 0) where dnighes is the current largest height of the object. The third stage is denoted task completion, and reward Tcompletion bonus when the agent completes the task. Tcompletion 1(Task is completed) .For example, in the pickup task, the task completion condition is that the object height exceeds the required threshold. Note that at one time, the robot can only be at one stage and receive one stage reward of the above three, ensuring that different task stages are not intervened in. The assistant rewards are designed to a) smooth the robot's behavior to be easier to transfer to the real world, like penalty huge action commands and joint acceleration; b) prevent deviation and sampling useless data, such as acquiring the robot's body and gripper to point the object, which guarantees that the camera does lose tracking objects. 2) Beyond these stage rewards, we also design set of assistant reward functions that help accelerate the learning procedure. We put all reward functions along with their weights of training the high-level policies
I

---

Table 3: Details of high-level rewards, where q denotes arm joint velocities; vi denotes base velocity command; doij. dee and chase are the direction from robot body to the object, the direction from end-effector to the object and the direction of the robot body orientation, separately; Xabj and Xhase are the position coordinates in world frame of the object and the robot body. Yapponchs Progress and commllttion are defined Section 3.2.
Definition
Weights
Stage Rewards
I Tapproach proogrss
0.5
1.0
T ccompletion
3.5
Tace
exp( - hc-1
9:
-0.001
Assistant
Temd
0/ + 0.25 exp(
10
1.0
Taction = 1 exp( (o0||||:-- -1 arl
-0.001
Rewards
Tee orn cos (dab) dee)
0.01
Pbsse om = cos dobj. dease)
0.25
basse appreach
(1 tanh( 10 Fodd
Dbase 0.6l)))
0.01
in Tab. 3. Except for "progress ? ccommplttonn and "completion that is determined by the task stage, we define set of assistant rewards to help the algorithm converge faster. Among them, Face limits the arm joints velocities' change rate; Temi is designed to encourage the robot to slow its velocity when it is close to the object to be picked., Taction helps smooth the output of the high-level policy to prevent it from changing quickly; Ice orn and /base encourages the arm and the robot body to be aimed at the object; Thase appoouc encourages the robot to be close to the object when the robot is far away from the object.
Teacher policy network architecture. Our privileged teacher high-level policy network includes pre- pre-trained PointNet encoder for modeling object pointclouds, followed by two-layer MLP that gives the high-level actions.
Student policy network architecture. Our vision-based high-level policy network comprises a CNN and a two-layer MLP that outputs the high-level actions. We concatenate the images and other states as observations. We stack 4-step images which are passed through the two-layer CNN network with kernel size being 5 and 3, respectively, and then encoded as 64-dim latent vector. Then, we flattened vector. This vector is then passed to
concatenate this latent vector with other states two-layer MLP with hidden size of 64. We use ELU as the activation.
Simulation We use the GPU-based Isaac Gym simulator [27]. As for the teacher, we train 10240 environments in parallel: regarding the student, we only utilize 240 parallel environments as there a huge GPU memory cost for rendering visual observations. Experiments are trained on a single GPU (we use GTX4090 and GTX3090). On a GTX4090, The teacher policy takes around 36 hours for training, and the student policy consumes around 48 hours.
Additional training techniques. We design several training techniques with our prior knowledge to help the high-level policy to be perf-performed and stable, not only in the simulation, but can also be transferred into the real world. We briefly introduce them in the following:
1. Action delay. We add one-step action delay to cover the inference and execution in the real world.
2. Command clip curriculum. We gradually clip the linear velocity command during training, find that large velocity helps accelerate the learning convergence but a slow velocity ensures safe, stable, and smooth behavior in the real world.
3. Reward curriculum. The command penalty reward Tead is added after the robot learns well to pick up objects.
4. Randomly changing the object position. During training, we take small probability (10%) randomly change the object position and pose. This helps the robot to learn the retrying behaviors when it fails during one trial.
5

---

5. Forcing stop when closing gripper. We forcibly set the velocity command to 0 when the gripper command is closing. This contributes lot to a stop-then-pick behavior and thus improves the performance.
These training techniques are adopted for both teacher policy and student policy training. When training the student policy, we additionally randomize the camera latency as the policy will never obtain the immediate images at the time infers, but pictures that few milliseconds before, similar the real world. We randomize the value by measuring the camera latency of the device used in our real-world experiments.
D Summary of Domain Randomization
We summarize and list all domain randomization here during the simulated training. Low-level policy. The randomization of the low-level policy is to train robot with robust walking skills that adapt to different surroundings, while also knowing how to adjust body poses to reach different end-effector goals.
Terrains, including flat planes and rough ones.
Friction between robots and terrains.
Mass of the quadruped and the arm.
Center of mass of the robot.
Motor strength.
High-level teacher policy. The randomization of the high-level part mainly focuses on learning the manipulation task with generalization. The high-level teacher policy follows the same randomization as the low-level policy, along with the following additional terms:
High-level frequency by involving random low-level policy calls during one high-level call. In particular, we randomize 6-8 calls considering the high-level inference time and signal transmis- sion latency, corresponding O. 0.12-0.166 (low-level policy at 50 Hz), i.e., 6.25-8.33H4
Kp and Kd parameters for the arm controller.
Table heights, 0-0.5m.
Initial position and pose of the robot.
Position and pose of objects.
High-level student policy. The high-level student policy follows the same randomization as the teacher policy, along with the following additional term:
Camera latency.
E
Motivations for IK-based Arm Control
In this work, we control the arm by solving the target joints with inverse kinematics, given the predicted target end-effector 6-DoF poses, rather than directly predicting the joints. Such design is motivated by lessons learned from the previous work, DeepWBC [8], which struggles to control the accurate end-effector position and orientation. To verify the advantage of our IK-based control predicting end-effector poses compared to predicting the target joints, we conduct a goal- reaching experiment, in which we randomly sample end-effector goals as in training, and end the episode once the goal is reached (the distance of end-effector and the goal is less than 2cm) or the episode time is out. We quantitatively evaluate the average error of position and orientation in Tab. 4. The results demonstrated clearly better precision compared to the target joint prediction solution in Fu et al. [8], which contributes lot training effective high-level policy complete complicated manipulation tasks.
F Sim-Real Visual Observation Comparison
We compare the trajectories and segmented depth images in simulation and rorl-world sampled by VBC, shown in Fig. 11.

---

Trajectory
Segmented Depth
Segmented Mask
Conci
REAL
Figure 11: Comparison between trajectories and segmented depth images in simulation and real-world sampled by VBC. Left: The real-world behaviors match well with the simulation, in- dicating less sim-to-real gap in our method. Right: Our TrackingSAM-basee working pipeline allows us to obtain the segmentation mask precisely during deployment (not the same object ob- served). The depth images pre-processed by hole-filling and clipping.
RGBD
Cameras
DepthColor
Imagee
Quadruped
High-Level Policy
Requeat
Goal Recaiver Process
Quadruped Rebot Sontrol Process
Arm Control Process
High-leeel Action
THZ
Commands
50HZ
800H2
Info
Computer Server
Computer 2, Client
The server-elient multi-processingg system design. Modules work asynchronously under different frequencies. The high-level planning policy (left) and the low-level control modules (right, controlling arm and legs) are deployed on different devices. To be more specific, the goal receiver requests the high-level policy to take the visual input and receive the new commands; the low-level policy sends joint position commands to the quadruped runs at 50Hz, given the input of high-level commands; the IK-based arm controller runs and send the joint position command the arm constantly at 800Hz, given the target end-effector pose. Therefore, the low-level policy and the arm controller keep utilizing the old commands control the robot until the high-level policy updates the commands (i.e., target robot velocities and end-effeecte pose).
Figure 12:
G Real World System
Hardware setup. As visualized in Fig. 4, we mount two cameras our robot platform, which are both RealSense D435 cameras. B1 itself has an onboard computer, and we also attached an additional onboard computer, Nvidia Jetson Orin, to the robot. The B1, Z1, and the Orin computer are all powered by Bl's onboard battery. The inference of the low-level policy and the control of the arm conducted at the onboard computer of B1, and that of the high-level vision-based planning policy is made on Nvidia Jetson Orin. It worth noting that the computation and the power of our robot system are fully onboard and, therefore, untethered by any PCs.
Software system design. During deployment, the low-level policy that controls the quadruped works at a frequency of 50Hz; in the meantime, the high-level policy runs at ~-8Hz, which is more than five times slower than the low-level policy; besides, the arm is required to work at a frequency
Approach
Status
Position Error (m) L
Orientation Error (rad)
Mean
Min
Mean
Joini prediction (18))
Standing
0.236 0.0
0.146 1 0.0
0.325 D.D
1.154 1 0.0
0.616 1 0.0
1.686 0.0
prediction-IK (curs)
0.047
0.027
0.0
0.850 0.01
2.269
Joint prediction (18I)
Walking
0.1141 0.0
0.182 = 0.0
0.558
0.0
2,015 = 0.0
1.165 = 0.03
3.198 0.01
EE prodiction-IK (ours)
0.0
0.008 = 0.0
0.050
0.332 0.02
0.071 0.03
0.743 0.08
Table 4: Precision of reaching goals, evaluated in simulation with 3 seeds, each over 100 episodes. We record the results while the robots standing and walking. In both status solution achieves the best performance.

---

High-Level
Low-Level
TrackingSAM KOT Init
Pre-Process
Model Inference
Model Inference
SAM chicking
AOT Tracking
Time
1.369 0.33
0.323
0.72
0.083 + 0.0
0.031 0.01
0.003 0.0
0.011
Stage
Initialization
Autonumous
Device
External NVIDIA Jetson Orin 64GB
Internal NVIDIA Jetson Xavier NX
Table 5: Component runtime analysis. All components are run on real robots, and the results are averaged over 100 times.
of more than 800Hz. Since these policies work on different computers, along with the quadruped and the arm should work under different frequencies, we design a client-server multi-processing system for this work, as shown Fig. 12. We set up three processes at the low-level policy side: a quadruped robot control process, an arm control process, and goal receiver process behaving as client. The high-level policy combines the vision inputs from the depth cameras and the information from the low-level request as the observation works server. Every 0.1 seconds, the goal receiver sends request with the necessary information to the server; the high-level policy makes the latest inference and sends the high-level action back to the robot; at the same time, the quadruped and the arm control processes gather the latest high-level commands to execute, while keep updating the necessary information for the goal receiver process.
Component runtime analysis. We record the runtime of all our components running in real-wordh experiments, shown in Tab. 5. Before an autonomous working pipeline, users are required to click on images to annotate the object be grasped with SAM, which is further processed by AOT to initialize the first reference frame to track. After initialization, the high-level process reads in new frame from two cameras, and pre-process them by resizing, hole-filling, and clipping; the images will be further processed by Tracking AM to track and provide segmentation masks; the masks and segmented images are concatenated as observations along with the proprioception states sent by the low-level process, and call the high-level model to predict the command. The high-level process finally sends the command back to the low-level process, which calls the low-level model predict the leg joints. Note that we did not record the time cost of process communication between the high- level and the low-level, each of which runs different devices and communicates through HTTP. During training, we gave a slightly larger randomization range than the recorded total time (also see in Appendix D). Overall, the high-level runtime limits the control frequency of the high-level policy to 6-8Hz. We also fix the low-level control frequency as 50Hz, even if the policy inference is rather fast.
H Details of Used Objects
For the simulation, we included 34 objects adapted from the YCB dataset [28]. According to the object shapes, we roughly divided them into categories: ball, long box, square box, bottle, cup, bowl and drill. In the real-world experiments, we choose 14 objects, including four irregular objects, six common objects, and four regular shapes. Objects are illustrated in Fig. 13
I Failure Analysis and Future Works
Although we showed great success picking various objects, we found several failure cases due to the limitations of the system: Compounding error. Our pipeline is hierarchical and requires each module to be coherent and precise make the robot work well. 2) In In-precise depth estimation. The RGBD camera does not reflect perfect depth estimation, especially on reflective objects. 3) Bad gripper. The gripper of the Unitree Z1 Arm is beak-like gripper instead of a parallel one, which tends to push objects away and hard for accurate manipulation. 4) Unstable vision inputs. The tracking SAM that is used to provide the mask of annotated objects, may lose track due to camera deviation and occlusion, or be confused when the selected objects have similar color to the other things. This is the most common cause leading to failure in our real-world experiments. In our cur- rent pipeline, before the robot autonomously picks up objects, must manually specify the target object in both cameras mounted on the robot. Thanks to the generalizability of trackingSAM,

---

Simuation
Square Box
Long Box
Ball
Bowl
Cup
Drill
Bottle
Real World
Regular Shapes
Common Objects
Irregular Shapes
Cuboid
Cube
Mouthwash
Bucket
Hammer
Broken Box
Rope
Ball
ToyPeach
Peanutbutter
Cup
Sand Bag
Broken Can
Duck
Figure 13: Examples of simulated training objects and real-world objects.
find that long the object is visible one camera (either the head or the wrist camera), the annotation mask can be propagated another view for automated picking, and free the robot from being specified in the between of the range of the two cameras, which improves the usability of our system. Since the requirements of annotation and object visibility are totally decoupled from our sim-to-real framework, in the future we can further improve and replace these modules with automated annotation algorithms and a search/exploration module to further improve the automa- masks and segmented depth
tion of the whole system. Our choice of visual excludes RGB images that provide color information. This limitation restricts the system from performing tasks that rely color differentiation, such distinguishing between Fanta and Sprite cans, sorting garbage by category, or pressing green button. During the training of our high-level policy, we utilized 33 distinct objects. While this approach demonstrates strong generalization capabilities on unseen objects in real-world scenarios, future work could enhance this by introducing randomized object sizes to further improve generalization. Future works should also consider upgrading hardware, like replacing a better gripper and RGBD cameras that provide better depth estimation; and simplify the working pipeline.
J Additional Details of TrackingSAM
TrackingSAM [24] provides very simple and intuitive way to provide segmentation masks which only requires minimal manual intervention. It invokes SAM [26] to generate an initial segmentation mask of objects before the tracking begins, where the user only needs to make a few clicks (as populate two instances of trackers
shown in Fig. 14). Given the initial segmentation, we use for images from the wrist camera and the head camera, respectively. The tracker used is AOT (Associating Objects with Transformers) [25], using the most lightweight AOT-Tiny variant, which is implemented based on MobileNet and can achieve nearly 10 fps on the onboard computer.

---

D
Figure 14: Segmented examples of the Tr tool.
10


Reviews:
[{"review_id": "JCVbW6Asvs", "reviewer": "Reviewer_FL2x", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 3, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "The authors propose VBC, a three-phase cycle that maximally utilizes sim-to-real to zero-shot transfer policies to hardware. The main idea revolves around learning a generalized low-level controller that takes high-level goals, and a high-level controller that provides goals to complete a task. The novelty of the approach is twofold: 1) distilling the high-level controller to a visual observation, emerging recovery behaviors, and improving results compared to a static height, and 2) training all tasks in sim and zero-shot transferring them to reality. The methodology does not constrain itself with real-world constraints to define tasks and collect data, therefore, theoretically scaling for many tasks. The downside of the methodology is sim training and hardware deployment, as the objects of interest need to be in the view of the robot. There are no ablations if that assumption is relaxed and the methodology would work. The authors have addressed these limitations. Though Figure 1 shows the methodology worked on different environments, it is not experimentally validated.", "summary_of_recommendation": "The paper presents a vision-based control framework for legged loco-manipulation. The hierarchical approach, successful sim-to-real transfer, and emergent retrying behavior are key strengths. VBC has potential for future research and development in whole-body control, but there are limitations in gripper design, depth estimation, and tracking robustness.", "technical_quality": 5, "clarity_of_presentation": 5, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 2, "questions_for_rebuttal": "* Line 110 => rad/s\n* Line 110 => “task-planning policy and further clipped into smaller ranges”. What is the reason for the discrepancy in size? To what extent is the size difference? The appendix does not contain the information in question.\n* Line 114 => yaw\n* Line 117 => While the environment extrinsic vector has been introduced, its application in conjunction with low-level hardware controllers remains unclear. Appendix C.1 fails to provide adequate explanation regarding this aspect. Given the crucial role of the extrinsic vector in sim-to-real transfer, it would be greatly appreciated if you could provide a detailed explanation or direct me to a specific reference that addresses this topic.\n* Line 128 => wondering whether the authors have experimented with the damped least squares. Provide a reference, e.g., https://www.cs.cmu.edu/~15464-s13/lectures/lecture6/iksurvey.pdf. \n* Line 165-169: “Moreover,  …”. The sim-to-real transfer strategy mentioned in this document requires further clarification. Kindly provide a more detailed explanation and ensure that any verbose information is moved to an appendix.\n* Line 182: Type after a_{t-1}.\n* Line 185: In the appendix, there is a reference to DAgger. Kindly provide that reference.\n\n\n* One of the fundamental presumptions in training and deployment is that the objects of interest must be visible within the viewport. If this assumption is not met, can the methodology still function effectively?\n* While the paper employs the term \"autonomous,\" the methodology is technically \"semi-autonomous.\" A human user must supply an augmentation mask and ensure the robot faces the object. To accurately reflect this, consider removing the word \"autonomous\" or adding the adjective \"semi-\" before it.\n* Fig. 6. Is the labeling of minimum and maximum values correct? The description mentions two baselines, but the results only show one baseline. This inconsistency should be addressed.\n* What are the primary factors contributing to the average simulation success rate hovering around 60%? What specific strategies or techniques can be implemented to enhance the simulation success rate beyond the current 60% average?\n* Table 1: please provide the standard error. \n* The experimental setup in the real world seems to be confined to a singular environment. Although Fig. 1 depicts activities in various environments, the results do not validate this. Thus, Fig. 1 can be considered misleading, particularly regarding the stringent requirements imposed on TrackingSAM and the intricate nature of task completion. \n* Is the implementation based solely on tabular rasa, or have other libraries been utilized? If other libraries have been incorporated, kindly provide the required attributions for acknowledgment. \n* Appendix B.4: please provide the references.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2d0QUQGvi9", "reviewer": "Reviewer_KPpy", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper proposes a bi-level approach for achieving fully-autonomous real world vision-based grasping on a quadruped manipulator platform, trained using RL in real.", "summary_of_recommendation": "The paper demonstrates novel fully-autonomous capabilities, contains interesting details across the stack for achieving good manipulation object generalization and Sim2Real, has a thorough evaluation in both simulation and real, and is well written.", "technical_quality": 5, "clarity_of_presentation": 5, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- 1. The work proposed to train a RL-based high-level policy, and have put efforts into tuning the RL set up for pick up task. Could the authors expand on the scalability of using RL for learning new tasks, versus other high-level policy approaches, such as trajectory planning or behavior cloning?\n- 2. Using object masks is a very robust way to transfer to real, since as segmentation and tracking tools become very robust. However, this design choice, while sufficient for object picking, may corrupt information that's crucial for other task. Could the authors expand their limitation section on the applicability of this design choice to other tasks?\n- 3. I find it interesting that IK worked well for the author's applications, given that DeepWBC tried and reported much worse numbers for the MPC+IK baseline, where one common failure mode in their case were body arm collisions. Is this sidestepped in the author's approach by a better hardware design (mounting the arm closer to the front), or replacing MPC with RL trained WBC (to avoid collisions in training)?", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ifYjyU4rP1", "reviewer": "Reviewer_xf7y", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": 4, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "This paper tackles the problem of learning a pick-and-place task for a quadrupedal robot with an arm. The task is accomplished with a high-level RL policy trained with a teacher-student method that outputs end-effector commands and body velocities, and a low-level policy that tracks these commands. Training is performed just in simulation.", "summary_of_recommendation": "I gave a high ranking because I think that the authors propose an interesting and novel method with convincing experimental validation.", "technical_quality": 4, "clarity_of_presentation": 4, "potential_impact": 4, "robotics_focus": 4, "quality_of_limitations": 3, "questions_for_rebuttal": "- Section 3.1: \"any given end-effector pose and body velocity\".\n- Section 3.1: It is not clear in which frame v_{lin}^{cmd} and omega_{lin}^{cmd} are expressed. Are they expressed in base frame? Why are they scalar values and not vectors? Please improve the description here.\n- Section 3.1 - observations: It is not clear what the variable z_t is. The paper says that it encodes environment physics, but it should be described better and more explicitly.\n- Section 3.1 - Paragraph starting with \"Besides, following the work of [4], etc...\": please improve the readability.\n- Do the \"encoded shape features\" mentioned in 3.2.1 coincide with the latent vector obtained by applying PointNet++ to the objects' pointclouds?\n- Paragraph 3.2.1: it seems that the authors do not randomize the shape of the objects to pick up. Why? It seems that this would be an important variable to randomize.\n- Section 3.3. Can the authors provide more details about the framework used to obtain segmentation masks in real-time (\"TrackingSAM\")? Is there a publication regarding this? [24] is a PhD thesis. Also, how reliable is this framework? Does it output segmentation masks which are 100% correct?\n- Section 3.3: The authors state that all the computations happen on the robot on-board computers, which is quite impressive. Even though evaluating the networks used for the low-level and high-level policies is probably computationally feasible, I imagine that the model used to compute segmentation masks might be quite large. Can you comment about the computation time of all these modules?\n- Section 4.1: a failure is considered if the object is not picked up in \"150 high-level steps\". What does this corresponds to in seconds?\n- The authors state that the objects are manually annotated using trackingSAM. Can the authors specify why is this annotation needed and how is it performed? Does the user need to draw a bounding box around the object?\n- Supplementary material: In some videos, the authors show the RGB image, masked depth, and segmentation mask from the two camera views. For the sake of clarity, I would suggest to show only the masked depth and segmentation mask, because the RGB images are not used by the policy.", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The submission presents a new sim2real method for controlling locomotion and manipulation in parallel, leading to strong results and a detailed evaluation. 

The submission effectively breaks down loco-manipulation problems into sub-problems, uses visual observations (segmented depth images), demonstrates successful deployment of the trained policy on a physical robot without requiring real-world data, robust retrying grasping attempts when initial attempts fail, enhancing overall success rates, is rigorously tested through broad simulation experiments and real-world tests on unseen objects.

While effectively addressing a pick-up task, applicability to more intricate manipulation tasks remains unexplored. The reliance on a beak-like gripper restricts the range of objects that can be manipulated, necessitating pushing instead of grasping. Performance heavily relies on accurate depth estimation. More details are needed about the model used for object annotation and segmentation mask generation, computation time of each module. The paper's readability is limited by long paragraphs.

The paper has improved during the rebuttal via detailed communication between reviewers and authors. Based on the above the paper is recommended for acceptance. Please follow up on the remaining open points for a potential camera ready version.
Decision: Accept