UMI-on-Legs: Making Manipulation Policies Mobile with Manipulation-Centric Whole-body Controllers

Authors: huy  ha
yihuai  gao
zipeng  fu
jie  tan
shuran  song
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-05 20:16:53

Abstract: We introduce UMI-on-Legs, a new framework that combines real-world and simulation data for quadruped manipulation systems.  We scale task-centric data collection in the real world using a handheld gripper (UMI), providing a cheap way to demonstrate task-relevant manipulation skills without a robot. Simultaneously, we scale robot-centric data in simulation by training a whole-body controller.  The interface between these two policies are end-effector trajectories in the task-frame, which are inferred by the manipulation policy and passed to the whole-body controller for tracking. We evaluate UMI-on-Legs on prehensile, non-prehensile, and dynamic manipulation tasks, and report over 70% success rate for all tasks. Lastly, we also demonstrate the zero-shot cross-embodiment deployment of a pre-trained manipulation policy checkpoint from a prior work, originally intended for a fixed-base robot arm, on our quadruped system. We believe this framework provides a scalable path towards learning expressive manipulation skills on dynamic robot embodiments.

Main paper: 
 introduction: 
 Collecting robot data in simulation versus in the real world are two distinct pathways for scaling up robot learning approaches, each with its challenges. Real-world robot teleoperation allows for direct human demonstration of new tasks. However, it is often bottlenecked by the robot's physical hardware -in addition to the cost and safety concerns, the involvement of robot hardware usually makes the collected data robot-specific (e.g., uses body commands) or even irrelevant whenever the robot's physical form changes. In contrast, simulation provides safe exploration and infinite resets for any new robot design. However, it is often bottlenecked by the task diversity -accurately simulating diverse objects and their dynamics and defining all tasks and their associated rewards remains a significant challenge.In this work, we introduce UMI-on-Legs, a quadruped manipulation system that demonstrates an alternative way to combine real-world and simulation data:• Scaling task-centric data in the real world without a robot: We propose using a hand-held gripper (i.e., UMI [1] ) to demonstrate task-centric data without requiring robot hardware. We model this manipulation data using a diffusion policy [2] that is agnostic to the robot embodiment beyond the end-effector. By allowing human operators to collect the data directly, we can easily capture a diverse set of challenging tasks, bypassing the need for real-world robot trial-and-error.• Scaling robot-centric data in simulation without task simulations: We train a whole-body controller (WBC) in massively parallelized simulation [3] [4] [5] [6] [7] , whose sim2real transfer to a wide range of terrain conditions [7] [8] [9] is a well-studied problem. By training control policies to track end-effector trajectories instead of interacting with objects, we can bypass the difficulty of setting up realistic manipulation assets, engineering task rewards, and modeling object dynamics.In-the-wild Manipulation, on Legs. Featuring a self-contained iPhone odometry and onboard power/ compute, our mobile manipulation system is complete for in-the-wild manipulation.A key design choice in the framework is the interface between the two policies: it must be simple enough for nonexperts to use, yet expressive enough for complex manipulation tasks. Most prior legged manipulation systems use body velocity commands with single-step, body-frame end-effector targets [8, [10] [11] [12] , which are embodiment-specific and insufficient for dynamic manipulation. We propose using endeffector trajectories in the world-frame, where the manipulation policy infers these from gripper-centric image observations. The whole-body controller then converts these trajectories into embodiment-specific controls, enabling cross-embodiment generalization of the manipulation policy and providing preview for anticipating motions for the controller.To support system deployment, we present an accessible, real-time odometry solution, based on the Apple's ARKit. Instead of assuming a tightly integrated and precisely calibrated/synchronized system [13] [14] [15] , our odometry approach takes the self-contained, compact form factor of an iPhone.We tested UMI-on-Legs with complex tasks requiring prehensile actions (pick & place), non-prehensile whole-body actions (weight pushing), and dynamic actions (tossing) in challenging scenarios with on-board sensing and in-the-wild generalization. Thanks to our interface design, our asynchronous bi-level policy formulation robustly executes these tasks, achieving over 70% on all tasks. In summary, the main contributions of this work are:• UMI-on-Legs, a framework for combining real-world and simulation data, which enables learning expressive manipulation skills on mobile robots using no real-world robot data. Namely, real-world data collected without any robots [1] can be distilled into manipulation skills for different mobile robot platforms using robot-specific controllers trained entirely in simulation.• Within this framework, we propose a manipulation-centric whole-body controller, using an end-effector trajectory interface. This simple interface enables zero-shot cross-embodiment deployment of existing manipulation policies while being expressive enough to represent complex manipulation skills.• A real-world deployment system featuring a real-time, robust, and accessible odometry approach for in-the-wild task-space tracking, addressing a common bottleneck in achieving fast and precise manipulation in mobile manipulation systems. 
related work: 
 Mobile Pick-and-Place Systems. Prior works have shown that pick-and-place quadrupeds trained in simulation can be deployed in real-world environments using object detectors for visual domain transfer [12, 16] . Wheeled systems demonstrate more complex manipulation tasks using carefully engineered primitives [17] [18] [19] [20] [21] , enabling perception and planning [17, 20, 22] with foundation models [23] [24] [25] [26] , online learning [18, 19] , or trajectory optimization [21] . Recently, Mobile-ALOHA [27] proposed a wheeled-bimanual platform for demonstration collection. However, these systems rely on quasi-static manipulation or heavy bases with a low center of mass.Learning-based Controllers for Robot-specific Teleoperation. Reinforcement learning [28] (RL) with massively parallel simulators [3, 7] is the dominant approach for learning dynamic locomotion [6, [29] [30] [31] and manipulation skills [32] [33] [34] in legged robots. While domain adaptation methods [7] [8] [9] allow deployment in different environments, mobile-manipulation on quadrupeds still typically requires human teleoperation/AprilTags [35] during deployment for teleoperation [8, 10, 11] . Fully-autonomous deployment is limited due to the high cost of collecting real world robot data, which depends on robot-specific commands and requires the robot's physical presence. Recently, He et al. combines behavior cloning with an RL controller, but their approach relies on robot-specific demonstrations generated heuristically in simulation, limiting scalability, expressivity, and usability. Furthermore, their system is constrained by a tethered setup with external cameras for April Tag tracking.Cross-embodiment Manipulation. In pursuit of robust and generalizable manipulation, behavior cloning works has innovated on policy architectures [2, 37, 38] , sensor placements [37] , action spaces [38] , and data considerations like quality, size, and cost [37, [39] [40] [41] [42] [43] . Key design choices have emerged, including pre-trained visual encoders [1, 25, 39] , diffusion-based action decoding [2, 39, 44] , and end-effector sequence prediction [1, 38, 44] . Recent work [1, 45] shows zero-shot cross-embodiment visuo-motor policy transfer using only ego-centric visual inputs. However, by predicting target pose sequences, these approaches assume precise low-level controllers for tracking.3 Method: Universal Manipulation Interface on Legged Robots UMI-on-Legs (Fig 3 ) comprises of two main components: (1) a high-level diffusion-based manipulation policy [2] which takes as input wrist-mounted camera views and outputs sequences of end-effector pose targets into the future in the camera frame, and (2) a low-level whole-body controller which tracks end-effector pose targets by outputting joint position targets for both the legs and arm. We train the manipulation policy using data collected in the real world using UMI [1] , a hand-held gripper data collection device and train the WBC entirely in simulation using massively parallelized simulators [3] .Our choice of using world-frame end-effector trajectories as the interface has the following benefits: • Intuitive demonstration: Using end-effector trajectories instead of robot-specific low-level actions, we allow intuitive task demonstration by non-expert users using hand-held devices like UMI [1] . • High-level intention from preview horizon: With the preview horizon of future targets, the whole body controller can anticipate upcoming movements. For instance, the robots should brace accordingly if a high-velocity toss is coming up. Meanwhile, if the target is moving within the arm's reach range, the body should lean instead of taking a step, which risks shaking the end effector. • Precise and stable manipulation in the world-frame: Unlike most of legged manipulation systems that uses body-frame tracking, our controller tracks action in the task-space (Fig 4 ) that is persistent regardless of base movement, enabling precise and stable manipulation. • Asynchronous multi-frequency execution: The interface defines a natural inference hierarchy that allows a low-frequency manipulation policy (1-5Hz) to coordinate with a high-frequency low-level controller (50Hz) for handling drastically different sensor and inference latencies. • Compatible with any trajectory-based manipulation policy: Our interface supports the plug-and-play of any trajectory-based manipulation policy [1, 38, 39, [43] [44] [45] . With the rise of policies trained on diverse datasets [39-41, 45, 46] , our manipulation-centric WBC can accelerate the porting of existing "table-top" manipulation skills to "mobile" manipulation. 
manipulation policy with behavior cloning: 
 Following the default configuration from Chi et al., we use a U-Net [47] architecture diffusion policy [2] (Fig 3a ) with the DDIM scheduler [48] and a pre-trained CLIP vision encoder [25] to map from a context of visual observations to end-effector pose and gripper command trajectories. We use a longer action horizon of 64 to provide more information into the future for the low-level controller. For our cup rearrangement task, we directly use UMI [1] 's cup rearrangement checkpoint. For the pushing and tossing task, we collect data and train diffusion policies from scratch. Tasked with processing high-dimensional visual data, this network runs much slower than the controller. However, its output pose trajectory can be densely interpolated to get inputs for the controller, which outputs position targets for 12 leg joints and 6 arm joints. Meanwhile, its output gripper command trajectory can be interpolated can fed directly into the gripper joint. Our system takes as input RGB images from a GoPro and infers a camera-frame end-effector trajectory using a diffusion policy (a), trained using real-world UMI demonstrations. We transform this trajectory into the task-space, and use it as the interface to the WBC. This controller (c) outputs joint position targets at 50Hz, which PD controllers subsequently tracks.To track end-effector trajectories predicted from the manipulation policy, we propose to train a whole body controller with reinforcement learning in simulation to infer arm and leg joint targets. Notably, setting up a simulation to track these manipulation end-effector trajectories does not require setting up the manipulation task and environment. This design greatly alleviates a key bottleneck in using simulation data.World-frame Trajectory Tracking of Manipulation Trajectories. Prior works [8, [10] [11] [12] train WBCs by sampling end-effector poses in the body-frame, simplifying policy optimization but failing to teach whole-body coordination to compensate for base movements (Fig 4b ). This limitation is more pronounced when the arm's momentum affects the base, especially with lightweight systems or dynamic movements. In contrast, we train our controller to track trajectories in the world-frame (Fig 4a ), teaching the robot to maintain its gripper pose by compensating for body movements. We use task trajectories from UMI [1] , which are more representative of real manipulation tasks. In § 4, we show our controller generalizes across tasks without sacrificing tracking performance. Deployment details are in § 3.3.Observation space. The observation space includes robots' 18 joint positions and velocities, the base orientation and angular velocity, the previous action, and the end-effector trajectory inferred by the manipulation policy. We represent end effector pose using a 3D vector for the position and the 6D rotation representation [49] . We densely sample target poses from -60ms to 60ms at 20ms interval relative to the current time, which informs the controller of current velocity and acceleration. In addition, we include the target 1000ms into the future, which helps the controller prepare to take foot steps if necessary.Rewards. The task objective rewards the policy for minimizing the position error ϵ pos and orientation error ϵ orn to the target pose as follows: exp(-( ϵpos σpos + ϵorn σorn )), where σ represents scaling terms based on precision requirements. This formulation entangles position and orientation, resulting in more ideal behavior compared to treating them separately, which often leads to high precision in only one aspect. We also found that implementing a σ curriculum for both position and orientation is essential for enabling exploration early on while ensuring high precision with further training. In addition, we incorporate common regularizations [7] [8] [9] 12] , as detailed in the supplementary material.Policy network architecture. We train a multi-layer perceptron (MLP) controller which maps from observations to target joint positions for both the legs (12 DOF) and the arm (6 DOF) (Fig 3c ). A forward pass for this controller takes ≈0.06ms on the robot's on-board CPU, and is called at 50Hz during deployment. The joint position targets are tracked using separate PD controllers for the legs and the arm.Robot system setup. The robot system (Fig. 2 ) consists of a 12-DOF Unitree Go2 quadruped robot and a 6-DOF ARX5 robot arm, both powered by the battery of Go2. We customize the ARX5 arm with Finray grippers and a GoPro to match the UMI gripper [1] . Our whole-body controller runs on the Go2's Jetson while our diffusion policy inference runs on a separate desktop's RTX 4090 over an internet connection. We mount an iPhone for pose estimation and connect it to the Jetson through an Ethernet cable.Sim2Real Transfer. To improve robustness, we apply random push forces and randomize joint friction, damping, contact frictions, body/arm masses, and centers of mass. Domain randomization of the end-effector's mass and center of mass simulates the effects of grasped objects without explicitly modeling them. Modeling a Accessible real-time odometry. The lack of real-time onboard task-space tracking is a common limitation in prior quadruped manipulation works. By assuming external tracking with motion capture [32] and/or AprilTags [8, 11, 36] , their systems can't be deployed fully-autonomously in the wild. In our system, we address this drawback with an iPhone, mounted on the robot's base. Our choice of the rear mounting site avoids adding extra weight on the robot arm, prevents arm-phone collisions, and minimizes motion blur and visual occlusion. In contrast to many existing robust, real-time odometry solutions [13] [14] [15] , our odometry solution has a self-contained, compact form-factor and uses only ubiquitous consumer electronic devices.Asynchronous Transform Book-keeping. During deployment, the manipulation policy, WBC, and iPhone pose estimator run asynchronously. Let t be the current timestep and T the action horizon of the manipulation policy. The iPhone ARSession's frame serves as the world-frame, initialized when the iOS app opens. For each camera-frame action trajectory a t+T from the manipulation policy, we transform it into the world-frame using the camera-pose recorded at time t. This pose is derived from the iPhone's timestamped world-frame pose and joint state forward kinematics. For all timesteps t ′ ∈[t,t+T ], the controller densely interpolates the end-effector trajectory then transform it into the gripper's frame for controller observations. 
experiments: 
 We designed a series of experiments in simulation and the real world to validate our key design decisions, which aims to answer the following key questions:• Capability. Is our system UMI-on-Legs able to learn complex and challenging manipulation skills such as whole-body dynamics actions (e.g., tossing)? • Robustness. Can our whole-body controller handle unexpected external perturbations and object dynamics, encountered during manipulation? • Scalability. Can we use cross-embodiment manipulation data collected entirely absent of the quadruped?Metrics. In simulation, we report average position error (cm), orientation error (radians), survival (%), and electrical power usage (kW) over 500 episodes. Here, survival is 1 for the episode only if the policy lasts for 17 seconds without a terminal collision (any robot link contact that is not the robot's feet or gripper). In real-world experiments, we report average success rate over 20 episodes unless specified otherwise.Training Data. We train a WBC and a manipulation policy for each task on only data from those tasks: 100 trajectories for tossing, 200 for pushing, and 1000 for cup rearrangement. An exception is the cup task manipulation policy, where we directly use the public checkpoint from UMI [1] , and only train a WBC for this task. To investigate the effects of adding more task trajectories, we finetuned "Ours (Tossing)" on 300 trajectories, 100 from tossing, pushing, and cup rearrangement each, leading to "Ours (Unified)".Comparisons. We ablate our design choices for the WBC in Table 1 . Starting with our approach, Ours (Tossing), we remove preview (i.e., trajectory observations), world-frame tracking formulation, and tossing strategy, despite limited arm strength and body inertia, in three stages. First, as the arm accelerates forward, the back legs lift, creating a leap and toss motion (1a, green). To avoid falling forward, the robot tucks its arms and legs inwards (2a, orange), generating backward torque that shifts the front legs' contact points ahead of its center of mass for a soft landing (3a). In contrast, the controller without preview information (bottom row) attempts to leap after the fast target acceleration but lacks foresight, resulting in a dropped ball. Check out our robot videos at our website! manipulation trajectory training trajectories. Removing all three simultaneously leads to a baseline similar to DeepWBC [8] . Finally, to investigate the efficacy of our iPhone-based odometry system, we compare against an OptiTrack motion capture system in all our real-world experiments, which serves as the oracle odometry. For real-world tossing, we also show the no-preview baseline's behavior in real (Fig. 5 ).Our first experiment aims at validating the system's capability, i.e., whether our policy interface can capture complex manipulation skills. For this, we chose a particularly challenging task -dynamic tossing. The robot, with the center of its body 150cm away from and facing the target bin, must toss pre-grasped balls towards a target bin 90cm away, where a trial is successful if it lands within 40cm of bin's center. Task difficulties. Most of existing tossing systems assume a carefully designed primitive [17, 50] . Recently, Chi et al. [1] demonstrated that learning from hand-held gripper demonstrations also results in capable tossing skills, but has only been shown to work with analytical controllers on afixed-based arm. For legged systems, reliable tossing depends on the dynamic body coordination. Thus, in order to achieve a successful toss, the controller must discover the right stance, utilize power from all of its degrees of freedom, and learn the coordination required to achieve precise tracking throughout the dynamic tossing motion.Dynamic tossing requires dynamic whole-body coordination. Through learning to track end-effector trajectories, our whole-body controller discovered a whole-body tossing strategy (Fig 5 ). The strategy involved dynamic whole-body coordination, involving power from all joints during the toss, momentarily balancing on one foot/two feet, and effective usage of body mass inertia to regain balance. Overall, our system achieves 75% and 70% when using motion capture and iphone odometry, respectively.Effects of the interface design. From our ablations and baseline comparisons (Table 1 ), we observed that training on world-frame manipulation trajectories was crucial -removing either world-frame tracking or manipulation trajectories ((-) Task-space and (-) UMI Traj) significantly hurts position and/or orientation precision. Meanwhile, removing trajectory information from the WBC's observation prevents the controller from anticipating future motions, resulting in dangerous reaching behaviors (-5.4% survival) and more jittery actions (+130W ) compared to our approach, as demonstrated in Fig 5 . Improved tracking from diverse UMI trajectories. When finetuned on additional trajectories from pushing and tossing, we observe that our tossing-pretrained WBC not only retained its tracking abilities, but controller continuously tries to push forwards. This leads to a wiggling motion along with increased pushing force until static friction breaks into a dynamic slide, at which point the robot catches itself by dropping its front feet forwards. We observed this push-and-catch strategy while the robot pushes both 10lbs and 20lbs (b), despite the latter succumbing to static friction more often. We also report comparable performance with the motion capture oracle (c). also learned safer behavior. Quantitatively, its collision rate (1 -survival) improves over the tossing only WBC by 25% (from 1.6% to 1.2%, Table 1 ) while reducing its power usage by 100W .Failure analysis. Despite the highly-dynamic nature of this motion, the controller manages safe landings during all motion capture evaluation trials. To further investigate whether tossing misses were due to imprecise WBCs or noisy end-effector trajectories, we tested our controller with calibrated tossing trajectory replay with motion capture, which effectively acts as an "oracle" manipulation policy, and report a +10% improvement (Fig. 5 ). We hypothesize that scaling up tossing data collection further could close this gap.Our second task stress tests the controller's ability to handle unexpected and unseen external dynamics in zero-shot fashion. To evaluate this capability we studied kettlebell pushing (Fig 6 ), where goal of this task is to push a 10lbs kettlebell, such that it slides upright into a goal region. Between each episode, the kettlebell's distance from its goal is varied within the range [80cm,120cm], and the robot starts off with its center 60cm behind the kettlebell with its grippers grasping the kettlebell's handle.Task Difficulties. Although the end effector motion involved may be simple, the whole body controller need to adapt to significantly changed dynamic models. In order to be successful, the WBC must be robust to feet slippage, overcome static friction, and adapt to when static friction suddenly breaks into a slide. In addition, small imprecision in the pushing contact point could lead to the kettlebell toppling over.Robustness to large disturbances. To our surprise, even with such a large disturbance, our controller was able to maintain balance and finish the task, without ever training on simulated forces or weights. Our controller completes the task with 95% and 90% success rates for motion capture and iPhone deployment respectively. We observed that as soon as the WBC observes a larger position tracking error caused by the weight, it changes its strategy to lean forward in order to exert more force -a strategy which would cause the robot to fall forwards in absence of the kettlebell. By sending only end effector trajectories without any base velocity, the controller can also move its legs forward to reach faraway target poses. In addition, we also tested the policy with a 20lbs kettlebell, and observed 4 successful episodes out of 5.Failure analysis. The most common failure case in this task was due to the kettlebell toppling sideways. In addition, we observed that our hardware system often overheats during this task's evaluation, and requires frequent cool-down breaks. Besides obvious hardware improvement, we hypothesize training with simulated forces [10] could lead to more elegant, energy efficient pushing behavior.In our third experiment, we validate the feasibility of deploying a pre-trained manipulation policy from prior work [1] with our WBC, without fine-tuning, demonstrating zero-shot generalization to an unseen embodiment * . We evaluate this on the In-the-wild Cup Rearrangement Task, where the robot must place an espresso cup on a saucer with the handle facing left. The cup and saucer are randomly positioned within ), the In-the-wild cup rearrangement policy from Chi et al. was able to achieve 80-85% success rates zero-shot on our quadruped system with our learned whole-body controller when tested on hard ground (center figure). In out-of-distribution (OOD) terrain settings (outdoor environments with deformable/unstable terrain), our system with iPhone tracking achieves 60% success rate. a 20cm radius, with random cup orientation. The robot starts 30cm from the table edge, facing the table. Success is defined as the cup being upright on the saucer, with the handle within ±15 • of left. This task is always performed in an unseen environment for both the manipulation policy and WBC.Task difficulties. The key challenge for this experiment is to effectively leverage cross-embodiment manipulation policy. In the original UMI paper [1] , the policy was trained on data collected by humans, and intended for industrial robot arms (e.g., UR5 and Franka) with a fixed-base and analytical controller. In this experiment, we are directly deploying this pre-trained policy on an 18 DoF legged manipulator. Moreover, the intrinsic difficulty of the pushing and rearranging task also tests the system ability to perform precise 6 DoF movements, including prehensile and non-prehensile manipulation. This kind of precise and gentle actions have been traditional difficult for low-cost legged manipulation systems.Results and Findings. Without fine-tuning the manipulation policy, our system achieved success rates of 85% and 80% using motion capture and iPhone odometry on hard flat ground, respectively, despite the cross-embodiment gap. The manipulation policy, originally designed for a UR5e system with a larger kinematic reach than our ARX5 arm, often predicts action sequences that span a larger workspace. To accommodate this, our system dynamically tilts and leans its base to increase reach and counterbalance the arm. We evaluated our system with four different cups in five locations, including various indoor and outdoor terrains like soft carpet and unstable surfaces, which were out-of-distribution (OOD) for the WBC, trained only on hard flat ground. Despite this, we achieved a 60% success rate (27/45 trials), with 72% (13/18) of failures attributed to manipulation errors (e.g., dropping or pushing the cup out of view) and 28% (5/18) due to controller shakes on OOD terrain.Although UMI-on-Legs has demonstrated various manipulation skills, several limitations remain that future work could address. First, our system currently supports only gripper-based manipulation. A promising direction is to extend our interface to facilitate whole-body manipulation [32, 51, 52] . Second, while our embodiment-agnostic interface is scalable, incorporating embodiment-specific constraints into the high-level manipulation policy is crucial. Lastly, for a complete mobile manipulation platform, our system could benefit from features like collision avoidance [53] , force feedback, and force-based control [10] . 
conclusion: 
 We introduce UMI-on-Legs, a framework that combines real-world human demonstrations and simulationbased reinforcement learning. With world-frame end-effector trajectories as the interface, our formulation allows intuitive demonstrations from hand-held devices yet is expressive enough to represent complex manipulation skills such as dynamic tossing and pushing, and precise pick and place.Although, as community, we've yet to have a unanimous foundation manipulation policy, we believe our framework is an important step towards enabling the transfer of such policies to different robot platforms. Looking forward, we're optimistic that tackling limitations in our interface ( § 4.4) could take us even closer towards a future where separate people can develop ever more general visuo-motor policies and robust WBCs, knowing that they can be plugged-and-played together in the end. 
things that did not work: 
 In this section, we discuss things we tried, but did not improve the performance or caused other issues. We hope that through openly discussing our unsuccessful attempts, the community could gain a more complete understanding of the project, learn from our mistakes, and improve upon our attempts. To complement this section, we've also included robot failure videos on our website.We tried to include privileged information (kp, kd, friction, damping, ground truth poses, base velocities, etc.) to train a privileged policy and distill it through supervised learning and online regularization. However, it didn't provide performance boost and introduce instability when we include observation history longer than 1 step. This could be because that the Python ROS2 timer is imprecise, thus the observation history can easily be out of distribution given the wrong history timestamps. This was less of an issue on the Unitree A1 platform, which ships with a real-time kernel.To train diffusion policies for grasping and tossing, we collected 500 episodes of human demonstration. However in the grasping phase, the small shakes from the controller results in distributional drift in visuo-motor manipulation policy. We hypothesize that robust, fully-autonomous grasping and tossing could be achieved with more data.Reliability for Fully-untethered Deployment.Although the system is capable of various tasks in the real world without any external cables, the reliability is in general not high enough. We summarize some hardware and system level challenges we encountered as follows. Given a small base-to-arm weight ratio, the joints of the dog are likely to overheat in 10 to 30 minutes according to the robot posture and then goes to an error mode. Therefore, we needed to frequently cool down the motors and fine-tune the policy to a more energy-efficient posture. On the power front, the battery supplies different voltages depending on how full it is. When fully charged, the voltage is too high for the arm, and the setup requires a voltage adaptor. When closer to empty, the policy's behavior is more dampened. long. We optimize the run time of all callback functions and detach the callback functions that take too long to other ROS2 nodes. This allows the joint observation update to run closer to 200Hz and policy inference closer to 50Hz. A C++ implementation with more precise timer and lower latency can achieve better performance given the same checkpoint and evaluation setup. 
safety: 
 We observed that directly deploying the whole-body controller checkpoints in the simulation section led to some safety issues and addressed them by fine-tuning the checkpoints with more conservative reward schemes:• Shaking: To reduce the shaking and oscillation behavior of the robot, we increased the action rate regularization. The controller will get less reward if the predicted action is farther away from the previous action. We also disabled the on board lidar that introduces shaking behavior.• Overheat Shutdowns: The calf joint of the robot uses a linkage configuration rather than directly applies the output torque. Therefore, the required output torque of the motor is higher than the other joints, leading to frequent overheating and emergency shutdown. We observed that increased torque regularization during training allows the controller to run up to 30 minutes continuously ‡• Unsafe Configurations: In the simulator, the controller is likely to twist the legs to achieve higher precision with less movements, but this makes it unsafe to deploy in the real world. We added reward terms to regularize the body to a more balanced pose, so that the center of mass can roughly stay in the center. 
manipulation policy: 
 In this section, we report hyperparameters used for training our manipulation diffusion policy. Visual observation and proprioception horizon means how many history image and robot states (with 0.1s interval) are used as input to the policy. Output Steps means the action length of the diffusion policy output. Execution Steps and Execution Frequency indicate how many steps of the diffusion policy output is actually executed in the real-world robot. To enable more stable end-effector trajectory updates, we add linear interpolation between the executed trajectory and the newly-updated trajectory for a duration of "Trajectory Update Smoothing Time".We increased the inference and execution steps for a smoother toss, since the trajectory is less likely to update during the dynamic toss. We also increased the execution frequency to get farther toss. Please refer to Table 2 for detailed parameters.Since the trajectory motion is much simpler, we collected fewer human demonstrations for the diffusion policy. We increased the "Trajectory Update Smoothing Time" to prevent heavy shakes due to the trajectory update. Please refer to Table 2 We evaluated the whole-body controller on the cup rearrangement task in the wild. We tested it in some challenging scenarios including unstable terrain (grass, dirt, collapsible table), direct sunlight which made the robot easily overheated, and unseen tables and cups to show the generalizability of the diffusion policy. Please checkout the supplementary video for more details.During RL training of the whole body controller, we include the following reward terms:• Joint Limit, Joint Acceleration, Joint Torque, Root Height, Collision, Action Rate: We use the same definition as prior works [7, 8] .• Body-EE Alignment: We regularize joint 0 and joint 3 of the arm (the joints that mostly affect the yaw angle of the gripper) to stay close to their initial pose. This allows the arm to be aligned with the body.• Even Mass Distribution: We regularize the standard deviation of the force of the four feet to be lower. The motors are less likely to overheat if the body mass is distributed more evenly on the four legs.of the three tasks studied. Power usage report is electrical power usage based on real hardware's voltages, manufacturers' reported torque constants, and the simulation's motor torques.11 Misc.Cost of the system. We report the cost of our entire robot system based on the market price in  
: 
 Fig 1 : UMI-on-Legs. We achieve fully autonomous, expressive, real-world manipulation skills on quadrupeds by combining real-world demonstrations using hand-held grippers (left) and simulation-trained whole-body controllers (right). Our framework allows the porting of existing "table-top" manipulation policies to mobile manipulation while enhancing mobility and power from the quadruped's legs. 
 
 Appendix: Training * Zero-shot refers to the manipulation policy, which is not trained on the target embodiment 


Decision: **Pre-rebuttal**

This paper receives a mixed review from three reviewers.

The reviews acknowledge the following strengths:
- Effectiveness of the whole-body controller trained with RL (**ZZoE**, **H1aK**).
- Showcasing a way of leveraging hand-held gripper data for quadruped manipulator (**ZZoE**, **H1aK**).
- Versatility of hardware setup (**H1aK**).
- Clarity of presentation (**ZZoE**, **H1aK**).
- Potential infrastructure contribution if open sourced (**ZZoE**).

The following weaknesses are mentioned:
- Lack of details on training (**ZZoE**, **uNw1**).
- Insufficient demonstration on tasks that require a legged base (**ZZoE**).
- Concern and confusion on the "zero-shot cross-embodiment" claim (**ZZoE**, **uNw1**).
- Limited technical novelty (**H1aK**).
- Lacking on the evaluated task number and variation (**H1aK**, **uNw1**).

---
**Post-rebuttal**

The paper initially received two weak accept and one weak reject. The reviewer gave the weak reject acknowledged the merit of the rebuttal and raised the assessment to weak accept. This leads to a positive consensus among three reviewers.

Overall the paper has delivered a novel and interesting framework for combining quadruped whole-body control with task frame human demonstrations. AC agrees with the assessments of the reviewers and thereby recommends accept.

Reviews: review: title: title: Review
paper_summary: paper_summary: This paper proposes UMI-on-legs which integrates the Universal Manipulation Interface (UMI) from prior work with a quadruped robot base. This is enabled by substituting the task-space end-effector controllers used in UMI for a RL whole-body controller that controls the legs and arm jointly to track task-space manipulator commands.
main_review: main_review: Strengths
- The task-frame tracking formulation for the RL controller is novel, intuitive, and demonstrated to be helpful for manipulation.
- The ability to use cross-embodiment data is valuable since there is much more demonstration data available for fixed-base manipulators than for legged manipulators.
- The infrastructure contributions such as ARKit VIO app and precisely calibrated URDF for the system will be valuable tools for diverse legged manipulation applications if open sourced.
- The writing is very clear and evaluations are thorough.

Weaknesses
- I couldn't find some details about the trajectories used during RL policy training.
- It’s not quantitatively evaluated to what extent, and in what aspects, the demonstrated tasks take advantage of the base having legs.
- As acknowledged in the limitations, it's not clear how to propagate embodiment-specific constraints back to the high-level manipulation policy, which would be necessary to ensure the scalability of the decoupled plug-and-play approach.
questions: questions: “To provide the controller with relevant reference trajectories, we use trajectories collected with UMI [1].” .. “removing either task-frame tracking or manipulation trajectories ((-) Task-space and (-) UMI Traj) significantly hurts position and/or orientation precision.” I would like more details about the trajectories used to train the RL controller. How many trajectories are there and what kinds of tasks do they represent? E.g. are the trajectories in 4.2 and 4.3 also seen during training or are they out-of-distribution for the RL policy? If I wanted to add a new task with UMI, would it also be important to retrain the WBC on some task-related trajectories, or does the policy learn a very general tracking capability?

It would be really cool to see the system accomplish tasks that specifically require legs. For example, can the robot walk over some uneven terrain (inaccessible to a wheeled robot) before performing the manipulation, while carrying the UMI payload? This could even be done by switching to Unitree’s default controller, if it’s hard to include non-flat terrain in the RL training.

Further thoughts on this topic: “For legged systems, reliable tossing depends on the body placement and the tossing motion. Thus, in order to achieve a successful toss, the controller must discover the right stance, utilize power from all of its degrees of freedom, and learn the dynamic whole-body coordination required to achieve precise tracking throughout the dynamic tossing motion.” Can this be quantified? For example, you could consider training an ablation policy (a) where the robot’s legs have fixed targets and the action space only controls the arm joints. Or you could consider an ablation (b) where the legs have fixed targets and the gripper is controlled with IK. I think it’s reasonable to expect (b) will not be performant because the base will shake — but this would be nice to show. I think (a) could actually perform better because the policy can still learn to compensate the arm’s commands for the motion of the base. If the full system delivers the best performance, the gap would quantify how much the legs are really contributing power or stabilization towards the task, vs. primarily acting as a complication to the dynamics that needs to be overcome. I think both are suggested in different parts of the paper and we don't really know which is being evidenced here.

"With an expressive embodiment-agnostic interface in place, we can separately develop ever more general visuo-motor policies and robust WBCs, knowing that they can be plugged-and-played together in the end." As mentioned in the limitations section, there is the outstanding problem of making the high-level policy aware of the embodiment-specific constraints. With this problem unsolved, can we really claim yet that developing more general versions of these components will lead to a plug-and-play solution?
limitations: limitations: 3
review_summary: review_summary: The paper makes a clear contribution to the field of legged mobile manipulation. It would be enhanced by clarifying points of the training procedure and ablating claims about the role of the legged base in the realized behaviors.
title: title: A practical framework well designed and well communicated
paper_summary: paper_summary: The paper presented a framework for training mobile manipulation policies using UMI decide that uses task-space trajectory and was tested on prehensile, non-prehensile, and dynamic actions
main_review: main_review: Strengths:
- The paper tackled the learning of challenging, dynamic whole-body tossing and pushing tasks enabled by UMI's free-range data collection ability.
- The paper showcased that policies trained using UMI data in end-effector trajectories can be generalized to other robot embodiments, and the robot dog's low-level policy is good enough to bridge the gap in embodiment.
- Thanks to the whole-body controlled trained using RL, the robot learned its three-foot jumping style of tossing that balances its weight.
- The figures are thoughtfully illustrated to aid understanding and videos are well made.
- The hardware set up (including iPhone odometry) is versatile for a range of whole-body manipulation tasks. The paper also included results showing that the performance is not significant from using the mocap (ground-truth) pose tracking method.

Weakness:
- It's an integration of existing tools and methods. It showcased the ease to collect data using UMI and its use in training whole-body policy. And the idea of training a whole-body controller to achieve end-effector poses is also known.
- The experiments involved very limited object and initial scene variation.
questions: questions: 1. What exactly is task frame? The world frame? If so, why don't you call it the world frame? I was expecting to see it described in more detail in page 4, section 3.2 line 142. 
2. How do you adapt the trajectory-based policy from table-top to floor-top? For example, if you have a policy trained for tossing on the table using UMI data that a human collected while standing. How do you use that policy on the robot dog with its limited height? From the figures, it looks like the human has to kneel down to collect tossing and pushing data.
3. How much data was collected to train for each task that achieved > 70% success rate? What's the distribution of initial states (e.g. height of tables in the picking task, range of robot initial base configuration and ball poses in tossing task) in demonstrations, and how different are the initial states in testing? I see in the supplementary that grasping and tossing took 500 demonstrations. How much would performance drop if only a subset of those demonstrations is used for training?
4. Is the robot's initial base positions obtained using April Tags in the environment?
5. The low-level RL policy is only used to train to each any end-effector poses, as opposed to follow certain demonstration trajectories for tasks? Would it be helpful to include task trajectories (like tossing) in training the low-level policy?
limitations: limitations: 3
review_summary: review_summary: I recommend accepting the paper for its thoughtful hardware setup, choice of challenging tasks, thoroughness in experiments, and clarity in communication.
title: title: Solid framework for quadruped manipulator control. Possibly under-explained and oversold.
paper_summary: paper_summary: The paper describes a learning-based framework for manipulation with a robot arm (gripper) mounted on a quadruped base. A diffusion policy is trained via demonstrated motions to map end-effector camera input to target position trajectories for the end-effector in the task frame.  A full-body low-level controller is then trained to map the target trajectories to joint poses for the robot. Experiments are provided on three tasks both in simulation and the real-world.
main_review: main_review: The proposed framework is a natural decomposition of the problem --- produce target trajectories and then track those trajectories. The decomposition itself isn't that novel from a general robotics perspective, though it may be novel for this particular robot configuration. 

One claimed novelty is that the produced trajectories are in the task frame, rather than the robot body frame. The claimed advantage is that this allows for the low-level controller to adjust to body shifts during execution, rather than the trajectory planner needed to adjust. This can be important when the low-level controller is operating at a much higher frequency compared to the trajectory planner. 

Another claimed novelty that is mentioned a number of times is "zero-shot cross-embodiment deployment". I found the emphasis on this point to be a bit oversold. I was confused at first by this emphasis, but it appears to mean that the target trajectories can come from a number of means and not just the robot body used at deployment. Yes, I agree, but this is very different from what "zero-shot cross-embodiment" will be interpreted by many people. Many would consider "zero-shot cross-embodiment" to mean that you take your full system and it is immediately useful on a new robot (not sure what that would even mean, but it sounds impressive). 

I was unable to get a full picture of how the system was trained from the paper. In particular, the paper is not self-contained in describing how data is produced for the diffusion policy learning. It is also unclear what data is used to train the low-level policy in simulation. Is training only done on the tasks considered in experiments, or is a richer set of trajectories used for training? 

It wasn't clear what was controlling the end-effector opening and closing. From the description, it appeared that the trajectory data only included the position and rotation of the end-effector. 

The authors say "Notably, setting up a simulation to track these manipulation end-effector trajectories does not require setting up the manipulation task and environment."  While this is notable, I don't understand how it can lead to good performance for a wide range of manipulation tasks. For example, in simulation, a tossing motion requires very different control depending on whether the gripper is empty or holding something heavy. But from the description it did not appear such a range of situations would be part of the simulations used for training. More description of this is required to provide intuition about why this works. 

Given the claimed generality of the approach and the easy of creating trajectories for new tasks, I would have expected a wider variety of tasks to be explored. Currently we only see the system evaluated on 3 tasks.

Post Rebuttal:

Thanks for the clarifications. I agree that the work is making a nice contribution and those outweigh the weaknesses. I'm raising my rating to "weak accept".
questions: questions: The gripper state did not appear to be part of the observation space or controllable by the learned policy. How is the gripper handled in this work? 

What exactly is "survival %"? Is it the same as success? For example, in tossing one might miss the goal but still not fall over. 

Why is the low-level controller able to learn object manipulation when the simulations do not appear to involve objects or simulated disturbances that would be similar to such object interactions? 

Am I misunderstanding what you mean by "zero-shot cross-embodiment"? 

What trajectories are used to train the low-level controller?
limitations: limitations: 3
review_summary: review_summary: Overall the paper is considering an interesting problem and has developed a natural and potentially effective approach. We are shown successful sim-to-real performance on 3 tasks. Given the claims of generality, the paper would have been more convincing if a wider variety of tasks was considered. The description of the system and intuition behind the results can be improved significantly.