Visual Manipulation with Legs

Authors: xialin  he
chengjing  yuan
wenxuan  zhou
ruihan  yang
david  held
xiaolong  wang
u c san
carnegie  mellon university
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-05-20 14:15:29

Abstract: Animals have the ability to use their arms and legs for both locomotion and manipulation. We envision quadruped robots to have the same versatility. This work presents a system that empowers a quadruped robot to perform object interactions with its legs, drawing inspiration from non-prehensile manipulation techniques. The proposed system has two main components: a visual manipulation policy module and a loco-manipulator module. The visual manipulation policy module decides how the leg should interact with the object, trained with reinforcement learning (RL) with point cloud observations and object-centric actions. The loco-manipulator controller controls the leg movements and body pose adjustments, implemented based on impedance control and Model Predictive Control (MPC). Besides manipulating objects with a single leg, the proposed system can also select from left or right legs based on the critic maps and move the object to distant goals through robot base adjustment. In the experiments, we evaluate the proposed system with the object pose alignment tasks both in simulation and in the real world, demonstrating object manipulation skills with legs more versatile than previous work.

Main paper: 
 introduction: 
 In robotics, manipulation is often associated with robot arms and locomotion with legs. Typically, quadruped robots use dedicated arms for manipulation, while legs are used for walking [1, 2, 3] . However, animals seamlessly use their limbs for both tasks. Primates adeptly utilize all four limbs for walking, climbing, and object manipulation, suggesting quadruped robots could leverage their legs for a broader range of tasks, simplifying design and expanding manipulation capabilities. This work bridges the gap between locomotion and manipulation by leveraging non-prehensile manipulation for leg-based tasks. Non-prehensile manipulation, which involves tasks without grasping, aligns with the capabilities of quadruped robot legs [4] . Unlike previous efforts focused on simpler tasks like kicking a ball or pushing a button [5, 6, 7, 8, 9] , our task demands more precise control.Our system for solving object manipulation tasks with legs comprises a learned visual manipulation policy and a model-based loco-manipulation controller. The manipulation module, trained with reinforcement learning in simulation using point cloud observations, dictates how the leg interacts with the object to achieve the goal pose. We utilize an object-centric action space based on point cloud data [10] , allowing precise leg manipulation. The policy determines a contact point and motion vector, and critic values assist in choosing between the left or right leg.The model-based loco-manipulation controller synchronizes leg movements and body adjustments to execute commands. It translates high-level actions from the visual manipulation policy into lowlevel torque commands. The manipulation leg uses impedance control, while other legs maintain stability with Model Predictive Control (MPC). The controller adjusts the base pose when the object is out of reach, facilitating effective pushing toward distant goals. We evaluate the system on various object pose alignment tasks, demonstrating the visual manipulation policy's effectiveness against baselines, its generalization to unseen objects, and the advantages of leg selection. Real-world experiments assess the feasibility of zero-shot sim2real transfer and the system's precision in moving objects toward distant goals, showing improvements over previous work [5, 6, 7, 8, 9] . 
related work: 
 Quadrupedal Locomotion: Legged robots have traditionally relied on proprioceptive senses for navigating complex terrains, using model-based methods that ensure stability but require significant computational power [11, 12, 13, 14, 15, 16, 17, 18] . Recent advancements in modelfree reinforcement learning have reduced computational demands while enhancing generalization [19, 20, 21, 22, 23, 24, 25] . Although these techniques have primarily been applied to locomotion, recent work has integrated exteroceptive perception, such as elevation maps and depth sensing, to enhance high-level planning [26, 27, 28, 29, 30, 31, 32, 33, 34] . While most research uses these approaches for locomotion, our work uniquely applies them to dexterous manipulation, leveraging model-based controllers for precise end-effector positional control.6D Manipulation with Manipulators: Previous work in 6D manipulation has focused on dexterous hands and grippers on fixed bases designed for manipulation [35, 36] . In contrast, we use the leg of a robot dog for 6D manipulation, an underexplored approach.Non-Prehensile Manipulation: Non-prehensile manipulation, which involves interacting with objects without grasping, has been a focus for tasks like pushing, sliding, and flipping. Foundational work by Mason [4] established the mechanics of these tasks, while more recent approaches have integrated reinforcement learning and model-based methods to enhance precision and adaptability [37, 38] . For instance, Zhou et al. [39] developed learning frameworks for robust manipulation in unstructured environments. Building on these advances, our work applies non-prehensile techniques in legged robots, achieving 6D object control through precise actions like pushing and flipping.Manipulation with Legged Robot: Legged robots are increasingly recognized for their potential in manipulation tasks. Leveraging research that integrates leg and arm mechanisms [40] and biomimetic designs, these robots enhance movement and object transportation [41] . Existing methods predominantly use mounted arms [1, 2, 42] or the robot's body [5, 6] for basic tasks, with few employing legs. In this context, some works have extended quadruped capabilities by attaching small grippers at the end-effectors of their legs to facilitate prehensile skills [43] . Others have utilized the legs for broader, large-scale manipulations, such as door pushing and basket lifting [3, 44, 45, 9] or rotating large yoga balls [46] but sacrificing mobility. In contrast, our approach emphasizes non-prehensile manipulation, using the quadruped's legs to directly control objects with precise actions like pushing and flipping smaller items, achieving 6D object control without relying on additional appendages and maintaining full mobility. Our system employs visual cues for detailed manipulation, an area not extensively explored in legged robotics. By combining advanced perception technologies with agile quadruped legs, we enhance the robots' precision and adaptability for complex tasks. 
visual manipulation with legs: 
 We define our task as Object Pose Alignment; further details can be found in appendix A. An overview of the proposed system is summarized in Figure 2 . Our proposed system includes two modules: 1) a learned visual policy that takes a point cloud observation of the object and outputs an action consisting of a contact location and a vector of motion parameters. 2) a model-based loco-manipulation controller that uses the front leg of the robot to interact with the object while maintaining the robot's balance and moving the robot for long-horizon object manipulation.Our system integrates the visual policy and the model-based loco-manipulation controller to achieve object manipulation tasks. The model-based loco-manipulation controller operates within a framework that alternates between motion and manipulation states using a finite state machine structured around MPC. This approach requires multiple transitions between these states to accomplish a task. Every locomotion state is guided by the previous observation, setting a target position for the robot to approach. Upon reaching the target, the system transits into the manipulation state. Here, our learned visual policy computes and executes the necessary actions based on the current object and the goal object pose. Once the action is completed, the system reverts to the locomotion state to continue towards the next segment of the task, repeating this process until the task is completed.Our visual manipulation policy is trained with reinforcement learning using point cloud observations similars to [47] . The policy operates in an object-centric action space, selecting a contact location among the observed object points and a vector of motion parameters that define the post-contact movement [10] . Upon action selection, the robot moves its foot to the chosen contact location on the object. Once close enough, the robot executes the motion parameters, a 3D vector defining the push direction after contact. The range of motion parameters exceeds the robot's leg reach, allowing the policy to learn to apply varying force magnitudes for object manipulation.The policy is trained with off-policy reinforcement learning algorithms based on TD3 [48] . We use a segmentation-style network architecture (e.g., PointNet++ [49] ) for both the actor and the critic. Given a point cloud observation, the actor outputs per-point motion parameters (actor map), and the critic outputs per-point Q-values (critic map). The policy output is determined by selecting the point with the highest Q-value from the critic map and the associated motion parameters.We propose using the critic values to select between the left or right legs during manipulation. The legs on quadruped robots usually have a restricted range of motion due to limited degrees of freedom and joint limits. Choosing between the front legs during manipulation can broaden the available motion possibilities. As our experiments will show, using the right leg for tasks on the right side of the robot is more efficient and results in a higher success rate.We implement a policy capable of selecting the appropriate leg for tasks by modifying the structure of the actor map and critic map. Each point is associated with two motion parameters and two Qvalues corresponding to the left and right legs. We select the point with the highest Q-value and its corresponding motion parameter and leg to complete the task. At testing time, a single point cloud is processed once, facilitating decision-making for both legs simultaneously.Our loco-manipulation controller serves two purposes: 1) interact with the target object using the learned visual policy, and 2) move the robot for long-horizon object manipulation.Built on an MPC controller [11] and [50] , our controller divides the process into stages represented by a finite-state machine (FSM) in Figure 3 : QP Stand, COM Shift, Manipulation, Walking, and Post-Act COM Shift. At each control step, the robot starts in QP Stand, where it stands with four legs. The visual policy computes the action from the object point cloud. The robot then shifts to COM Shift, lifting one leg to prepare for interaction. In Manipulation, the lifted leg moves to the precontact point and manipulates the object. MPC calculates torque for the stance legs, and impedance control is used for the swing leg. After manipulation, the swing leg returns to the ground in Post-Act COM Shift state, preparing the robot for walking. If the object is out of reach, the controller moves the robot closer.Model Predictive Control. Our MPC, based on Bledt et al. [11] and Chen et al. [50] , uses ground reaction force over a finite horizon k to determine optimal control inputs and trajectory:min x,u k-1 i=0 ||x i+1 -x i+1,ref || Qi + ||u i || Ri subject to x i+1 = A i x i + B i u i , c i ≤ C i u i ≤ c i , D i u i = 0,Here, x i is the state, u i the control input, Q i and R i are weight matrices, A i and B i describe system dynamics, and C i , c i , c i set control constraints. D i identifies forces for feet not in contact.MPC commands include roll, pitch, yaw, Cartesian position, and target velocities. We primarily manipulate yaw (Z-axis) and provide linear velocity for X and Y directions. More details in Bledt et al. [11] .Impedance Control. Our impedance control adjusts mechanical impedance for optimal leg movement, enabling precise object interaction. Control torques (τ ) are computed as:τ = J T (K p (p des -p foot ) + K d (v des -v foot )) ,where p des and v des are the desired foot position and velocity, p foot and v foot are the actual position and velocity, K p and K d are proportional and derivative gains, and J maps foot force to joint torques.This control algorithm achieves precise control over the robot's manipulating leg, enabling optimized interaction with the environment and task execution.To perform accurate 6D object pose alignment, our system relies on the relative transformation between the target and current poses. While simulation provides accurate 6D poses, obtaining them in the real world is challenging, especially with egocentric visual observation.We integrate RPM-Net [51] , a learning-based point-cloud registration method, into our framework. RPM-Net processes source and target point clouds with per-point normals, outputting the transformation from source to target. We fine-tune the pre-trained RPM-Net on a subset of ModelNet40 [52]  
experiments: 
 We evaluate our system and compare it with various baselines in simulation and the real world on object pose alignment and present quantitative and qualitative results in this section.Task. We evaluate all methods on the following 5 tasks.• Box push (Fixed goal): Push a box forward (along x-axis in the robot frame) by 15cm.• Box push (Random goals): Push a box to a target position (in the robot frame) on the ground. The X and Y coordinates of the target position are uniformly sampled from (10cm, 20cm) and (-5cm, 5cm) respectively. • Box flip + push (Random goals): Flip a box left or right by 90 degrees and push to a target position sampled as in the Box push (Random goals) task. • Multi-object push (Fixed goal): Push an object forward (along x-axis in the robot frame) by 15cm. • Multi-object push (Random goals): Push an object to a target position sampled as in the Box push (Random goals) task.For Box push and Box flip + push (Random goals) task, we randomize the length, width and height of the box within (5cm, 10cm), (5cm, 10cm) and (4cm, 8cm) respectively. For Multi-object push tasks, we use 37 objects from Liu et al. [54] , where we use 27 objects for training and 10 objects for evaluation. The object visualization is provided in the supplementary section. Sampled trajectories in the real-world experiment are visualized in Figure 5 .Locomotion Strategy. In the real deployment, our robot uses MPC to approach the object. MPC calculates foothold positions based on gait settings, allowing the robot to walk stably to the target. Initially, the target is placed in front of the robot without base movement. Later, the object's position is obtained through point-cloud registration. Commands for MPC are calculated using the robot's current position and the object's position, as described in Sec 3.2. In the simulation, we directly set the robot's position in front of the object with small variations, eliminating the need to walk over.Baselines. We compare with the following baselines to verify the necessity of learning contact location and motion parameters:• Random Location Baseline: Selects a contact location uniformly from the object point cloud and learns motion parameters for manipulation as our method does. • Flow Baseline: Maintains contact location selection but replaces motion parameters with the flow between the object and goal point cloud at the selected point. • Planning Baseline: For pushing tasks, strategically select the center point of an object's side as the contact point; for flipping tasks, select the midpoint of the edge in the direction of the top side flip. Tuned motion parameters ensure effective interaction with minimal unintended movements or rotations. These parameters are then applied to other objects.Training and Evaluation. Our policy (except for leg-selection) and all baselines are trained in IsaacGym [55] for 50,000 steps, while the leg-selection policy requires 100,000 steps for optimal performance. Each policy is trained with three different seeds. The main evaluation metric is the success rate, defined as the mean flow between the object and goal being smaller than 3 cm. The mean flow measures the average distance between corresponding points in the current and target point clouds, considering both rotational and translational errors. By default, the evaluation uses the front left leg. Section 4.2 explores the advantages of selecting either the left or right leg. We first evaluate our method on single-box manipulation tasks, with results shown in Figure 6 (left).Our method achieves nearly 100% success on Box push (Fixed goal), and over 80% success on the harder Box push (Random goals) and Box flip + push (Random Goals) tasks. Our method significantly outperforms all baselines on all three tasks.Learning to select a contact point significantly outperforms random selection. The performance gap between the flow baseline and our method highlights the importance of learning motion parameters for accurate object manipulation, even with a simple box. When the robot needs to push the box forward, our visual manipulation policy outputs a vector pointing forward and slightly downward, enabling the robot to obtain larger friction and control the box's movement. The flow baseline's forward-only vector cannot effectively control the box, leading to poor performance. When we expand our object set from a single box to diverse objects, our method still achieves over 80% on both Multi-object push (Fixed goal) and Multi-object push (Random goals) tasks.To further illustrate the superiority of our method, we compare the trajectory of our method and the flow baseline manipulating the same object in simulation in Figure 5c . When the policy manipulate a tall bottle, our method learned to use motion parameters that result in a gentle push. In contrast, without adjusting the motion parameters according to the object, the flow baseline flipped over the bottle and failed to push it forward.Generalization to unseen objects. To evaluate the generalization capability of our proposed system across object shapes, we evaluated the policy trained with 27 objects on 10 novel objects and provided the results in Figure 6 (right). We found that our learned visual manipulation policy generalized well to novel objects without a significant performance drop, while the performance of other baselines dropped significantly.Ablation of leg selection. As stated in Section 3.1.1, our model outputs actions for both legs, selecting the one with the highest Q-value.We compare our method with a variant that only uses the left front leg for the Box Flip + Push (Random Goal) task. The object's location is randomized laterally in front of the quadruped, and the success rates for different object locations are shown in Figure 7 (left). When the object is close to the left front leg (offset ≥ -5cm), both variants perform similarly. However, the left-legonly variant fails when the object is farther away (offset ≤ -5cm), while our method consistently outperforms it. This shows our leg selection strategy significantly enlarges the robot's operation space for manipulation tasks.To better understand our system, we visualized the critic map in our learned visual manipulation policy, which scores each contact location on the object (Fig. 8 ). These Critic Maps capture goal-conditioned object affordances, providing insights into manipulation strategies. Figure 8a shows two scenarios involving pushing different objects, while Figure 8c focuses on flipping and pushing a box to the target pose.We deploy our system in the real world for all 5 tasks using Unitree Go1 [56] . We quantitatively evaluated our method on Box Push (Fixed Goal), Box Push (Random Goal), and Box Flip + Push (Random Goal). Each task had 10 trials, and the average success rates are provided in Figure 7 (right). We compared our method with flow and planning baselines in real-world scenarios, omitting the random location baseline due to poor simulation performance. Our method achieved up to 60% success rate, while other baselines barely succeeded. Despite a smaller simulation gap, the flow baseline performed significantly worse in real-world tests. We hypothesize it is more sensitive to environment parameters like friction, object mass, and the sim2real gap in low-level control.Figure 9 illustrates this gap with a scenario where the robot needs to push a box forward by 10 cm. The flow baseline outputs a 10 cm forward vector, but control errors cause the actual foot trajectory to move upward, losing contact with the object. In contrast, our method compensates for this error by maintaining contact and completing the task.Moving the object to a distant goal. Our system combines locomotion and manipulation, allowing the robot to move objects beyond its leg's reach. In an experiment, the robot pushed a box 1 meter forward, maintaining minimal lateral deviation. The manipulation policy, trained with close goals, repeatedly commands the robot to push the object 10 cm forward until it reaches the 1-meter target.After each action, the robot updates its observation using RPM-Net and adjusts its position for the next action with MPC. We evaluate performance with two metrics: steps to reach 1 meter and y-axis error at the final destination. Results in Table 1 show our system achieves higher accuracy and lower y-axis error compared to the baseline. 
conclusion: 
 We propose a system that allows quadruped robots to manipulate objects with their legs using visual input. Our RL policy leverages point cloud observations for object interaction, with actions implemented via impedance control and MPC. We evaluate the system in both simulation and real-world, demonstrating significant advancements in legged manipulation skills not seen in previous work.Limitation: While our system explores using a quadruped leg as a manipulator, it has limitations. For example, using an iPhone's lidar introduces accuracy issues in point cloud observations due to distortion from reflective surfaces, affecting the registration results of RPM-nets. Additionally, our reliance on MPC during training, though beneficial for sim-to-real transfer, faces challenges due to the lack of parallel processing for the Quadratic Programming (QP) problem, which slows down training. The current state estimator, relying on the robot's internal IMU, is susceptible to translation and rotation errors, which can result in inaccuracies in motion parameters and contact location during execution. Additionally, the front-mounted camera's inability to capture the object's side point cloud further restricts the robot's action space.Future work: To further improve our approach, we could accelerate the QP solution by reducing the number of state variables through motion parameterization as quintic splines, as proposed in [57] . Additionally, incorporating friction estimation [58] or utilizing a friction dataset as a prior [59] could enhance the system's ability to generalize manipulation tasks to more complex terrains, addressing some of the challenges we identified.MPC mentioned earlier. The weights in the table, from left to right, are for roll, pitch, yaw, position (XYZ-Axis), angular velocity (XYZ-Axis), velocity (XYZ-Axis), and gravity place holder. With the MPC illustrated earlier, The dynamics can be succinctly expressed as:EQUATIONwhere X ∈ R 13k represents the vector encompassing all state variables over the prediction horizon, and U ∈ R 3nk denotes the vector of all control inputs within the same period.The objective function, aiming to minimize the weighted least-squares discrepancy from a reference trajectory alongside the weighted magnitude of forces, is formulated as:EQUATIONwhere L ∈ R 13k×13k and K ∈ R 3nk×3nk are diagonal matrices containing weights for state deviations and force magnitudes, respectively. Here, U and X represent the control input and state vectors over the prediction horizon. We assign equal weighting to forces with K = α1 3nk .More details can be found in Bledt et al. [11] . 
 
 Appendix: In terms of our employment of RPM-Net [51] , we initiated training with the first 20 categories of the ModelNet40 [52] dataset employing a pre-trained model provided by the RPM-Net authors. This model was trained with cropping augmentation to perform registration on the incomplete point cloud. We opted not to use the normals from ModelNet, instead performing normal estimation on all objects using Open3D [61] . We then performed further augmentation by introducing visual occlusion through hidden point removal algorithm, increasing the extent of noise and cropping and better aligning with deployment. Training continued until convergence, which was achieved after roughly 1300 epochs. Subsequently, we fine-tuned the model previously trained in the described process on the YCB [53] dataset and our own scanned objects, continuing until convergence was once again attained. The fine-tuning process achieved roughly 8 degrees in rotation mean absolute error (MAE) and 0.05 in translation MAE. The visual cropping was conducted from random viewpoints, making it challenging to guarantee the remaining percentage of the object, though it was at most 70%.This approach leverages the capabilities of RPM-Net for robust feature extraction and point cloud processing, enabling us to perform object pose estimation with the egocentric camera. The application of occlusion, noise augmentation, and fine-tuning on diverse datasets underscores our method's effectiveness in improving model generalization and accuracy in real-world scenarios. Our approach extends the Twin Delayed DDPG (TD3) algorithm [48] , utilizing the framework provided by Stable-Baselines3 [62] as a foundation. For the segmentation-style network, we adopt PointNet++ segmentation architectures for both actor and critic networks, leveraging PyTorch Geometric's implementation. Details on hyperparameters can be found in Table 5 . Both the actor and the critic networks are configured with identical architectures and learning rates. One issue we observe is that noisy point cloud observations in the real world often lead to unreasonable outputs from our policy, as shown in Fig. 12 . This presents challenges for both RPM-net and our policy. Additionally, in our real robot experiments, a common failure case is registration error, particularly when objects lack distinct features. As illustrated in Fig. 11 , the registration result can incorrectly suggest that the object should be flipped, even during simple planar motions such as pushing an object forward. This error causes the pushing policy, which is trained on correctly oriented objects, to produce unexpected behaviors and ultimately fail the task. Control errors also contribute to failures, particularly in tasks where precise contact point accuracy is critical, such as in flipping tasks. As shown in Fig. 13 , the intended contact point (marked by the red dot) differs from the actual contact location (marked by the blue dot) captured by the camera. This discrepancy between the planned and executed contact points can lead to ineffective manipulation, resulting in the task's failure. These control errors highlight the challenges of achieving high precision in real-world scenarios, where even small deviations can significantly impact performance. Actual Contact 


Decision: The reviewers point out several strengths and weaknesses of the paper. While the method has good results and this indeed a challenging task, more baselines could be considered and the method uses some heuristics which take away from the generality of the approach. Please see reviewer comments for more details.

-----Post-rebuttal comments----
The rebuttal addresses some of the key issues brought up by reviewers and the paper appears to be above the bar for acceptance.

Reviews: review: title: title: Improving quadruped robot non-prehensile manipulation by sim-to-real high level visual policy and low-level model-based controller
paper_summary: paper_summary: The paper shows a system that lets four-legged robots use their legs to move objects. The system has two parts: a visual policy trained to see and decide how to move objects, and a controller that moves the robot's legs and body. This allows the robot to push or flip objects to new places using either leg. Real-world settings show the system works better than a few baselines. The paper's key contributions are combining high level visual policy with low level model-based movement control and testing on different tasks.
main_review: main_review: Strengths
- The system effectively combines visual 3D point clouds and control techniques to allow quadruped robots to manipulate objects with their legs. The high level policy outputs object-centric actions in terms of target object motion direction and desired contact location on the object. The low level controller is a model-based controller to transform the target motion on a single front leg (left or right) into torques. 
- The approach demonstrates successful real-world application, showing the system's practical use with performance conparison over a few baselines shown in the paper.

Weaknesses
- In the evaluation comparison, the task seems to be limited in terms of the object diversity, goal definition, and object-environment interactions (frictions, etc.). The robot intends to either push objects to a location or flip objects on the ground.
- The learned policy seems to be not very autonomous with a few heuristically-defined behavior transitions among standing and walking. The task seems not affecting robot dynamics too much, which could reduce the impact of this work when considering applying it to legged robots.
questions: questions: Expand the range of tasks and objects to include more diverse and complex interactions, addressing limitations in object diversity, goal definition, and object-environment interactions (e.g., friction variations). This could involve tasks beyond pushing and flipping, like lifting or rotating objects or using legs to do side motions than just forward push.
limitations: limitations: 2
review_summary: review_summary: The paper introduces a promising system for enabling quadruped robots to use their legs for object manipulation, combining visual data and control techniques. However, the current tasks are limited to pushing and flipping objects, and the system relies heavily on predefined behavior transitions, which reduces its autonomy. Expanding the task diversity and improving the system's adaptability to dynamic tasks would enhance its real-world applicability. Addressing these issues would strengthen the paper's contributions and make the system more robust and impactful.
title: title: Systems-type paper lacking novelty for method and task
paper_summary: paper_summary: This paper presents a system for doing non-prehensile manipulation with the legs of a quadruped. The target task is pushing objects along the ground to a target pose using a single leg of the robot. This is done by using a MPC controller to balance the robot on 3 legs so it can move the 4th with an impedance controller to push the object. Employing the HACMan method, the manipulation leg is controlled by a learned visual policy, which takes in point cloud observations and outputs an object contact location along with motion parameters to define the pushing motion. To do pose alignment RPM-Net is used to do point cloud registration and to get the transformation between the current object pose and the target pose. Evaluations are done in simulation and on hardware, comparing against different heuristic baselines. The proposed method outperforms all baselines and can generalize to unseen objects on the real robot.
main_review: main_review: Overall I think the work is a bit minor. The novelty is pretty limited and is more of a system putting together different previously researched techniques. The task is not particularly novel either, with plenty of other works doing non-prehensile manipulation with quadruped legs, pushing boxes, pushing buttons, opening doors, etc. What seems to be the main point of the work, doing learned visual manipulation, is just the previously published HACMan method applied to the author’s specific quadruped platform. The only main differences seems to be an improvement in the point cloud registration method, using RPM-Net as opposed to an out of the box global registration method with ICP. As such the contribution of this work is mostly just serving as another example of visual (non-AprilTag) manipulation for quadrupeds, of which there are still few. 


## Strengths:

Though minor, the leg selection module is a good logical extension of the HACMan per point Q-value setup. Having a “per-point-per-leg” Q-value makes sense. 

The system does seem to perform pretty well. When it does succeed, visually the objects end up quite close to the final goal pose. Generalization to different objects and even different ground surfaces are shown in the video results as well (though no accuracy evaluations are given for different ground surfaces). 

The paper is presented well and is very understandable. I had zero technical questions while reading. Exact details of the method are a bit light, but they are heavily referenced and hyperparameters are given in the appendix, so it is still clear exactly what is being used and in sufficient detail. Enough explanation is given that someone could recreate this work. The technical details of the methods seem to be all correct to my understanding.

A lot of hardware experiments are done, which is very appreciated. Multiple hardware trials for each task are also done.

## Limitations:

The novelty in this work seems quite limited, outside of these specific techniques being applied to this specific task. All of the individual techniques used, such as the learned visual manipulation policy, the MPC system, and object pose estimation and alignment are all pretty much just the referenced works (your references [10], [16], and [54] respectively) directly applied to your task. The leg selection for the manipulation policy is just a minor extension of the per-point Q-value and highest Q-value selection that the HACMan method does. 
There is also not much novelty to the task, as there are prior works doing learned controllers for quadruped non-prehensile manipulation, see [1-5] (these should also probably be part of your related works section). On the visual side of things [6] also uses PointNet++ for manipulation tasks (they use it for grasping rather than non-prehensile manipulation, but is a very similar example of using PointNet++ to go from point cloud observation to grasping/contact points).
This main novelty is in incorporating vision in to these manipulation tasks, and is more of a systems paper of applying HACMan to a quadruped rather than a fixed based manipulator (though once the robot is standing on 3 legs and balanced through the MPC controller it is basically liked a fixed base arm). The point cloud registration part is improved upon, but the main idea of training and system setup is the same as HACMan. 

There is a pretty large drop in performance between the simulation results and hardware results. What are the causes of failures on hardware? Does the robot not execute the motion parameters correctly, does it not contact the object at the right location, is the object pose transformation estimation not accurate enough (i.e. the RPM-Net point registration has errors), etc.? You can test some of these in sim as well, how does the system perform when you add noise in the point cloud observation and/or object pose estimate (speaking of which, is the policy trained with any of these noise/randomizations?) I think it’s good to report not just the performance and accuracy of your system, but to also try and show why the failures happen if possible. Right now only point cloud observation noise is mentioned in the limitations. These are also things that you can add to there as well. You can also probably add in/look at the limitations mentioned in HACMan since you use the same method with no changes here as well. 

The Flow baseline is a good baseline to compare against to isolate the effect that the learned contact selection has, but I think having a baseline that shows the effect of the learned motion parameters is good/necessary to have as well. Basically my question is “what is more important for this task, selecting the contact location well or outputting good motion parameters”? The “Random Location Baseline” is part of answering this, but I would like to see something like the planning baseline with the learned motion parameters. If we have a decent heuristic for what contact location to use, and then just learn the motion parameters, how well does it do?


References:

[1] He, Zhengmao, et al. "Learning Visual Quadrupedal Loco-Manipulation from Demonstrations." arXiv preprint arXiv:2403.20328 (2024).

[2] Shi, Fan et al. “Circus ANYmal: A Quadruped Learning Dexterous Manipulation with Its Limbs.” 2021 IEEE International Conference on Robotics and Automation (ICRA) (2020): 2316-2323.

[3] Arm, Philip, et al. "Pedipulate: Enabling manipulation skills using a quadruped robot’s leg." 41st IEEE Conference on Robotics and Automation (ICRA 2024). 2024.

[4] X. Cheng, A. Kumar and D. Pathak, "Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion," 2023 IEEE International Conference on Robotics and Automation (ICRA), London, United Kingdom, 2023, pp. 5106-5112, doi: 10.1109/ICRA48891.2023.10161470.

[5] Lin, Changyi, et al. "LocoMan: Advancing Versatile Quadrupedal Dexterity with Lightweight Loco-Manipulators." arXiv preprint arXiv:2403.18197 (2024).

[6] P. Ni, W. Zhang, X. Zhu and Q. Cao, "PointNet++ Grasping: Learning An End-to-end Spatial Grasp Generation Algorithm from Sparse Point Clouds," 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France, 2020, pp. 3619-3625, doi: 10.1109/ICRA40945.2020.9196740.

## Formatting comments:

Can you increase the text size in Fig. 2. Right now it is too small to easily read at a reasonable zoom level. 

Would it be possible to increase the image resolution of Figure 4 and 5? Right now it’s hard to see the individual points and get an idea of what is going on in the point transformation and the critic map. Same for Figure 6, it’s almost useless to have because it’s so small and low resolution that can hardly tell how the object is moving. This kind of goes for all images that aren’t generated plots actually. Even figure 2 is pretty low res, why does it look like that when the same figure in your video looks much better? There might be some pdf compression issues going on, perhaps any image that isn’t an SVG is just getting really really compressed. Something to double check. 

On line 241 the sentence “Our method achieved up to 60” ends abruptly, seems like there is some missing text here? Judging from Figure 8 I’m guessing you meant something like “Our method achieved up to 60% success rate”?

If space allows could you move Figure 5 to be later in the text so it’s closer to the where you reference it in line 227? It was weird seeing early and not having the context for what it means/is supposed to so, and then not reading anything about it until 2 pages later.
questions: questions: In section 4.2 what is “Post-Act COM Shift”? Why is this step needed? The explanation of each fo the FSM states in lines 115-122 seems to end after the “Walking” state.

The visual policy takes in a point cloud observation of just the object correct? So it is already segmented out from the raw point cloud observation? What is used for the point cloud segmentation?

For the Planning Baseline, what are the “tuned motion parameters” used? Is this something the heuristic used in the Flow baseline, where it’s the flow between the object and goal location? Or is just some fixed push direction? Is this why performance drops so much when the goal is randomized vs. when it is fixed?

Along a similar line, you mention that your visual manipulation policy outputs a vector pushing the object forward and slightly downward, which performs much better than the Flow baseline’s forward-only vector. If you add some downward direction to the motion parameter to the Flow baseline (and the Planning baseline as well) how well does it perform? This seems like a very reasonable heuristic to use. Overall I am surprised that for what seems like (at least to me) a relatively simple task of pushing an object to a goal, a simple heuristic of selecting the object center point and then pushing forward and slightly down would perform so bad. 

How does the MPC controller decide how much closer to move the robot to the object in the long horizon cases? Looking at the videos it seems like the robot could move much closer to the object and thus push the object farther in each manipulation step, moving it to the goal much quicker. Is there just some set target distance of how far the robot should be in relation to the current pose of the object?
limitations: limitations: 2
review_summary: review_summary: The work lacks a lot of novelty in both method and task and mostly serves an having another example of visual non-prehensile quadruped manipulation. I am not convinced enough that this is a large enough contribution for acceptance, but having more (especially hardware) examples in the field is not a bad thing when there are still relatively few. I would lean towards reject but can swing either way given the hardware experiments shown.
title: title: An intuitive path forwards for legged manipulation
paper_summary: paper_summary: This paper presents a learning-based approach for non-prehensile manipulation using the legs of a quadrupedal robot. The manipulation policy is trained in simulation and outputs motion vectors for the quadruped’s leg. This output is then tracked using an impedance controller while MPC is used to maintain balance and center-of-mass control. The approach is validated across experiments involving pushing and flipping different objects.
main_review: main_review: Overall, the paper is presented clearly and concisely and represents an interesting but intuitive path forwards for legged manipulation. The authors have done a good job presenting the proposed method, however there are concerns regarding the thoroughness of the literature review as well as issues regarding figure quality and the thoroughness of the experimental validation.

Major Comments:
1) Figures:

a) Figures 1, 2, 4, 5, 6, 9, and 10 are all low-quality and pixelated, making inferring information from them challenging. In particular, Figures 4,5, and 6 are illegible. Saving the original images in svg or pdf format and importing these should fix this issue.

b) Figure 10 in the Appendix is unlabeled and does not contain a caption.

c) The point clouds in Figure 2 are very blurry and unrecognizable as point clouds.

2) While the experiments show improved results for the proposed method when compared to the baselines, the range of object goal translations is limited in the y-direction. I would be interested in seeing how the proposed method behaves when the robot is required to push an object sideways, for instance in a circle around the robot, or pull the object closer to itself.

3) In Section 5.3, the second paragraph suddenly ends.

4) In Section 5.3, the authors indicate that “[...] our method compensates for this error by maintaining contact and completing the task.” While I agree with the authors that the proposed method outperforms the flow-based baseline, I disagree that this is because the policy has learned to compensate for controller error. From the authors descriptions, this control error does not appear to be present in the IsaacGym simulation environment, and so the policy has learned nothing about this error. I would appreciate a more in-depth examination of why the proposed method outperforms the baselines.

5) The author’s breakdown of the literature in Section 2 is confusing. There is no contribution to the blind locomotion literature, and so this area should not be included. Furthermore, there is no discussion of the non-prehensile manipulation literature, an area of research which this paper claims to advance.

6) Section 6, regarding the limitations of the work, is not thorough. There appear to be several possible limitations which the authors do not address, neither in this section nor the experimental sections. First, the proposed approach assumes the object to be manipulated is light enough to be pushed or flipped by the robot, which may not be the case in the real world. I would be interested to see what happens when such a failure case occurs. Second, it appears the impedance control for the leg is open-loop with respect to contact forces. If the robot does not make contact with the object, what happens? 

Minor Comments:
1) It may be helpful to parameterize each of the state variables in the MPC optimization using a quintic spline (refer to Section III.A of [1]). This minimized the number of optimization parameters especially when there is a fine time discretization.

2) The authors may be interested in parallel QP software such as this library.

3) The authors may be interested in recent work regarding physical property (friction, mass, etc) estimation from vision [2][3]. While it is not necessary to integrate into this work, combining the proposed approach with these methods may be an interesting future research direction.

[1] Dario Bellicoso, Fabian Jenelten, Christian Gehring, and Marco Hutter. "Dynamic locomotion through online nonlinear motion optimization for quadrupedal robots." IEEE Robotics and Automation Letters 3, 2018.
[2] Parker Ewen, Adam Li, Yuxin Chen, Steven Hong, and Ram Vasudevan. "These maps are made for walking: Real-time terrain property estimation for mobile robots." IEEE Robotics and Automation Letters, 2022.
[3] Donghun Noh, Hyunwoo Nam, Min Sung Ahn, Hosik Chae, Sangjoon Lee, Kyle Gillespie, and Dennis Hong. "Surface material dataset for robotics applications (SMDRA): A dataset with friction coefficient and rgb-d for surface segmentation." International Conference on Pattern Recognition (ICPR), 2020.
questions: questions: I would like to see an addition experiment involving pushing an object through a non-linear path which requires the robot to push the object sideways. Furthermore, it would be interesting to see what happens if the object is unable to be pushed by the robot, for instance if the object was too heavy.

Additionally, the authors must present a more thorough literature review regarding non-prehensile manipulation to better ground their work in the this field.

Lastly, the authors should present a more thorough evaluation of the limitations of this work.
limitations: limitations: 1
review_summary: review_summary: This paper presents an intuitive research direction for legged manipulation which I have not seen in the literature, but several issues need to be addressed within the paper