HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation

Authors: jin  wang
rui  dai
weijie  wang
luca  rossini
francesco  ruscelli
nikos  tsagarakis
Venue: robot-learning_org_CoRL_2024_Conference
Published: 2024-06-07 01:08:02

Abstract: Enabling robots to autonomously perform hybrid motions in diverse environments can be beneficial for long-horizon tasks such as material handling, household chores, and work assistance. This requires extensive exploitation of intrinsic motion capabilities, extraction of affordances from rich environmental information, and planning of physical interaction behaviors. Despite recent progress has demonstrated impressive humanoid whole-body control abilities, they struggle to achieve versatility and adaptability for new tasks. In this work, we propose HYPERmotion, a framework that learns, selects and plans behaviors based on tasks in different scenarios. We combine reinforcement learning with whole-body optimization to generate motion for 38 actuated joints and create a motion library to store the learned skills. We apply the planning and reasoning features of the large language models (LLMs) to complex loco-manipulation tasks, constructing a hierarchical task graph that comprises a series of primitive behaviors to bridge lower-level execution with higher-level planning. By leveraging the interaction of distilled spatial geometry and 2D observation with a visual language model (VLM) to ground knowledge into a robotic morphology selector to choose appropriate actions in single- or dual-arm, legged or wheeled locomotion.  Experiments in simulation and real-world show that learned motions can efficiently adapt to new tasks, demonstrating high autonomy from free-text commands in unstructured scenes. Videos and website: hy-motion.github.io//

Main paper: 
 introduction: 
 Humanoid robots with behavioral autonomy have consistently been regarded as ideal collaborators in our daily lives and promising representations of embodied intelligence. Compared to fixed-base robotic arms, humanoid robots, due to their configuration characteristics, offer a larger operational space while significantly increasing the difficulty of control and planning. Despite the rapid progress towards general-purpose humanoid robots [1, 2] , most studies remain focused on locomotion with few investigations into learning whole-body coordination, which results in simplistic control designs that struggle to adapt to new tasks and environments, thus limiting the potential to demonstrate longhorizon tasks under open-ended instructions. One major challenge is how to explore actionability of humanoid robots and diversify their behaviors, while learning to infer affordances and spatial geometric constraints, thus making plans for various tasks with learned skills like humans.Recently, vibrant advances in robotics learning have made it a promising avenue for manipulation and locomotion [3, 4, 5, 6] . Learning-based methods, such as reinforcement learning (RL), have become effective tools for task-oriented action generation [7, 8, 9] while facilitating generalization across diverse scenarios. However, extending learning algorithms to humanoid robots remains a challenge stemming from the exponential increase in training costs induced by high degrees of freedom (DoF) and the difficulty of deployment on real robots under dynamic constraints. Meanwhile, the rise of large language models (LLMs) and their remarkable capabilities in robotic planning [10, 11, 12] have made it possible to perform logical reasoning and construct hierarchical action sequences for complex tasks. By integrating observations from different modalities, those models can be used for extracting features of objects and environments for robot perception and decisionmaking. Nevertheless, limitations to the utilization of LLM in humanoid robots exist, particularly in complex whole-body motion control and precise coordination between body parts.To address these issues, we first recognized that directly outputting whole-body trajectories for a real-world multi-joint system through simulation training is inefficient and impractical. Therefore, we adopt a decomposed training strategy that modularly selects the actuation components related to given tasks, and project lower-dimensional space trajectory on the whole-body space with a unified motion generator. The trained actions are stored as skill units in the motion library. We utilize the LLM's ability to decompose complex semantic instructions consisting of multiple sub-tasks and design a modular user interface as model's input. The LLM selects skills from the motion library and arranges a sequence of actions, referred to as task graphs. Furthermore, the 3D features extracted from captured 2D images and depth data can be integrated with the visual language model (VLM) and robotic intrinsic characteristics, acting as a robotic motion morphology selector.We refer to this study as HYPERmotion, a framework that tackles behavior planning for humanoid robot autonomous loco-manipulation using language models. By leveraging the interaction of distilled spatial geometry and 2D observation with VLM, it grounds knowledge to guide morphology selection combining robotic affordance. And bridge the gap between semantic space, robotic perception and action. We demonstrate a learning-based whole-body control to generate humanoid motion that adapts to new tasks and performs long-horizon tasks with primitive skills. We further illustrate through experiments how HYPERmotion can be learned and deployed on a high-DoF, hybrid wheeled-leg robot, performing zero-shot online planning under human instructions. 
related work: 
 Planning and Reasoning via Language Model Grounding pre-trained language models has become a promising avenue for robotic studies. Extensive prior works [13, 14, 15, 16] focus on planning and reasoning of robotic tasks with LLM, using it as a tool for code generation [10, 17] , reward design [18, 19, 20] , and interactive robotic learning [21, 22, 11, 23] . Several transformer-based architectural planners [24, 25, 26] showcase the potential for embodied usage of generating robot action. And with the integration of multi modalities such as visual and auditory [27, 28, 29, 30, 31] , perception and behavior can be directly bridged with semantic commands. Further research involves creating a customized skill library [32, 33, 12] to link robot execution and high-level planning. A related line of works [34, 35, 36, 37] has also explored grounding affordances with foundation mod-els to enable spatial reasoning and guide manipulation. However, most efforts focus on employment of fixed robotic arms, with few attempts made to extend language modes to humanoid robots, due to their complex dynamics and precise coordination between different components. [2] presents an end-to-end humanoid manipulation towards speech reasoning but lacks the cooperation of mobility. [38][39] use LLMs for decision making and learning, while demonstrating solely in simulation scenes. In contrast, we realize language model based online planning and humanoid motion bootstrapping, distilling spatial knowledge for robotic morphology selection using only onboard sensing.Task-orient Humanoid Control As the practical value of general-purpose humanoid robots becomes evident, a substantial amount of research has focused on the hardware of humanoid robots [40, 41, 42] , as well as gait generation and balance control [43, 44, 45, 46] . Methods based on learning and model predictive control (MPC) have significantly enhanced the mobility of such robots [47, 48, 49, 50, 51, 52, 53] . Some demonstrate motion generation through teleportation [54] and imitation learning [55] , while these often lack autonomy and struggle to organize learned short-horizon skills. Recent works on long-horizon tasks [56, 57] and bimanual coordination [58, 59] show dexterity and stabilization, but these are often limited to specific tasks to utilize their characteristics. Our approach enables the robot to perform composite tasks involving locomotion and manipulation in unstructured environments, engaging in rich physical interactions with various objects. Our work can also decompose tasks into sub-modules based on verbal instructions and autonomously perform tasks using pre-trained motions.Recent research has demonstrated significant advancements in robust walking [60, 61] , trotting [62, 63, 64] , and parkour [7, 8, 9] for legged robots using end-toend RL. The combination of learning-based locomotion policy with model-based manipulation of attached arm shows feasible whole-body motion on rough terrain [5] . However, most learning-based controllers are implemented on quadruped robots with few DoFs, while highly redundant humanoids are rarely addressed. For the latter, optimization-based control is still necessary to ensure the safety and adherence to constraints due to limited reactive frequencies. Nowadays, learning-based MPC demonstrates capabilities in system dynamics identification [65, 66] , closed loop performance [67, 68, 69] and safety assurance [70, 71, 72] . [73] shows a whole-body MPC on legged manipulators, but the tasks are limited by manually defined trajectories. In this work, we leverage RL to enable a wide range of motion skills without relying on predefined trajectories and employ a low-level optimization-based controller to ensure the feasibility of whole-body motion. 
methodology: 
 In this section, we illustrate how the HYPERmotion framework enables the humanoid robot to autonomously perform loco-manipulation guided by semantic instructions (Sec. 3.1). We then provide the method for task-orient whole-body motion learning policy and how we build the humanoid motion skills library (Sec. 3.2). We further describe how to achieve robotic morphology selection based on spatial reasoning by integrating multi-modality language models, and map the long-horizon task to hierarchical behavior structure using learned motions (Sec. 3.3).To address the autonomous loco-manipulation challenge for complex robotic platforms such as humanoid robots. This work proposes a method to perform language-guided behavior planning, motion generation and selection towards different scenarios. As shown in Fig 2 , we divide the pipeline into four interrelated sectors that are learned and deployed sim-to-real manner. The motion generation sector selects RL training configurations for specific tasks and conducts training in parallel. The trajectory obtained from the training is provided as a reference to the optimizer, which ultimately generates whole-body motion skills and the skills will be stored in the motion library. The user input sector contains a user interface as well as pre-defined basic prompts, function options, and motion library, all of which together constitute the textual material fed to the LLM. After receiving a command, the task planning sector first generates a hierarchical task graph that includes task logic, condition determinations, and actions using the LLM. Once the task graph is loaded, it is interpreted as a Behavior Tree to guide the robot and to pass actions to lower-level execution. When a task Figure 2 : Overview of HYPERmotion.We decompose the framework into four sectors: Motion generation is assigned for learning and training whole-body motion skills for new tasks and storing them in the motion library. User input includes received task instructions and initialization prompt sets. Task planning generates a task graph that guides the robot's behavior through reasoning and planning features of LLM and passes action commands to the real robot. Morphology Selector is used for action determination in specific sub-tasks, selecting the appropriate morphology for locomotion and manipulation based on grounded spatial knowledge and robot intrinsic features. requires selecting the robot's morphology, depth-sensing information is invoked and distilled into 2D images and geometric features. These data, along with the task state and prompts, are fed to the VLM, which then selects the morphology capable of achieving the goal. Through the coordination of these sectors, HYPERmotion facilitates semantic command understanding and zero-shot behavioral planning and action execution for humanoid robots, as shown in Fig 1.Tasks Learning with RL In this section, we show the details of learning whole-body motion generation based on different tasks, as shown in Fig. 3 . To reduce the action space, we separate the robot's upper body from its legs. To guarantee feasibility, the floating base motion is heuristically limited to avoid generating unfeasible motions for the legs once projecting the trajectory on the whole-body space of the robot. For single-arm tasks, the action space A 1 ⊆ R 14 consists of the 6-DoF right arm joint angles, 6-DoF floating base translation distances and Euler angles, one torso yaw, and one gripper joint angle. The left arm is fixed during these tasks. For the dual-arm picking task, the action space A 2 ⊆ R 19 appends the 6-DOF left joint angles, with the gripper joint closed in this case. The observations include the states of the corresponding targets and the joint states of the robot's upper body, detailed in the Appendix along with the policy settings. All tasks utilize proximal policy optimization (PPO) [2] because of its efficiency. The output in the RL layer is a joint position trajectory q * ∈ R 20 of the upper body. We train all skill policies separately using a general reward formulation:EQUATIONwherer l reach = ( 1 1+d 2 l ) 2 and r r reach = ( 1 1+d 2 r) 2 with d l and d r representing the distance of the operational target to the left and right end effectors, respectively. The termr rot = sign(d x ) * d 2 x + sign(d z ) * d 2z is the reward for aligning the gripper's orientation with the task's object (e.g. drawer handle, door handle, drill). Here d x and d z are the dot products of the gripper's forward and up axes with the object's inward and up axes, respectively. The term r f inger = β -(d t + d b ) encourages the gripper to grasp the objects, where β a fine-tuning parameter related to the size of the operational object, and d t and d b are the distances from the top and bottom links of the gripper to the task's object, respectively. r penalty = -∥a∥ 2 penalizes excessive actions a to ensure smooth operation. Finally, r task denotes the specific reward for task completion, which will be detailed in the Appendix along with the specific settings of the parameters α 1 to α 6 and the axes for different tasks. The reduced robot space from RL should be mapped into the whole body joint space to generated feasible trajectory. To fit the gap between the upper body trajectory and the whole body action, we solve an optimal control problem to merge them enforcing the whole-body dynamics of the robot to guarantee the feasibility of the resulting motion. In this case, we choose a unified whole-body trajectory generator [75] , specifically, we use the framework presented in [75] to solve the following non-linear problem:       min x(.),u(.) T 0 L(x(t), u(t), t)dt s.t. ẋ(t) = f (x(t), u(t), t) g 1 (x(t), u(t), t) = 0 g 2 (x(t), u(t), t) ≤ 0 (2) where x(t) = [q, v] ∈ R nx , n x = 93 is the state number,u(t) = [ v, f c ] ∈ R nu , n u =58 are vectors of state and input variables, q, v are the generalized coordinates and generalized velocities, f c is foot force. L(x, u, t) = ∥q uq * ∥ 2 + ∥u∥ 2 is intermediate cost, q u is the upper body joint variables.As for constraints, ẋ(t) = f (x(t), u(t), t) is the whole body dynamics, g 1 represents the set of equality constraints, g 2 the set of inequality constraints consisting of joint limits, velocity limits, and unilaterality of the contact forces. Finally, we could get the whole body motion combining upper body reference trajectory and whole body dynamic feasibility to complete the given tasks.Motion Library After learning task-orient motion skills, we constructed a motion library to host these primitives, which consists of attribute and functional descriptions of these actions, and the corresponding learning-based whole-body policies. Then, the LLM can reason the attribute-function descriptions to create sequences of actions to be executed based on different tasks and generate a task graph to invoke the execution of each node without additional training or demonstration.Migrating foundation models from a fixed robotic arm to a humanoid robot with a floating base presents numerous issues and challenges. The addition of robotic components not only imposes complex dynamic constraints, making it difficult to coordinate and control various parts. It also requires addressing the potential for different manipulation modes inherent in human-like structures, as well as the increased DoF for spatial mobility by the addition of wheels and legs. Due to the construction of our motion library, the usage of the LLM for planning no longer requires additional considerations for constraints such as self-collision or self-posture balance maintenance. This allows more focus on the decomposition of given tasks, and the selection of the robot's morphology.Humanoid Motion Morphology Selection Humans utilize common sense and learned experiences to extract the affordances of objects they manipulate and select appropriate movement based on the estimation of geometric constraints of the environment. Inspired by this, we leverage VLM to implement similar functionalities in robots. First, we include descriptions of the robot's structure and functions, and the robot's achievable range of motion in the prompts p V . While determining the morphology for a manipulation task T m , the robot utilizes 2D and depth images from its head camera. Object detection and pose estimation algorithms [76, 77] are invoked to acquire the position and orientation of the target object v c ∈ R 6 , which is then transformed into the robot's coordinate system v R ∈ R 6 . The VLM V, based on the current task state s, the scene's 2D images I h scene , and the target object's 6D pose v R , generates the robot's manipulation morphology x m for the task scenario. For locomotion tasks, the robot uses the depth information from its pelvis depth camera to generate the point cloud P c , which is down-sampled to create a voxel grid V g . This spatial information containing the current moving path together with the 2D image I p scene and the task state are finally fused to select the robot's locomotion morphology x l using the VLM V.EQUATIONEQUATIONFigure 4 : Robotic Morphology Selector extracts spatial geometric data and 2D observations from the physical environment upon receiving the language-conditioned task state and interacts with the VLM incorporating the grounded robot's affordances, so as to provide the optimal motion morphology that meets the requirements of given task scenario during manipulation and locomotion process.To obtain the desired response from LLM, it is necessary to impose constraints on the input. In the user interface, we define three types of constraints. The basic prompt provides a description of task background and characteristics of the robot, as well as interpretation of the user command and the output format. The motion library offers a catalog of learned skills and their description. Function option module offers specifications of the added functions developed for humanoid robot and determines whether these predefined functions are invoked during planning. Such as, if the morphology selection is chosen, the LLM will incorporate the morphology selector based on the task scenario; otherwise, this function will not be considered (See the Appendix). This approach allows for systematic construction of prompts and modular addition of constraints, thereby enhancing the flexibility of planning. We utilize BehaviorTree (BT) [78] as an intermediate bridge to convert high-level instructions into executable low-level skill sequences. BT provides a hierarchical structure for guiding actions and making decisions for the robot, which is composed of nodes with different effects. With a pre-defined motion library, LLM can generate a task graph consisting of learned motions and BT nodes, build it in an XML file, which constructs the complete BT. Thus realizing the robot's behavior planning with LLM by giving verbal instruction. 
experiment: 
 We demonstrate HYPERmotion's ability to learn, plan, and select behaviors for different tasks in both simulation and real-world experiments using objects that can be commonly found in daily life.Our robot is a centaur-like humanoid robot, supported by four legs with wheels. The robot has two arms with one claw gripper on its right arm. There are two depth cameras, one is on the head and another is in the pelvis position. We use Xbot [79] to achieve real-time communication between the underlying actuators and the control commands. We use Isaac Gym [80] as a training environment and validate the planned whole-body motion using Gazebo [81] . We access the gpt-4o model as the LLM planner and the gpt-4v model as the VLM from OpenAI API [82] .We trained individual motion primitives based on everyday tasks, deploying these skills on the robot to verify their feasibility (Sec. 4.1). We tested the adaptability and versatility of morphology selector in various task scenarios (Sec. 4.2) and conducted further study to validate the behavioral planning capability and performance of our framework in response to open-ended instructions (Sec. 4.3).We pick two representative motions and compare the trajectories in Isaac Gym, Gazebo and the real world, respectively, as illustrated in Fig. 5 , using the end effector position trajectories as an example.In the drawer opening task, the end effector first approaches the drawer handle and then pulls it out, demonstrating an initial increase followed by a decrease in the x-axis distance. In the door opening task, the end effector reaches the door handle and pushes it down. Self-collision is not considered during the training period because we still need to account for collisions between the upper body and legs, causing the arm to reach the target point at a faster speed. Instead, the whole-body planner projects the trajectories within constraints to avoid self-collision of the entire body. The trajectories from training are effectively tracked, with smoother results achieved after applying the whole-body controller as a filter. This indicates the successful deployment of our methods on the real robot. We investigated whether a VLM can zero-shot determine robot's morphology based on task scenarios. We picked ten scenarios each for manipulation and locomotion in both simulation and real-world environments (See the Appendix). We compared the success rate of the VLM's morphology selection using 2D image input only versus image combined with spatial geometric as input. Each scenario was tested 10 times under both inputs, as shown in Fig. 6 . We found the morphology selector effectively chooses the optimal mode for everyday object manipulation and mobile environment with a high average success rate. Compared to solely image input, adding spatial information improves the selector's accuracy, particularly in determining locomotion modes and adapting to complex scenarios (paths with obstacles of varying types and heights), thus leveraging the robot's affordances and leading to robust execution.Tasks with human instructions We validate the ability of LLM to plan motion primitives for different loco-manipulation tasks, as well as the effect of the modular user input designed for humanoid robots regarding reasoning and planning. Experiments were conducted on tasks requiring a combination of perceptions and actions. We recorded the success rate and the impact of different errors of 4 representative tasks and provided quantitative evaluations in Fig. 8 . The results show the LLM based planner can effectively plan semantic instructions based on learned skills and guide the robot to complete a variety of tasks according to the action sequences, achieving a desired success rate Long-horizon Task We further explored whether HY-PERmotion can enable behavior planning for a humanoid robot towards long-horizon tasks. We orchestrated a collaborating task scenario and input verbal instruction as shown in Fig. 1 . Qualitative results including time-lapse shots of robot motion execution and a Behavior Tree mapped out by LLM are shown in Fig. 7 . We demonstrate that our framework can synthesize sequences of motion primitives based on designed user input and accurately infer the logic of semantic knowledge while selecting robotic morphology of locomotion and manipulation according to the environment and state of the task. We found that language-based behavior planner exhibits greater versatility and adaptability to more complex tasks compared to existing methods. 
conclusion: 
 We present HYPERmotion, a framework that enables humanoid robots to learn, select, and plan behaviors, integrating knowledge and robotic affordance to perform embodied tasks. We evaluate the framework's efficiency and versatility through real-world experiments and long-horizon tasks. Despite achieving expected results, there are limitations: the motion library's size restricts the range of task commands, and learning of new skills requires separate training optimization, hindering generalization from existing actions. Moreover, the system struggles to handle external disturbances and collisions, lacks real-time linguistic interaction during the task and has limited capability for re-planning in response to unexpected tasks. Future work will focus on enriching the robot's action skills, enhancing LLM dynamic planning ability, and improving robot navigation and perception to achieve close-loop humanoid motions and safe human-robot collaboration. 
 
 Appendix: Our robot is a centaur-like robot platform. The upper body of the robot is humanoid in design and is similar in size to the average human to adapt to both dual-arm and single-arm manipulation. The robot's mobility relies on its quadrupedal lower body and maintains whole-body balance to cope with a variety of terrain conditions and perform loco-manipulation tasks. Moreover, to improve the robot's mobility on flat ground, wheel modules are integrated underneath each leg and can control the direction and steering of the wheels.The robot's whole body consists of 38 actuatable joints. The robot's torso is mounted on the pelvis of the lower body via yaw joints, allowing the upper body to rotate in the transverse plane. Each arm of the robot includes 6 DoF, where the right hand gripper contains one extra DoF that controls its opening and closing. The robot's legs are designed to provide an omni-directional wheeled motion and articulated legged locomotion, with each leg containing six degrees of freedom, allowing for positioning, orientation, and rotation of the wheeled-leg module.The perception system of the robot consists of two on-board RealSense Depth Camera D435i, one located in the robot's head and the other in the robot's pelvis, which are used to provide 2D images and depth information of the surrounding environment and objects. The complete computing system consists of two on-board computing units (ZOTAC-EN1070K PC, COM Express conga-TS170) for system communication and real-time robot control and an external pilot PC (Inter Core i9-13900HX CPU @3.90GHz, NVIDIA GeForce RTX 4090) for task planning and sensory data processing as well as a user interface. We use XBotCore, a cross-platform, real-time, open-source software designed for interfacing with low-level hardware components of robots [1] . This innovative tool enables effortless programming and management of various robotic systems by offering a standardized interface that conceals the intricacies of the hardware. Additionally, a proprietary CartesI/O motion controller [2] handles higher-order motion instructions. It is capable of managing multiple responsibilities and restrictions, prioritized according to the demands of specific situations. Through solving a series of quadratic programming (QP) challenges, each linked to a unique priority tier, the controller ensures optimal performance across all preceding priority stages.We experimentally validate the efficacy of HYPERmotion in behavior learning, planning, and decision-making by implementing the framework and evaluating its performance on a wheel-legged humanoid robot executing long-horizon tasks in response to semantic commands.We conducted the experiment using objects that are commonly found in an office kitchen. The test environment was an open area inside the lab, with objects randomly placed. The AprilTag system [3] , which incorporates a vision-driven algorithm, was used during the long-horizon task to identify the relative objects' location and direction of recognized tags. Within the actual environment, we employ AprilTags to gather task-specific observations. A single visual marker on the door allows for the determination of the door handle's relative position. The robot searches for the tag if it doesn't exit the camera's field of view (FOV). Additionally, AprilTags enable the identification of the drawer's relative positions.A long-horizon task refers to a task that needs to be completed over an extended time period, typically involving the execution of multiple subtasks and decision-making across several steps. Such tasks usually require the system to possess strong planning capabilities and long-term strategies to effectively achieve objectives in complex environments. In this study, we define long-horizon tasks based on the number of robotic actions included in the task. When the task described by the given instructions comprises four or more robotic actions, we classify it as a long-horizon task. Here, robotic actions refer to those included in the action library. For example, the task 'Pick the box' consists of three actions: < ObjectDetect >, < M oveT arget >, and < DualP ick >, and is thus categorized as a simple motion task. In contrast, the long-horizon task described in Fig. 1 requires the robot to perform multiple actions, including < F indObject >, < M oveT arget >, < ObjectDetect >, < SingleP ick >, and < OpenDrawer > et., with a longer time span required for task completion.We initially conducted simulation experiments to compare the construction of behavior trees using traditional method [4] with LLM based planner. The experiments included a qualitative analysis of the capabilities of both approaches, as well as the planning results for various robotic tasks. Executable rate refers to whether the generated BT is logically consistent and executable by the robot. Compared to normal BT-planner, the integration of LLMs' reasoning and decision-making capabilities enables the direct generation of BTs from tasks described in free-text instructions. Moreover, BT can be restructured by simply altering the input instructions, significantly enhancing the autonomy of task planning. Quantitative analysis in Table A1 indicates that BTs generated by LLMs exhibit higher executability and task success rates compared to those generated by BT-planner. Additionally, the time required for robots to complete tasks in the simulation environment was reduced. These results underscore the advantage of LLMs' reasoning and text generation capabilities in constructing behavior trees, which rely on a deep understanding of task logic and adherence to fixed construction rules.After verified that the LLM-based planner is capable of effectively planning robotic tasks, we selected six loco-manipulation tasks to validate the performance and adaptability of the HYPERmotion framework performing different tasks. We compared its performance with traditional whole-body MPC control methods [5] and conducted 25 experiments for each tasks separately, the simulation results are shown in Table A2 . The experimental results indicate that using an LLM to plan pre-trained motion primitives for executing loco-manipulation tasks yields a higher success rate compared to WB-MPC, particularly for complex tasks such as 'pick and place object' and tasks requiring long-term planning such as 'open drawer and pick object.' However, for some simpler tasks, the time required by the LLM-Planner is slightly longer than that of WB-MPC, attributable to the time taken for online API requests and BT construction. Adding Morphology Selector (MS) into the task planning process results in a modest increase in the average planning time, but it concurrently enhances the success rate of tasks, especially those involving interaction with the surrounding environment and objects. We further investigated the relationship between the success rate of executing loco-manipulation tasks using HYPERmotion and the complexity of the tasks. We conducted ten separate simulation experiments for tasks involving 3, 4, 5, and 6 actions. The results show that as task complexity increases, the success rate of task completion gradually decreases, though the overall average success rate remains above 60%. Additionally, task success rates vary depending on the specific content of the tasks and the difficulty of the required robotic actions, with this variability being more pronounced in long-horizon tasks (those involving four or more actions).During the real-world experiment, failure of different cases happened in the process of the robotic tasks performing. In Fig. 8 , we recorded the average rate of failure caused by different type of errors. Planning errors primarily occur during the BT construction process by the LLM-Planner. When tasks are highly complex and require multiple estimations of the environment or object states, the task graph generated by the LLM may exhibit logical errors or missing actions, leading to task failure. To address this issue, we optimized the motion primitives by integrating the robot's selfstate detection or environmental perception into the primitives to reduce the logical complexity of constructing BT.Perception errors stem from inaccuracies in the robot's sensing system, particularly in tasks involving navigation and target or object detection. The perception system consists of AprilTag localization and deep object pose estimation. While using an RGB camera for AprilTag localization, there is a margin of error of 2-4 cm, especially when the robot is in motion. The accuracy of the object pose detection algorithm is also influenced by the object's orientation and potential occlusions, which can affect the robot's manipulation posture.When performing robotic tasks in the real world, execution errors are the most common and can arise from a variety of factors. During task execution, since the actions are pre-trained and optimized for full-body motion, unexpected disturbances can significantly impact performance. For instance, while opening a door, if the robot's leg collides with the doorframe, it may struggle to adapt its trajectory in real-time. In the case of long-horizon tasks, errors tend to accumulate with the increasing number of actions performed. For example, after pulling open a drawer and retrieving an object, any deviation in the grasping pose can lead to failure in placing the object, causing it to collide with the environment and drop. Additionally, tasks with extended durations can result in overheating of the robot's actuators, further compromising task performance.Applying LLMs to a high-DOF, floating-based humanoid robot, where manipulation must be coordinated with locomotion, is a complex and challenging endeavor. Due to the low real-time performance of LLMs, they are unable to update plans in real-time during task execution and cannot directly generate control trajectories for multiple joints simultaneously. To address this, we consider the integration of a task graph and motion library, where the LLM is responsible for reasoning and decision-making, constructing a task graph composed of motion primitives. This task graph then issues control commands to enable the robot to complete long-horizon tasks.During the acquisition of motion primitives, the prioritization in the whole-body motion optimization phase significantly influences action generation. For instance, in actions like 'opening door' and 'open drawers', the optimization must not only account for the accuracy of the motion trajectory but also maintain robot balance and avoid self-collision. Additionally, to accommodate drawers of varying heights, we incorporate floating base adjustments in the optimization process.When dealing with long-horizon tasks, differences in pose solving can lead to issues with action continuity. For example, in the "Pick and place object" task, we introduced a homing action between the pick and place phases to ensure seamless transitions between actions to reduce the error rate.We utilize Proximal Policy Optimization (PPO) [6] for training our tasks, employing a multi-layer perceptron within an actor-critic framework. First, we define the frame of the drawer handle. The x-axis of the handle points towards the robot, while the z-axis points upwards. The handle's inward direction is aligned negatively along the xaxis, and the upward direction is consistent with the z-axis. The task reward is defined as r task = α 7 r around + l drawer * r around + l drawer (A1)where r around = 0.5 when the gripper's top link is above the handle's position and the bottom link is below the handle's position, otherwise r around = 0. l drawer represents the length by which the drawer has been pulled. We define the object's upward direction as aligning negatively along the x-axis, and the inward direction as aligning negatively along the z-axis. This orientation encourages the gripper to adopt a top-to-bottom pose, facilitating a proper grasp of the object. The task reward is defined asr task = α 7 r around + h (A3)where r around is the same setting as the previous tasks with the corresponding object frame and h = 1 if the object is been picked up, otherwise h = 0.The observations and reward parameters for this task are listed in Tab. A5 and A6. The prompt words used for the motion morphology selector are shown in the figures, where the prompt words for manipulation morphology selector will be fed into the VLM along with the received textual task instructions from the Behavior Tree.The motion morphology selector are packaged as one of the functions in 'User Input' module and it turned 'off' by default. When it needs to be invoked in task planning, it must be enable in 'Function Options' or specified to be set to 'on' when inputting the task instructions. The 'User Input' is the module that links the instructor to the language model and contains predefined prompts for initializing the language system environment and limiting the model output, as well as an interface for accepting task commands sent from the user side.Basic prompts provide a description of the task context and robot characteristics, as well as an explanation of user commands and output formatting requirements. As shown below:# ## Basic Prompts ### " You are now a robot controller , please output a XML file for constructing a behavior tree to control the robot under the requirements and given task . " " The robot you control is a centaur like robot , with a humanoid upper body and four legs , each leg has a wheel at the bottom . " " The robot has two arms , with a claw gripper on the right arm . It can manipulate objects with two ways : single -arm manipulation and dual -arm manipulation . " " The robot has two modes of movement : wheel motion and leg motion . The robot default manipulation and locomotion modes are ' single arm ' and ' wheel '. " " The robot has two depth cameras : one located on the head to view objects , and one on the waist to view the road and terrain ahead . "We designed a number of functions for the robot and packaged them into condition nodes for selective invocation by the LLM during the planning of the task. These functions include: 'Manipulation Morphology Selector', 'Locomotion Morphology Selector', 'Failure Detection and Recovery'. We add the descriptions of these functions acting as 'Function Options' inside the 'User Input', and set all functions to 'off' state by default. When the instructor expects a function to be added during a task planning, it can be manually set to 'on' or include a declaration to use the function in the instruction.# ## Function Options ### " The robot has the following functions , all of which are ' off ' by default . " " When a function is ' on ', it need to be involved in planning for the given task , and when it is ' off ', it should not be used . " " Functions : " " 1. ' m a n i p u l a t i o n _ m o d e _ s e l e c t o r ': this function allows the robot to add the condition node < WhetherSingleArm > to the planning of BehaviorTree , which is used to determine whether the current manipulation task should use the ' single_arm ' or ' dual_arm ' type of action . " " 2. ' l o c om o t i on _ m o de _ s e le c t o r ': this function allows the robot to add the condition node < WhetherWheelMove > to the planning of the behavior tree , which is used to determine whether the current locomotion task should use the ' wheel ' or ' leg ' type of action . " " 3. ' detection_recovery ': this allows the robot to add the condition node < IsActionSuccess > , which is used to determine whether the previous action has been successfully completed and , if not , to employ a recovery mechanism that repeat the action . " 


Decision: The paper has addressed my concerns. I encourage the authors to follow Ec5Y's advice and frame it as a systems paper. Overall, I found the paper interesting, and showcases an interesting application of VLM and LLM for humanoid control in the real world.


==============================

Strengths:
- Impressive system that showcases what integration of VLM, RL, and VLM can achieve
- Real-world experiment is appreciated.

Weaknesses:
- It lacks a scientific contribution and the paper reads as a tech report that tells the reader how the system can be integrated.

The authors are encouraged to pinpoint the technical challenges they faced and how they solved those challenges.

Reviews: review: title: title: Review
paper_summary: paper_summary: The paper tackles the loco-manipulation problem for a dual-arm quadruped robot. Specifically, the paper focuses on an LLM-based pipeline with various components to complete loco-manipulation tasks such as articulated object manipulation, grasping, and traversing different terrains, where low-level skills are trained with RL and provided in a skill library. Demonstrations are shown on real-robot hardware.
main_review: main_review: Strengths:

- The paper is in general clear to understand and figures are well-made to assist readers in understanding the framework.
- The real-world demonstration is impressive and refreshing to see.

Weaknesses:

- It is unclear whether the paper provides new insights for the community and it lacks a central focus. Specifically, the paper seems to leverage LLMs / VLMs in many components in the pipeline, most of which have been heavily studied in prior works, while using standard RL to train common skills. As a result, none of these may qualify for a unique contribution of the paper. Given the extent of the framework, many important details are missing or simply do not have the space to be discussed in the main paper.
- Similarly, given the extent of the framework, the evaluations appear to be severely lacking and missing important details. For example, what are the success rates of the learned RL policies alone? What is the evaluation procedure (e.g., objects and their initial and goal randomization)? What is the scope of the long-horizon tasks that are using LLMs as planners?
questions: questions: See weakness section above.
limitations: limitations: 3
review_summary: review_summary: While the paper provides impressive real-world demonstrations, it appears that it lacks a central focus to provide insightful contributions to the community. Additionally, given the large scope of the claimed framework, the evaluations and their details appear to be lacking as well. Therefore, I’m withholding a higher recommendation.
title: title: This paper showcases a robot system for long-horizon tasks from human commands, highlighting technical integration but lacking detailed experiments.
paper_summary: paper_summary: In this paper the authors present an fully integrated system demonstration that autonomous execution of long horizon tasks from human language commands on a platform capable of locomotion and manipulation.
main_review: main_review: In this paper the authors present an fully integrated system demonstration that autonomous execution of long horizon tasks from human language commands on a platform capable of locomotion and manipulation.

The robot demonstration on hardware are compelling. They relies on a great effort of integration of novel techniques (VLM, LLM, etc.) and classical ones whole-body controller, behavior trees, etc. It is a great achievement to integrate all of these technical building blocks together in a single autonomous system.

The authors offer valuable insight into the performance of their system with a breakdown of the failure modes between different modules. They also provide extensive details about their modular pipeline. I would say that this paper reads more like a technical report as it describes important technical choices. However, except for Fig 6, the paper does not focus on justifying these choices through thorough experiments, baseline comparisons, and ablations studies. 

Yet I still think this work is an informative data point for other researchers to understand how far we can go with the current tools at our disposal.  

Finally, the authors could further improve the clarity of the paper by concisely stating and pinpointing the contribution of the manuscript in the introduction section.
questions: questions: The authors could further improve the clarity of the paper by concisely stating and pinpointing the contribution of the manuscript in the introduction section.
limitations: limitations: 3
review_summary: review_summary: I would recommend a weak accept since this work is valuable for the research community but its contribution lies in system integration rather than the development of novel techniques.
title: title: HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation
paper_summary: paper_summary: This paper presents a framework for enabling humanoid robots to learn, select, and plan behaviors for loco-manipulation tasks. The approach combines RL for generating a motion primitive library, LLM for high-level task planning, and VLM for selecting appropriate robot morphologies. The authors demonstrate the system on a centaur-like humanoid robot in both simulated and real-world environments.
main_review: main_review: Paper Strengths:
1. A complex integrated system tying multi components like RL, LLM and VLMs.
2. Demonstration of LLMs capability to compose of learned skills.
3. Evaluation in both simulation and the real robot system.

Weaknesses:
1. Limited quantitative evaluation and comparison to baseline methods.
2. Lack of detail on some technical aspects, e.g. the whole-body motion optimization.
3. Disjoint upper body and lower body control and non-general approach for navigation using AprilTags.
4. A large number of system components making it difficult to test, reproduce and deploy.
5. Limited discussion of failure cases.


Suggestions:
- (Lines 19-20) "Experiments in simulation and real-world show that learned motions can efficiently adapt to new tasks, demonstrating high autonomy from free-text commands in unstructured scenes." - while experiments are described, the paper doesn't provide quantitative metrics for adaptation efficiency / capabilities, making this strong claim difficult to fully verify.
- (line 272-274) Please support claim like "We found that language-based behavior planner exhibits greater versatility and adaptability to more complex tasks compared to existing methods." with experimental data - currently the paper has no comparison with other methods
- (line 83 - typo) - teleportation -> teleoperation

Overall, this work presents an interesting approach for leveraging large model in a zero shot fashion for enabling more general and flexible behavior in humanoid robots. The integration of learning-based motion generation with language model planning is particularly promising. However, the experimental evaluation is somewhat limited in scope and more extensive comparisons and analysis would strengthen the paper. Nonetheless, this represents a meaningful step towards more capable and general-purpose humanoid robots.
questions: questions: 1. Can you provide more quantitative comparisons to baseline methods, e.g. planning without language models using a custom decision tree? It is hard to grasp the complexity of the domain for the limited text description of the tasks and how much role do LLMs and VLMs actually play.

2. Can you elaborate on failure cases or limitations observed in your experiments? Are there particular types of tasks or scenarios where the system struggles?

3. Can you provide more details on the whole-body motion optimization process? How do you ensure dynamic feasibility?

4. How did you approach the question of what primitives to include in the learning library and how easy it is to acquire a specific primitive?

5. Can you give a comment on what component of the system could be: combined, remove, added? and why?
limitations: limitations: 3
review_summary: review_summary: The proposed approach shows promise in zero-shot generalization to new tasks. While it represents a step forward in autonomous loco-manipulation, it lacks comprehensive quantitative comparisons and detailed technical explanations. The system's complexity raises concerns about usability and reproducibility. Despite these weaknesses, the integration of language models with robotic control is timely and potentially impactful. I recommend a weak accept, contingent on addressing these issues, particularly by providing more rigorous evaluation and critical analysis of limitations.