Title: Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Humanoid Robots

Locomotion

Loco-Manipulation

Reinforcement Learning
Abstract: Humans exhibit diverse and expressive whole-body movements. However, attaining human-like whole-body coordination in humanoid robots remains challenging, as conventional approaches that mimic whole-body motions often neglect the distinct roles of upper and lower body. This oversight leads to computationally intensive policy learning and frequently causes robot instability and falls during real-world execution. To address these issues, we propose Adversarial Locomotion and Motion Imitation (ALMI), a novel framework that enables adversarial policy learning between upper and lower body. Specifically, the lower body aims to provide robust locomotion capabilities to follow velocity commands while the upper body tracks various motions. Conversely, the upper-body policy ensures effective motion tracking when the robot executes velocity-based movements. Through iterative updates, these policies achieve coordinated whole-body control, which can be extended to loco-manipulation tasks with teleoperation systems. Extensive experiments demonstrate that our method achieves robust locomotion and precise motion tracking in both simulation and on the full-size Unitree H1-2 robot. Additionally, we release a large-scale whole-body motion control dataset featuring high-quality episodic trajectories from MuJoCo simulations. The project page is https://almi-humanoid.github.io.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nTraining curriculum for whole body humanoid controller has been an active area of research lately. This paper proposes ALMI (Adversarial Locomotion and Motion Imitation) . They address this by first treating lower and upper bodies separately. then by training upper and lower body contrllers separately while dilating the other policy to be adverserial to learning of the other policy.  the instantiatione implicitly assumes that lower bod precise footstep and movement is not needed and we only care about velocity. Additionally, the authors release ALMI-X, the first large-scale whole-body control dataset with 80K+ trajectories and language descriptions, enabling foundation model training for humanoid control.\n\nSTRENGTHS AND WEAKNESSES:\n*Strengths:*\n- Code has been provided and is clean with enough intructions for reproducibility. The details provided in the paper is also good. \n- Mujoco dataset shows sim2sim transfer and the dataset collected is good and showcases that such can be used as a dataset for training foundation model directly that can do sim2real transfer. training policy in high throughput sim, and using that to collect data in high fidelity sim to transfer to real world is a good approach. \n- having adversarial curriculum is an intuitive approach to having more robust policy. \n\n*Weakness:* \n- Overall the novelty of the paper boils down to iterative adversarial upper and lower body commands curriculum. It is not clear how much effective it is as compared to other innovations already done in whole body imitation controllers like Masked Humanoid Controller, ExBody2, OmniH20, Hover, MaskedMimic to name a few. It could be wonderful to see some of these baselines for fair comparison. \n- The iterative process takes longer time for training, the compute need of the approach is not clear. does all approaches in the baseline use the same amount of compute and training time, if not it has to be clear when comparing the baselines. \n- There should be a way to do the iterative approach in an integrated manner with a smaller window for switching between upper and lower bod policy. having a fixed number of iteration seems a bit restrictive and compute heavy.\n\nQUESTIONS:\nOne of my main reason for rejection is that I am not quite sold on the notion that upper and lower body needs to be *trained separately*. I do understand that having different policy heads or networks for upper and lower body may have some benefits and should be treated differently in the reward design, but I don't see why they cannot be trained together with some clever sampling of upper and lower body command directives. I am happy to hear some counter arguments to this and am willing to change my score in rebuttal.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents ALMI (Adversarial Locomotion and Motion Imitation), a novel framework for humanoid control that separately trains upper-body and lower-body policies in an adversarial setup. During training, the lower body learns to perform robust locomotion under adversarial disturbances from the upper body, while the upper body learns to track motions accurately despite unstable base movement. The algorithm is validated on the full-size Unitree H1-2 robot. The authors also introduce ALMI-X, a dataset with over 80K trajectories labeled with natural language descriptions. Experiments are conducted in both simulation and real-world settings.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\nThe core idea of treating upper-body and lower-body control as adversaries is novel and interesting. To the best of my knowledge, this is the first time such adversarial interplay has been used to encourage robustness in humanoid control.\n\nThe separation of motion roles (locomotion vs. imitation) helps stabilize policy learning.\n\nThe authors release a dataset (ALMI-X) with language annotations, which could potentially support future work in language-conditioned whole-body control.\n\nWeaknesses:\n\nWhile the adversarial framework is promising, it is inherently limited to motion decomposition (i.e., clear separation between upper- and lower-body tasks). It may not generalize well to agile whole body coordination.\n\nThe foundation model component feels disconnected from the core contribution. If I understand correctly, the model is trained in a supervised fashion on the collected ALMI-X dataset using language annotations, but the results are still far from a true \"foundation model\" for whole-body control.\n\nIn comparison, stronger motion generation performance might be achieved by combining diffusion-based human motion synthesis with a physics-based motion tracking controller.\n\nThe real-world robot experiments appear overly conservative, especially in locomotion behavior. The demo videos on the project page mostly showcase the ALMI policy, not the foundation model. Is that correct?\n\nQUESTIONS:\nThe policies are trained and tested in Isaac Gym, yet the dataset is collected in MuJoCo. What’s the motivation or specific consideration behind using MuJoCo for ALMI-X data collection?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ALMI (Adversarial Locomotion and Motion Imitation), a novel framework for humanoid robot control that addresses the challenge of whole-body coordination by separately learning policies for upper and lower body control through adversarial training. The key insight is that conventional approaches that mimic whole-body motions often fail because they don't account for the distinct roles of upper body (motion tracking) and lower body (stable locomotion). The authors formulate this as a two-player zero-sum Markov game where the lower body learns robust locomotion while the upper body tracks various motions, with each acting as an adversary to the other. They demonstrate their approach on the Unitree H1-2 robot, create a large-scale dataset (ALMI-X) with 80K+ trajectories and language annotations, and train a foundation model for end-to-end control.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n+ The paper includes extensive experiments in simulation and real-world deployment, demonstrating robust performance across various metrics including motion tracking accuracy and locomotion stability.\n+ The method achieves impressive real-world results on the Unitree H1-2, including loco-manipulation tasks with teleoperation, showing clear practical applicability.\n+ ALMI-X represents a valuable contribution to the community - a large-scale humanoid control dataset with language annotations that could enable future research.\n+ Dual curriculum mechanism for adversarial motion sampling is well-designed, starting with moderate disturbances and gradually increasing intensity.\n\nWeaknesses\n- The paper acknowledges simplifying the original max-min problem by sampling adversarial commands rather than updating both policies simultaneously. This may limit the effectiveness of the adversarial training.\n- The foundation model results are preliminary and show degraded performance when trained on the complete ALMI-X dataset versus subsets, suggesting scalability issues.\n- The paper only compares against Exbody and ablations of their own method. Comparisons with other state-of-the-art humanoid control methods would strengthen the evaluation.\n- While the method works well for the specific robot and motions tested, it's unclear how well it would generalize to different humanoid platforms or more complex manipulation tasks.\n\nQUESTIONS:\n* How does your approach compare to recent work on language-guided robot control, particularly by Mower, Bou Ammar and others (integrated into \"ROS-LLM: A ROS framework for embodied AI with task feedback and structured reasoning\")? While your method uses language annotations in the dataset, ROS-LLM integrates LLMs directly into the control loop for task reasoning and feedback. Could you discuss the trade-offs between your adversarial RL approach versus LLM-based reasoning for humanoid control?\n\n* The simplified framework samples adversarial commands rather than updating both policies. Have you experimented with true simultaneous policy updates? What are the computational and stability trade-offs?\n\n* Why does the foundation model performance degrade on the full dataset? Have you considered alternative architectures or training strategies to address this limitation?\n\n* How dependent is ALMI on the specific kinematics of the Unitree H1-2? Would it work with a TALOS or a G1 or a Booster? What modifications would be needed to apply it to other humanoid platforms?\n\n* The experiments focus on relatively short motion sequences. How does the method perform on longer-horizon tasks that require extended coordination between upper and lower body?\n\n* What are the primary failure modes of the trained policies? When does the adversarial training fail to find a good equilibrium?\n\nLIMITATIONS:\nThe authors partially address limitations in Section 7, but several key limitations deserve more discussion:\n\n- The paper doesn't discuss the computational requirements for the adversarial training process or how it scales with robot complexity.\n- The approach relies on motion retargeting from human data, which may limit the types of behaviors that can be learned.\n- While teleoperation is demonstrated, the paper doesn't discuss safety mechanisms for autonomous operation in human environments.\n- Although successful sim-to-real transfer is shown, the specific domain randomization required suggests potential brittleness.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents the ALMI framework, which introduces an adversarial learning paradigm for training upper and lower-body policies in humanoid control. The key idea is to formulate the coordination between upper and lower body as a two-player zero-sum game, thereby encouraging robustness and precision in full-body motion generation. The method is validated through comprehensive experiments both in simulation and on the real H1 humanoid robot. Additionally, the authors contribute a large-scale language-annotated dataset for whole-body control, which holds promise for advancing foundation policy learning in robotics.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n1. Novel adversarial learning design: The proposed adversarial formulation for upper-lower body coordination is a promising and innovative direction. Framing the interaction as a two-player zero-sum game, combined with curriculum learning for the lower body, introduces a structured and scalable approach to whole-body policy learning.\n2. Real-world deployment: The evaluation on the H1 humanoid robot in real-world settings is a major strength, demonstrating the practicality and robustness of the proposed method beyond simulation.\n3. Good writing and experimental design: The experiments are well-structured, with evaluation metrics covering motion tracking accuracy, command fidelity, stability, and survival rate. The ablation studies effectively highlight the benefits of different modules within the framework.\n4. Open-source contributions: The release of code and a large-scale dataset with language annotations adds substantial value to the community and paves the way for future research in language-conditioned whole-body control.\n\n## Weaknesses\n1. Reward design in Appendix: The reward structure, which plays a critical role in training, is relegated to the appendix. Including a concise version of it in the main text would greatly benefit the reader's understanding of the method.\n2. Symbol clarity: The preorder symbol introduced in Line 117 should be clearly defined to avoid ambiguity.\n3. Limited baseline comparison: The experiments compare only against Exbody. Including additional baseline methods, such as recent approaches in modular or hierarchical control, would strengthen the empirical validation.\n4. Minor typos:\n   - Line 90: The symbol should be \\( a^u \\) instead of \\( a_l \\).\n   - Line 159: The citation should refer to Eq. (1) instead of Eq. (2).\n\nQUESTIONS:\n1. In Lines 195–196, the adversarial signal to the upper-body policy consists solely of velocity commands from the lower body. Could the authors clarify why this information alone is sufficient? For instance, would incorporating additional signals such as gait type, gait frequency, or phase provide stronger adversarial influence and improve learning outcomes?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-13"}