HWC-Loco: A Hierarchical Whole-Body Control Approach to Robust Humanoid Locomotion
Authors: Sixu Lin
Guanren Qiao
Yunxin Tai
Ang Li
Kui Jia
Guiliang Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Humanoid
Reinforcement Learning
Whole-body Control
Primary area: applications to robotics, autonomy, planning
TL;DR: We introduce HWC-Loco, a hierarchical humanoid control algorithm designed to dynamically balance the trade-off between optimizing locomotion performance and ensuring safety across diverse deployment environments.

Abstract: Humanoid robots, capable of assuming human roles in various workplaces, have become essential to embodied intelligence. However, as robots with complex physical structures, learning a control model that can operate robustly across diverse environments remains inherently challenging, particularly under the discrepancies between training and deployment environments. In this study, we propose HWC-Loco, a robust whole-body control algorithm tailored for humanoid locomotion tasks. By reformulating policy learning as a robust optimization problem, HWC-Loco explicitly learns to recover from safety-critical scenarios. While prioritizing safety guarantees, overly conservative behavior can compromise the robot's ability to complete the given tasks. To tackle this challenge, HWC-Loco leverages a hierarchical policy for robust control. This policy can dynamically resolve the trade-off between goal-tracking and safety recovery, guided by human behavior norms and dynamic constraints. To evaluate the performance of HWC-Loco, we conduct extensive comparisons against state-of-the-art humanoid control models, demonstrating HWC-Loco's superior performance across diverse terrains, robot structures, and locomotion tasks under both simulated and real-world environments.

Published as a conference paper at ICLR 2026
HWC-Loco: A HIERARCHICAL WHOLE-BODY CON- TROL APPROACH TO ROBUST HUMANOID LOCOMO-
TION
Sixu Lin', Guanren Qiao', Yunxin Tai?, Ang Li', Kui Jia', Guiliang Liu 1 School of Data Science, The Chinese University of Hong Kong, Shenzhen, 2 DexForce Technology, SShenzhen Loop Area Institute.
ABSTRACT
Humanoid robots, capable of assuming human roles in various workplaces, have become essential to embodied intelligence. However, as robots with complex physical structures, learning control model that can operate robustly across diverse environments remains inherently challenging, particularly under the discrepancies between training and deployment environments. In this study, we propose HWC- Loco, robust whole-body control algorithm tailored for humanoid locomotion tasks. By reformulating policy learning as robust optimization problem, HWC- Loco explicitly learns to recover from safety-critich scenarios. While prioritizing safety guarantees, overly conservative behavior can compromise the robot's 's ability complete the given tasks. To tackle this challenge, HWC-Loco leverages a hierarchical policy for robust control. This policy can dynamically resolve the trade-off between goal-tracking and safety recovery, guided by human behavior norms and dynamic constraints. To evaluate the performance of HWC-Loco, we conduct extensive comparisons against state-of-the-att humanoid control models, demonstrating HWC-Loco's superior performance across diverse terrains, robot structures, and locomotion tasks under both simulated and world environments.
INTRODUCTION
Humanoid robots, with physical structure sembling that humans, can be scamlessly inte- grated into humans" workspace and take roles in completing different tasks Saeedvand et al. (2019). This capability makes them reliable em- bodiments of artificial intelligence in various forms Paolo al. (2024). In recent years, with advancements in both hardware capabilities and control algorithms, humanoid robots have be- come an increasingly significant type of robot with growing impact on practical applications across various environments, such factories, homes, and offices (Nasiriany et al., 2024; Tao et al., 2024; Zhang et al., 2025b).
Disturb - Detect Stability Recover
Figure 1: An example of recovering from a Hard
Kick: The humanoid robot withstands external disturbance by automatically detecting hazardous states and adjusting its motion regain stability.
In the development of humanoid control, classic methods commonly rely model-based optimization (Sakagami et al., 2002: Sentis & Khatib, 2006; Gouaillier et al., 2009; Radford et al., 2015; Chignoli et al., 2021). However, these methods require precise and comprehensive modeling of the robot's structure, kinematics, and dynamics across different environments. In practice, obtaining such data requires significant computational resources manual effort. These limitations significantly influence the scalability of these approaches. To develop an end-to-end solution with promising generalizability. recent studies (Zhuang al., 2024; Radosavovic al., 2024: Long et al., 2024; He et al., 2024b: Gu et al., 2024c: Qiao et al., 2025) adopted Reinforcement Learning (RL) methods by training a neural model control based on human demonstrations and interactions with the environment.
"Corresponding author: Guiliang Liu, liuguiliang www.w.eec..c.....

---

Published as a conference paper at ICLR 2026
While these learning-bas based approaches show potential for effective control policies across various tasks, they are typically trained in simulated environments Qiao et al. (2023). which often differ considerably from the deploying environment. To bridge the Simulation-to- -e-aal (Sim2Real) gap. previous works often introduce additional regularization to constrain the robot's movements or employ domain randomization to account for variations in the robot's 's physical properties (Lai al., 2023; Rudin et al., 2022; Nahrendra et al., 2023). However, excessive regularization can greatly affect the efficiency of control policy, and unstructured randomization often fails capture safety-critical patterns in real-world applications.
To develop a reliable locomotion policy capable of generalizing from the training to the deployment environment, we propose formulating policy optimization as robust optimization problem under misspecified environmental dynamics. However, traditional robust optimization primarily focuses on worst-case control, requiring the policy to ensure safety under the most adverse environmental dynamies within an uncertainty set. Under this context, the learned policy tends to be overly conservative, inducing sub-optimal control performance in terms of achieving the given task.
In this paper, to learn a control policy that can dynamically solve the trade-off between task perfor- mance and maintaining safety under different deployment environments, we present Hierarchical Whole-body Control for Robust Humanoid Locomotion (HWC-Loco), where a high-level planning policy switches between a goal-tracking policy for task exccution and a rafoty-r policy for stability under disturbances. The goal-tracking policy ensures task performance and human-like motion patterns, while the safety-recovery policy enforces ZMP-based constraints under extreme scenarios. A high-level planner coordinates these policies by dynamically selecting the appropriate one based on the deployment situation. Figure illustrates an example of HWC-Loco, where high-level planning policy detects hazardous states and seamlessly switches from the goal-tracking policy to a safety-recovery policy.
We conduct extensive experiments to evaluate the performance of HWC-Loco from four fundamental perspectives: 1) effectiveness, demonstrated across diverse terrains, 2) robustness, evaluated under varying scales of disturbances, 3) naturalness, exhibited in its ability to imitate human behavioral norms, 4) scalability evaluated in different robot embodiment and motion tracking tasks. These validations are performed in both simulated and real-world settings, demonstrating HWC-Loco as foundational advancement for achieving reliable humanoid locomotion in safety-critical scenarios.
2 RELATED WORK
Learning-based Legged Robot Locomotion. Various learning-based methods have been proposed for legged locomotion, such as quadrupedal locomotion (Rudin et al., 2022; Shi et al., 2022: Chen et al., 2023; Nahrendra et al., 2023; Margolis & Agrawal, 2023; Zhuang et al., 2023; He et al., 2024d) and bipedal locomotion (Li et al., 2021; Kumar et al., 2022; Li et al., 2023; Duan et al., 2024). Despite their success, these approaches cannot be directly applied to humanoid robots because of the complex physical structure and higher degrees of freedom (DOF). Recent advances in hardware and learning methods have enabled humanoid robots to move through diverse environments. Some humanoid control policy navigates across diverse terrains using only proprioceptive information (Radosavovic al., 2024; Gu et al., 2024b;e). Others enhance performance by incorporating additional sensors, such as vision or LiDAR, to collect detailed environmental data, enabling robots to perform complex tasks like stair climbing and parkour jumps (Zhuang et al., 2024; Long al., 2024; Sun et al., 2025). While these approaches demonstrate promising locomotion performance, they lack mechanisms to handle safety-critical scenarios humanoid robots may encounter during real-world deployment.
Whole-body Control for Humanoid Robots. Learning effective whole-body control remains
central challenge for humanoid robotics. Prior works (Luo et al., 2023a; 2024a;b) have demonstrated impressive results in simulated avatars, showcasing the feasibility of lcarning-basee control strategies. More recently. studies such as (Cheng et al., 2024; He et al., 2024b;a; Fu et al., 2024; He et al., 2024c; 2025; Ze al., 2025; Li et al., 2025; Liao et al., 2025; Huang et al., 2025; Zhao al., 2025) have enabled realistic robots to imitate complex human-like motions, such as boxing and dancing by leveraging motion priors from human demonstrations. Additionally, previous methods employed multi-stage reward designs to accomplish complex tasks (Kim et al., 2024; Zhang et al., 2024a). However, the success of these methods often relies on intricate reward shaping and struggles to generalize across different humanoid robots, particularly when motion priors are unavailable.
2

---

Published as a conference paper at ICLR 2026
3 PROBLEM FORMULATION
Learning Environment. We formulate the environment of the humanoid locomotion task with a Partially Observable Markov Decision Process (POMDP) M = AS, PT Mo, 7), where: 1) Within the state space S, a state S records the environmental information and the robot's internal states. 2) A denotes the action space, and action E A denotes the target joint angles that a PD controller uses to actuate the DOFs. 3) O denotes an observation, which provides partial information about the state of the agent due to sensory limitations and environmental uncertainty. At time step t, or includes velocity command e umna E R and proprioception of that records humanoid's ss internal state. Appendix A.2 introduces the details. By incorporating the temporal observations of = 00: Ot-1, -t-Hr), HI, the control model can summarize a state as 8 = =oo Pt). where Pt is the privileged information which the robot can't It access in realistic deployment, including base velocity Vt, terrain height, external disturbance and ZMP features (see Section 4.3). 4) Pr E A denotes the transition function mapping from state-action pairs states. 5) r denotes the reward functions, which typically consist of penalty, regularization, and task rewards. These reward signals significantly influence the optimality of the control policy, for which we provide a detailed introduction in the following. 6) 1s E A denotes the initial state distribution. 7) 7 (0, 1J denotes the discounting factor. The robot's policy stops terminating state 5, and the corresponding terminating time is denoted as T E (0, oc).
distribution of future
Under this POMDP. our goal to learn a policy x E A according to the objective as follows:
max [(r, max E PT. > [3ers at) + Ser at) + (s at)
(I)
In humanoid locomotion, the total reward is typically expressed weighted sum of task rewards rg, penalty rewards rp, and regularization rewards rox (He et al., 2024b;a; Zhuang et al., 2024), each serving distinct purposes:
Task rewards rgg evaluate goal achievement, e.g., velocity tracking Zhuang al. (2024), contact management Zhang et al. (2024a), or expressive motions Cheng et al. (2024). They directly reflect policy optimality and thus serve the main objective to maximize.
Penalty rewards rp discourage undesirable outcomes such as falls or violations of torque and joint limits. Recent work (He et al., 2024b;a) assigns large weights 3ep, effectively acting as Lagrange multipliers Gu et al. (2024a) but fixed rather than optimized. A more prineipled formulation is therefore constrained RL problem.
Regularization rewards i't encourage motions aligned with human preferences for style safety. Since such specifications are task-dependent and expert-driven Chen et al. (2024), we instead align robot motion with human datasets (e.g., AMASS Mahmood al. (2019)), which embed rich behavioral priors.
Constrained RL for Whole-Body Control. Inspired by the aforementioned analysis, we formulate the body control locomotion objective for humanoid robots as follows:
max E.o.PT.
Σ Srtr(sss at) s.t. (D (P S ef and (p- [o(T)] S cs
where 1) we replace the hard regularization in rot with a mimic learning objective Pp 2 (PTT)) that constrains the distributional divergence between the learned policy's occupancy measure Piv and the expert policy's occupancy measure PTr In this study we implement D, by Wasserstein distance (Section 4.1). 2) Instead of relying on the penalty reward rp. we use á to capture the feasibility of the current trajectory generated by the policy π under the environment M. Within the objective. only the task rewards rgg are subject to maximization; other desired characteristics of humanoid robots are captured by constraints learned from human motion datasets. More importantly, this approach effectively eliminates the need for regularization the humanoid robot's 's upper or lower pose, significantly enhancing the learning of whole-body locomotion policies.
Robust Locomotion in Humanoid Robot. A common approach to learning humanoid locomotion policies involves training in a simulated environment before deploying the policies in a real-world setting. Due to the complexity of real-world environments, there is often a significant discrepancy between the training and deployment environments. In this study, we characterize this discrepancy by the mismatched POMDPs defined follows:
3

---

Published as a conference paper at ICLR 2026
Definition 3.1. (Mismatched POMDPs) We define POMDP M := (S, A. O,r, Mo, 7), to be mismatched with another POMDP M' := (S, A, Py. O, r, /o, 7) if they differ only in transition functions (i.e., Pr * P7). We define a set of mismatched POMDPs as Mm = {M fT M "r , where their transition functions differ from each other.
Note that we follow previous studies Moos et al. (2022) and primarily focus the mismatch in transition dynamics during learning and deployment. The uncertainty set of transition function Viano et al. (2022) is represented as:
P +
VP E
where a specifies the scale of mismatch, P+ denotes the transition function in the learning environ- ment and & S A represents the set of all candidate transition functions. While other factors, such as divergence in state-action spaces and initial state distribution /4., can influence the performance during deployment, we find that the majority of factors affecting Sim2Real performance in humanoid locomotion tasks can be characterized by the mismatch in transition dynamics. For example, Py denotes Gaussian function, the underlying sensor noise can be captured by transitions in F Additionally, if Py denotes a projection from one spatial state to another, the variations terrains in the deploying environment can be modeled by transitions in P4 Long al. (2024).
To solve a robust RI. problem, previous studies often consider a max-min objective max, Mi, J(H, .)) where [TT, denotes standard RL objective (1). With this ob- jective, while the agent can learn conservative policy to ensure worst-case control performance, this policy often compromises the control performance, making the humanoid less effective at tracking the given commands. To address this issue, we focus on ensuring a worst-case feasibility constraint, thereby reducing the influence of mismatched POMDP on maximizing task rewards. Accordingly, we update the CRL objective (2) to the robust humanoid locomotion objective represented as:
max min E Bpo,Pf.n Pr Ppp
Σ C'rs(s),
s.t. Dr MP4 M S Ef and E (r- [Ø(T)] S Es
llp"
where P. represents the set of mismatched dynamics from the learning environment dynamics Pf (defined in 3) and 2) P5g (8, a) = (1 - 7) (a]s) -'=0,"p(s s/m, M) defines the normalized occupancy measure of policy æ under the environment M. Intuitively, we aim to learn robust
control policy that ensures feasibility across all mismatched transition functions Pr E P2, thereby However, for the eeward-maximizing and mimic learning
guaranteeing the safety of policy objectives, we optimize the policy based on the transition dynamics in the learning environment Pf. rather than concentrating solely on worst-case guarantees.
4 HIERARCHICAL WHOLE-BODY CONTROL FOR HUMANOID LOCOMOTION
In practice, developing an end-to-end solver for the robust locomotion problem (4) is challenging, as it involves multiple deployment environments and constraints that must be adhered to. To tackle this issue, we propose dividing the objective into two stages: goal-tracking and safety recovery. The goal-tracking policy maximizes rewards while mimicking human behaviors (see Section 4.1). When mismatched environmental dynamics pose a risk to the safety of the control policy. HWC-Loco switches the safety recovery policy, which manages safety-critical events to ensure the feasibility of humanoid control (see Section 4.2). To dynamically identify the "sweet spot" for policy switching or activation, we introduce high-level policy that coordinates the low-level policies based on historical observations and the robot's current status (see Section 4.3). Figure 2 illustrates the training pipeline of our HWC-Loco.
4.1 LEARNING
POLICY FOR EFFICIENT HUMANOID LOCOMOTION
The goal-tracking policy primarily focuses on tracking the provided command within the learning environment efficiently and naturally. This is achieved by optimizing the following objective function:
max E PT T1
Σ
Trr ar) s.t. Dy of
(5)
Note that compared to the complete objective (4), this goal- trakktra objective focuses only on maximizing performance in the training environments' dynamics P.P. Within this objective, the scale
4

---

Published as a conference paper at ICLR 2026
(4) Stage 1:
(b) Stage 2:
Gesl-mmkiigg
Disaiiiali
Relovnse
Vandiealy Updke
CRIS
D
Unter
Bapky Balkr
crgrerty
Pullay
Uver Commund
Pollcy
Sarakian
13 coznela
Action
Safty Recovery
Polley
Figure 2: Overview of HWC-Loco: The framework consists of two stages: (a) Training goal-tracking policy to effectively enable human-like locomotion across diverse terrains (Section 4.1) and safety recovery policy recover from safety-critical states (i.e., extreme-case) (Section 4.2). (b) Training the high-level planning policy to select between the two pre-trained low-level policies (Section 4.3), thereby ensuring stable and consistent locomotion.
of the cumulative task rewards, Et=o t= y'rs reflects how effectively the policy tracks locomotion- related commands. Appendix A.2 specifies the configuration of our task reward.
In addition to task rewards, a crucial aspect of the goal-trace tracking objective (5) is evaluating the distance between the occupancy measures of the trained and expert policies. In the following, striving for concise representation, we represent o Mr"4 and pT " by p²* and pT. Driven by information theory, previous works Hussein al. (2017) primarily implement D, using metrics such Kullback-Leii Leibler (KL) divergence or p= and p" has limited zero overlap. In this study. consider implement D, with Wasserstein-1 1 distance under the Santorovich-Rubinstee duality Villani et al. (2009) such that:
(JS) divergence. However, these metrics fail when the support of
Dr 100))
sup E Ex [Ja(x)] En-pri [fa(x)]
1elll
where the bounded Lipschitz-norm [l.allz I ensure the smoothness of functions fa. Inspired by adversarial imitation learning (Ho Ermon, 2016: Zhang et al., 2020), an effective and intuitive implementation of fa is a discriminator for measuring whether x is generated by human experts, such that ʃa X + R. It can be learned by maximizing the following objective:
max E 2~00 = [fa(x)] - Exwprr [fa(=)] + B B- 2 =0,== ,0=1 [(I)afa())ll - 1)²]
(7)
fa
where weighted by 8, the gradient penalty term enforces the Lipschitz continuity required by the Wasserstein distance (i.e., [|la 1). During training, p²² represents the density of expert demonstration dataset DE, which is built from the CMU MoCap dataset cmu. Additionally, we include data about humans standing, walking, and running learn various human behaviors. We then retarget the motion data into humanoid robot-compation format, which we use as the training dataset. By substituting Dy with equation (6), the waal-trccking objective (5) can be simplified as:
max EE Ep
Σ (rr at)
s.t.
E E [fa(s)]
Epr Sey
(8)
where sd records the necessary state information for the discriminator. (See details in Appendix A.5). Given that this formulation of constrained RL problem has duality gap Paternain et al. (2019), we can transform it into an unconstrained objective by analyzing the Lagrange dual form of the original constrained RL problem:
max MT
E
T
r' at) {fa(s")]
where denotes the optimal Lagrange multiplier. In practical implementation, the policy и and discriminator fa can intuitively represent the generator and the discriminator under the adversarial
5

---

Published as a conference paper at ICLR 2026
learning framework. In implementation. we utilize the proximal policy optimization algorithm Schul- man et al. (2017) to update the goal-tracking policy. By alternatively updating (7) and (9), we can develop a goal-tracking policy that effectively maximizes task rewards while closely mimicking the expert's hehavior.
4.2 LEARNING SAFETY RECOVERY POLICY FOR HANDLING SAFE-CRITICE EVENTS
Our safety recovery policy primarily focuses on handling emergency events, thereby recovering the robot from safety-critical situations and preventing failures of locomotion tasks, such as loss of balance or control policy malffucctons. To learn the safety-recowe policy, we mainly study the complete objective (4). Similarly, as it is introduced in Section 4.1, we implement the divergence metric D with Wasserstein-1 distance, deriving the following objective:
max min E 32 PrEPE Pr.
Σ Ars(st, ar) Xfa(se.
s.t. E, Br-(un.Pr.a)) [Ø(T)) S co. (10)
where c is a hyperparameter. By comparing this objective with the goal-tracking objective (9), while both consider task rewards and human mimicking performance, their key differences lie in the incorporation of uncertainty in environmental dynamics and feasibility constraints in policy updates. These additions necessitate the construction of an extreme-case uncertainty set and the modeling of safety constraints by learning the feasibility function o(.).
Extreme-case Uncertainty Set. Real-world deployment has complex terrains, external disturbances, hardware malfunctions, and sensor noise. Those factors may lead to extreme cases like falling or unexpected behaviors. To model them in the simulation (i.e., modelling PTs), We: 1) apply multi- scale external forces and torques to the humanoid body, 2) introduce random, high-intensity noise to the proprioceptive information and PD gains, 3) resample velocity commands to simulate malicious velocity inputs Shi et al. (2024) within the goal-tracking policy, and 4) apply domain randomization to the training environment. Together, these dynamic variations construct the uncertainty set P/, under which we learn the safety-recover policy. Appendix A.6 shows our implementation details. Zero-Moment Point (ZMP) Constraint. A bipedal robot can be generally modeled by linear inverted pendulum (Morisawa et al., 2006; Harada et al., 2006), where 7MP refers to the point where the ground reaction force has no horizontal moment. If the ZMP exits the support polygon (typically the foot's contact area), the robot will quickly lose balance (Wieber, 2006; Feng et al., 2016; Scianca et al., 2020; Xie et al., 2025). Stability is governed by gravity and the Center of Mass (CoM) accelerations, which also account for disturbances like obstacles and slippery surfaces. Under this assumption, implement the feasibility indicator in the safety recovery objective (10) as follows:
((s, a) = [[zMp(ss (s, a) Pacll2 where pZMP (s, a) = PCoM a) =CoM(s,
PCOM(s, a) (11)
where 1) the vector PCOM denotes the x and y-coordinates of CoM position. 2) #CoM is the height of the CoM. 3) g represents the acceleration due to gravity. 4) PCOM indicates the accelerations of the CoM in the x and y-directions. 5) Pac represents the center of the support polygon (specific to each robot and known in advance). Intuitively, & provides real-time insights into the robot's stability and varies in its representation depending on the current support phase. Therefore, the humanoid robot can assess its stability real time and use whole-body coordination to satisfy the ZMP constraint.
4.3 HIGH-LEVEL PLANNING FOR POLICY TRANSITION
While the low-level policies can achieve varying levels of op- timality, fundamental challenge is coordinating these policies to ensure that a humanoid robot can efficiently complete tasks while adhering to safety and robustness requirements. We intro- duce a high-level planning policy that dynamically determines which policy activate based the specific state of humanoid robots. The task of policy selection has a discrete action space, and we learn this high-level policy To based on the following objective:
203
Prpprttss
Figure 3: Average activation ratio of goal-tracking policy.
max E
Y |nr a.) 1(ā,11 # a) al(ss)]
(12)
1=0

---

Published as a conference paper at ICLR 2026
where 1) the policy selection action a, E {0, 1)2 denotes one-hot vector featuring whether the goal-tracking policy Ti and safety-recovery policy T2. 2) [ * a.) denotes the continuality
identifier for discouraging the over-frequent switch of policies, and 3) (s;)) denotes the termination identifier for preventing the failing of locomotion tasks. (2) and (3) are both rp defined in Section 3. More importantly, within this objective, by adjusting o, practitioners can adjust the trade-off safety guarantee and task complction. Training and algorithmic details can be found in Appendix A.2. We report the average activation ratio of я during training in Figure 3. For a {0, 20, 50), I dominates and T2 is rarely invoked. In contrast, a 200 destabilizes training and induces overly conservative behavior due to sparse termination rewards.
Practical Deployment. The training of humanoid control policy commonly relies on the privileged information Pt, such as external disturbances, terrain dynamics, etc. We follow Nahrendra al. (2023) and infer P2 by training a VAE-based estimator P(ex, ze 24 where 2; captures the contextual embedding of the y,,, and e, consists of body velocity and 7MP features. To enhance the representation of <()), we introduce frequency encoding, similar to the approach in Mildenhall et al. (2021), to capture the subtle variations in o(.) for the policy, Additionally, instead of the zero-mean Gaussian prior, we introduce a learnable prior similar to Luo al. (2023b). Moreover, we include certain degree of domain randomization in the learning of HWC-Loco policies. The details of domain randomization are in Appendix A.6.
5 EXPERIMENT
Experiment Settings. To conduct comprehensive evaluation, quantify the control performance of HWC-Loco in simulated (Isaac Gym Makoviychuk et al. (2021)) I)) and realistic environments from the following perspectives: 1) Effectiveness: How effectively does HWC-Loco navigate across diverse terrains? 2) Robustness: How well can HWC-Loco stabilize the humanoid robot under varying levels of disturbance? 3) Naturalness: How well does HWC-Loco imitate human-likk movement norms? 4) Scalability How effectively does HWC-Loco generalize to diverse embodiments and tasks?
The proportion of successful traverse across different scenarios. Cui et al. (2024) b) Goal Tracking performance: The ability to accurately follow velocity commands by maximizing task rewards rı detailed in Appendix A.2 Chen et al. (2024). e) Human- Like behavior: Measured as the Wasserstein-1 1 distance between the robot's 's and human motions. d) ZMP Deviation: The proportion of time the ZMP lies outside the support polygon.
Our evaluation metric includes a) Success Rate:
To evaluate the effectiveness of different components in HWC-Loco, we compare against domain- randomized policy baselines isolate the benefits of our hierarchical design: 1) HWC-Loco-I sets to a lower value, thereby reducing the sensitivity of the high-level policy to failure events. 2) Toal-Tracking Policy removes the safety-recovery mechanism and relies solely on the goal-tracking policy T, (Section 4.1), which is trained under the same domain randomization as HWC-Loco. 3) Dream DreamWaQ-An -umaanid removes the human imitation objective D, from objective (5), yielding a adaptation of Dream WaQ Nahrendra et al. (2023) for humanoid control. 4) To further highlight the advantages of our hierarchical policy, we compare against a recent history- aware extension of Dream WaQ. called AHL Cui et al. (2024), which employs two- two-phase training. Implementation and tuning details for all baselines are provided in Appendix A.7.
5.1 EFFECTIVENESS: LOCOMOTION ACROSS DIVERSE TERRAINS
Effectiveness in Simulated Environments. We evaluate the policies on challenging terrains, specif- ically slopes and stairs. To enable a comprehensive comparison, evaluations are performed un- der both low-speed and high-speed command conditions. Low-speed commands are sampled as 2t [0.0, 1.0], of ~ [-0.3, and w, ~ [-0.5, 0.5], while high-speed commands follow 2f ~ 2.0]. oe ~ [-0.6, 0.6], and w ~ [-1.0, 1.0). Here, of and of denote the commanded linear velocities along the x- and y-axes, respectively, and wy denotes the commanded angular velocity around the z-axis. Table 1 shows that our HWC-Loco achieves the highest success rate across all evaluated scenarios, while maintaining promising goal tracking and human mimic performance. Notably, in the high-speed stair terrain setting, HWC-Loco has significantly higher success rate than all other policies, although its goal-trackin trackina ability slightly decreases. This indicates that the policy prioritizes stability over aggressive goal pursuit. For example, when given high-speed command, HWC-Loco enables the policy to adapt dynamically in safety-critica situations (c.g., navigating stairs mid-air) rather than rigidly maintaining high velocity. Comparably, when downplaying the sensitivity
7

---

Published as a conference paper at ICLR 2026
to safety- crittcal events and removing the safety-recoven policy, the success data drops significantly from nearly 85% to around 60% in the testing environment with stairs and high-speed commands.
Effectiveness in Realistic Environments. Figure 10 (see Appendix B.6) demonstrates the robot's ability to climb 15 cm stairs and 20 slopes. When faced with challenging terrain, the robot prioritizes safety by activating the recovery policy. After regaining stability, it smoothly transitions back to goal tracking, showcasing its adaptive control in dynamic and uncertain environments. Figure 11 (see Appendix B.6) further illustrates the robot's effectiveness in real-world outdoor settings, where successfully navigates across diverse terrains, including flat ground, grassy surfaces, and slopes.
Table 1: Locomotion performance in simulated environments. Each evaluation runs for 1200 steps, which is equivalent to 12 seconds of real clock time. + corresponds to the standard deviation of the performance on random seeds. The best result of each setting is marked as bold
Method
Slopes
Stairs
Success Rate
Goal Tracking t Human like
Success Rate
Goal Tracking 1
Human like 1
(a) Low Speed
PreamWWa
92.31 0.40
1.19 0.02
3.52 + 0.05
74.32 1.30
1.19 0.03
3.53 +
AHL
98.83 = 0.15
1.21 : 0.01
3.41 + 0.04
93.73 = 0.70
1.21 = 0.04
3.44 + 0.01
Goal-tracking
99.90 0.04
1.31 + 0.00
3.30 + 0.05
96.60 = 0.14
1.31 :: 0.00
3.31 + 0.01
HWC-Loco-I
100.00 - 0.00
1.23 1.23::0.02
3.32 + 0.03
99.80 0.08
1.21 0.02
3.31 + 0.06
HWC-Loce
100.00 0.00
1,22 0.02
3.32 +: 0.05
99.98 + 0.01
1.19 0.03
3.34 + 0.03
High Speed
DreunWaQ
1.05
3.63
60.58 = 0.64
1.06
3,66 + 0.02
AHI.
97.36 0.23
0.02
3.76 +: 0.02
67.48 + 0.77
1.09 + 0.01
0.01
3.68 + 0.04
Goal-tracking
0.45
1.13
3.52
72.60 0.97
1.11
3.43 0.02
HWC-Loco-1
99.95 0.02
0.02
3.43 0.04
78.92 = 0.45
1.10
3.39 + 0.05
HWC-Loco
100.0 0.00
0.02
3.44 1: 0.09
84.34 1 0.43
1.07 + 0.03
3.46 +: 0.03
5.2 ROBUSTNESS: STABLE CONTROL UNDER DISTURBANCES
Robustness under Simulated Disturbances. To demonstrate the robustness of HWC-Loco, we conduct disturbance tests in simulation. The robots commanded follow sampled velocities within the training distribution while navigating uniformly mixed terrain comprising flat ground, obstacles, slopes, and stairs. Under these general locomotion settings, we design three types of disturbances as follows: 1) External force/torque disturbances. where random forces and torques (up to 200 and 200 N-m) are applied each robot link, either low frequency of 1 Hz or as constant perturbations lasting 0.5 seconds, following prior setups (Weng et al., 2023; Lee et al., 2020). 2) Impulse disturbances on the CoM, implemented by directly altering the robot's CoM velocity (Weng et al., 2023; Gu et al., 2024b). We simulate both low-impulse and high-impulse scenarios, with linear velocity perturbations of 0-1 m/s or 1-2 m/s, and angular velocity perturbations of 0-0.5 rad/s 0.5-1.0 rad/s, respectively. 3) Payload disturbances, where an additional mass uniformly sampled from either 0-5 kg or 0-10 kg is added to upper- upper-body links (Margolis & Agrawal, 2023; Radosavovic et al., 2024), along with internal perturbations from sensor and actuator noise (see Table 11 in Appendix A.6), to simulate unmodeled dynamics and varying mass distribution scenarios.
As shown in Table 2, HWC-Loco consistently achieves the best overall robustness across disturbance types, attaining the highest success rates while maintaining the lowest 7MP Deviation. Under constant disturbances, reaches success rate of 75.95% % with ZMP Deviation of 6.61 %. evidencing effective resistance to strong, persistent forces. For low-impulse perturbations, all methods remain relatively strong, yet HWC-Loco still leads 94.84 %. Under more challenging high-impulse pushes, the gap widens: HWC-Loco attains 81.27 % with 7.90% ZMP, indicating that its recovery mechanism (e.g., rapid body lowering and arm swing) mitigates sudden impacts and expedites stability recovery. In payload scenarios, despite not being explicitly trained for added- added-mass conditions (e.g., +10 kg in the hands), HWC-L Loco maintains the highest success rates with consistently low 7MP. This highlights the robustness of HWC-Loco to unseen disturbances. In summary. HWC-Loco couples high task success with consistently low ZMP deviation, yielding safer contact stability across disturbances.
real humanoid robot and evaluate its performance under various external force disturbances. Figure 12 (see Appendix B.6) shows policy switching and associated changes in roll and pitch angles under continuous perturbations, including pushes, pulls, and kicks. Upon encountering disturbances, the robot promptly switches to the recovery policy, adjusting its posture and gait to regain stability. Importantly, the controller does not rely solely
Robustness in Realistic Deployment. We deploy HWC-Loco
8

---

Published as a conference paper at ICLR 2026
Table 2: Robustness of locomotion under different disturbances.
Policy
External Force/Toqque (Low-freq.)
Impulse CoM (Low-imp.)
Payload (Low)
Success Rate 1
ZMPP
Success Rate
ZMP
Success Rate
ZMP
DrcamWaQ
85.92:+1.44 87.15+22.86
5.94=0.08
85.24=0.37
6.88+00088
67 07..3++33110
18.35+00.21
AHI.
6.42+0.10
85.87+1.44
7.53+0.11
79 79.299+0.65
17.50+0.19
Goal-tracking
90.00
7.64
88.90
8.24:10.11
78.34.1:..3
10.41
HWC-Loco-]
92. 02.69-10.66
5.59 08
92.79-1.77
6.16:10:10
84 84.11122.0 2.66
10.37:10.11
HWC-Loco
95.88:0.37
5.53-0.00
94.8410.54
6.13::0009
87. 87.43:0..92
9.33:10.12
Policy
External Force/ Teree/ (Constant)
Impulse on CoM (lligh-imp.)
Payload (Highh
Success Rate 1
ZMP |
Success Rate
ZMP J
Success Rate
ZMP L
DrcamWaQ
51.31=0.97
1.71=0.09
45.34=0.41 62.94+1.299
8.97+00.1
55.04+22.55
18.67=00.27
AHI.
80.72+0036
7.96+0.07
8.99++0.10
59.16++1.00
17.74+0..1
Goal-tracking
61.20:10.61
8.60:1..8
58.13-10.78
10.17:10.10
61.44:11.000
10.81110.16
HWC-Loco-]
68.60110.28
6.6210.07
07.09-0.79
8.18110.09
63.96.10.09
10.6810.13
HWC-Loco
75,9510.60
6, 6.61
81.27:10.80
7.90=1008
69.86=1.00
9.6911.111
on recovery mode but dynamically switches between goal-tracking and recovery policies, thereby adapting the action distribution to environmental changes. Figure 13 (see Appendix B.6) further demonstrates the robot's robustness disturbances outdoor settings.
5.3 NATURALNESS: IMITATING HUMAN-LIKE MOVEMENTS
As shown in Figure 4, the robot remains standing at O m/s, and exhibits natural walking at 1.0 m/s, characterized by straight elbows and coordinated arm swings. At 2.5 m/s, it transitions to running, with longer strides and increasingly bent elbows, closely resembling human running. As the velocity increases, both stride length and step frequency adjust dynamically, mimicking typical human locomotion. In contrast, prior approaches (Gu et al., 2024b;e; Cui et al., 2024; Zhang al., 2024b) often maintain fixed step frequency regardless of the commanded velocity. Some even exhibit in-place stepping m/s (Zhang et al., 2024b; Cui et al., 2024), resulting in unnatural behavior for tasks requiring steady stance, such manipulation.
Standlig
5.4 SCALABILITY: EXTENDING HWC-LOCO TO NEW TASKS
Figure ior.
Human-like behav-
Cross Embodiment Validation. We deploy HWC-Loco the
Unitree GI humanoid robot to evaluate its generalization capability under the same locomotion settings. As shown in Table 3, G1 outperforms H1 across all metrics. A detailed analysis is provided in Appendix B.7.
Table Comparison of Embodiments
Table 4: Motion Tracking under Disturbances
Sucvess Rate
Gool Tracking
Human-like
Panching
Damcing
Expressive Walking
HWC-Laco
0.49
86.44 0.60
94.53
Unitree 98.14 + 0.35
1.12 4 0.01
3.11 0.03
Motion-tracking
Expressive Motion Tracking. Following ExBody (Cheng et al., 2024; Ji et al., 2024), we set expressive motion HWC-Loco's goal and train the model using the same locomotion pipeline. domainn-andommized motion-tracking policy serves as the baseline. We evaluate both on three representative motions under impulse disturbances, and report success rates in Table 4.
LIMITATION
Our approach has three limitations. First, policy switching is handled by discrete module while low-level controllers are frozen. Jointly optimizing the hicrarchy could yield smoother transitions. Second, the humanoid robot in real-world deployment has only 19 DOF, which limits whole-body coordination and constrains the expression of complex recovery behaviors. Third, the recovery policy is trained in simulation and may not capture the full diversity of real-world disturbances. Incorporating more realistic or adversarial perturbations could further improve robustness.
CONCLUSION
We introduce HWC-Loco, a hierarchical control framework for humanoid robots that incorporates an embedded safety recovery mechanism. This framework has been validated across various tasks and

---

Published as a conference paper at ICLR 2026
scenarios, demonstrating exceptional effectiveness, robustness, naturalness, and scalability. Notably, the safety mechanism HWC-Loco extends beyond locomotion, enabling reliable performance in complex tasks through dynamic task-safety balance. This ensures robust operation in weal-world deployments, positioning HWC-Looco as foundational solution for safety-critical applications such industrial automation and assistive robotics. A promising direction for future research is integrating HWC-Loco with loco- aeco-manipulation skills, enabling safety-critical control across broader range of tasks involving different objects.
ETHICS STATEMENT
This research concerns robotic control and does not involve human participants, personally identifiable data, or sensitive attributes. It presents no harmful insights, dual-use risks, or discrimination concerns. We disclose conflicts of interest or external sponsorship; accordingly, do not foresee ethical issues.
REPRODUCIBILITY STATEMENT
We provide the implementation of HWC-Loco in the supplementary material, and the hyperparameter settings are detailed in Appendix A, enabling faithful replication of our results.
ACKNOWLEDGMENT
This work is supported in part by Shenzhen Science and Technology Program un- der grant 07702024090310400000 Shenzhen Science and Technology Program under grant ZDCY20250901 010550000111000000 113000001, CUHK-CUHK(SZ-GDSTTI 2025A0505000053, GuangDong Key Laboratory of Big Data Computing (2021B122 121200000))) and Guangdong Provincial Key Laboratory of Mathematical Foundations for Artificial Intelligence (2033B1212010001).
Joint Collaboration Fund No.
REFERENCES
CMU, Cmu graphics lab motion capture database, htttp://mocast cs. cmu. cma.edu/.
Shuxiao Chen, Bike Zhang, Mark W Mueller, Akshara Rai, and Koushil Sreenath. Learning torque control for quadrupedal locomotion. In 2023 IEEE-RAS 22nd International Conference Humanoid Robots (Humanoids), pp. 1-8. IEEE, 2023.
Zixuan Chen, Xialin He, Yen-Jen Wang, Qiayuan Liao, Yanjie Ze, Zhongyu Li, Shankar Sastry, Jiajun Wu, Koushil Sreenath, Saurabh Gupta, et al. Learning smooth humanoid locomotion through lipschitz-constrannge policies. arXiv preprint arXiv:2410.11822 2024.
Xuxin Cheng, Yandong Ji, Junming Chen, Ruihan Yang, Ge Yang, and Xiaolong Wang. Expressive whole-body control for humanoid robots. arXiv preprint wrXhv:2402.16796 2024.
Matthew Chignoli, Donghyun Kim, Elijah Stanger-Jones, and Sangbae Kim. The mit humanoid robot: Design, motion planning, and control for acrobatic behaviors. In 2020 IEEE-RAS 20th International Conference Humanoid Robots (Humanoids), pp. 1-8. IEEE, 2021.
Wenhao Cui, Shengtao Li, Huaxing Huang, Bangyu Qin, Tianchu Zhang, Liang Zheng, Ziyang Tang, Chenxu Hu, NING Yan, Jiahao Chen, et al. Adapting humanoid locomotion over challenging terrain via two- wwo-phase training. In 8th Annual Conference Robot Learning, 2024.
Helei Duan, Bikram Pandit, Mohitvishnu S Gadde, Bart Van Marum, Jeremy Dao, Chanho Kim, and Alan Fern. Learning vision-based bipedal locomotion for challenging terrain. In 2024 IEEE International Conference Robotics and Automation (ICRA), pp. 56-62. IEEE, 2024.
Siyuan Feng, X Xinjilefu. Christopher G Atkeson, and Joohyung Kim. Robust dynamic walking using online foot step optimization. In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 5373-5378, IEEF, 2016.
10

---

Published as a conference paper at ICLR 2026
Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans. In 8th Anmual Conference on Robot Learning, 2024.
David Gouaillier, Vincent Hugel, Pierre Blazevic, Chris Kilner, Jérôme Monceaux, Pascal Lafourcade, Brice Marnier, Julien Serre, and Bruno Maisonnier. Mechatronic design of NAO humanoid. In IEEE International Conference on Robotics and Automation, ICRA, pp. 769-774, 2009.
Shangding Gu, Long Yang. Yali Du, Guang Chen, Florian Walter, Jun Wang, and Alois Knoll. A review of safe reinforcement learning: Methods, theories, and applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12): 1 2024a.
Xinyang Gu, Yen-Jen Wang, and Jianyu Chen. Humanoid- -ymm: Reinforcement learning for humanoid robot with zero-shot sim2real transfer. arXiv preprint arXiv:2404.055695, 2024b.
Xinyang Gu, Yen-Jen Wang, Xiang Zhu, Chengming Shi, Yanjiang Guo, Yichen Liu, and Jianyu Chen. Advancing humanoid locomotion: Mastering challenging terrains with denoising world model learning. arXiv preprint arXiv:2408.1447,, 2024c.
Kensuke Harada, Shuuji Kajita, Kenji Kaneko, and Hirohisa Hirukawa. An analytical method for real-time gait planning for humanoid robots. International Journal of Humanoid Robotics, 3(01): 1-19, 2006.
Tairan He, Zhengyi Luo, Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous human-to-humanoid whole-body teleoperation and learning. arXiv preprint arXiv:2406.08885, 2024a.
Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning buman-to-humanoid real-time whole-body teleoperation. arXiv preprint arXiv:2403. 04436, 2024b.
Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Jan Kautz, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, and Yuke Zhu. HOVER: versatile neural whole-body controller for humanoid robots. arXiv preprint 2rXiv:2410.21229, 2024c.
Tairan He, Chong Zhang, Wenli Xiao, Guanqi He, Changliu Liu, and Guanya Shi. Agile but safe: Learning collision-free high-speed legged locomotion. arXiv preprint arXiv:2401.17583 2024d.
Tairan He, Jiawei Gao, Wenli Xiao, Yuanhang Zhang, 7i Wang. Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbab, Chaoyi Pan, et al. Asap: Aligning simulation and real-world physies for learning agile humanoid whole-body skills. arXiv preprint arXiv:2502.01144 2025.
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. In Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (eds.), Advances in Neural Information Processing Systems, NeurIPS, pp. 4565-4573, 2016.
Tao Huang, Huayi Wang. Junli Ren, Kangning Yin, Zirui Wang, Xiao Chen, Feiyu Jia, Wentao Zhang. Junfeng Long, Jingho Wang, et al. Towards adaptable humanoid control via adaptive motion tracking. arXiv preprint arXiv:2510. 14.14444 2025.
Ahmed Hussein, Mohamed Medhat Gaber, Fyad Flyan, and Chrisina Jayne. Imitation learning: A survey of learning methods. ACM Computing Surveys, 50(2):111122:335 2017.
Mazeyu Ji, Xuanbin Peng, Fangchen Liu, Jialong Li, Ge Yang, Xuxin Cheng, and Xiaolong Wang. Exbody2: Advanced expressive humanoid whole-body control. arXiv preprint 21X 2024.
Dohyeong Kim, Hyeokjin Kwon, Junseok Kim, Gunmin Lee, and Songhwai Oh. Stage-wise reward shaping for acrobatic robots: constrained multi-objective reinforcement learning approach. arXiv preprint arXiv:2409. 15755, 2024.
Ashish Kumar, Zhongyu Li, Jun Zeng, Deepak Pathak, Koushil Sreenath, and Jitendra Malik. Adapting rapid motor adaptation for bipedal robots. In 2022 IEEE/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 1161-1168 IEBE, 2022.
11

---

Published as a conference paper at ICLR 2026
Hang Lai, Weinan Zhang, Xialin He, Chen Yu, Zheng Tian, Yong Yu, and Jun Wang. Sim-to- real transfer for quadrupedal locomotion via terrain transformer. In 2023 IEEE Interaational Conference Robotics and Automation (ICRA), pp. 5141-5147. IEEE, 2023.
Joonho Lee, Jemin Hwangbo, Lorenz Wellhausen, Vladien Koltun, and Marco Hutter. Learning quadrupedal locomotion over challenging terrain. Science robotics, 5(47)xeabc5986, 2020.
Jialong Li, Xuxin Cheng, Tianshu Huang, Shiqi Yang, Ri-Zhao Qiu, and Xiaolong Wang- Amo: Adaptive motion optimization for hyper-dexterous humanoid whole-body control. arXiv preprint arXiv:2505.03778, 2025.
Zhongyu Li, Xuxin Cheng, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, and Koushil Sreenath. Reinforcement learning for robust parameterized locomotion control of bipedal robots. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 281 2811-281 1-2817. IEEE, 2021.
Zhongyu Li, Xue Bin Peng, Pieter Abbeel, Sergey Levine, Glen Berseth, and Koushil Sreenath. Robust and versatile bipedal jumping control through reinforcement learning. arXiv preprint arXiv:2302.094550 2023.
Qiayuan Liao, Takara E Truong, Xiaoyu Huang, Guy Tevet, Koushil Sreenath, and C Karen Liu. Beyondmimic: From motion tracking to versatile humanoid control via guided diffusion. arXiv e-prints, pp. arXiv-2508, 2025.
Junfeng Long, Junli Ren, Moji Shi, Zirui Wang, Tao Huang, Ping Luo, and Jiangmiao Pang. Learning humanoid locomotion with perceptive internal model. arXiv preprint arXiv:2411.14386 2024.
Zhengyi Luo, Jinkun Cao, Kris Kitani, Weipeng Xu, et al. Perpetual humanoid control for real-time simulated avatars. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10895-10904, 2023a.
Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, and Weipeng Xu. Universal humanoid motion representations for physics-based control. arXiv preprint arXiv:2310.04582, 2023b.
Zhengyi Luo, Jinkun Cao, Rawal Khirodkar, Alexander Winkler, Kris Kitani, and Weipeng Xu. Real- time simulated avatar from head-mounted sensors. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 571-581, 2024a.
Zhengyi Luo, Jiashun Wang, Kangni Liu, Haotian Zhang, Chen Tessler, Jingbo Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang, et al. Smplolympics: Sports environments for physically simulated humanoids. arXiv preprint arXiv:2407.001/ 007877 2024b.
Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. AMASS: archive of motion capture surface shapes. In IEEE/CV International Conference on Computer Vision, ICCV, pp. 5441-5450. IEEE, 2019.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint arXiv:2108. 10470, 2021.
Gabriel B Margolis and Pulkit Agrawal. Walk these ways: Tuning robot control for generalization with multiplicity of behavior. In Conference on Robot Learning, pp. 22-31. PMLR, 2023.
Ben Mildenhall, Pratul P Srinivasan, Matthew Tancik, Jonathan T Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Commmicatiins of the ACM, 65(1):99-106, 2021.
Janosch Moos, Kay Hansel, Hany Abdulsamad, Svenja Stark, Debora Clever, and Jan Peters. Robust reinforcement learning: A review of foundations and recent advances. Machine Learning and Knowledge Extraction, pp. 276-315, 2022.
12

---

Published as a conference paper at ICLR 2026
Mitsuharu Morisawa, Kensuke Harada, Shuuji Kajita, Kenji Kaneko, Fumio Kanehiro, Kiyoshi Fujiwara, Shinichiro Nakaoka, and Hirohisa Hirukawa. A biped pattern generation allowing immediate modification of foot placement in real-time. In 2006 6th LEEE-RAS International Conference on Humanoid Robots, pp. 581-586. IEEE, 2006.
I Made Aswin Nahrendra, Byeongho Yu, and Hyun Myung. Dreamwaq: Learning robust quadrupedal locomotion with implicit terrain imagination via deep reinforcement learning. In 2023 IEEE International Conference Robotics and Automation (ICRA), pp- 5078-5084. IEFF, 2023.
Soroush Nasiriany, Abhiram Maddukuri, Lance Zhang, Adeet Parikh, Aaron Lo, Abhishek Joshi, Ajay Mandlekar, and Yuke Zhu. Robocasa: Large-scale simulation of everyday tasks for generalist robots. arXiv preprint arXiv:2406.025523 2024.
Giuseppe Paolo, Jonas Gonzalez-Billandon, and Balázs Kégl. Position: A call for embodied AI. In Forty-first International Conference on Machine Learning, ICML, 2024.
Santiago Paternain, Luiz F. O. Chamon, Miguel Calvo-Fullana, and Alejandro Ribeiro. Constrained reinforcement learning has zero duality gap. In Advances Neural Information Processing Systems, NeurlPS, pp. 7553-7563, 2019.
Guanren Qiao, Guiliang Liu, Pascal Poupart, and Zhiqiang Xu. Multi-modal inverse constrained reinforcement learning from a mixture of demonstrations. In Advances in Neural Information Processing Systems, (NeurIPS), 2023.
Guanren Qiao, Sixu Lin, Ronglai Zuo, Zhizheng Wu, Kui Jia, and Guiliang Liu. Signbot: Learning tommanto- sign language interaction. arXiv preprint 2rXiv:2505.24266, 2025.
Nicolaus A. Radford, Philip Strawser, Kimberly A. Hambuchen, Joshua S. Mehling, William K. Verdeyen, A. Stuart Donnan, James Holley, Jairo Sanchez. Vienny Nguyen, Nguyen,Lyndon Lyndon B. Bridgwater, Reginald Berka, Robert O. Ambrose, Mason Myles Markee, J. Fraser-Chanpong, Christopher McQuin, John D. Yamokoski, Stephen Hart, Raymond Guo, Adam Parsons, Brian Wightman, Paul Dinh, Barrett Ames, Charles Blakely, Courtney Edmondson, Brett Sommers, Rochelle Rea, Chad Tobler, Heather Bibby, Brice Howard, Lei Niu, Andrew Lee, Michael Conover, Lily Truong, Ryan Reed, David Chesney, Robert Platt Jr., Gwendolyn Johnson, Chien-Liang Fok, Nicholas Paine, Luis Sentis, Eric A. Cousineau, Ryan W. Sinnet, Jordan Lack, Matthew Powell, Benjamin Morris, Aaron D. Ames, and Jide Akinyode. Valkyrie: Nasa's 's first bipedal humanoid robot. Journal of Field Robotics, (113):977-199 2015.
Ilija Radosavovic, Tete Xiao, Bike Zhang, Trevor Darrell, Jitendra Malik, and Koushil Sreenath. Real-world humanoid locomotion with reinforcement learning. Science Robotics, :89)=eadi95,,, 2024.
Nikita Rudin, David Hoeller, Philipp Reist, and Marco Hutter. Learning to walk in minutes using massively parallel deep reinforcement learning. In Conference on Robot Learning, pp. 91-100. PMLR, 2022.
Saced Sacedvand, Masoumeh Jafari, Hadi S. Aghdasi, and Jacky Baltes. A comprehensive survey on humanoid robot development. Knowledge Engineering Review, 34:e20, 2019.
Yoshiaki
Sakagami, Ryujin Watanabe, Chiaki Aoyama, Shinichi Matsunaga, Nobuo Higaki, and Kikuo Fujimura. The intelligent ASIMO: system overview and integration. In IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, IROS, pp. 2478-2483 IEFE, 2002.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06344 2017.
Nicola Scianca, Daniele De Simone, Leonardo Lanari, and Giuseppe Oriolo. Mpc for humanoid gait generation: Stability and feasibility. IEEE Transactions on Robotics, 36(4): 1888 2020.
Luis Sentis and Oussama Khatib. A whole-body control framework for humanoids operating in human environments. In IEEE International Conference on Robotics and Automation, ICRA, pp. 2641-2648. IEFE, 2006.
13

---

Published as a conference paper at ICLR 2026
Fan Shi, Chong Zhang, Takahiro Miki, Joonho Lee, Marco Hutter, and Stelian Coros. Rethinking robustness assessment: Adversarial attacks on learning- learning-based quadrupedal locomotion controllers. arXiv preprint ar arXhv:2405.12424, 2024.
Haojie Shi, Bo Zhou, Hongsheng Zeng, Fan Wang, Yueqiang Dong, Jiangyong Li, Kang Wang. Hao Tian, and Max Q-H Meng. Reinforcement learning with evolutionary trajectory generator: general approach for quadrupedal locomotion. IEEE Robotics and Automation Letters, 7(2): 3085-3092, 2022.
Wandong Sun, Baoshi Cao, Long Chen, Yongbo Su, Yang Liu, Zongwu Xie, and Hong Liu. Learning perceptive humanoid locomotion over challenging terrain. arXiv preprint arXiv:2503.0069) 2025.
Stone Tao, Fanbo Xiang, Arth Shukla, Yuzhe Qin, Xander Hinrichsen, Xiaodi Yuan, Chen Bao, Xinsong Lin, Yulin Liu, Tse- Chan, et al. Maniskill3: Gpu parallelized robotics simulation and rendering for generalizable embodied ai. arXiv preprint arXiv:2410.00423 2024.
Luca Viano, Yu-Ting Huang. Parameswaran Kamalaruban, Craig Innes, Subramanian Ramamoorthy, and Adrian Weller. Robust learning from observation with model misspecification. In International Conference on Autonomous Agents and Multiagent Systems, pp. 1337-1345, 2022.
Cédric Villani et al. Optimal transport: old and new, volume 338. Springer, 2009.
Zhicheng Wang, Wandi Wei, Ruiqi Yu, Jun Wu, and Qiuguo Zhu. Toward understanding key estimation in learning robust humanoid locomotion. arXiv preprint urXiv:2403.05860 2024.
Bowen Weng, Guillermo A Castillo, Wei Zhang, and Ayonga Hereid. On the comparability and optimal aggressiveness of the adversarial -cenario-based safety testing of robots. IEEE Transactions on Robotics, 39(4):3299-3318 2023.
Pierre-Brice Brice Wieber. Trajectory free linear model predictive control for stable walking in the presence of strong perturbations. In 2006 6th IEEE-RAS International Conference on Humanoid Robots, pp. 137-142. IEEE, 2006.
Weiji Xie, Chenjia Bai, Jiyuan Shi, Junkai Yang, Yunfei Ge, Weinan Zhang, and Xuelong Li. Humanoid whole-body locomotion on narrow terrain via dynamic balance and reinforcement learning. arXiv preprint arXiv:2502.17219 2025.
Yanjie Ze, Zixuan Chen, JoAGo Pedro AraAšjo, Zi-ang Cao, Xue Bin Peng, Jiajun Wu, and C Karen Liu. Twist: Teleoperated whole-body imitation system. arXiv preprint 2xxiv:2505.02833 2025.
Chong Zhang, Wenli Xiao, Tairan He, and Guanya Shi. Wococo: Learning whole-body humanoid control with sequential contacts. In Annual Conference on Robot Learning, 2024a.
Qiang Zhang, Peter Cui, David Yan, Jingkai Sun, Yiqun Duan, Gang Han, Wen Zhao, Weining Zhang, Yijie Guo. Arthur Zhang, et al. Whole-body humanoid robot locomotion with human reference. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 11225-11231. IEEE, 2024b.
Tong Zhang, Boyuan Zheng, Ruiqian Nai, Yingdong Hu, Yen-Jen Wang, Geng Chen, Fanqi Lin, Jiongye Li, Chuye Hong, Koushil Sreenath, et al. Hub: Learning extreme humanoid balance. arXiv preprint arXiv:2505.072 07294, 2025a.
Xin Zhang, Yanhua Li, Ziming Zhang, and Zhi-Li Zhang. f-gail: Learning f-divergence for generative adversarial imitation learning. In Advances in Neural Information Processing Systems, NeurIPS, 2020.
Ye Zhang, Qi Chen, Longsen Gao, Rui Liu, Linyue Chu, Kangtong Mo, Zhengjian Kang, Wenyou Huang, and Xingyu Zhang. Invertible liquid neural network-basse learning inverse kinematics and dynamics for robotic manipulators. Scientific Reports, 15(1):42311, 1, 2025b.
Haoyu Zhao, Sixu Lin, Qingwei Ben, Minyue Dai, Hao Fei, Jingbo Wang, Hua Zou, and Junting Dong. Smap: Self-supervised motion adaptation for physically plausible humanoid shole-body. control. arXiv preprint arxiv:2505. 19463, 2025.
14

---

Published as a conference paper at ICLR 2026
Ziwen Zhuang, Zipeng Fu, Jianren Wang, Christopher Atkeson, Soeren Schwertfeger, Chelsea Finn, and Hang Zhao. Robot parkour learning. arXiv preprint arXiv:2309.05666. 2023.
Ziwen Zhuang, Shenzhe Yao, and Hang Zhao. Humanoid parkour learning. In 8th Annual Conference on Robot Learning, 2024.
15

---

Published as a conference paper at ICLR 2026
A ADDTIONAL DETAILS OF HWC-LoCO
A.1 MOTION RETARGETING
Our motion data is derived from the CMU Mocap dataset cmu, which exclusively encompasses human locomotion data, featuring a diverse array of walking, jogging, and running styles. aggregate, we have loaded 318 motion sequences, totaling 3729. 17 seconds in duration. Our motion retargeting methodology closely follows the approach outlined in Cheng et al. (2024). We have meticulously aligned the humanoid robot's skeleton with that of humans, incorporating adjustments such as rotation, translation, and scaling. Additionally, we map human joints onto the humanoid robot's structure. Specifically, the human motion data is adapted to the humanoid robot's framework by mapping local information. Both the Unitree HI and GI robots possess shoulder and hip joints that are functionally equivalent to spherical joints. During the retargeting process, the three hip and shoulder joints are treated as a single spherical joint, corresponding to the human's 's spherical joint. For one- me-dimensioonal joints such as the elbow and torso, the rotation angle projected onto the corresponding rotation axis of the joint angles.
Table 5: Double-DQN Parameters
Parameter
Value
Table DQN Network Structure
Batch Size
128
Learning Rate
le-4
Layer
Details
Gamma
0.99
Layer
[input_dins dim, 256]
Max Grad Norm
1.0
Layer
Activation: ReLU
Replay Buffer Capacity
2000000
Layer
Regularization: Dropout [256, 128]
Switch Penalty Coefficient
0.001
Layer
Target Update Frequency
50
Layer
[128, 64]
Optimizer
Adam MSELoss
Layer
Activation: ReLU
Loss Function
Layer
[64, output_ dim]
Initial Epsilon
0.1
Minimum Epsilon
0.001
Epsilon Decay Rate
0.999
A.2 IMPLEMENTATION DETAILS
Proprioception: For the Unitree H1, proprioception of R&5 which denotes the internal state of the robot, including the base angular velocity w. E 3 base roll r; E R base pitch E R degrees of freedom (DOF) positions qi IR.9 DOF velocities ÿ. R19 previous actions a;- E 19 and projected gravity g. E R The projected gravity refers to the component of gravity expressed in the robot's 's local coordinate system. For the Unitree GI, proprioception of R?T includes base angular velocity velocities qi 23 previous actions a;-1 1 E 23 and projected gravity g: R As with the H1, the projected gravity denotes the component of gravity in the robot's 's local coordinate system.
R-3 base roll E R base pitch E R DOF positions E R DOF
The policy outputs continuous actions a, E R. which are utilized as target positions for PD controller to compute joint torques. The actions correspond to the robot's actuated degrees of freedom: specifically, a, 1 (19 DoF)) for the Unitree HI and a, E 222 (23 Doff) for the Unitree GI.
Action Space:
Terrain details:
Terrains in the training environment simulated by Isaac Gym Makoviychuk et al. (2021) include flats, obstacles, slopes, and stairs. An example of these terrains is shown in Figure 5.
Low-level Policy Training: For all the low-level policies training, the commands are sampled from ranges: o ~ [-0.6, 2.5). of 1-0.6, 0.6) and w ~ [1.00, 1.0]. During the initial training of the humanoid robot on uneven terrain, the robot often remains stationary due to the high difficulty of navigating complex surfaces. To address this, we introduce terrain curriculum method Makoviychuk et al. (2021). The training terrain consists of various types, including flat planes, rough surfaces, steps, and slopes. As the robot achieves goal-tracking performance of 70% of the commanded
16

---

Published as a conference paper at ICLR 2026
Figure 5: Visualization of different terrains. From left to right, the terrains are flats, obstacles, slopes, and stairs
velocity, the terrain difficulty increases. Conversely, the goal-tracking performance drops below 40% of the commanded velocity, the terrain becomes easier.
Learning for Policy Transition: High-level planning policy is utilized to switch between low-level policies, ensuring compliance with robustness constraints. For the High-level policy, the input is the same set of observations as used by the low-level policies, with the output being a two-dimensional Q-value. During training, trained low-level policies loaded and rolled out to generate training data for optimizing the high-level policy. We train this high-level policy in the simulation environment mentioned detailed Section 4.2, focusing on locomotion goals as the primary task. The objective is to enable the robot to track goal commands across variety of terrains. The details of the training process is in Algorithm
Algorithm High-level Policy Training
Input: Environment env, Selector network S, Target network Stagett Goal-tracking policy L. Recovery policy R
Initialize: Learning rate a, Discount factor y, Exploration rate c, Replay buffer B, Target update
frequency furget. Switch penalty Aswith. Exploration decay Cdecay, Minimum exploration Guin
Suagett S (Initialize target network]
repeat
Observe initial state s from env
Reset cumulative reward Riotal step count n. previous action Opeev f- None while not done do
Compute (s,,-) <- S(s), select action a with e-greedy
Compute switch penalty: Tpennaly < Awwitch 1 F (prev]
Execute a, observe reward r, next state s', and done Store (s, a, T + , s', d.) in B
Update s s', Aprev <
Riotal + Reotal + +- n if B.size() batch size then Sample (s,a,T, r, from B
a' arg max S(s',a') (Online network selects action]
Quaret (s', a') Compute target: Quagel (s,a) f- + d) Qurgue (s', d')
(s (s',a') (Target network evaluates
Update S by minimizing: L = (S(s, a) Quaree (s. a))²
end if
if n. mod furget = 0 then
Suuret < (Syne target network)
end if
end while
Decay exploration rate: +- max(e Edecay, Emin until convergence criteria met
Output: Trained Selector Network S
17

---

Published as a conference paper at ICLR 2026
Reward Details: To better facilitate efficient humanoid locomotion, task reward is defined Gu et al. (2024b;c):
1 Pxyll2 I
wz
13 = 01 exp
+ 02 exp
aan
where 1) by denotes the commanded linear velocity along the x and y axes, while wy is the commanded yaw velocity. 2) Exy and w, represent the corresponding base velocities of the humanoid robot. 3) aj and ag indicate the hyperparameters that are utilized adjust the importance of the different velocity terms. 4) ji and "ing control the precision of the expected command tracking. Smaller values of oi and 2ng enhance the precision of command tracking for humanoid robots. However, they may hinder reward acquisition during the initial stages of training.
The recovery policy shares the same task reward formulation the goal-tracking policy. However, it uses larger values the The and dang: which implies greater tolerance for velocity tracking errors. As result, this reward term serves as back-tracking reward for the safety recovery mechanism, encouraging it to return to stable goal-tracking state. To further promote stable posture restoration and enable smooth transitions back the goal-tracking policy, we introduce an additional stand reward, defined as:
Istand
|1q
Idefault
122
where qdefauht represents default standing pose.
For safety realistic deployment, we add some energy and safety reward for the low-level policies training. These rewards are shown in Table.
Table Safety & Energy Reward
Term
Expression
Weight
DoF position limits
1(gr é [mins 4max])
- lee
DoF acceleration
|q
le
DoF velocity
||qu
5e
Action rate
||a; a a
- 2e 33
Torque
||
-l-
Collision
Eiccontace
1 [
> 1.0)
-10
A.3 PPO
In order to enhance the understanding of historical observation information, we have made some improvements to the Actor-network. We first process this information using encoder network to preliminarily extract the features of the observation values at each time step, and then use a merger network to integrate the observation values different time steps. Finally, obtain fixed- dimensional feature information, which is then input into the Actor backbone network together with other information.
18

---

Published as a conference paper at ICLR 2026
Table 8: PPO hyperparameters
Hyperparameter
Value
Actor Layer
[512. 256, 128]
Critic Layer
[512, 256, 128]
Activation Function
ELU
Discount Factor
0.99
GAE Parameter
0.95
Epochs per Rollout
5
Minibatch
4
Entropy Coefficient
0,005
Value Loss Coefficient (a²)
1.0
Clip Range
0.2
Learning Rate
2.e-4
Environment Steps per Rollout
24
A.4 VAE
The VAE Network consists of encoder and decoder. The encoder takes historical observations input and outputs latent representation and estimation of privileged information. The decoder takes latent input and outputs the reconstruction of the next observation. The structure of VAE is shown in Figure
Aecosotzetinn Loss
Table 9: VAE hyperparameters
Enirnation Loss
Hyperparameter
Value
Encoder Layer
[256, 128]
Decoder Layer
[256, 128, 64]
Bnooder
Decoder
Activation Function
ELU
KL Coefficient
1.0
Learning Rate
1.e-4
Figure 6: VAE Structure
A.5
DISCRIMINATOR
The input to the discriminator s is defined as [q. v. w.r.p. k), where & represents the local positions of key points. Specifically, we define key points, including the hips. shoulders, hands, ankles, knees, and elbows. For the Unitree H1, the total input dimension is 63, while for the Unitree G1, it is 67.
The discriminator's hyperparameters are shown in Table. 10.
Table 10: Diseriminator hyperparameters
Hyperparameter
Value
Discriminator Hidden Layer Dim
[1024.512]
Replay Buffer Size
500000
Expert Buffer Size
200000
Expert Fetch Size
512
Learning Batch Size
4096
Learning Rate
2e-5
Gradient Penalty Coefficient
1.0
lambda
1.0
19

---

Published as a conference paper at ICLR 2026
A.6 EXTREME INITIALIZATION
To simulate extreme scenarios that may occur in real-world deployment, we identify three primary causes of such cases: external disturbances, hardware malfunctions, and malicious commands. Additionally, we retain domain randomization to enhance the generalization of physical dynamics within the simulation environment. The specific details are outlined follows:
To enhance robustness and generalization in simulation, we simulate both extreme deployment scenarios and apply domain randomization. The extreme scenarios include external disturbances, sensor and actuator failures, and adversarial commands, while domain randomization targets physical variability the environment.
1) External disturbances: We apply external disturbances to every component of the robot. The external forces and torques are applied along the x, y, and z axes, with each direction sampled from the range of -200 N to 200 N for forces and -200 Nm to 200 Nm for torques. These disturbance forces and torques are updated at frequency of Hz.
2) Hardward magnification: Random, high-intensity noise is introduced to the proprioceptive information and PD gains. The noise added to the proprioceptive data simulates sensor errors that may occur during real-world deployment. The noise applied the PD gains simulates fluctuations in motor strength, reflecting potential issues in the robot's 's actuators. The parameters for this noise are detailed in Table 11.
3) Malicious velocity:
Velocity commands are resampled from the command range, with updates occurring at frequency of Hz. Notably, this resampling mechanism is activated once the terrain curriculum exceeds half of its total progression.
Domain randomization (for environment variability): Domain randomization is applied to all the training environments. This technique introduces variability in the environment's parameters, such friction and gravity. The specifics of the domain randomization are provided in Table 12.
To improve the generalization of recovery ability, we utilize random initialization with extreme case initialization. The parameters are shown in Table 13
Table 11: Intense Noise Parameters
Parameter
Range
Operator
DOF Position Noise
0.04
additive
DOF Velocity Noise
0.30
additive
Angular Velocity Noise
1.00
additive
IMU Noise
0.40
additive
Gravity Noise
0.10
additive
PD Gains
0.20
additive
Table 13: Random Initialization Parameters
Table 12: Domain Randomization Parameters
Parameter
Range
Parameter
Range
Operator
Linear Velocity (x)
1.0,
Friction
[0.6, 2.0]
additive
Linear Velocity (y)
1.0
Base Mass
[-1.0, 5.0]
additive
Linear Velocity (z)
-0.4,0.4
Base CoM
[-0.07, 0.07]
additive
Angular Velocity
-1.0,
Motor Strength
[0.8, 1.2]
scaling
DOF Position
0.2 default, 1.8 x default]
Action Delay
[0
DOF Velocity
0..
Link Mass
[0.7, 1.3]
scaling additive
Pitch Angle
0.25,
Gravity
1-0.1,
Yaw Angle
0.4,0.4]
Position
2
Extreme- Caremee-asseaa Analysis: We record nearly 60000 normal state data with randomly sampled com- mands in diverse terrain settings in 7. We record nearly 10000 extreme cases in the 2/ Some of
20

---

Published as a conference paper at ICLR 2026
the terminal states visualized in Fig. 8. For the extreme states, the distribution of velocity is close to the command ranges. Roll pitch and their velocity are in a common range, which is near zeros. For the normal states, the distribution of linear velocity overlaps with the command velocity range, with a dense cluster around the high-velocity value of 2 m/s. Regarding yaw velocity, most values fall within the commanded range. However, there are instances of high speed, with some approaching 10 m/s. This may be due to the robot's ' extreme behaviors when encountering unseen situations influenced by external forces and random noise. Regarding the roll-pitch velocity, the pitch velocity shows a higher degree of fluctuation. Similarly, in the roll pitch distribution, for pitch, there concentrated distribution at a pitch of m/s, which is one of the terminal conditions. These two figures suggest that the robot is tripped by obstacles during its motion and thus terminates, for instance, when hits steps or gravel during high-speed motion.
Velacity
Hermet Rot l Plach Velecily
14
Figure 7: Normal State: Policy rollout's state distribution in the original learning environment
Uneer Velectt
Yie Valuciit Distriuuiiss
Terminal foll & Picch Velecity
velecity
6.1
Figure 8: Extreme State: Policy's state distribution in the extreme cases
A.7 BASELINE REPRODUCIBILITY
We adapt baseline implementations (DreamWare Nahrendra al. (2023), AHL Cui al. (2024)) as faithfully as possible while aligning embodiment and simulator specifics for fair comparison and reproducibility.
Common adaptation principles. 1) Proprioceptive inputs, action space, PD gains, control fre- quency, and domain domain-randomiin randomization ranges are aligned with HWC-Loco. 2) Physics time step, contact parameters, and terrains are matched to our Isaac Gym setup. 3) Training/validation splits, evaluation episodes, and disturbance samplers are shared across methods.
Dream WaQ. We retain the original reward structure and network design, with minimal adaptations account for humanoid morphology and dynamics: 1) reward scaley are slightly adjusted (see Table 14); 2) VAE-based estimator predicts velocity and base height from short observation histories; 3) following the original setting, the policy consumes a single observation frame (no temporal stacking). All other simulation and control parameters follow the common principles above.
AHL AHL adopts a gait-centric objective with phase tracking, foot slip, and air time. We reuse the same estimator DreamWaQ, while the policy ingests short histories of observations (temporal stacking). We employ a two-stage curriculum: 1) train on simple terrains with strong gait-shaping terms; 2) once the success rate exceeds 80%, remove the gait-shaping bonus and increase terrain difficulty and command ranges.
Reproducibility notes. We use shared disturbance samplers and evaluation protocols for all base- lines; policy observation formats, actuator models, PD targets, and randomization ranges doeu-
21

---

Published as a conference paper at ICLR 2026
Table 14: Dream WaQ reward terms and weights.
Reward term
Equation
Weight
Linear vel. tracking
exp(
Marg
1.0
Angular vel. tracking
exp
5 (wyyw
0.8
Velocity mismatch
exp
(-,)) exp( /hott 113
0.5
Orientation
||
1.0
Default joint position
exp(
Owmo 113
0.5
Body height
4
1.0)
-1.0
Root accelerations
exp
|
0.2
Joint accelerations
0012
-x1007
Joint velocity
1012
- -5x10-4
Joint energy
|T|
1x10-5
Action smoothness
aa de 11 ||a: 10tn-22111tt--2
0.002
Feet clearance
pf (1)
pf(1 2 (1 (t)))
0.01
Feet & Knee distance
eepp 100 |min(dd dmin: 0)1) + exp( 100 m dmax, 0))
0.2
Table 15: AHL gait-centric reward terms (additional to the shared terms in Table 14).
Reward term
Equation
Weight
Gait phase tracking
(1-Ix(t) Ie(t) THr Ic(f)
-0.001
Feet air time
10.0
Feet slip
Tz(() exp( -2 Ounzenl)
0.011
Joint position tracking
1.0
Other terms
Linear/Angular velocity. Height, Feet clearance/distanee, etc.
same as Table
mented in Appendix...2 and Appendix Code will be released with configuration files specifying seeds, observation keys, disturbance schedules, and reward weights corresponding to Tables 14-15.
A.8 COMPUTATIONAL COST
All policies are trained on a single NVIDIA RTX 4090 GPU with 4,096 parallel environments. Our framework involves training three policies. The goal-tracking policy converges in roughly 10,000 iterations (about 10 hours), while the recovery policy requires around 8,000 iterations (about hours). The high-level policy converges fastest, needing approximately 6,000 iterations (under hours). For comparison, Dream WaQ and AHL require about 8 and 9 hours of training, respectively, under the same simulation settings.
A.9 DEPLOYMENT DETAILS
We utilize the Unitree HI humanoid robot as our deployment platform. This full-sized humanoid robot weighs 47 kg. stands approximately 1.8 meters tall, and features 19 degrees of freedom. The control frequency
100 Hz for both simulation and real-world deployment. The PD gains, characterized by stiffness and damping values, used in the real-world deployment are detailed in Table 16.
For smoother policy transitions, we apply Butterworth low-pass filter (cutoff frequency: 5 Hz), following He al. (2024b), during real-world deployment to attenuate switching-induced transient oscillations. During deployment, inference operates through hierarchical structure involving both high-level and low-level policies. To meet the strict latency and resource constraints of onboard systems, the policies optimized for lightweight designs and exported JIT format. The high-level policy model size is below 2MB, and the low-level policies remain under 3MB. Those designs ensures that real-time inference 100 Hz. is feasible on the onboard computer of the Unitree H1 robot.
22

---

Published as a conference paper at ICLR 2026
Table 16: Stiffness, Damping and Torque Limit
Joint Names
Stiffness [N#m/rad]
Damping [N*mds/rad]
Torque Limit (Nm)
hip yaw
200
170
hip roll
200
5
170
hip pitch
200
5
170
knee
300
6
255
ankle
40
2
34
torso
200
5
170
shoulder
30
1
34
elbow
30
1
18
B EXPERIMENT
B.1 SALIENCY ANALYSTS OF OBSERVATIONS
We adopt an integrated gradients method similar to Wang et al. (2024) to assess the importance of different parts of the observation the policy, as shown Figure 9.
For the recovery policy, the ZMP encoding proves to be the most critical factor, followed by the estimated velocity. The robot's current and historical proprioceptive states also contribute substantially to the policy's 's decision-makern only marginal effect. This hierarchy of importance reflects the necessity of strictly adhering to ZMP constraints to maintain dynamic stability and ensure the safety of the robot during recovery.
process. In contrast, the environmental latent representation exhibits
For the high-level policy, the ZMP encoding remains the most influential component, indicating that it provides critical information for decision-making and substantially enhances the policy's 's awareness of the robot's stability. This observation implies that the high-level policy prioritizes stability-relevan features over task-in inseevrteeevant ones, which advantageous for promoting generalization across diverse scenarios. Morcover, the environmental latent variable exhibits greater importance compared to that in the recovery policy, suggesting that high-level decisions more sensitive to environmental factors such as terrain variations.
0.5
0.3
Balabvv
0.1
Hatory P
Current
Latent
Valocity
ZHP Encoding
Hiltory P
Current
Lature
Velecity
ZHP Eecoding
Figure Importance of Observations
B.2 SENSITIVITY TO HYPERPARAMETERS
Sensitivity to a. To quantify the sensitivity to the mismatch scale a, we scale all disturbance and parameter ranges by factors (0.5, 1.0, 2.0), where the original setting corresponds to 1.0. As shown Table 17, smaller a improves nominal tracking but reduces robustness, while larger a yields more conservative yet more robust behavior.
23

---

Published as a conference paper at ICLR 2026
Table 17: Sensitivity of HWC-Loco the mismatch scale a.
Policy
Success Rate
Goal-tracking t
Human-like
HWC-Loco (0.5)
97.88 + 0.26
1.14 = 0.24
3.10 + 0.11
HWC-Loco (1.0)
98.14 + 0.35
1.12 = 0.01
3.11 + 0.03
HWC-Loco (2.0)
98.20 + 0.33
1.10 0.05
3.12 + 0.14
Sensitivity to A. To quantify sensitivity to imitation weight A. The imitation weight controls the strength of the Wasserstein imitation objective for πT We start from the nominal value used in the main experiments and perform a small sweep around it by scaling À by factors (0.5, 1.0, 2.0). As command tracking but produces less human-
summarized in Table 18, smaller À slightly improves like, stiffer motion. Larger À yields smoother and more human-like gaits, at the cost of a modest increase in tracking error. When combined with the same सı and mo, robustness under disturbances changes only marginally across this range, since recovery capability is primarily governed by Ta and the switching behavior of mo. Overall, HWC-Loco is not overly sensitive to the exact choice of within this range.
Table 18: Sensitivity of HWC-Loco to the imitation weight A.
Policy
Success Rate
Goal-tracking t
Human-like J
HWC-Loco (0.5)
98.21 + 0.14
1.12 + 0.01
3.23 + 0.03
HWC-Loco (1.0)
98. 14 + 0.35
1.12 = 0.01
3.11 + 0.03
HWC-Loco (2.0)
98.08 + 0.26
1.12 0.02
3.08 + 0.03
B.3 COMPARISON TO STRONG DOMAIN RANDOMIZATION POLICIES
To further clarify the benefit of our structural safety-recovery hierarchy, we compare HWC-Loco against strong dondin-randomized (DR) baselines under extreme disturbance settings. addition to our standard DR baselines, Hist, which uses a single history-awar policy trained with large-scale domain randomization and regularization rewards, and is evaluated at randomization scales 1, 2, and 4. This baseline shares the same randomized dynamics and disturbance distributions as our safety-recovery policy. As summarized in Table 19, under comparable DR configurations HWC-Loco achieves clearly superior performance in most scenarios, with substantially fewer catastrophic failures and more successful recoveries, especially under constant external forces and high-impulse disturbances. These results highlight the value of the proposed safety-critics hierarchy on top of DR.
introduce strong non-hiene aon-herarcchial baseline, denoted Large-DR -RR-
Table 19: Comparison with strong DR baselines under extreme disturbances.
Policy
Low-freq.
Constant
Low-imp.
High-imp.
Low Payload
High Payload
Large-DR Hist (1.0)
87.15 0.23
61.72 - 0.36
85.45 0.87
62.38 + 1.03
79.53 4 0.76
60.23 1 0.61
Lange-DR-Hist (2.0)
88.45 + 0.56
65.32 1 0.28
88.47 + 1.75
68.36 + 0.79
82.32 + 0.87
65.21 + 0.57
Large-DR- (4.0)
90.35 +: 0.43
70.53
: 0.42
90.67 + 1.31
71.36 + 0.59
85.77
+ 0.93
70.36 : 0.69
HoCo-Loco
95.88 0.37
75.95 0.66
94.84 0.54
81.27 1 0.80
87.43 0.92
69.86 4
B.4 ROBUSTNESS TO VAE ESTIMATION NOISE
To quantify the quality of the VAE estimates, we report the mean squared error of the reconstructed velocity and ZMP-related features under different disturbance scenarios in Table 20. The errors remain small across all settings, and only increase moderately under strong impulses or heavy payloads.
We further evaluate the robustness of Fy to additional synthetic noise by injecting Gaussian noise with standard deviation o into the VAE-estimated features at test time. As shown in Table 21, for all realistic noise levels (a < 1.0). the success rate remains high (> 76%) and the number of switches (Flip Count) increases only mildly. Performance degrades substantially only under extreme noise
24

---

Published as a conference paper at ICLR 2026
Table 20: VAE estimation accuracy across disturbances.
Scenario
Velocity
ZMP Features
Normal
0.0250 + 0.0008
0.0491 + 0.0005
Low-Frequency
0.0242 0.0009
0.0493 0.0008
Constant
0.0294 + 0.0009
0.0507 + 0.0009
Low-Impulse
0.0408 0.0013
0.0518 0.0016
High-Impule
0.0719 = 0.0014
0.0531 = 0.0014
Low Payload
0.0497 0.0012
0.0577 0.0012
High Payload
0.0510 + 0.0013
0.0751 + 0.0018
(o= which is well beyond the observed VAE error. These results indicate that To is robust to realistic estimation noise and does not exhibit significant chattering.
Table 21: Robustness of the high-level policy To injected VAE estimation noise.
Noise Level
Success Rate (%)
Flip Count 4
0.0
78.65 + 0.53
80
a 0.1
77.77 0.62
81
0,2
77.36 0.61
82
σ 0.5
77.33 + 0.64
84
1,0
76.99 = 0.83
96
O 2.0
21.81 3.41
126
B.5 ABLATION ANALYSIS
Switching Strategies Ablation. We compare our learned high-level planner with a fixed-threshold.
7MP-based heuristic that triggers recovery whenever the estimated ZMP exceeds margin (0.2 or 0.4). Although intuitive, this heuristic relies VAE-based ZMP estimate and thus poise-sensitive, often over-triggering recovery otherwise recoverable perturbations. As summarized in Tables 22, 23, 24, and 25, the fixed-0.2 rule attains success close to ours but incurs very high switch counts and degraded tracking, while fixed-0.4 reduces switching and improves tracking at the cost of success. In contrast, our learned planner exploits temporal context and jointly optimizes success, tracking. and switching: HWC-Loco achieves the best overall trade-off (highest success with competitive switching), whereas HWC-Loco-1 minimizes switching and attains the best tracking.
Table 22: Low-Frequency Disturbance
Method
Success Rate t
Goal Tracking
Switch Count
HWC-Looc-fixed-0.22
94.83 + 0.31
0.93 + 0.01
461
HWCCLooc-fixed-a
91.54 + 0.32
1.00 0.01
190
HWC-Loco-I
92.69 + 0.56
1.04 0.01 0.01
21
HWC-Loco
95.88 + 0.37
1.02 + 0.01
183
Table 23: Constant Disturbance
Method
Success Rate f
Goal Tracking
Switch Count
HWC-Loco-fied
70.71 + 0.63
0.86 + 0.01
459
HIWC-Locoffxed-ae
64.86 + 0.78
0.95 + 0.01
206
H-C-Loco-l
68.60 + 0.28
0.99 +: 0.01
45
HWC-Loco
75.95 = 0.66
0.97 0.01
195
25

---

Published as a conference paper at ICLR 2026
Table 24: Low-Impulse Disturbance
Method
Success Rate 1
Goal Tracking
Switch Count
HWC-Loco-fixeee
-ieco-fieee-.22
95.04 0.31
0.81 + 0.01
468
WWWC-Loc-ffied-OO
91.81 = 0.83
0.92 0.01
209
HWC-Loco-I
92.79 + 1.75
0.94 0.01
47
HWC-Loco
94.84 = 0.54
0.92 + 0.01
198
Table 25: High-Impulse Disturbance
Method
Success Rate f
Goal Tracking f
Switch Count
WWWC-Locoffieed-a
80.31 + 0.54
0.80 + 0.01
469
WWWC-Loco-fixed-O0
75.84 0.41
0.90 + 0.01
214
HWC-Loco-1
77.09 = 0.79
0.92 + 0.01
61
HWC-Loco
81.27 = 0.80
0.89 + 0.01
215
Robust-aptimization Ablation. We train the recovery policy with the 7MP constraint but without the extreme-case uncertainty set. As shown in Table 26, using only the ZMP constraint preserves good performance in nominal settings (e.g.. low-ffequency disturbances), but robustness degrades under challenging conditions such as high-impulse perturbations. This indicates that adversarial training is key for handling real-world uncertainty. Overall, the ZMP constraint promotes baseline safety, while adversarial perturbations are essential for robustness in high-noise or extreme scenarios; the combined effect yields the best performance.
Table 26: Robust-optimization ablation on success rate (%). "w/o" denotes training without the extreme-case uncertainty set.
Policy
Low-freg, 1
Constant f
Low-imp. f
High-imp.11 1
Low Payload 1
High Payload
HWC-T (w/o)
94.67 0.31
72.23
0.43
92.67 + 0.75
76.29 + 0.34
84.76 + 2.66
65.21 0.83
HWC-Loco
0.37
81.27 0.80
87.43
69.86 1.00
Non-robust constrained ablation. To isolate the effect of the max-min robust optimization term in Eq. 4, we compare HWC-Loco with two non-robust baselines: 1) CRL, a constrained RL variant that optimizes Eq. 2 with the same safety constraint but without the adversarial extreme set; and 2) RL-Penalty, standard RL baseline trained with a large safety penalty under the same domain randomization CRL. As shown in Table 27, both CRL and RL-Penalty either sacrifice tracking performance or converge to less natural, regularied gaits. In contrast, the full robust CRL formulation used by HWC-Loco matches or slightly improves the success rate while achieving better command tracking and more human-like motion. This ablation indicates that the max-min robust optimization in Eq. provides a clear advantage over standard constrained RL and penalty-based reward shaping.
Table 27: Performance comparison with non-robust constrained methods.
Policy
Success Rate
Goal-tracking t
Human-like +
CRL
98. 12.1266 + 0.32
1.06 + 0.08
3.56 +: 0.14
RL-Peenlty
96.24 + 0.13
1.11 + 0.18
3.45 + 0.23
HWC-Loco
98.14 + 0.35
1.12 + 0.01
3.11 = 0.03
History Length Ablation. As shown in Table 28, investigate the impact of observation history length on HWC-Loco's performance. Setting 10 achieves the best overall trade-off across the three metries. While increasing H beyond 10 slightly improves the success rate, it leads to
26

---

Published as a conference paper at ICLR 2026
noticeable decline in goal- toal-nrac accuracy and miman-likenees. This suggests that excessively long history may hinder the effective use of privileged information, resulting in more conservative and less adaptive behaviors.
Table 28: Performance comparison with different history lengths (H)
History Length
Success Rate t
Goal-tracking
Human-like 4
95.13 + 0.51
1.07 0.00
3.26 + 0.06
10
96.73 0.40
1.10 0.00
3.18 0.01
20
96.53 + 0.40
1.08 + 0.00
3.18 + 0.02
40
97.38 0.33
1.02 - 0.00
3.21 + 0.02
B.6 REAL WORLD EXPERIMENTS
Effectiveness Tests in the Real World. Figure 10 demonstrates the robot's ability to traverse multi-terrain course consisting of 15 cm steps, 20 slopes, and wooden boards at a speed of 0.3 m/s. Figure 11 shows the ability to stability in real-world outdoor locomotion, including flat ground, grassy surfaces, and slopes.
Figure 10: Climb Stairs Test. The blue segments indicate the activation of the goal trackinackig policy, while the orange segments correspond to the safety recovery policy.
Robustness Tests in the Real World. As shown in Figure 12, we apply various disturbances to Unitree HI lasting for one minute. Four representative disturbance segments were selected for analysis. The time is measured in control cycles, with each cycle lasting 0.01 seconds. Furthermore, we conduct an outdoor disturbance test by applying pushes, pulls, and kicks to the robot, as shown in Figure 13. The robot waves its arms and adjusts its gaits maintain balance. Notably, the high control frequency (100 Hz) ensures smooth transitions between policies, and isolated activations of the recovery policy have minimal impact on overall stability.
Failure Mode Analysis. Although HWC-Loco robust across diverse scenarios, is not failure- free. In real- world tests, typical failures arise when operating conditions lie far outside the training distribution:
Extreme surface conditions. On very low-friction surfaces (c.g., ice-like friction far below training values), the feet cannot generate sufficient ground reaction forces, and the robot may slip and fall even when wg frequently invokes T2.
Aggressive, out-of-range commands. When commanded velocities or abrupt turns signifi- cantly exceed the training range, क. can drive the robot into configurations from which m can no longer recover.
Unmodeled hardware issues. Severe communication delays, sensor drop-outs, actu- ator faults are not fully captured by our uncertainty set; in such cases, violated feedback assumptions can cause failures despite switching to π2.
27

---

Published as a conference paper at ICLR 2026
Walking on flat terrain.
(b) Traversing grassy surfaces.
(c) Climbing a slope.
(d) Descending a slope.
Outdoor Deployment across Diverse Terrains: (a) flat ground, (b) grassy surface, (c) upward slope, and (d) downward slope.
Figure 11:
28

---

Published as a conference paper at ICLR 2026
Pull
Backward
Kick
Forward
Kick
Sideward
Kick
M
(a) Pall
(b) Backward Kick
(c) Forward Kick
(d) Sideward Kick
Figure 12: Disturbances in Realistic Deployment: Policy Switching value of 0 corresponds to the goal-tracking policy, while value of denotes the safety recovery policy.
Figure 13: Robustness in Outdoor Settings: The robot responds to external disturbances in an outdoor environment by waving its arms and adjusting its gaits to maintain balance.
29

---

Published as a conference paper at ICLR 2026
B.7 SCALABILITY EXPERIMENTS
Case Study on Motion Tracking. We visualize the robot's performance on tracking punching motion in Figure 14. During the execution, the robot experiences an impulse impact. In response, it dynamically activates the recovery policy, switching to more conservative and safer actions. Specifically, it lowers its body, adjusts gait, and stabilizes itself to satisfy ZMP constraints, rather than strictly following the original target motion. As a result, it generates backtracking motion to maintain balance and ensure robustness.
Figure 14: Robustness in Motion Tracking: The green dots represent the target keypoints, the blue dots indicate the keypoints corresponding to the DoFs, and the red dot denotes the ZMP.
Table 29: Comparison of Humanoid Configurations
Parameter
H1
G1
Height
178 cm
127 cm
Mass
47 kg
35 kg
DoFs
19
23
General and Flexible Locomotion. G1 outperforms HI across all three metrics (success rate, goal-tracking accuracy, and human-likeness) primarily due to the following factors:
As shown in Table 29, GI's smaller size and lighter mass contribute to lower inertial loads and reduced torque demands. This facilitates greater stability, especially during dynamic motions, and enables faster corrective actions in response to disturbances.
Size and Mass:
Degrees of Freedom (DoFs): With 23 DoFs compared to HI's 19, GI offers enhanced joint-level flexibility. The additional Dol', particularly the upper body and feet, allow for better whole-body coordination. This increased redundancy leads to improved disturbance rejection and more human- behavior during complex locomotion tasks.
Notably, as illustrated in Figure 15, GI is capable of achieving straight-kner walking, which remains significant challenge for HI due its larger size and reduced agility. The additional DoFs in GI enable finer control of limb trajectories, which is critical for maintaining balance during such biomechanically demanding gaits.
Figure 15: GI walk with straight knees
Robust Balance under Disturbances. To qualitatively evaluate balance and recovery challenging motions, deploy HWC-Loco the Unitree GI and command demanding single-leg stance under external pushes, using motion data from HuB Zhang et al. (2025a). As shown in Fig. 16, the robot is able maintain one-leg balance while compensating for the induced disturbances through
30

---

Published as a conference paper at ICLR 2026
coordinated whole-body responses. By leveraging its safety- structure to preserve stable ZMP region and avoid catastrophic falls, HWC-Loco demonstrates superior robustness in safety-critical regimes.
Figure 16: Single-leg balancing under external disturbances.
31


Reviews:
[{"review_id": "Mrr3ht662q", "reviewer": "Reviewer_L28V", "summary": "This paper, \"HWC-LOCO,\" proposes a novel hierarchical whole-body control (HWC) approach for robust humanoid locomotion. The core contribution is a hierarchical policy designed to dynamically resolve the trade-off between aggressive goal-tracking and conservative safety-recovery, particularly under environmental disturbances and Sim2Real mismatch. The authors frame the policy learning as a robust constrained reinforcement learning (CRL) problem, maximizing task rewards while ensuring worst-case feasibility constraints across an uncertainty set of transition dynamics. The hierarchical structure consists of a high-level planner that switches between a task-oriented goal-tracking policy (trained with a mimic learning objective for natural, human-like motion) and a stability-focused safety-recovery policy (enforcing ZMP-based constraints). The method is evaluated extensively in simulation and on a real-world humanoid platform, demonstrating superior robustness and performance compared to state-of-the-art baselines across various terrains and", "questions": "1. the paper compares HWC-Loco against standard baselines. Could the authors provide an ablation study comparing the full HWC-Loco (trained with Robust CRL, Eq. 4) against a version trained with the simpler Constrained RL (Eq. 2) or even a standard RL with a large penalty for constraint violation? This would isolate the benefit of the max-min robust optimization component specifically.\n\n2. The high-level planner is key to the dynamic trade-off. What is the exact trigger mechanism for switching from the goal-tracking policy to the safety-recovery policy? Is it a simple threshold on a stability metric (e.g., ZMP distance from the support polygon), or is it a learned policy itself? Please elaborate on the input features and the decision logic of the planner.\n\n3. The safety-recovery policy enforces ZMP-based constraints. Is this policy trained to be general across all tasks and terrains, or is it specialized? If it is a fixed, model-based controller, please state this clearly. If it is a learned policy, how is its robustness guaranteed, and how does it interact with the goal-tracking policy's learned dynamics?\n\n4. The Sim2Real success is impressive. Were any specific system identification or domain randomization techniques used to tune the simulation parameters to match the real robot before training? If so, please detail these steps, as they are often critical for successful Sim2Real transfer in complex systems like humanoids.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper presents a novel and well-motivated combination of existing concepts. The formulation of humanoid locomotion as a robust constrained RL problem (Equation 4) is a significant and original contribution, moving beyond simple reward-shaping for safety. The hierarchical architecture, which explicitly and dynamically switches between a task-maximization mode and a safety-guarantee mode, is a practical and elegant solution to the classic robustness-vs-performance trade-off in safety-critical systems.\n\nThis work is highly significant for the field of humanoid robotics. Robust and reliable locomotion is a critical bottleneck for real-world deployment. By providing a principled way to integrate safety guarantees (via ZMP-based constraints in the recovery policy) with high-performance task execution (via the goal-tracking policy), HWC-Loco offers a foundational advancement. The demonstrated Sim2Real success and superior robustness under external disturbances suggest a practical and deployable control framework.", "weaknesses": "1. While the overall formulation is novel, the concept of a hierarchical controller switching between a task-policy and a recovery-policy is not entirely new in robotics (e.g., in model-based control or even some prior RL works). The paper's novelty rests heavily on the robust constrained RL formulation that trains this hierarchy. The authors should more explicitly discuss and contrast their hierarchical training approach with prior hierarchical execution methods to better highlight the distinction.\n\n2. The robust optimization objective (Eq. 4) depends on the uncertainty set parameter $\\alpha$ and the feasibility constraint $\\epsilon$. The paper mentions that $\\alpha$ specifies the scale of mismatch, but the sensitivity of the final policy's performance and robustness to the choice of $\\alpha$ is not thoroughly explored. A poor choice of $\\alpha$ could lead to either an overly conservative or insufficiently robust policy. More detailed analysis or guidance on selecting these critical parameters would strengthen the work.\n\n3. Robust RL methods, especially those involving a max-min objective or sampling from an uncertainty set of dynamics, are typically computationally expensive. The paper does not provide sufficient detail on the training time or computational resources required for HWC-Loco compared to the baseline methods. Given the complexity of whole-body humanoid control, this is a crucial practical consideration that should be addressed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gWi5Gv7LwM", "reviewer": "Reviewer_qydQ", "summary": "The authors present a learning based hierarchical control framework for humanoid robots that ensures task completion while maintaining safety requirements. A high level planning policy, trained using double DQN, selects between a goal tracking policy and a safety recovery policy based on robots state and historical observations. The paper presents impressive results in simulation as well as real world experiments, across various tasks and terrains. The extensive evaluation demonstrates superior performance over comparable baselines.", "questions": "1) What are some of the failure cases and what caused them?\n2) What sort of domain randomization was applied in simulation?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is well written, structured and easy to understand. The paper tackles a challenging problem and the proposed approach is appropriately motivated and positioned well among related work. Real world robust humanoid locomotion is a challenging task and the results presented in this paper is quite impressive. \n\nThe paper provides sufficient experimental details and extensive validation. Tests across a wide range of conditions such as terrains, task commands and disturbances provide insightful details about the policies performance. Comparisons against relevant baselines and ablation studies shed light on the effectiveness of the proposed approach well.", "weaknesses": "One of the weaknesses I can think of is the complex framework - this involves training two lower level policies separately, then a high level planner , GAN discriminator, and to enable real world deployment a VAE encoder that estimates privileged information that the policy has during training in simulation. These design choices might be hard to reproduce or deploy.\n\nSince the networks are not trained jointly, there could be switching behavior between the two low level policies due to errors in the system. For example, if the VAE estimator has noisy estimates of privileged information.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A21ffMH0NP", "reviewer": "Reviewer_qF8c", "summary": "The paper proposes HWC-Loco, a hierarchical approach for humanoid locomotion balancing safety and agility. The policy is switching between two policy classes: a goal-tracking policy which imitate human motion and a recovery policy trained with robust objective. Then, a high-level planner is learned to switch between two policies optimizing for overall tracking performance. The results is verified in sim and real world.", "questions": "- how sensitive the tracking/robustness trade off to the imitation multiplier lambda? would the increase of human-like behavior would lead to less recovery agility?\n    \n- is it possible to distill two policies into a single one to enable smooth transition by teacher-student training?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- the hierarchical approach that target for safety and performance trade-off is natural and well-motivated.\n    \n- the performance of controller is evaluated comprehensively with diverse metrics to show is robustness and naturalness.", "weaknesses": "- comparison mainly ephasize dreamwaq and hal, where recent strong baselines for humanoid locomotion without hierarchical design like [1] as well as the same style switch controller like [2] is missing.\n    \n- it would be beneficial if the author could further discuss the sim2real gap identified in the real world deployment and explains with quantitative result on how those sensing and actuation gap hurts the standard policy design and why the hierarchical design can improve on it.\n    \n\n[1] AdaMimic: Towards Adaptable Humanoid Control via Adaptive Motion Tracking, Huang et al\n\n[2] Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion, He et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lZscfLq5De", "reviewer": "Reviewer_tg3Q", "summary": "This paper presents HWC-Loco, a hierarchical whole-body control framework for robust humanoid locomotion. It combines a goal-tracking policy for task performance and human-like motion with a safety policy for handling safety-critical scenarios with a high-level planner that can dynamically switches between them to keep safe and precise. By formulating policy learning as a robust optimization problem under mismatched environmental dynamics, HWC-Loco ensures safety while maintaining efficiency, demonstrating superior performance across diverse terrains, robots, and tasks in both simulation and real-world deployments.", "questions": "1. Why does the baseline Goal-tracking consistently outperform AHL/DreamWaQ?\n2. Some videos on the project website (e.g., Soft and Slippy Terrain) were not visible at my review time. Is this an accessibility issue on my side, or were those videos not yet uploaded?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. HWC-Loco quantitatively improves task success rates and demonstrates reasonable high-level policy switching to maintain safety.\n2: The high-level policy and hierarchical architecture are well-designed, effectively balancing goal-tracking and safety recovery.\n3. The paper presents extensive experiments, and the overall structure and clarity of the manuscript are strong.", "weaknesses": "1. The paper lacks comparison to commonly used baselines such as larger domain randomization or larger domain randomization combined with history-aware policies. While the authors argue that “excessive regularization can greatly affect the efficiency of control policy,” they do not provide evidence that HWC-Loco achieves a better trade-off between efficiency and safety relative to these methods. (If these issues are addressed with stronger evidence, I’d be happy to raise my score.)\n2. Performance/Evaluation Concerns: 1) The terrain experiments do not clearly illustrate when or how the high-level policy switching occurs, making it difficult to understand the specific contribution of the safety policy. 2) The terrains and challenge cases used are relatively simple. The paper should include scenarios that highlight the necessity of the hierarchical design—i.e., cases where success cannot be achieved without it or with standard domain randomization alone. Providing qualitative comparisons would help distinguish HWC-Loco from these baselines. \n3. The reliance on ZMP constraints is not that generalizable or robust. ZMP-based control depends heavily on accurate robot dynamics modeling and sensing, which can undermine the reliability (though I agree that integrating it within hierarchical, rather than enforcing hard constraints, is a more flexible design). Moreover, for tasks involving rapid or aerial motions (e.g., jumping) or requiring more agile locomotion, enforcing ZMP-based safety objectives may conflict with task goals, thereby restricting the effectiveness of the safe policy. Finally, its applicability to loco-manipulation under external forces remains limited", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)