From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots
Authors: Yuxuan Wang
Ming Yang
Ziluo Ding
Yu Zhang
Weishuai Zeng
Xinrun Xu
Haobin Jiang
Zongqing Lu
Venue: NeurIPS 2025 spotlight ()
Keywords: robotics
Primary area: reinforcement_learning

Abstract: Achieving general agile whole-body control on humanoid robots remains a major challenge due to diverse motion demands and data conflicts. While existing frameworks excel in training single motion-specific policies, they struggle to generalize across highly varied behaviors due to conflicting control requirements and mismatched data distributions. In this work, we propose BumbleBee (BB), an expert-generalist learning framework that combines motion clustering and sim-to-real adaptation to overcome these challenges. BB first leverages an autoencoder-based clustering method to group behaviorally similar motions using motion features and motion descriptions. Expert policies are then trained within each cluster and refined with real-world data through iterative delta action modeling to bridge the sim-to-real gap. Finally, these experts are distilled into a unified generalist controller that preserves agility and robustness across all motion types. Experiments on two simulations and a real humanoid robot demonstrate that BB achieves state-of-the-art general whole-body control, setting a new benchmark for agile, robust, and generalizable humanoid performance in the real world.

From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots
Yuxuan Wang 12+
Ming Yang!2- Ziluo Ding Yu Zhang
Weishuai Zeng' Xinrun Xu²
Haobin Jiang Zongqing Lu 12f
!eeking University BeingBeyond
https://beinggbyeee github. i BumbleBee/
Start
End 135s
Figure 1: A general whole-body control policy in the real world tracks the motion of two consecutive long-duration actions, with total duration of approximately 135 seconds. A boxing sequence appears at the top of the image, with a Charleston dance shown at the bottom.
Abstract
Achieving general agile whole-body control on humanoid robots remains a major challenge due diverse motion demands and data conflicts. While existing frame- works excel in training single motion-specifie policies, they struggle to generalize across highly varied behaviors due to conflicting control requirements and mis- matched data distributions. In this work, we propose BumbleBee (BB), an expert- generalist learning framework that combines motion clustering and sim-to-real adaptation to overcome these challenges. BB first leverages an autoencoder baaseddder---dedd clustering method to group behaviorally similar motions using motion features and motion descriptions. Expert policies are then trained within each cluster and refined with real-world data through iterative delta action modeling sim-to-real gap. Finally, these experts are distilled into a unified generalist con- troller that preserves agility and robustness across all motion types. Experiments on two simulations and a real humanoid robot demonstrate that BB achieves state- of-the-art general whole-body control, setting new benchmark for agile, robust, and generalizable humanoid performance in the real world.
bridge the
"These authors contributed equally to this work. CCorrespondence to Zongqing Lu <zongging Iu @ pku.edu.cn>.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Humanity has long dreamed of using humanoid robots to assist or replace humans in completing various daily tasks, including grabbing objects [Lenz al., 2015, Joshi al., 2020, Miller and Allen, 2004, Du et al., 2021, Shao et al., 2020], carrying goods [Pereira et al., 2002, Dao et al,, 2024, Agravante et al., 2019], etc. At present, single agile whole-body humanoid skill can be trained to a remarkably high level of performance. The ASAP [He et al., 2025] framework first trains motion-specific tracking policies in simulation, then refines them using real-world data through motion-specific dynamic policy learning. This framework enables the robot to effectively bridge the sim-to-real gap and achieve a wide repertoire of agile and expressive behaviors. However, it is hard to directly apply this framework to general motion tracking, as different motions require the robot to focus on different aspects of control. For example, aggressive actions like jumping or fast walking demand precise, high-torque control, while conservative motions prioritize balance and smoothness. These differences lead to mismatched data distributions across motion types. It can lead to conflicting gradients during training, ultimately degrading the performance of the policy.
Intuitively, mitigate the issues caused by mismatched data distributions, there two main direc- tions. One is at the model level (Be more powerful): employing more expressive architectures such as Transformers Diffusion Models [Shridhar et al., 2023, Kim et al., 2021, Brohan et al., 2022, Radosavovic et al., 2024], which are capable of capturing complex motion distributions across di- verse skills. The other is the data level (Decompose the complexity): introducing structure into the training data, such as clustering motions by type or difficulty, balancing data across motion cate- gories, or using curriculum learning strategies to gradually expose the policy to increasingly diverse behaviors [Ji et al., 2024]. These approaches aim to reduce interference between conflicting samples and improve the generalization capability of the learned policy. Inspired by the success of mixture of experts (MoE) [Shazeer et al., 2017, Lepikhin et al., 2020] in LLMs, we follow the second path to build a general whole-body control policy.
To this end, we propose BumbleBee (BB), an expert-generalist pipeline that enables general agile humanoid whole-body controller. Since both tracking and dynamic policies benefit from training on behaviorally similar motions, the dataset is first clustered using an autoencoder (AE) [Hinton and Salakhutdinov, 2006] trained on kinematic features, including additional leg-related features, and motion text descriptions. This design reflects the fact that different leg movements, such as walking, jumping, or crouching, require distinct torque control patterns, making leg dynamics key factor in categorizing motion. A general tracking policy is initially trained on the full dataset and then fine-tuned within each cluster to obtain specialized expert policies. These experts are deployed on the real robot to collect real-world trajectories. For each cluster, a delta action model is trained to compensate for sim-to-real discrepancies by modeling the difference between simulated and real- world state transitions. The expert policies are further refined using the corresponding delta model in an iterative manner. Finally, a unified generalist is obtained through a combination of expert knowledge distillation.
To summarize, our main contributions are as follows: 1) We propose an expert-to-generalis frame- work for training agile whole-bodi control policies, which effectively mitigates interference across diverse motion types. 2) We introduce an rugo-regressive clustering method that leverages leg- specific motion features and text descriptions to structure the dataset, enabling better specialization of control policies. 3) We demonstrate that our method, BumbleBee, achieves superior control per- formance in both simulation and real-world experiments, outperforming baselines in terms of agility, robustness, and generalization.
2 Related Work
2.1 Humanoid Whole-Body Control
Whole-body control (WBC) [Chignoli et al., 2021, Darvish et al., 2019, Dallard et al., 2023, Hutter et al., 2016, Kajita et al., 2001, Moro and Sentis, 2019, Li et al., 2025, Ding et al., 2025] is essential for high DoF humanoid robots to perform coordinated and stable movements complex tasks. Recent advances in reinforcement learning have enabled robots to acquire diverse whole-body skills, yet training general WBC policy remains challenging due to hardware constraints and the inherent complexity of the robotic action space.
2

---

Text information I.
Step 1. AE clustering
Step 3. Generalist
A with right head A person squats in place.
jumps up only
Text Encoder
S2s
S22
Sgn
Motion Decoder
M
BUMBLEBFE WHC Salley
Snt
Saz
Smn
Motion
Mation Encoder
DAgger
Six Clusters
Tranfformrr
Stift Ratargating
PHC Fiiter
Step 2. Experts
k times
Expert
Sup
Spnce
Show
Delta
jump
Sart
Pipuld
Now
General Motion
Robot
Tracking Policy
Figure 2: Overview the BumbleBee framework. The left section illustrates the data curation stage, which includes motion retargeting and PHC-based filtering. The upper middle part shows the autoencoder-baase clustering process, where motions are grouped via their semantic and kinematic characteristics. The lower section depicts the iterative delta fine-tuning of expert policies for each cluster. Finally, as shown on the far right, all experts are distilled into a single general WBC policy using a Transformer-bar based architecture.
Previous works address this different ways: the H2O series [He al., 2024a,b] incorporates global keypoint positions but suffers from eтror accumulation in long- morizon tasks; HumanPlus [Fu et al., 2024] uses only joint angles for better real-wor world adaptability; Hover [He et al., 2024c] introduces observation masking for flexible control mode switching. However, these methods are all trained on the full AMASS [Mahmood et al., 2019] dataset, where diverse motion distributions may cause gradient conflicts and reduce generalization. Exbody2 [Ji et al., 2024] attempts mitigate this through progressive learning based on task difficulty.
In contrast, we propose clustered expert-to- poprrttt-geane to-generalist paradigm. By classifying motion types and training specialized expert policies on each cluster before distillation, our approach reduces cross- task interference and improves both training efficiency and generalization.
2.2 Sim-to-Real Transfer
The sim-to-real transfer challenge primarily arises from factors: (I) inaccuracies in robot mod- cling, including structural simplifications and parameter errors, and (2) the complex, nonlinear, and time-varying dynamics of real-world environments, e,g., contact events, friction variations, and elas- tic deformations, that exceed the fidelity of modern physics simulators.
Traditional approaches address this gap through System Identification (SysID) [Áström and Eykhoff, 1971, Kozin and Natke, 1986, Yang et al., 2024, Yu et al., 2018, 2020], which uses real-world ob- servations to calibrate simulator parameters for improved physical realism. Domain Randomization offers an alternative by perturbing environment parameters during training enhance policy robust- ness and generalization.
ASAP proposes a different strategy by learning action delta model to directly compensate for simulation-reality discrepancies using real-world data. Building on this idea, our method extends the action delta model by training it separately for each motion cluster. Compared to single unified model, this cluster-specific approach enables improved sim-to-real performance across diverse range of motions.

---

3 Method
Training a general whole-body control policy directly is challenging, primarily due to conflicts among diverse motion types. These motions differ significantly in their control objectives, for ex- ample, jumping requires high-torque control, whereas in-place movement emphasizes balance and continuity. To address this challenge, take new perspective on organizing motion data and policy learning by introducing an expert-to-generaaist framework as illustrated in Figure 2. Rather than training a single policy kinematic characteristics of each motion type, thereby reducing cross-task interference and enabling more targeted learning.
heterogeneous dataset, first partition the data based the
Dataset Curation. Following prior work [He et al., 2024a], we first retarget SMPL-format [Loper et al., 2015] human motion sequences from the AMASS dataset into robot-specific representations, including global translations and joint axis-angle rotations. Given that AMASS contains wide range of motions, such as crawling and climbing, we perform an additional data cleaning step to ensure quality. We apply PHC [Luo al., 2023] filter the dataset [He al., 2025], resulting in 8,179 high-quality trajectories for clustering and training.
3.1 AE Clustering
We leverage intermediate representations generated within a self-supervised framework, i.e. an autoencoder, to cluster motion data from the entire dataset. We aim to categorize the data based on motion types, for example, grouping jumping movements into one cluster and in in-place movements into another.
Our method not only relies solely on autoencoders to reconstruct full motion sequences, but also incorporates textual annotations align motions the semantic level. This ensures that motions with diverse patterns but shared semantics are positioned closer in the latent space. For example, the act of walking may take the form of linear movement or circular paths. These patterns would be hard to associate with based only on motion alignment. However, things can be different when textual semantic alignment is introduced. Note that the text information for training is from the open-source HumanML3D [Guo et al., 2022] dataset, which offers textual annotations and corresponding frame ranges for most of the sequences AMASS.
In addition, our method does not reconstruct motion sequences represented by the SMPL format. Since SMPL primarily contains joint angles and root transformations, it fails to represent the kine- matic dynamics essential for distinguishing motion types. To address this, we first apply forward kinematics convert joint rotations and root positions into 3D coordinates of joints in the world frame. Subsequently, we prune redundant joints and introduce foot velocity relative to the world frame, enhancing the model's capacity differentiate between motion types such as jumping, stand- ing, and walking.
For the encoding process, we adopt Transformer for motion encoding, following the previous work [Petrovich et al., 2023, 2022]. The motion encoder takes as input a motion sequence Miul = {pt.r,, ry. Ct, v/eet 1 and outputs motion representation zn where pa e Nx3 represents the 3D positions of all joints, F, E I23 is the root translation in 3D space, I, E IR3 is the root velocity, c E {0, 1) denotes the binary contact states of F feet, and vffet Fx3 represents the 3D velocities of the F feet. Textual data is first processed through a BERT model [Devlin et al., 2019] for serialization and then passed through Transformer to obtain latent representation 2! with the same dimensionality as z
The decoding module uses the same Transformer as the motion encoder, but with different input and output dimensions. The reconstruction focuses only on subset key joints (i.e. head, pelvis, hands, and feet), allowing the model to concentrate reconstructs the same features of the motion inputs from the latent representation z' or zm except that some redundant joint 3D positions mentioned above removed. The loss function is defined as:
the core motion features. Specifically, it
Letnster LIMONNEE( = + (2(2', (2(=',z"') =") )++ + Lhaber (M'. , M) + Cruber (M'', ,M),
where z! and nm denote the intermediate latent variables for the text and motion modalities, re- spectively. Lhuber refers to the Huber loss function, M represents key features selected from the

---

ground-truth -tunt motion sequence Minn M' and Mm are the reconstructed features from the text and motion modalities, respectively. The first two terms are used to align the motion latent space with the semantic latent space, and the last two terms are reconstruction losses to train the autoencoder. Finally, we apply the K-means algorithm to cluster the latent variables of all motion data, generated by the learned motion encoder.
3.2 Experts
To improve motion specialization and sim-to-real transfer, we introduce expert policies, i.e. motion tracking and delta action policies, trained on motion clusters derived from AE embeddings. All models in this part are implemented using three-layer multilayer perceptron (MLP) and are trained via reinforcement learning (RL). More details of reward design can be found in the Appendix B.
3.2.1 Motion tracking training
Initially, general motion tracking policy is trained on the entire dataset base model. Note that both joint angles and selected keypoint positions of the tracking/reference motion are included in the policy's observation. More details about the observation design of framework included in the Appendix A.
Instead of training from scratch, all expert motion tracking policies are derived from the base model. This because we expect each expert to retain some generalization capability for other motion clusters. Each expert policy is then fine-tuned specialize behaviorally consistent cluster of skills, e.g., walking, standing, or jumping. This fine-tuning significantly improves motion fidelity and effectively resolves training conflicts caused by diverse motion types.
specific motion cluster, allowing the policy to
3.2.2 Multi-stage delta action training
To further overcome the sim2real gap for tracking policy, we adopt the delta action fine-tuning framework. By applying the delta action the simulator dynamics and continuing fine-tune the tracking policy within this modified environment, the training process effectively approximates the training directly in the real world. Our key contribution lies in specializing this framework by training expert delta models for each motion cluster, rather than relying on a single unified model.
In more detail, each expert motion tracking policy is deployed on the real robot to collect its corre- sponding motion trajectories the real-world environment. At each timestep 4, using the onboard sensors of the 29-DoF Unitree G1 robot (23 DoF actively controlled, excluding the wrist joints), we record the following: base linear velocity ef huse E R3, the robot base orientation as a guater- nion aa 1 the base angular velocity hase R3, the vector of joint positions 9: R23, and joint velocities q: E 23 For each expert, we randomly sample several dozen reference motions for repeated execution, resulting in the collection of over hundred real-world trajectories in total. Subsequently, we further train expert delta action models based on the data collected for each expert following ASAP [He et al., 2025]. We find that, due to the consistent dynamic characteristics within each cluster of motion, this expert-specific training significantly improves the fitting accuracy of the delta action models and enables more efficient correction of the sim-to-real gap. Compared general delta action model, the expert models exhibit notable advantages in motion compensation accuracy and overall control performance.
With the learned delta action models, π ( ar), we reconstruct the simulation environment as follows, s1+1 (st a, t at)) and fine-tune the pretrained expert motion tracking policies within this modified environment. This process can be performed iteratively until both kinds of expert policies converge.
3.3 Generalist
After optimizing the expert policies, we employ knowledge distillation to integrate the knowledge of each expert policy and generate general whole-body control policy. We adopt DAgger [Ross et al., 2011] to implement multi-experts distillation. The distillation loss function is defined as:
Laisill EoND [KI. (el.(PPencnt (Pgencral (a s) (espon,M(e s)]]
5

---

Table 1: Main results evaluated in IsaacGym and MuJoCo. We assess performance using three key metrics: Success Rate (SR), Mean Per Joint Position Error (MPIPE), and Mean Per Keypoint Position Error (MPKPE). BB demonstrates superiority over baselines.
IsaacGym
MuJoCo
Method
SR1
MPKPE!
MPJPEJ
SR1
MPKPEL
MPJPEJ
OmaiH20 [He et al., 2024a]
85.65%
87.83
0.2630
15.64%
360.96
0.4601
Exbody2 al., 2024]
86.63%
86.66
0.2937
50.19%
272.42
0.3576
Hover [He al., 2024c]
63.21% %
105.84
0.2792
16.12%
323.08
0.3428
General Init
88.69%
92.88
0.2628
33.01%
342.68
0.3467
BumbleBee (BB)
89.58%
83.30
0.1907
66.84%
294.27
0.2356
where E.ED denotes the expectation over the states s in the training dataset D, KL(-II-) is the Kullback-Leibler divergence, ((x)ortA(s)) is the expert policy corresponding to state 8, and Pgeneral is the general policy. Here, k(s) denotes the index of the expert that corresponds to the cluster which state s belongs.
However, we observed that the capacity of the three-layer MLP is limited, making it insufficient effectively learn behaviors of multiple expert policies. To address this issue, we adopt a more expressive architecture, the Transformer, to serve as the backbone for the final general policy model. The Transformer enables better modeling of complex patterns across diverse states, allowing for more effective fusion of expert knowledge. More details of the architecture can be found in the Appendix C.
4 Experiment
4.1 Experiment Setup
Our models are trained in IsaacGym. Since there large gap between IsaacGym and the real world, MuloCo serves more reliable proxy for evaluating the capability of the model. As result, most of our evaluations are conducted in MuloCo. We assess performance using the filtered AMASS dataset described in Section 3. For real-world testing, we further include long-range mo- tions to evaluate the generalization and tracking capabilities of BumbleBee (BB). All training and deployment are performed on the Unitree GI robot, which has 29 DoF, 23 of which are actively controlled, excluding the wrist joints.
Baselines. To evaluate the effectiveness of our method, we compare with three state-of-the-ar (SOTA) methods: OmniH20 [He et al., 2024a], Exbody2 [Ji et al., 2024], and Hover [He et al., 2024c]. For the fair comparison, we either use the officially released code or closely follow the official implementation when code is unavailable, adapting each method to the Unitree GI robot (instead of H1/H1-2). addition, we keep the training dataset consistent with that used for BB. For Hover, we use unmasked observations during the evaluation to ensure optimal performance.
Metrics. We assess performance using three key metrics: Success Rate (SR), Mean Per Joint Position Error (MPJPE), and Mean Per Keypoint Position Error (MPKPE). SR reflects the overall capability of the policy. MPJPE measures its accuracy in tracking retargeted joint angles, while MPKPE evaluates the precision of tracking retargeted keypoints the world coordinate frame. Among all metrics, SR is the most critical, as it reflects the overall viability and stubility of the policy, Other metrics, such as MPJPE and MPKPE, are only meaningful when the policy can successfully complete the task, indicated by high SR. Detailed definitions of these metrics can be found in Appendix A.
4.2 Cluster Analysis
How do we determine the number of clusters? We determine the number of clusters using the Elbow Method [Thomdike, 1953], which identifies the trade-off point between using fewer clusters and

---

Table Statisties for each cluster. We measured the motion sequences in terms of the displacement distance (Displacement), the movement speed (Speed), the average displacement along the z z-axis (Z-Movee, the average per-step axis-angle change of joints (Joint), as well as the joint variations of the upper and lower body (Lower and Upper Joint).
Cluster
Disp (m)
Z-Move (mm)
Speed (m/s)
Joint
Lower Joint
Upper Joint
Keywords
Jump
2.32
9.21
0.329
0.379
0.346
0.405
jumps, jumping
solk-slow
3.42
1.60
0.353
0.279
0.358
0.215
jogs,
Walk-fast
3,24
5.63
0.429
0,430
0.472
0.396
walks, forward
Stand-up
0.89
0.68
0.061
0.230
0.136
0.305
something, hand
Stand-mid
1.33
0.82
0,119
0.198
0.170
0.221
arms, hand
Stand-low
1.84
1.52
0.148
0.274
0.281
0.267
foot, log
minimizing the within-cluster sum of squares (total square distance from all points to their respective
cluster centers). Based on this analysis, we set the number of clusters to six, as shown in Figure 3. Whether the clustering results meaningful in terms of both kinematics and semantics? Table 2 summarizes the kinematic features of the six clus-
[Baniw wiP)
Rlbow
Optinal
SS8E (Witin Cluster Sqen of Sqaree)
3.5-
ters. The first cluster has the highest Z.MMvve value (average displacement along the 7-axis), and is there-
3.0-
fore categorized as the jump cluster. The second and third clusters exhibit the largest ranges of displace- ment and are classified as walking behaviors. How- ever, the third cluster demonstrates higher speed com- pared to the second. The remaining three clusters pri- marily consist of in-place movements, based on their lower speeds and smaller displacement ranges. More specifically, the fourth cluster involves larger upper- body movements, the sixth is characterized by more prominent lower-body movements, and the fifth falls between the two. From the semantic perspective, extracted the top keywords from each cluster and find that their meanings closely align with the cor- responding kinematic characteristics, demonstrating that the clusters are semantically meaningful. For convenience, all clusters are named exactly as specified in Table 2.
1.b-
1.0
Figure 3: Elbow method showing the trade-off between cluster number and within-cluster sum of squares, with K selected as our selected point.
4.3 Generalist Performance
Does expert-to-generaais framework demon- strate superiority over existing methods? We an- alyze the performance of BB compared to existing SOTA methods in both IsaacGym and MuloCo sim- ulators. As shown in Table BB outperforms all baselines across nearly all evaluation metrics. In IsaacGym, BB shows advantages over other meth- ods in terms of success rate, MPJPF, and MPKPF: however, the minor differences observed are within reasonable variation and likely attributable to differ- ences in hyperparameter tuning rather than the al- gorithm itself. In contrast, the performance gap be- comes more pronounced MuloCo, more realis- tic simulator that better reflects real-world dynamics. In this setting, BB achieves a success rate of 86%,, significantly outperforming Exbody2 (50.19 19%) and all other baselines, which fall below 40%. This highlights the strong generalization capability of BB, which effectively integrates the spe- cialized strengths of multiple experts into a unified policy, enabling stable and accurate execution of whole-body motions across different domains.
Table 3: Comparison of BB with gen- eral policy trained without experts and one trained on experts from randomly split clus- ters, evaluated by success rate.
IsaacGym
MuJoCo
General Init
88.69%
33.01%
Random
86.25%
35.36%
BB
89.58%
66.84%
7

---

Hond
wilk
Hond
Sland
slaw
up
slaw
Hond
sloe
atond
Jump
dland tow
jomp
viand Jow
sbad lou
yoerp
Maadd Jow
(a) Stand Low
(b) Jump
(c) Stand Up
(d) Walk Slow
General Init
Expert Init
General Final
Expert Final
Figure 4:
Evaluation of expert vs. generalist models in MuJoCo, measured by success rate.
Whether the clustering is beneficial for training? To explore this question, conduct two ablation studies. The first baseline, General Init, directly trains a generalist policy without any expert special- ization. The second, Random, trains generalist policy based experts derived from six randomly partitioned subsets of the data. As shown in Table 3, the Random offers no clear advantage over General Init, highlighting the limited benefit of naive data partitioning. The superior performance of BB demonstrates that reasonable clustering plays a crucial role in policy learning. Even within randomly divided subsets, conflicting motion patterns may impede effective RL exploration.
4.4 Expert Performance
Do expert policies outperform the generalist within their respective motion clusters? To answer this question, evaluate the performance of expert policies. In Figure 4, show performance improvements across four motion clusters during multi-stage delta action training, with results for the remaining clusters included in Appendix D. Specifically, across four variants: the generalist policy without expert specialization (General Init), expert poli- cies before delta action fine-tuning (Expert Inif), expert policies after fine-tuning rounds (Expert Final), and the final distilled generalist (General Final).
compare success rates in MuJoCo
From these results, make several key observations. clusters leads to stronger tpsk-spec performance compared to the General Init, validating the effectiveness of autoencoder-basse clustering and the importance of specialization. Second, experts maintain a degree of generalization to motions outside their clusters, likely due to being initialized from the General Init. Third, delta action fine-tuning significantly improves each expert's performance. Surprisingly, the final generalist (General Final), distilled from all experts, sometimes outperforms the individual experts, i.e. jump and walk-slow. These motions are initially difficult to execute stably. We hypothesize that the generalist benefits from inheriting stable control behaviors reliably.
First, training expert policies motion
from multiple experts, allowing it to execute challenging motions
How does iterative delta fine-tuning affect the motion tracking policy? ? We adopt an iterative ap- proach to the optimization process, based the intuition that each refinement improves the tracking policy, which enables the collection of higher-qualit real-world data, thereby allowing the delta ac- tion policy be further enhanced and, in turn, further improve the tracking policy. To validate this, we evaluate policy performance across three training stages in MuJoCo and also verify the results on real humanoid robot. As shown in Table 4, the mean success rate steadily increases from 51.49% (Iter 0) to 60.33% (Iter /), and further to 70.37% (Iter 2), clearly demonstrating the effectiveness of iterative refinement. While additional iterations could potentially yield even better results, our experiments are constrained by limited computational resources.
We take stand-low without delta action fine-tuning (Iter 0), the policy fails to stabilize the foot, resulting in landing failure and deployment breakdown. After the first iteration (Iter 1), landing stability improves sig- nificantly, although the robot still struggles to lift its feet and exhibits visible trembling. After the second iteration (Iter 2), the policy track the motion smoothly and maintain balance. More detailed visualizations can be found in the Appendix
example to provide more intuitive understanding. As shown in Figure 5,
Whether the delta action model needs to be trained in specialized manner? To address this question, we conduct an ablation study using collected real-world data to train a general delta action model. The results are summarized in Table 5, where Expert Gen Final represents the final

---

Table Mean success rates of ex- perts evaluated on their respective motion clusters. The evaluation is in MuloCo.
Table Ablation Results of the General Delta Action Model on three clusters. Expert Gen Final represents the final expert after fine-tuning on the general delta model.
Cluster
Expert Init
Expert Gen Final
Expert Final
Iter 0
lter
leer
Jump
59.64%
50.71% %%
68.92%
Stand-up
64. 64.11%
75.85%
77.32%
SR
51.49%
60.33%
70.37%
Walk-slow
15.71%
42.32%
56.50%
(a) Itertion
(b) Iteration 1
(c) Iteration 2
Figure 5: Visualization expert performance across iterations in the real world. We deploy the stand-low policies from three training iterations perform the same right-stepping motion on a real robot, and observe a clear improvement in foot stability with each iteration.
expert after fine-tuning the general delta model. The general delta action model is trained using all real-world data collected in each iteration loop.
We observe that the general delta model improves performance in two motion clusters but struggles significantly on Jump. This aligns with our hypothesis that delta action learning is sensitive to distributional shifts across motion types, making the single general delta action model less effective in highly diverse categories. Furthermore, we find that the general model is harder to train and less stable compared to its specialized counterparts.
Why does a single delta action model strug- gle to generalize across motion clusters? To further investigate the variation in delta actions across different motion clusters, we analyze the magnitude of the delta action outputs. As shown in Table 6, Stand-up exhibits the small- est delta values, which is expected given that its motions are largely static. In contrast, Walk-fast shows the highest delta magnitudes due to its inherently large motion amplitudes. These dis- tributional shifts across clusters introduce con- flicts when attempting to train single delta action model, further highlighting the need for cluster-specific modeling.
Table 6: Magnitudes of delta action outputs for the ankle joint across different clusters. Higher values indicate greater motion adjustments.
Jump
Walk-slow
Walk-fast
Yaw Roll
0.2107
0.2865
0.3399
0.0589
0.0956
0.0843
Stand-up
Stand-mid
Stand-low
Yaw Roll
0.1556 0.0278
0.2098
0.2599
0.0509
0.1273
5 Conclusion
We present BB, framework for training agile and general whole-body control policies on humanoid robots. follows an expert-to-generaliss framework and uses auto-regressive encoder to cluster motions by semantics and leg dynamics, enabling effective specialization and knowledge transfer. Extensive experiments show that BB outperforms baselines agility, robustness, and generalization, highlighting its potential for real-world deployment.

---

Limitation Currently. BB does not utilize additional figh-precision localization sensors such as GPS or Visual-Inertial Odometry (VIO) [Huai and Huang, 2022]. As result, lacks access to global positioning information, which may introduce biases when aligning with the reference mo- tion sequence. We believe that equipping BB with additional sensors like high-precision IMU could significantly improve its performance in real-world scenarios by providing more accurate and reliable pose estimation. Moreover, the complexity of the overall pipeline in BB constrains its scal- ability, particularly when integrating real-world training feedback.
Acknowledgments
This work was supported by NSFC in part under Grant 62450001 and 62476008. The authors would like to thank the anonymous reviewers for their valuable comments and advice.
10

---

References
Don Joven Agravante, Andrea Cherubini, Alexander Sherikov, Pierre-Brice Wieber, and Abderrah- mane Kheddar. Human-humanoid collaborative carrying. IEEE Transactions on Robotics, 35(4): 833-846, 2019.
Karl Johan Aström and Peter Eykhoff. System identification-a survey, Automatica, 72))123-162, 1971.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-l: Robotics transformer for real-world control at scale. arXiv preprint arXiv:22 2668177, 2022.
Matthew Chignoli, Donghyun Kim, Elijah Stanger-Jones, and Sangbae Kim. The mit humanoid robot: Design, motion planning, and control for acrobatic behaviors. In 2020 IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), pages 1-8, 2021. doi: 10.1109/HU MANOIDS47582.2021..99 9555782.
Antonin Dallard, Mehdi Benallegue, Fumio Kanehiro, and Abderrahmane Kheddar. Synchronized human-hu motion imitation. IEEE Robolics and Automation Letters, 87):4155-4162, 2023. doi: 10. 2A.0.......338000.7.
Jeremy Dao, Helei Duan, and Alan Fern. Sim-to-real learning for humanoid box loco-manipulation. In 2024 IEEE International Conference on Robotics and Aulomation (ICRA), pages 16930- 16936. IEEE, 2024.
Kourosh Darvish, Yeshasvi Tirupachuri, Giulio Romualdi, Lorenzo Rapetti, Diego Ferigo, Francisco Javier Andrade Chavez, and Daniele Pucci. Whole-body geometrie retargeting for humanoid robots. In 2019 IEEE-RAS RAS 19th International Conference on Humanoid Robots (Humanoids), pages 679-686, 2019. doi: LLL19//Hummnoi444999990999999999
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pages 4171-4186, 2019.
Ziluo Ding, Haobin Jiang, Yuxuan Wang. Zhenguo Sun, Yu Zhang, Xiaojie Niu, Ming Yang. Weishuai Zeng, Xinrum Xu, and Zongqing Lu. Jaeger: Dual-level humanoid whole-body con- troller. arXiv preprint arXiv:2505.06588 2025.
Guoguang Du, Kai Wang, Shiguo Lian, and Kaiyong Zhao. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: review. Artificial Intelligence Review, 5473) 2021.
Zipeng Fu, Qingqing Zhao, Qi Wu, Gordon Wetzstein, and Chelsea Finn. Humanplus: Humanoid shadowing and imitation from humans. In Conference on Robot Learning (CoRL), 2024.
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating diverse and natural 3d human motions from text. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5152-5161, June 2022.
Tairan He, Zhengyi Luo. Xialin He, Wenli Xiao, Chong Zhang, Weinan Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Omnih2o: Universal and dexterous noman-to-humande whole- body teleoperation and learning. 2024a.
Tairan He, Zhengyi Luo, Wenli Xiao, Chong Zhang, Kris Kitani, Changliu Liu, and Guanya Shi. Learning human-to-humanoid real-time whole-body teleoperation. arXiv preprint arXiv:2403.0mm 04436 2024b.
Tairan He, Wenli Xiao, Toru Lin, Zhengyi Luo, Zhenjia Xu, Zhenyu Jiang, Changliu Liu, Guanya Shi, Xiaolong Wang, Linxi Fan, and Yuke Zhu. Hover: Versatile neural whole-body controller for humanoid robots. arXiv preprint 4r1.222410.21229, 2024c.

---

Tairan He. Jiawei Gao, Wenli Xiao, Yuanhang Zhang, Zi Wang. Jiashun Wang, Zhengyi Luo, Guanqi He, Nikhil Sobanbabu, Chaoyi Pan, Zeji Yi, Guannan Qu, Kris Kitani, Jessica Hodgins, Linxi "Jim" Fan, Yuke 7hu, Changliu Liu, and Guanya Shi. Asap: Aligning simulation and real- world physies for learning agile humanoid whole-hody skills. arXiv preprint arXiv:2502.01143, 2025.
Geoffrey E Hinton and Ruslan Salakhutdinov. Reducing the dimensionality of data with neural networks. science, 313(5786):504-507, 2006.
Zheng Huai and Guoquan Huang. Robocentric visual-inertial odometry. The International Journal of Robotics Research, 41(7)x667-689. 2022. doi: 10. 10.11770078864999883331 URL https: ///oi. www.orrg11... 1111117777777999911
Marco Hutter, Christian Gehring, Dominic Jud. Andreas Lauber, C. Dario Bellicoso, Vassilios Tsoumis, Jemin Hwangbo, Karen Bodie, Peter Fankhauser, Michael Bloesch, Remo Diethelm, Samuel Bachmann, Amir Melzer, and Mark Hoepflinger. Anymal a highly mobile and dynamic quadrupedal robot. In 2016 IEEE/RSI International Conference on Intelligent Robots and Systems (IROS), pages 38 44, 2016. doi: 10.1 109//ROS.201..777500
Mazeyu Ji, Xuanbin Peng, Fangchen Liu, Jialong Li, Ge Yang, Xuxin Cheng, and Xiaolong Wang. Exbody2: Advanced expressive humanoid whole-body control. arXiv preprint arXiv:2412.13193 2024.
Shirin Joshi, Sulabh Kumra, and Ferat Sahin. Robotic grasping using deep reinforcement learning. In 2020 IEEE 16th International Conference on Automation Science and Engineering (CASE), pages 1461 1461-1466. IEEE, 2020.
Shuuji Kajita, Fumio Kanehiro, Kenji Kaneko, Kazuhito Yokoi, and Hirohisa Hirukawa. The 3d lin- ear inverted pendulum mode: A simple modeling for a biped walking pattern generation. In Pro- ceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No. 01 01CH37180), volume 1, pages 239-246. IEEE, 2001.
Heecheol Kim, Yoshiyuki Ohmura, and Yasuo Kuniyoshi. Transformer-based deep imitation learn- ing for dual-arm robot manipulation. In 202/ JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 9975- IEEE, 2021.
F Kozin and HG Natke. System identification techniques. Structural safety, 3(3-4):299-316, 1986.
Ian Lenz, Honglak Lee, and Ashutosh Saxena. Deep learning for detecting robotic grasps. The International Journal of Robotics Research, 34(4-5):705-725 2015.
Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding, 2020. URL petpss://arxtt wwt.eiiww.i.rrrooooooooooo abs/2b0sss22 166 68.
Jialong Li, Xuxin Cheng, Tianshu Huang, Shiqi Yang, Rizhao Qiu, and Xiaolong Wang. Amo: Adaptive motion optimization for hyper-dexterous humanoid whole-body control. Robotics: Sci- ence and Systems 2025, 2025.
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), 34(6):248.12248116 October 2015.
Zhengyi Luo, Jinkun Cao, Alexander W. Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. In International Conference on Computer Vision (ICCV), 2023.
Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. AMASS: Archive of motion capture as surface shapes. In International Conference on Computer Vision, pages 5442-5451, October 2019.
12

---

Andrew T Miller and Peter K Allen. Graspit! a versatile simulator for robotic grasping. 1EEE Robotics & Automation Magazine. 11(4):110-122, 2004.
Federico Moro and Luis Sentis. Whole-body control of humanoid robots. Humanoid robotics: reference, pages 1161-1183, 2019.
Guilherme AS Pereira, Bruno S Pimentel, Luiz Chaimowicz, and Mário FM Campos. Coordination of multiple mobile robots in an object carrying task using implicit communication, Proceed- ings 2002 IEEE International Conference Robotics and Automation (Cat. No. 02CH37292), volume 1, pages 281-286. IEEE, 2002.
Mathis Petrovich, Michael J. Black, and Gil Varol. TEMOS: Generating diverse human motions from textual descriptions. In European Conference on Computer Vision (ECCV), 2022.
Mathis Petrovich, Michael J. Black, and Gül Varol. TMR: Text-to-motion retrieval using contrastive 3D human motion synthesis. In International Conference on Computer Vision (ICCV), 2023.
Ilija Radosavovic, Jathushan Rajasegaran, Baifeng Shi, Bike Zhang, Sarthak Kamat, Koushil Sreenath, Trevor Darrell, and Jitendra Malik. Humanoid locomotion as next token prediction. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https: wpwprevieeeeeeeeeeeeeee/e..oeewm .et/ /orr/foonnn forum? id-miicc0GGG.....
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and struc- tured prediction no-regret online learning. In Proceedings of the fourteenth international con- ference on artificial intelligence and statistics, pages 627-635. JMLR Workshop and Conference Proceedings, 2011.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06377, 2017.
Lin Shao, Fabio Ferreira, Mikael Jorda, Varun Nambiar, Jianlan Luo, Eugen Solowjow, Juan Apari- cio Ojea, Oussama Khatib, and Jeannette Bohg. Unigrasp: Learning a unified model to grasp with multifingered robotic hands. IEEE Robotics and Automation Letters, 5(2):2286-22) 22933, 2020.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gan gated mixture-of-expprts layer. arXiv preprini (001::/700.05533 2017.
Mohit Shridhar, Lucas Manuelli, and Dieter Fox. Perceiver-actor: A multi-task transformer for robotic manipulation. In Conference on Robot Learning, pages 785-799, PMLR, 2023.
Robert I.. Thorndike Who belongs in the family? Psychometrika, 18(4))267-276, 1953. doi: 10.1007/BF02288966.
Yuxiang Yang, Guanya Shi, Changyi Lin, Xiangyun Meng. Rosario Scalise, Mateo Guaman Castro, Wenhao Yu, Tingnan Zhang, Ding Zhao, Jie Tan, al. Agile continuous jumpiin in discontinuous terrains. arXiv preprint arXiv:2409.1099 10923, 2024.
Wenhao Yu, C Karen Liu, and Greg Turk. Policy transfer with strategy optimization. arXiv preprint veXiv:1810.05555 2018
Wenhao Yu, Jie Tan, Yunfei Bai, Erwin Coumans, and Sehoon Ha. Learning fast adaptation with meta strategy optimization. IEEE Robotics and Automation Letters, 5(2):2950-24) 2957, 2020.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The paper accurately states the main contributions and research scope in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discuss our limitations the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specificatton, asymptotic approximations only holding locally). The au- thors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the ap- proach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to ad- dress problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't it acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
We don't include any theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theo- rems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be comple- mented by formal proof's provided in appendix or supplemental material. Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclu- sions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yss]
Justification:
We provide the information in both appendix and experiment section.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary either make it possible for others replicate the model with the same dataset, or provide access to the model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all sub- missions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or a way to re- produce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case au- thors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [No]
We commit to open-sourcing all code and data upon the acceptance of our paper.
Justification:
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https: htt//niss.. ca/
public/ guides Code as ss celooooollcy for more details.
While encourage the release of code and data, understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https:
//n i pss. cc/ pubb c/ c/gu i des CodeSubrn ssi onPol icy for more details. The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We deseribe our training details in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropri- ate information about the statistical significance of the experiments?
Answer:
[Yes]
Justification:
Yes, we report them in our experiment section.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (c.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state The authors should prefer- ably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide detailed computer resources in the supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary failed experiments that didn't it make it into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: /ttes:///eeuue /twurisuup.pp///e//eee//e
Answer: [Yes]
Justification: Our work avoids harm, respects privacy, promotes fairness, and adheres to responsible reporting and data handling practices.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there is a special consid- eration due to laws or regulatiOons in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
This paper doesn't discuss anything relative to societal impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact spe- cific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitiga- tion strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have a high risk for misuse or dual-use should be released with necessary safeguards allow for controlled use of the model, for example by re- quiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, all external assets used this work are properly credited, and their licenses and terms of use are explicitly acknowledged and fully respected.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, if possible, include URL.
The name of the license (c.g.. CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use the pack- age should be provided. For popular datasets, paperswil hhode com/ cat set has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documenta- tion provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the pa- per include the full text of instructions given to participants and sereenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve crowdsourcing or research with human partici- pants, so this question is applicable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contri- bution of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, cura- tion, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve human participants, so this question is not appli- cable.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equiva- lent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: This research does not involve LLMs as any important, original, or non- standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: ineuriss wwtpuriss.ccc/ooooeee 5/LLM) for what should should not be described.
20

---

A Environment Details
A.1 RL environment
We provide detailed training and test environment setting in this subsection.
Observation For Privileged observation, we use proprioception, including linear velocity, angular velocity, joint position, joint velocity, and last action, and task-relevant observation, including target joint positions, target keypoint positions, target root translations, and target root rotations in the global coordinates. For student policies, we use all proprioception observation, except for linear velocity. For task-relevant information, we only preserve target joint positions, root translation, and root rotations in the local coordinates. For teacher policy, we take observations from 5 timesteps as input, and for student policies, we take observations from 10 timesteps as input.
For the delta action policy, we use the full proprioception of the teacher policy mentioned above, as well as the tracking policy actions. Note that we don't 't use the global information like root position and keypoint positions.
Action We use roportional derivative (PD) controller to control the 23 DoF of the GI (totally 29 DoF). And the policy outputs are the target joint position for PD controller.
In addition to falls, we added an additional termination condition during the training and testing process, where the position of the keypoints must not exceed threshold. During training, threshold set from 0.8 down to 0.3 using curriculum learning. During testing, a threshold of 0.8 is used for walking, and 0.4 is used for the other tasks.
Termination
Table Reward design for tracking policy.
Term
Expression
Weight
Penalty
Torque limits
( & [rmins Tmax))
10
DoF position limits
[ds & [sming gmax]] )
DaF velocity limits
1 [d, & limin: [mnx]]
10
Regularization
DaF' acceleration
||
-3 x 10
Action rate
||a, 0--1
-2
Action smoothness
||á, d
-2
Torque
1 pay r|ll
-0.0001
Stumble
-0.00125
Feet orientation
Σ lgravity xy!!
-2.
Task reward
Body position
exp ( (-4
Pl Pret
1.0
Root rotation
exp (-4
Aoot
Ovex |]
0.5
Root angular velocity
exp (-4 Wbody whet
0.5
Root velocity
exp 4 ||
Ueat 1)
DoF position
exp l|
doel
0.5
DoF velocity
exp
l|d drell)
0.5
A.2 Deployment
The policy runs at an inference frequency of 50 Hz. The low-level interface operates at 200 Hz, ensuring smooth real-time control. Communication between the control policy and the low-level interface is facilitated via Lightweight Communications and Marshalling (LCM).
21

---

A.3 Metrics
Success Rate (SR). SR measures whether policy successfully completes a rollout in the test en- vironment. A rollout is considered successful if the agent does not fall and is able to follow the reference motion with reasonable aceuracy. Merely maintaining balance without tracking the refer- ence trajectory is treated as failure, since such behavior deviates substantially from the intended motion.
Mean Per Joint Position Error (MPJPE).
MPJPE quantifies the average discrepancy between
the predicted and target positions of all joints:
N Σ
MPJPE
J.
J
where N is the number of joints, J. E IR$ denotes the target position of the i-th joint, and J; E R corresponding predicted position. The unit measurement radians.
Mean Per Keypoint Position Error (MPKPE). MPKPE evaluates the average error between predicted and target positions body keypoints:
МPКРЕ
K. K,
K
where K is the number of keypoints, K, R& represents the target position of the i-th keypoint, and K; E IR3 the predicted position. The unit of measurement is millimeters.
B
Training Details
B.1 Reward Design
We have listed the rewards used for training the WBC policy and the Delta Action model separately in Table 7 and Table 8, respectively. It is worth noting that when training the Delta Action model, compared to the rewards in ASAP, we used the translation of the root position rather than the posi- tions of all body joints. This is because did not motion capture system, but instead relied on odometry.
Table 8: Reward design for delta action model. Expression
Term
Weight
Penalty
Torque limits
1(r, & [imin: Tmax)]
DoF position limits
[d, & [aminr Amax)
10
DoF velocity limits
1 1(d, & [mmin:.max.]] 1(termination]
10
Termination
200.0
Regularization
DoF acceleration
-3 x 10-8
Action rate
||a. a-- 111
-2
Action norm
láella
-2
Torque
||
0.0001
Task reward
Root position
exp
Proce
Pret 2)
1.0
Root rotation
exp
4 Iror
qoet 1|
0.5
Root angular velocity
exp
Whoot
Whet
Root velocity
Preet User
0.5
position
exp
4 l|d drerl 2)
DoF velocity
exp
4 l|d drwell
0.5
22

---

B.2 Domain Randomization
Detailed domain randomization setups are summarized in Table
Table 9: The detailed domain randomization implementation. Our types of domain randomization include mechanical properties, external disturbances, and terrain.
Term
Value
Dynamics Randomization
Friction
24(0.5 1.25)
CoM offset
0.11
Link
(.2) default kg
Gain
((0.75, 1.25) default
D Gain
(00.55 1.25) default
Control delay
44220, 40)
External Perturbation interval = 10s, May 0.5m/s
Push robot
Randomized Terrain
Terruin type
flut, rough
B.3 RL Hyperparameters
The RL training progress is aligned with standard PPO [Schulman et al., 2017]. provide the detailed training hyperparameters in Table 10. We also list the hyperparameters used during the distillation process Table 11.
Table 10: Hyperparameters for PPO
Hyperparameter
Value
Optimizer
Adam
f1.8g
0.9. 0.999
Leannnng Rate
x 10-4
Batch Size
4096
Discount factor (?)
0.99
Clip Param
0.2
Entropy Coef
0.001
Max Gradient Norm
1
Learning Epochs
5
Mini Batches
4
Value Loss Coef
Table 11: Hyperparameters for DAgger
Hyperparameter
Value
Optimizer
Adam
Learning Rate
1 x 10
Batch Size
4096
Max Gradient Norm
Learning Epochs
2
Mini Batches
4
B.4
Delta Action
For each cluster in each iteration, we randomly sample 20 deployable motions and perform 8 rollouts in the real world. Similar to ASAP, we only train the DoF of the ankles. The average duration per motion is approximately seconds.
B.5 Training Resource
We used two desktop computers for training. Each was equipped with an Intel 19-13900 CPU, an NVIDIA RTX 4090 GPU, and 64 GB of RAM for policy training.
23

---

C
Model Details
MLP We use 3-layer MLP model with hidden layer sizes of 1024, 1024, and 512, respectively. The activation function used is ELU.
Transformer Our general WBC controller is built upon the Gated Transformer-XI. architecture, adapted from the open-source implementation at http://gghhu..oommmvvooo/d//////
The model integrates attention mechanisms with GRU-based gating enhance memory retention and capture long-range temporal dependencies, making it particularly effective for sequential con- trol tasks in reinforcement learning. The controller takes as input sequence of 10 consecutive observations and processes them through one Transformer block. Each block employs six attention heads and has hidden size of 128 and embedding dimension of 204. The memory length is maintained at 10 to preserve temporal context across sequences.
D Additional Results
Expert Comparison As shown in Figure 6, visualized the comparison between generalists and specialists across six types of clusters. The same trend can still be observed in the remaining two clusters (Sland Mid and Walk Fast). In both of these two clusters, the policy of the specialists outperforms that of the generalist. However, the final generalist still retains favorable properties and significantly outperforms the initial generalist.
walk Dast
stand mid
walk Dast
stbad mid
Fast
well
stund
up
Welli
sland up
walk slow
stand
up
60.0
stand
jump
Rland Bow
stand low
(a) Jump
(b) Stand Low
(c) Stand Mid
walk faet
stand mid
walk East
stand mid
walk
stand mid
walk
standd
ap
Molg
stand
wlme
stand
Jump.
stand Bow
jumap
stand low
jump
stand low
(d) Stand Up
(c) Walk Slow
(1) Walk Fast
General Init
Expert Init
General Final
Expert Final
Figure 6: Complete Iterative Comparison.
Clustering We clustered total of the following number of data samples for each category: Jump 351, Stand Low 229, Walk Slow 3355, Stand Mid - 578, Stand Up-2378, and Walk Fast 307. It is evident that the data distribution across the entire AMASS dataset is imbalanced. To balance the distribution for the general strategy, we ensure that each category is equally represented with ratio of 1/6 during the distillation process.
Statistical Tests better support our experimental results, we have supplemented Table with the complete confidence intervals (CI) test of the different methods.

---

Table 12: Main results with confidence interval (CI) statisties reported over samples single reference trajectory.
IsaacGym
MaJoCo
Method
SRI
MPKPEL
MPJPEL
SR1
MPKPE]
MPJPL
Ommil120 [[lc et al, 2024a]
35.65% (1.1145))
87.83 (0.6389)
0,2630 (0 (00009)
15.64% (1.40%))
(6..9666.619))
0.4601 (0.0010)
Exbody2 [Ji et 2024]
86.63% (1.10616)
86,66 (0.6013)
0.2937
(1.517%)
272.42(7.029)
0.3576 (0.0011)
Hover [Ho al, 2024c]
63.21%% (1.45159)
105 84 (0.93500)
0.2792 (0.0019)
16.12% (1.4915)
323.08 (6.919)
0.3428 (0.0010)
Rumbleeece
66 (1.2625))
0.2356
25


Reviews:
[{"review_id": "y35zYI5F18", "reviewer": "Reviewer_4Wt3", "summary": "The paper proposes BUMBLEBEE (BB), a method for learning real-robot motion tracking tasks. The method analyzes human motion capture data, autoencoding motion data and textual descriptions in order to cluster motions into 6 clusters.\n\nBB then trains a general motion tracking policy, then for each motion cluster, it finetunes an “expert” for specifically those motions, starting from the general policy. It also trains action delta policies to reduce the sim-to-real gap, using data from rolling out simulated policies on real robots.\n\nFinally, BB distills the experts into a single policy, which is shown to perform better than prior methods on motion tracking metrics. The final real-robot policy successfully performs highly dynamic actions.", "questions": "1) Based on SR as show in Table 3, “General Init” already outperforms all baselines on IsaacGym, and all baselines except Exbody2 on MuJoCo. How much of the benefit of BB comes simply from improved NN architectures/other hyperparameter choices, vs. the clustering, expert, and distillation steps emphasized in the paper? It would be good to include the full set of metrics for “General Init” in Table 1. If most of the benefit is coming from differences other than the clustering approach, these differences should be described in more detail.\n\n2) While the paper includes ablations showing that the use of multiple experts improves performance over the generalist policy (e.g. Table 3), it’s still not clear to me why this is the case. Do the generalist and 6 experts all use the same model architecture described in Appendix C? If so, maybe the greater total capacity of the experts allows better performance, which can then be distilled. What if, instead of 6 separate experts, you trained a single “General Init” using a model with 6x the total parameter count, then distilled this down to a single model of the original size? This would eliminate the need for the clustering models and significantly simplify the full method; unclear based on the paper whether we should expect this to work?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n1) The real-robot videos are very impressive! The robot stays balanced even during very rapid motions.\n2) BB is shown to improve motion tracking performance compared to three other recent methods: OmniH2O, Exbody2, and Hover.\n3) Detailed analysis of the method: each component is explained and ablated.\n\n\n**Weaknesses:**\n1) The full method is highly complex. While the paper’s ablations show that each component adds at least some performance compared to its absence, the importance of certain design decisions is still unclear; see questions.\n2) Results lack CIs and statistical tests for group differences (Tables 1-6).\n3) Some unclear environment descriptions: what is “Success Rate”? The paper says “Among all metrics, SR is the most critical, as it reflects the overall viability and stability of the policy” and “Detailed definitions of these metrics can be found in Appendix A.”, but I’m not able to find these definitions in the provided appendix A.", "was_revised": true, "final_justification": "The authors added statistical testing, addressing my concern.\n\nAs discussed with the authors, insofar as the IsaacGym results are demonstrating anything, they're primarily demonstrating the effectiveness of the authors' hyperparameter tuning compared to prior work, rather than the performance of the BB algorithm per se. I believe the authors should make this clearer in the writeup. Including General Init in Table 1 as discussed will help here.", "rebuttal": "## To reviewer 4Wt3\n\nWe sincerely thank Reviewer 4Wt3 for the insightful comments. We mainly clarify the points that may have caused confusion and address the weaknesses.\n\n### Clarification for design decisions\n\n> How much of the benefit of BB comes simply from improved NN architectures/other hyperparameter choices, vs. the clustering, expert, and distillation steps emphasized in the paper? It would be good to include the full set of metrics for “General Init” in Table 1. If most of the benefit is coming from differences other than the clustering approach, these differences should be described in more detail.\n\nThank you for the thoughtful question. To ensure fairness, we kept the MLP architecture consistent across all models, including ablations and baselines. The only exception is **BUMBLEBEE**, where we employed a Transformer-based architecture **specifically for the purpose of distilling multiple experts more effectively**. And thank you for your remind, we will include the \"General Init\" into Table 1. We also conduct an additional experiment that use the same MLP as backbone of BB and get the success rate result of 89.56% in IsaacGym and 61.04% in MuJoCo. The MuJoCo result is slightly lower than BB, but it is still significantly higher than the baseline, which proves that while the architecture does have an impact, the main influence comes from the expert and clustering strategies.\n\n\n### Statistical tests\n\nWe observe that the variance of policies trained in IsaacGym is very small, consistent with prior works like ASAP and HOVER. This is likely due to the large batch size (envs=4096) used during training, which minimizes randomness across different seeds, resulting in more stable performance. Importantly, the small variance does not cause any overlap and does not affect the results. To keep the main text concise, we have not included full statistical tables, but the complete statistical results are provided in the table below.\n\n|         |     SR     |    MPKPE    |    MPJPE    |     SR     |   MPKPE   |    MPJPE    |\n| ------- | :--------: | :---------: | :---------: | :--------: | :-------: | :---------: |\n| OmniH2O | $\\pm$0.14% |  $\\pm$0.10  | $\\pm$0.0007 | $\\pm$0.16% | $\\pm$0.80 | $\\pm$0.0016 |\n| ExBody2 | $\\pm$0.10% |  $\\pm$0.15  | $\\pm$0.0001 | $\\pm$0.41% | $\\pm$0.42 | $\\pm$0.0025 |\n| Hover   | $\\pm$0.27% |  $\\pm$0.10  | $\\pm$0.0060 | $\\pm$0.71% | $\\pm$1.11 | $\\pm$0.0089 |\n| BB      | $\\pm$0.16% | $\\pm$0.2252 | $\\pm$0.0005 | $\\pm$0.17% | $\\pm$0.58 | $\\pm$0.0011 |\n\nAnd we will add it into the final version.\n\n### Parameter count\n\n> While the paper includes ablations showing that the use of multiple experts improves performance over the generalist policy (e.g. Table 3), it’s still not clear to me why this is the case. Do the generalist and 6 experts all use the same model architecture described in Appendix C? If so, maybe the greater total capacity of the experts allows better performance, which can then be distilled. What if, instead of 6 separate experts, you trained a single “General Init” using a model with 6x the total parameter count, then distilled this down to a single model of the original size? This would eliminate the need for the clustering models and significantly simplify the full method; unclear based on the paper whether we should expect this to work?\n\nAs requested, we conducted an additional experiment by increasing the parameter size by 6× while keeping all other settings identical to the \"General init\" configuration. The success rates were 85.17% in IsaacGym and 37.36% in MuJoCo, showing minimal performance improvement over the original model. This further confirms that simply increasing the model size has limited impact on performance.\n\n### Detailed description of the metric\n\n> Some unclear environment descriptions: what is “Success Rate”? The paper says “Among all metrics, SR is the most critical, as it reflects the overall viability and stability of the policy” and “Detailed definitions of these metrics can be found in Appendix A.”, but I’m not able to find these definitions in the provided appendix A.\n\nThank you for the feedback. In our setting, \"Success Rate\" (SR), shown in line 813 of appendix, refers to the policy successfully completing the task without failure. More specifically, a trial is considered successful only if the agent maintains balance (i.e., does not fall) and closely follows the reference motion. A policy that remains stable but deviates significantly from the target motion would still be considered a failure. We will include SR and the two additional metrics in Appendix A.\n\nThe **Mean Per Joint Position Error (MPJPE)**:\n\n$$\n\\text{MPJPE} = \\frac{1}{N} \\sum_{i=1}^{N} \\left\\| \\hat{\\mathbf{J}}_i - \\mathbf{J}_i \\right\\|_2\n$$\n\n- $N$: Number of joints  \n- $\\hat{\\mathbf{J}}_i \\in \\mathbb{R}$: Target joint position of the $i$-th joint  \n- $\\mathbf{J}_i \\in \\mathbb{R}$: Joint position of the $i$-th joint.\n- Unit: rad \n\nThe **Mean Per Keypoint Position Error (MPKPE)** :\n\n$$\n\\text{MPKPE} = \\frac{1}{N} \\sum_{i=1}^{N} \\left\\| \\hat{\\mathbf{K}}_i - \\mathbf{K}_i \\right\\|_2\n$$\n\n- $N$: Number of keypoints  \n- $ \\hat{\\mathbf{K}}_i \\in \\mathbb{R}^3 $: Target position of the $i$-th keypoint  \n- $\\mathbf{K}_i \\in \\mathbb{R}^3 $: Position of the $i$-th keypoint  \n- Unit: mm"}, {"review_id": "qpOXAo1mGY", "reviewer": "Reviewer_sTVB", "summary": "This paper studies the problem of learning generalist whole-body control policies for humanoid robots. Recent work has demonstrated that whole-body control policies trained to imitate reference motions (re-targeted to a robot embodiment) in simulation can be transferred to real humanoid robots, either as-is (zero-shot sim2real) or with a small amount of finetuning/adaptation to real world data (residual action/dynamics modeling). While several works have considered generalist (multi-clip) whole-body control / tracking for simulated human embodiments (e.g. MoCapAct), there is comparatively little work on transferring such generalist policies to real humanoid robots. This work considers exactly that problem, and proposes an expert-to-generalist learning framework -- dubbed Bumblebee -- that successfully learns multi-clip whole-body control policies that transfer to the real world. The general idea is to (1) cluster all reference motions by semantic and behavioral similarity using an autoencoder, and (2) train a generalist tracking policy on the full set of motions which can then (3) be finetuned on each motion cluster. Then, (4) each cluster expert is deployed in the real world and (5) a per-cluster delta control policy is obtained with real data. Finally, (6) the cluster expert and sim2real delta policies are then distilled into a single Transformer-based generalist policy via DAgger. Experiments indicate that Bumblebee successfully learns a multi-clip generalist policy that can perform a variety of motions on a real humanoid robot, while also performing comparably or better in simulated motion tracking tasks than recent related works.", "questions": "I would like the authors to please address my comments listed under \"weaknesses\" in the previous section. I believe that they are all rather concrete and actionable.", "limitations": "As per my previous comment:\n> The discussion of limitations is rather minimal and merely points out that additional sensors are likely to improve results. I strongly suggest that the authors revise their limitation section to discuss a few key limitations pertaining to the proposed method itself, e.g. the way that clustering is currently performed, or the scalability of the generalist-to-expert-to-generalist training pipeline.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "This paper was a pleasant read overall. While my preliminary assessment leans positive, I identify several aspects of the paper that can be improved. I summarize my thoughts as follows:\n\n**Strengths**\n- The paper is generally well organized and easy to follow. The problem of generalist whole-body humanoid control on real hardware is an interesting and timely problem, and the paper is thus likely to be of interest to researchers at the intersection of machine learning and robotics. The paper is self-contained and contains sufficient discussion of recent related work. Figures and illustrations contribute positively to my understanding of the approach and results.\n- The proposed method, while rather complex due to it consisting of multiple stages, is composed of a number of intuitive techniques that are well established in the literature (clustering, generalist-to-expert finetuning, residual policies, DAgger, etc.). The overall approach is thus fairly intuitive to readers familiar with the area; I consider this a strength. I appreciate that the method is described in great detail.\n- The experimental setup appears to be fairly rigorous and includes several standard (quantitative) performance metrics in simulation (success rate, joint/coordinate tracking errors). Although the real-world evaluation is rather limited, I do appreciate the inclusion of qualitative results on real hardware.\n\n**Weaknesses**\n- As alluded to above, it would be rather insightful if the authors could provide some quantitative evaluation metrics for the hardware experiments. I understand that computing the same performance metrics as in simulation may be difficult, but on the other hand it is also difficult to evaluate the effectiveness of the expert refinement + final generalist performance without any metrics or baselines / comparisons.\n- I am somewhat surprised to see such a large gap in performance between methods in MuJoCo when numbers are rather close in IsaacGym. The authors argue that this is because MuJoCo is a more realistic simulator, but I find that to be a rather handwavy explanation for such a dramatic difference in results. Could the authors please elaborate on why there might be such a discrepancy between the two on a more technical level, and e.g. include some videos of each method evaluated in MuJoCo? Do baselines tend to overfit to the IsaacGym simulator? If so, could that possibly be alleviated by improving policy robustness with domain randomization or added control noise?\n- Given that clustering places a key role in this work, I expected to see an ablation on the number of clusters. I understand that this could potentially be quite time consuming if it includes real world policy refinement, so perhaps a simpler ablation could be set up purely in simulation. Even so, this might not be feasible within the strict time frame of a rebuttal. It would be helpful to include some discussion on the (expected) effect of the number of clusters on the overall approach, ideally backed by empirical evidence for e.g. the camera-ready version.\n- Unfortunately, I have not been able to view any of the supplemental videos. I tried two different browsers (Chrome, Firefox) and have also tried downloading the repo but it keeps failing. I give the authors the benefit of the doubt here but it is rather unfortunate nonetheless.\n- I like the general idea behind Figure 4; it neatly summarizes the empirical expert vs. generalist results. However, the different methods are currently difficult to visually distinguish due to the choice of colors. I would recommend refining this figure to better separate the different methods visually.\n- The discussion of limitations is rather minimal and merely points out that additional sensors are likely to improve results. I strongly suggest that the authors revise their limitation section to discuss a few key limitations pertaining to the proposed method itself, e.g. the way that clustering is currently performed, or the scalability of the generalist-to-expert-to-generalist training pipeline.", "was_revised": true, "final_justification": "The majority of my concerns have been addressed by the authors' rebuttal. I am willing to raise my score and recommend acceptance on the condition that the authors fully incorporate all reviewer feedback into their camera-ready version.\n\n**Score change: 4->5**\n\n**Confidence change: 3->4**", "rebuttal": "## To reviewer sTVB\n\nThanks for your detailed reviews.\n\n### Hardware experiments\n\n> - As alluded to above, it would be rather insightful if the authors could provide some quantitative evaluation metrics for the hardware experiments. I understand that computing the same performance metrics as in simulation may be difficult, but on the other hand it is also difficult to evaluate the effectiveness of the expert refinement + final generalist performance without any metrics or baselines / comparisons.\n\nWe agree with that it is important to provide quantitative evaluation metrics for the hardware experiments. \n\nAlso, it is challenging to provide precise quantitative comparisons for real-world deployment, primarily due to the lack of publicly available deployment code for baseline methods, which makes fair sim-to-real comparisons difficult. To address this, we present extensive videos demonstrating the capabilities of our final policy and comparisons with and without delta fine-tuning in each cluster. Additionally, we report results for our final general policy on 100 randomly sampled motions from the AMASS dataset, achieving a success count of 59 and an MPJPE of 0.2896 — slightly lower than its performance in MuJoCo, but still demonstrating strong real-world generalization.\n\n   ## Gap between IsaacGym and MuJoCo\n\n> - I am somewhat surprised to see such a large gap in performance between methods in MuJoCo when numbers are rather close in IsaacGym. The authors argue that this is because MuJoCo is a more realistic simulator, but I find that to be a rather handwavy explanation for such a dramatic difference in results. Could the authors please elaborate on why there might be such a discrepancy between the two on a more technical level, and e.g. include some videos of each method evaluated in MuJoCo? Do baselines tend to overfit to the IsaacGym simulator? If so, could that possibly be alleviated by improving policy robustness with domain randomization or added control noise?\n\nWe appreciate the reviewer’s thoughtful question regarding the performance discrepancy between MuJoCo and IsaacGym. We believe several factors contribute to this difference. \n\n1. **The physics engine**.  The discrepancy in results between MuJoCo and IsaacGym can be partially attributed to differences in their physical simulation fidelity. Specifically, MuJoCo provides more accurate and tunable contact modeling, which are often approximated more coarsely in IsaacGym. Additionally,  options such as `collapse_fixed_joints=True` are often used to reduce model complexity and improve simulation speed. While this can be beneficial for large-scale training, it effectively removes certain physical structures that are present in the original URDF models. Furthermore, joint-level control in IsaacGym may suffer from limited temporal precision, making it challenging to accurately regulate the duration and timing of torque application. In contrast, MuJoCo provides fine-grained control over both the timing and magnitude of applied torques, which is essential for precise motor behaviors. Finally, MuJoCo offers detailed control over joint friction and damping. All above can significantly impact performance in tasks especially **involving locomotion or contact-rich behavior** that account for a significant portion of the AMASS dataset.\n\n2. **Observation space.**   We hypothesize that different observation space designs exhibit varying levels of robustness in domain transfer scenarios. Compared to the local-coordinate-based observation space used in our method and ExBody2, world-frame-based observations—such as absolute keypoints positions—are significantly more fragile. This is because even minor perturbations or mismatches in the environment (e.g., initial pose differences) are directly reflected in the observations, amplifying their impact on policy performance. In our experiments, MuJoCo initializes the robot to a default pose that approximates real-world joint configurations, while IsaacGym resets the robot based on the reference motion. This discrepancy introduces an initial state mismatch between the two simulators. As a result, policies relying on world-frame position, such as absolute keypoints positions, suffer greater performance degradation under domain transfer, as also observed in ExBody2 for contrast. This issue becomes even more critical in real-world deployment, where precise resetting to a reference pose is generally infeasible.\n\n3. **Overfitting in IsaacGym**.  We acknowledge that overfitting during training in IsaacGym is a potential concern. In our experiments, we applied observation noise and domain randomization to the same extent as in ASAP. However, excessively large noise or overly aggressive domain randomization can significantly degrade training performance, and identifying the optimal trade-off remains a well-known challenge across all robot control algorithms. ASAP itself has been widely recognized as an effective approach for sim-to-real transfer. Building on this foundation, we extend the framework by introducing motion category diversity, enabling the learned general tracking policy to achieve better domain transferability.\n\nWe apologize for not being able to include additional figures at the rebuttal stage. However, as shown in Fig. 5, similar issues are observed in the MuJoCo environment as well — the policy often exhibits noticeable foot instability, struggles to accurately follow the reference motions, and in some cases, crashes early due to significant initial state mismatch. These issues may be attributed to the factors discussed above.\n\n### Number of clusters\n\n>- Given that clustering places a key role in this work, I expected to see an ablation on the number of clusters. I understand that this could potentially be quite time consuming if it includes real world policy refinement, so perhaps a simpler ablation could be set up purely in simulation. Even so, this might not be feasible within the strict time frame of a rebuttal. It would be helpful to include some discussion on the (expected) effect of the number of clusters on the overall approach, ideally backed by empirical evidence for e.g. the camera-ready version.\n\nThank you for the suggestion. We agree that exploring different numbers of clusters is valuable, but it also involves substantial additional effort. As illustrated in Fig. 3, we selected a moderately balanced number of clusters that offers a practical trade-off between task granularity and implementation complexity. In essence, clustering is used to simplify the learning problem: with more clusters, each expert is trained on a narrower and easier subtask, which typically improves expert performance. However, increasing the number of clusters also significantly raises the overall workload and complicates the distillation process. Given this trade-off, we opted for a reasonable balance in our current setup. That said, we acknowledge the importance of this parameter and will make an effort to investigate its impact in the final version.\n\n### Supplemental videos\n\nWe apologize for the inconvenience. You may try accessing the alternative link provided in the anonymous link, or download the video directly. This is currently the best method we can offer to share the video anonymously.\n\n### Suggestions for the paper\n\n> I like the general idea behind Figure 4; it neatly summarizes the empirical expert vs. generalist results. However, the different methods are currently difficult to visually distinguish due to the choice of colors. I would recommend refining this figure to better separate the different methods visually.\n\n1. Thank you for pointing this out. The data presented in Fig. 4 is indeed information-dense, and the color scheme may be too similar to distinguish clearly. We will revise the figure in the final version to improve clarity.\n\n> The discussion of limitations is rather minimal and merely points out that additional sensors are likely to improve results. I strongly suggest that the authors revise their limitation section to discuss a few key limitations pertaining to the proposed method itself, e.g. the way that clustering is currently performed, or the scalability of the generalist-to-expert-to-generalist training pipeline.\n\n2. We appreciate your valuable feedback. In the revised version, we add some of the method's limitations  pertaining to the proposed method itself, including the two issues you mentioned, the overall methodological complexity, and the significant amount of real-world iteration that may be required for practical deployment."}, {"review_id": "TV7g5UTlnv", "reviewer": "Reviewer_a9GS", "summary": "The paper describes an approach for the learning of classes of movements for a humanoid robot which is based on a combination of a clustering of the training movements using an autoencoder approach, and the learning of controller models for the movement subclasses, whose outputs are combined using a mixtures of expert approach. The subclass movement policies are learned by reinforcement learning, where robustness and sim-to-real transfer is ensured by an action delta learning approach. The movements are implemented on a real Unitree G1 robot. It is demonstrated convincingly that the new approach outperforms other SOTA methods for different types of walking, standing and jumping movements. Also the efficiency of the movement class-specific delta action learning is demonstrated in comparison with approaches that learn only a single motion policy model.", "questions": "Does it always make sense to take semantic similarity into account? It appears that it might be somewhat dependent on the quality of the semantic labeling and the specificity of semantic labeling. For example, 'turning' might include a whole range of movements that are much more variable than different types of walking (slow fast etc.) Can it be guessed how the method would, extend for such more heterogeneous motion classes ?\n\n\nMinor:  \n- What is the meaning of k(s) in L. 191. Is k an index?\n\n- L. 319: There is the class 'RUN' mentioned that seems not to appear anywhere before. Is this fast walking?", "limitations": "yes in principle; one could say a few words about scaling to larger and more heterogeneous data sets, where I fully understand that implementing all this on real robots wold be a lot of work.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is very well written and organized. The chosen approach is straight forward and makes sense. The superiority of the methods is convincingly shown, and there is a careful analysis also of the reasons why the mixture of experts method is superior to other approaches.\nAlso the presented comparison with other baseline methods using multiple different simulators is convincing. \n\nA thing that might be interesting to explore i the future is how the quality of the method changes with the number of clusters, and \nmaybe even more interesting, how the methods scales for more classes of substantially different movements.", "was_revised": true, "final_justification": "My concerns were adequately answered, and so were most remarks of the other reviewers. I thus maintain my score (accept).", "rebuttal": "We appreciate the reviewer’s feedback and have focused our response on addressing the aspects of the paper that may require improvement.\n\n> A thing that might be interesting to explore i the future is how the quality of the method changes with the number of clusters, and maybe even more interesting, how the methods scales for more classes of substantially different movements.\n>\n> Does it always make sense to take semantic similarity into account? It appears that it might be somewhat dependent on the quality of the semantic labeling and the specificity of semantic labeling. For example, 'turning' might include a whole range of movements that are much more variable than different types of walking (slow fast etc.) Can it be guessed how the method would, extend for such more heterogeneous motion classes ?\n\nAs illustrated in Fig. 3, we selected a moderately balanced number of clusters that offers a practical trade-off between task granularity and implementation complexity. In essence, clustering is used to simplify the learning problem: with more clusters, each expert is trained on a narrower and easier subtask, which typically improves expert performance. However, increasing the number of clusters also significantly raises the overall workload and complicates the distillation process. Given this trade-off, we opted for a reasonable balance in our current setup. We acknowledge the importance of this parameter and will make an effort to investigate its impact in the final version.\n\nWe share the same concern regarding semantic similarity, as language-based labels can suffer from quality issues, ambiguity, and redundancy. For instance, the label \"playing basketball\" may correspond to a range of motions, such as dribbling while moving or standing still to shoot, which are kinematically distinct despite sharing the same semantic tag. To address this, we do not rely directly on semantic labels for clustering. Instead, we align motion and text into a shared latent space and perform clustering based on this representation. This allows for a more meaningful categorization that captures both fine-grained motion differences and underlying semantic relationships, as illustrated in Table 2.\n\n> - What is the meaning of k(s) in L. 191. Is k an index?\n\nHere, *k* refers to the index of the expert corresponding to the motion that the given state belongs to. Thank you for pointing out and we will revise it accordingly.\n\n> - L. 319: There is the class 'RUN' mentioned that seems not to appear anywhere before. Is this fast walking?\n\nWe appreciate your observation. In this case, the label \"run\" was indeed intended to represent a fast walking motion. We will revise it in the final version."}, {"review_id": "T21zmqcStb", "reviewer": "Reviewer_qeFQ", "summary": "This paper proposes BUMBLEBEE (BB), an expert-generalist learning framework that combines motion clusters, trying to overcome challenges that existing frameworks struggle to generalize across highly varied behaviors. Based on the ASAP method, they categorized actions into six major classes and trained an expert policy for each class. Finally, they distilled the six policies into a single general policy using Dagger. The simulation results in the paper outperformed several existing baselines while also successfully completing sim2real experiments.", "questions": "1. The lower body in the merged motion demo appears somewhat unstable. What could be causing this behavior?\n2. Since the method is modified from the prior method ASAP, it should explicitly use ASAP as the experimental baseline for comparative evaluation. Although the paper compares some algorithms for migrating from H1 to G1 robots, I think a direct comparison with methods primarily designed for G1 robots would be more appropriate.\n3. While the proposed method categorizes motions into six classes, how should the system accommodate new action types (e.g., squat) – through additional class expansion or alternative architectural modifications?\n4. I observe that ASAP has a more natural jumping effect in a real-world experiment. Are there some examples of comparisons that BB can accomplish but ASAP can't? I think it's important to evaluate the contribution of the BB method.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed method demonstrates successful sim2real deployment for all six categorized action classes, achieving a merged motion combining Charleston Dance and Boxing demo.\n2. The paper presents clear explanations, and the methodological logic is fundamentally sound.\n\nWeaknesses:\n1. The merged motion demo appears to exhibit lower-body instability in real-world deployment. Without the support of suspension ropes, it remains uncertain whether the system can maintain stable standing.\n2. The method in the paper improves upon previous approaches, but it lacks comparison and discussion with prior methods.\n3. The experiment does not account for the impact of random seeds on the experimental results.", "was_revised": true, "final_justification": "recommended score: 3->4\nThe authors have provided clear responses to the questions I raised, which helped resolve my confusion, so I think an additional point should be awarded.", "rebuttal": "We thank the reviewer for their valuable suggestions. In our response, we primarily address potential misunderstandings, as well as the weaknesses and questions raised in the review.\n\n> Since the method is modified from the prior method ASAP, it should explicitly use ASAP as the experimental baseline for comparative evaluation. Although the paper compares some algorithms for migrating from H1 to G1 robots, I think a direct comparison with methods primarily designed for G1 robots would be more appropriate.\n\nWe first address the fundamental differences between our method and the ASAP approach. ASAP is inherently limited to single-motion settings, as it represents task information solely through the motion phase in the observations. This makes direct comparison with our method impossible. \n\nHowever, to validate the distinction between our method and ASAP, we conduct experiments in which the single delta action model, trained on the entire real dataset, is used to finetune the initial general policy, as described in the paragraph starting at L. 302. As discussed at L. 309, the result shows that this single delta model underperforms in diverse motion settings, supporting our claim that ASAP-style delta modeling does not generalize well across heterogeneous motion classes.\n\n>1. The merged motion demo appears to exhibit lower-body instability in real-world deployment. Without the support of suspension ropes, it remains uncertain whether the system can maintain stable standing.\n>2. The lower body in the merged motion demo appears somewhat unstable. What could be causing this behavior?\n\nThe relatively intense lower-body movements observed in the merged motion are primarily due to the task’s high difficulty and large motion range. To maintain whole-body balance, the robot naturally exhibits more compensatory behaviors, especially in complex motion sequences involving boxing. In contrast, for simpler categories such as stand-up, the policy produces highly stable motions, as demonstrated in the two accompanying videos.\n\nFurthermore, across all videos, the rope attached to the robot remains mostly slack, with the rear rope consistently loose. Even when the cabinet moves and the rope trails behind, the resulting force is predominantly horizontal and does not contribute any meaningful support to the robot’s standing behavior. Given that only a single robot was available for the demonstration, we have to employ a rope as a safety measure to protect the hardware.\n\n> While the proposed method categorizes motions into six classes, how should the system accommodate new action types (e.g., squat) – through additional class expansion or alternative architectural modifications?\n\nAMASS already covers a highly diverse range of human motions, and our current six-category division captures a broad spectrum of kinematic behaviors. Importantly, our clustering is based on kinematic characteristics rather than only semantic labels, meaning that any new motion can, in principle, be assigned to one of the existing clusters according to its movement profile—for example, squatting naturally falls under the “stand-low” category in our setting.\n\nMoreover, our expert-to-generalist framework is inherently modular and extensible. The MoE-style structure allows new motion types to be incorporated simply by training additional expert policies tailored to additional category, which can then be distilled into the unified generalist policy. Since only the final distilled model is used for deployment, expert architectures can even differ in network architecture without affecting integration.\n\n> I observe that ASAP has a more natural jumping effect in a real-world experiment. Are there some examples of comparisons that BB can accomplish but ASAP can't? I think it's important to evaluate the contribution of the BB method.\n\nAs noted in our previous response regarding the use of ASAP as a baseline, the task settings of ASAP and our method are fundamentally different. BumbleBee is explicitly designed for generalizing across a diverse set of motions, whereas ASAP is tailored for single-motion reinforcement learning. Its policy architecture does not support cross-task generalization, which would require training a separate model for each of the 10,000+ motion sequences in AMASS—an approach that is both computationally prohibitive and impractical.\n\nIn contrast, our method is inherently suited for multi-task learning. By clustering motion types based on kinematic characteristics and training expert policies accordingly, BumbleBee can generalize to a wide range of tasks. Moreover, since BumbleBee is trained on over 8,000 motion clips derived from real human data, it demonstrates robust generalization capabilities even without explicit supervision for each individual motion.\n\n> The experiment does not account for the impact of random seeds on the experimental results.\n\nWe observe that the variance of policies trained in IsaacGym is very small, consistent with prior works like ASAP and HOVER. This is likely due to the large batch size (envs=4096) used during training, which minimizes randomness across different seeds, resulting in more stable performance. Importantly, the small variance does not cause any overlap and does not affect the results. To keep the main text concise, we have not included full statistical tables, but the complete statistical results are provided in the table below.\n\n|         |     SR     |    MPKPE    |    MPJPE    |     SR     |   MPKPE   |    MPJPE    |\n| ------- | :--------: | :---------: | :---------: | :--------: | :-------: | :---------: |\n| OmniH2O | $\\pm$0.14% |  $\\pm$0.10  | $\\pm$0.0007 | $\\pm$0.16% | $\\pm$0.80 | $\\pm$0.0016 |\n| ExBody2 | $\\pm$0.10% |  $\\pm$0.15  | $\\pm$0.0001 | $\\pm$0.41% | $\\pm$0.42 | $\\pm$0.0025 |\n| Hover   | $\\pm$0.27% |  $\\pm$0.10  | $\\pm$0.0060 | $\\pm$0.71% | $\\pm$1.11 | $\\pm$0.0089 |\n| BB      | $\\pm$0.16% | $\\pm$0.2252 | $\\pm$0.0005 | $\\pm$0.17% | $\\pm$0.58 | $\\pm$0.0011 |\n\nAnd we will add it into the final version."}]

Author rebuttal:

Meta-review: Reviewers agree that this is a significant step forward in the field. The rebuttal and post-rebuttal discussion clarified initial questions. Congratulations, this submission is accepted.
Decision: Accept (spotlight)