Task Tokens: A Flexible Approach to Adapting Behavior Foundation Models
Authors: Ron Vainshtein
Zohar Rimon
Shie Mannor
Chen Tessler
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning
Hierarchial Reinforcement Learning
Behavior Foundation Models
Humanoid Control
Primary area: reinforcement learning
TL;DR: Task Tokens enable task-specific adaptation of behavior foundation models by learning a reinforcement-trained encoder, enhancing control without compromising generalization.

Abstract: Recent advancements in imitation learning for robotic control have led to transformer-based behavior foundation models (BFMs) that enable multi-modal, human-like control for humanoid agents. These models generate solutions when conditioned on high-level goals or prompts, for example, walking to a coordinate when conditioned on the position of the robot's pelvis. While excelling at zero-shot generation of robust behaviors, BFMs often require meticulous prompt engineering for specific tasks, potentially yielding suboptimal results. In this work, we introduce ``Task Tokens'' - a method to effectively tailor BFMs to specific tasks while preserving their flexibility. Our approach integrates naturally within the transformer architecture of BFMs. Task Tokens trains a task-specific encoder (tokenizer), with the original BFM remaining untouched. Our method reduces trainable parameters per task by up to $\times 125$ and converges up to $\times 6$ faster compared to standard baselines. In addition, by keeping the original BFM unchanged, Task Tokens enables utilizing the pre-existing encoders. This allows incorporating user-defined priors, balancing reward design and prompt engineering.
We demonstrate Task Tokens' efficacy across various tasks, including out-of-distribution scenarios, and show their compatibility with other prompting modalities. Our results suggest that Task Tokens offer a promising approach for adapting BFMs to specific control tasks while retaining their generalization capabilities.

Published as a conference paper at ICLR 2026
TASK TOKENS:
A FLEXIBLE APPROACH TO ADAPTING BEHAVIOR FOUNDATION MODELS
Ron Vainshtein
Zohar Rimon Technion zohar. www.ooommmmms i1
Technion
stonvecaapus..ecchiio.o
Shie Mannor
Chen Tessler
Technion
NVIDIA Research
teie@ee.technion
ctessler@nvidia.cmm
ABSTRACT
Recent advancements in imitation learning for robotic control have led transformer-based behavior foundation models (BFMs) that enable multi-modal, human-likk like control for humanoid agents. These models generate solutions when conditioned on high-level goals or prompts, for example, walking to coordinate when conditioned on the position of the robot's pelvis. While excelling at zero-shot generation of robust behaviors, BFMs often require meticulous prompt engincering for specific tasks, potentially yielding suboptimal results. In this work, we intro- duce "Task Tokens" method to effectively tailor BFMs to specific tasks while preserving their flexibility. Our approach integrates naturally within the transformer architecture of BFMs. Task Tokens trains task-specific encoder (tokenizer), with the original BFM remaining untouched. Our method reduces trainable parameters per task by up 125 and converges up to x6 faster compared to standard base- lines. In addition, by keeping the original BFM unchanged, Task Tokens enables utilizing the pre-existing encoders. This allows incorporating user-defined priors, balancing reward design and prompt engineering. We demonstrate Task Tokens' efficacy across various tasks, including out-of-distribution scenarios, and show their compatibility with other prompting modalities. Our results suggest that Task Tokens offer a promising approach for adapting BFMs to specific control tasks while retaining their generalization capabilities.
Recent advances in imitation learning have facilitated the emergence behavior foundation models (BFMs) designed for humanoid control (Peng et al. 2022 Won et al. 2022 Luo al., 2024a Tessler et al. 2024). These models, generate diverse behaviors when trained on large-scale human demonstra- tion data. In this work, focus specific type of BFM, which call CoallConditiooed Behavior Foundation Models (FC). Methods such as Masked Trajectory Models and MaskedMimic fall into this category (Wu al. 2023 Tessler et al. 2024). These methods use transformer architectures that process sequences of tokenized goals - high-level objectives such as "follow a path" or "reach with your right hand towards the object" are mapped to embedding tokens. These tokens condition the model's behavior generation. Specifically, we focus on MaskedMimic, which has manifested as particularly effective framework, demonstrating robust zero-shot generalization (ability handle new, unseen tasks without additional training) through its tosen-based goal conditioning mechanism.
For real-world usage, BFMs must be flexible enough to solve a variety of tasks, but the same time specialized enough to effectively solve complex tasks. Despite MaskedMimic's 's proficiency in generating diverse motions from high-level goals, significant challenges persist defining precise goal specifications, or prompts, for complex tasks. Typically, environment-specife reward can be designed, but this is prone to potential errors complex, long-horizon tasks. In contrast, GC-BFMs provide a "prompt-engineering" interface, where the user can specify high-level goals, which can result in a more stable motion, but might be less intuitive for some tasks. Consider game character tasked with walking to object and striking it. Even in this simple task, on the one hand, common emerging error of using reward design that the character walks backward to the goal, but on the other, specifying high-level goals for the striking motion to precisely hit the target is hard. This

---

Published as a conference paper at ICLR 2026
creates fundamental gap between the model's ability to generate robust and natural motions and the precise control needed for specialized tasks. A unified, flexible, and scalable paradigm for adapting BFMs for many complex downstream tasks, while retaining the original motion robustness, is needed.
To this end, we propose Task Tokens, a novel approach that integrates goal-based control with reward-driven optimization within GC-BEMs like MaskedMimic. Our method, illustrated in Figure] [ establishes a hybrid control paradigm: users provide high-level behavioral priors via goals (e.g.. "walk toward the object while facing forward"). These goals are encoded using the pre-existing GC-BFM. Concurrently, the system autonomously learns, via reinforcement learning, task-specific embeddings to optimize dense rewards (e.g.. "strike the target with maximum impact"). In this setting, Task Tokens serve to refine and enhance the user-defined goals: they build upon the priors by incorporating reward feedback, allowing the model achieve more precise and effective behaviors than either approach could alone. This integration leverages the inherent tokenization framework of GC-BFMs, enabling seamless combination of user-defined and learned conditioning tokens.
Our training paradigm preserves the pretrained BFM's extensive behavioral knowledge. During training, the system generates behaviors from the BFM, conditioned on both user-defined goals and the emergent Task Tokens, optimizing the encoder to produce tokens that align behaviors with task-specific rewards. This strategy ensures that the resulting motions remain consistent with the motion manifold defined by the frozen BFM, ensuring robustness and
capabilities. Inspired by parameter-effin -effcceete adaptation techniques in NL.P (Hu et al. 2021 Houlsby et al. 2019) Li and Liang) 2021) our method modifies the model's s behavior through lightweight, trainable module that leverages gradients from the frozen BFM. This allows the Task Token encoder to guide behavior without fine-tuning the full model, making it scalable for solving many downstream tasks.
Our experimental evaluation demonstrates that Task Tokens effectively balance MaskedMimic's ability to generate robust, human-likk motions with the precision required for task-specific control. This hybrid framework achieves rapid convergence and high success rates, surpassing traditional hier- archical reinforcement learning methods sample efficiency and requiring fewer learned parameters. Moreover, by adhering to the BFM's 's underlying motion manifold, Task Tokens conserve multi- prompting capabilities and exhibit stronger generalization across diverse environmental conditions, including variations in friction and gravity. These results demonstrate the potential of our approach to unify goal-based and reward-driven control, enhancing behavior optimization for complex tasks.
CONTRIBUTIONS
Task-Specific Adaptation: We propose Task Tokens, novel and parameter-efffcieet approach to adapt MaskedMimic, a Goal-Conditioned Behavior Foundation Model (GC- BFM), to specific tasks via tokenized control, without fine-tuning the foundation model, preserving its zero shot capabilities.
Scalability: Our approach is parameter-efficient requiring up to x125 less parameters and converges up to x6 faster than alternative methods.
Our method enables a seamless combination of user defineefined high-level priors (e.g., textual or joint-based goals) with learned reward-driven optimization.
Hybrid Control Paradigm:
Performance and Generalization: Task Tokens match the high task performance of full fine-tuning, while surpassing other methods terms of robustness to changes environment dynamics, such as gravity and friction.
RELATED WORK
Humanoid control is a challenging domain spanning both robotics and computer animation, with the shared goal of generating realistic and robust behaviors. In the animation community, physics simulation is used to ensure the generated motions are realistic and enable the characters to react to dynamic changes in the environment. To achieve this, they typically leverage imitation learning methods combined with motion capture data to learn and generate human- like behaviors in new and unseen scenarios (Peng et al. 2018 2021 Luo et al. 2023 Tessler et al.) 2023 Gao al. 2025). Similar approaches are observed in the robotics community, with the addition of sim-to-real adaptation used to ensure the controller can overcome the imperfect modeling of the
Humanoid Control:
world by the simulator (Viceconte et al. 2022
Lu al. 2024 Ji al. 2024).
2

---

Published as a conference paper at ICLR 2026
Our work builds on these foundations by preserving their flexibility and robustness while enabling precise task-specific control.
Behavior Foundation Models: Recent advances in reinforcement learning have led to the devel- opment of Behavior Foundation Models (BFMs) that can generate diverse behaviors for embodied agents. PSM (Agarwal et al. 2024) and FB representations (Touati and Ollivier (202T) provide framework for learning policies conditioned on a target stationary distribution. These models perform remarkably well when the requested behavior can be represented by a stationary distribution (for example, using a reward-weighted combination of data samples). However, covering the entire space of solutions in high dimensional control tasks remains challenge for these models. Methods such as Adversarial Skill Embeddings (Peng et al. 2022] ASE) and PULSE (Luo et al., 2024a) overcome this limitation by constraining the policy to reproducing human demonstrations. First, they compress a large repertoire of human reference motions into latent generative policy. Then using reinforcement learning they train a hierarchical controller (Sutton et al. 1999) to pick the latent (motion to perform) at each step and solve new and unseen tasks.
In this work, we focus on Goal Conditioned Behavior Foundation Models (Chen et al. 2021 Zitkovich et al.) (203) Tessler et al. 2024). In contrast to the aforementioned methods, GC-BFM BFM can solve new and unseen tasks without specific training by directly mapping from goals to actions. Their be seen form of inpainting, where the model attempts reproduce the most likely outcome given the training data for any provided objective. However, this strength is also a limitation, these models struggle when presented with out-of-distribution constraints, such as those defined manually by a user or task. Our Task Tokens approach addresses this limitation by providing mechanism to incorporate task speciifi optimization while preserving the model's ss ability to generate natural, human-like behaviors.
mode of operation
2 PRELIMINARIES
Our proposed method leverages MaskedMimic to effectively solve specific distribution of humanoid tasks by learning a "task encoder" with reinforcement learning.
2.1 REINFORCEMENT LEARNING
A Markov Decision Process (Puterman 2014 MDP) models sequential decision making as tuple M (S, A, P. R. 7). At each time step the agent observes state st S and predicts an action a, E A. As result, the environment transitions to new state Se+1 based on the transition kernel P and the agent provided a reward r, ~ R(s, az). The objective is to learn a policy æ S - that maximizes the expected discounted cumulative reward x² argmax rEnl Ex E. rrr.).
2.2 MASKEDMIMIC
MaskedMimic presents a unified framework for humanoid control, extending goal-conditioned rein- forcement learning through imitation learning. Goal-Conditioned Reinforcement Learning (GCRL) involves augmenting the state space with a goal g. allowing a policy T(s|s, .) to map states and desired goals to appropriate actions, effectively enabling a single policy to solve multiple tasks. Unlike traditional GCRL approaches that learn from reward signals, MaskedMimic learns directly from demonstration data through online distillation (Ross et al. [201T] DAgger). By combining trans- former architecture with random masking on future goals represented as input tokens, MaskedMimic learns to reproduce human-like behaviors from various modalities, such as future joint positions, textual commands, and objects for interaction. When trained vast amounts of human motion capture data, this goal-conditioned approach allows MaskedMimic to generalize to new objectives without additional training, all while preserving the similarity to the training data. This combination of architecture and control scheme makes it an ideal foundation for our Task Tokens method, which further enhances its capabilities by learning task task-specific tokens optimize for downstream tasks.
3

---

Published as a conference paper at ICLR 2026
Task'
Behavioral Priors
Irput
Jonn Condition
Prior Token
Environment
'A kicking"
GC-BFM (MaskedMiimic))
State Tokan
a
Task Encoder'
g
Figure 1: Task Tokens: Our approach combines three input sources: (1) Prior Tokens: optional tokens enabling user-defined behavioral priors from prompts or joint conditions, (2) Task Token: generated by our learned Task Encoder that processes the current goal observation g). and (3) State Token: representing the current environment state si. The prior and state tokens generated using the pre-trained encoders from the GC-BFM model. The frozen GC-BFM integrates these inputs to produce natural, task-optimized actions ai. During training. the policy gradient objective is computed with respect to the BFM's actions, with gradients flowing through the frozen GC-BFM to the Task Encoder, enabling task-specific optimization without modifying the foundation model's ' parameters.
METHOD
BFMs excel at producing a wide range of motions, but optimizing them for specific tasks presents significant challenges. Downstream applications often require specialized behaviors that fall outside the common distribution of motions. Traditional approaches to achieve such behaviors involve either time-consuming "prompt engineering" (Tessler al. (024) fine-tuning procedures that risk compromising the rich prior knowledge encoded in the BFM.
3.1
TASK TOKENS
The transformer-base architecture of MaskedMimic provides a natural mechanism for integrating new task-specific information. By design, transformers process sequences of tokens, allowing for flexible input composition. Task Tokens can be applicable to any transformer-basee BFM capable of attending over arbitrary token sequences. The only requirement is that the model can integrate additional tokens into its input, allowing the Task Encoder to optimize task-specific signals while the BFM itself remains frozen. This enables to seamlessly incorporate additional tokens without modifying the core network structure.
Our method, Task Tokens (Figure Dp. leverages the tokenized nature of the BFMs' objectives. We propose to train a dedicated task encoder to produce specialized token representations for each new task. This task token encapsulates the unique requirements and constraints of the target behavior, providing concise yet informative signal that can guide the foundation model toward task-specific outputs while preserving its general behavioral priors.
3.2 TASK ENCODER
The Task Encoder processes observations that define the current task goal g represented in the agent's egocentric reference frame and predicts a Task Token 7/ 52122 These observations vary by task - for instance, in steering task, they include the target direction of movement E IR², facing direction E IR? and desired speed E R, resulting in gi E R As MaskedMimic is trained to reach future pose goals, the task encoder is also provided with proprioceptive information. This aligns the encoder with the pre-trained representations, ensuring it can provide meaningful target objectives (see ablation studies in Section
4

---

Published as a conference paper at ICLR 2026
We implement the task encoder feed-forward neural network. Its output- the Task Token is concatenated with tokens from other encoders in the BFM's input space. This creates a token "sentence" where the task encoder's outputs represent specialized "words" that guide the model towards achieving the specific task while maintaining natural motion. We provide additional details on the architectural structure of the encoder in Section
3.3 TRAINING
To optimize the Task Encoder for new downstream tasks, we use Proximal Policy Optimization (Schulman al. 2017 PPO). During training, the BFM predicts action probabilities based on the combined input tokens (including the learned task token). We compute the PPO objective with respect to the task specific reward and the BFM's action probabilities. This approach ensures the BFM provides meaningful gradients for updating the task encoder parameters, while the BFM itself remains frozen. This design choice is fundamental - while fine-tuning the entire model might yield higher task-specific returns, it would compromise the BFM's prior knowledge, resulting in less natural and robust motions.
Leveraging MaskedMimic's token- based architecture, Task Tokens require only -200K parameters per task- compared to -20M for conventional methods-makinn them a highly parameter-efficient solution.
4 RESULTS
We evaluate the effectiveness of our Task Tokens approach through comprehensive set of experi- ments. We examine four critical aspects of our method to validate its performance and applicabil- ity. First, we assess the capability of Task Tokens to effectively adapt MaskedMimic for various downstream applications, demonstrating significant improvements in task-specific performance and efficiency (Section 4.1) over the original zero-shot method. Second, analyze whether the resulting controller preserves the robustness characteristics inherent to the original Behavioral Frequency Modulation (BFM) framework, confirming that stability under variable conditions remains consistent (Section 4.2) Third, we investigate the natural and human-like quality of the generated motions through human study (Section 4.3) Finally, we explore the synergy of Task Tokens and other prompting modalities, combining effects that further demonstrate the versatility of our method (Section[4.4 These experiments collectively demonstrate that our approach successfully balances lask-specific adaptation with the preservation of desirable properties from the foundation model, while requiring significantly less parameters than other baselines.
We provide accompanying video visualizations for all experiments in and the code reproduce all results can
sites google ee be found in the supplementary material.
Tasks: We evaluate our approach on a diverse set of humanoid control tasks, all simulated in Isaac Gym (Makoviychuk al. 202I). For all experiments, freedom. We focus on the following tasks: Reach, the agent must reach randomly placed goal with its right hand; Direction, the agent must walk in a randomly chosen direction; Steering, this task combines walking and orienting toward (look-at) random directions: Strike, here the agent must reach and strike target placed random location; and Long Jump, based on the SMPL-Olympics benchmark (Luo et al. 2024b), the objective is to run and jump as far as possible from a target location. Sample images shown in Figure 2 full technical details can be found in Section
simulate the SMPL humanoid Loper
cet al.] (015)) which consists 69 degrees
Baselines and Evaluation We compare our Task Tokens approach against several competitive baselines: Pure RL, policy trained directly using PPO without leveraging any foundation model; MaskedMimie Fine-Tune, using the reward signal to optimize all the MaskedMimic model without freezing; MaskedMimic (J.C. only), the original MaskedMimic model using only joint conditioning (J.C.) as the prompting mechanism. We use the J.C. defined in the original MaskedMimic for the Reach, Direction, and Steering tasks. In addition, humanoid control baselines: PULSE, a hierarchical approach that re-uses a latent space of skills from motion capture data; and AMP Peng et al. (2021), which uses discriminator ensure motion
compare against two state-of-the art
5

---

Published as a conference paper at ICLR 2026
Direction
Steering
Reach
Strike
Long Jump
Figure 2: Multi-task adaptation. Task Tokens is an effective approach to adapt BFMs to new downstream tasks, while preserving its prior knowledge. Task Tokens can be used alongside other prompting modalities to generate personalized and robust motions to solve new tasks.
Table Downstream tasks adaptation. We compare the success rates on the various tasks. While the reward provides proxy for the policy learn, the success metric measures the actual task outcome, For example, in Strike this is whether the target object is knocked down. We report the mean and standard deviation of the success rate across random training seeds, except J.C. only which uses zero-shot MisiedMimic and thus reports no variance.
Method
Reach
Direction
Steering
Long Jump
Strike
Task Tokens (ours)
94.88 + 1.99
99.26 + 0.79
88.69 + 4.04
99.75 = 0.57
76.61 + 3.49
MaskedMimic (J.C. only)
24.77
2.19
3.83
MaskedMimic Fine-Tune
93.70 + 4.59
99.10 # 1.29
87.44 6,79
47.36 + 54.78
83.07 = 5.71
PULSE
83.96 2.20
97.60 0.62
40.72 7.64
99.37 # 1.40
83.18 + 2.67
AMP
$7.14 4.80
5.14 0.68
4.28 +
76.59 43.42
52.21 47.58
PPO
89,90 3.25
97.74 1.40
32.64 40.21
61.91 + 52.26
81,36 + 1.41
quality while optimizing for task performance. Task Tokens is used with joint conditioning on the relative tasks. Long Jump and Strike pose great challenge in this sense, thus J.C. is not available for them neither Task Tokens nor in MaskedMimic.
For all experiments, we report the mean and standard deviation of the success rate across 5 random seeds, representing the variance across independently trained models. Trend lines in figures indicate the mean performance, with shaded regions denoting one standard deviation across these 5 random seeds. An exception is the J.C. Only model, for which no variance is reported as it represents the performance of a single, re-used MaskedMimic base model evaluated without task-specific retraining. Success rate definitions for each task are listed in Section B, and technical details on the training and evaluation setups available in Section C
4.1 TASK ADAPTATION
We first show that can Task Tokens effectively adapt MaskedMimic to downstream tasks. For each downstream task, we train a unique task encoder. In the Reach, Direction, and Steering environments, we also use the joint conditioning presented in MaskedMimic (we the effect of this choice in Section Visualizations of the resulting motions can be seen in Figure 2
We present the numerical results Table I The results show that Task Tokens obtains high score across the majority of environments, with PULSE, MaskedMimic Tune, and PureRL obtaining higher scores on the Strike task. Moreover, Figure 3 we present the evaluated success rate during training. Here, we observe that Task Tokens converges within approximately 50 million steps, while PULSE reaches the same performance around 300 million steps. To achieve these results, Task Tokens requires training encoder with ~200k parameters, whereas PULSE and MaskedMimic Fine-Tume require 9,3M and 25M parameters, higher by factors of x 46.5 and x125 respectively. This efficiency is critical in real-world settings where training large models is expensive. Our approach scales to many tasks with minimal additional overhead. These results show that Task Tokens can unseen tasks.
effectively and efficiently be used to adapt BFMs, like MaskedMimic,

---

Published as a conference paper at ICLR 2026
Sucoess Rate
0.6
0.4
0.2
0M
50M
100M
300M
350M
(ours)
MaskedMimic Fine-Tune
AMP
PureRL
Steps
Figure 3: Convergence for Strike. Task Tokens is sample efficient, adapting to new tasks in under 50M steps. Fine-tuning converges similarly, likely due its higher capacity, leading to overfitting.
100
Success Rate %
Success Rate 96
80
40
20
0.2
0.4
0.6
0.8
0.5
1.5
2
Friction
Gravity Multiplier
Taak Tokante (A4AS)
Taek Tokara (aurs) J.C.
Maskadhiimic Fire-
FULSE
AMP
FlastL
Figure 4: Out-of-distribution perturbations. We test the success rate the steering task when changing the ground friction (on the left) and gravity (on the right). Task tokens (both with and without J.C.) exhibit improved robustness.
4.2 OOD GENERALIZATION
The premise using MaskedMimic is that has been pre-trained on vast amounts of data and scenarios, which in turn should result in more robust behavior to new and unseen tasks. To test this, we compare on out-of-diittibution (OOD) perturbations, not seen during training both in the original BFM and in Task Tokens. We consider changes in both gravity and ground friction.
Indeed, the results, Figure show that by utilizing BFM, Task Tokens demonstrate highly improved robustness the and unseen scenarios. First, it performs almost well fully fine-tuning MaskedMimic on the baseline task (no perturbations), outperforming all other baselines. Then, performs significantly better compared the baselines the perturbation increases. Notably, Task Tokens exhibit asignificantly higher success rate in very low friction scenarios (e.g.. x0.4) and very large gravity (e.g. x1.5).
As can be seen in the gravity perturbations, fine-tuning the model seems to harm its built-in in robustness, leading to worse performance at higher gravity when compared our minimal intervention approach.
4.3 HUMAN STUDY
In some scenarios, such as animation, it is of interest to adapt and generate new behaviors (solutions to tasks) while preserving motion quality. We evaluate the realism of the generated motions by performing preliminary human study. In our study, presented -100 anonymous participants with video triplets. The participants were required to choose the motion that looked more human- mooking We provide additional details in Section
7

---

Published as a conference paper at ICLR 2026
Table 2:
Human study, Task Tokens win rate. We report the percentage by which Task Tokens was deemed more human-like. Higher values means Task Tokens was deemed more human-like.
Wins v.s. Algorithm
Direction
Steering
Reach
Strike
Long Jump
MaskedMimic (J,C. only)
95% + 2%
75% = 6%
53% + 5%
MaskedMimic Fine-Tumm Tune
99% 1%
90% = 4%
85% + 6%
85% + 5%
94% + 2%
MaskedMimic F.T. J.C.
96% + 3%
89% + 5%
82% = 6%
PULSE
15% 5%
46% + 6%
36% = 9%
24% + 5%
39% + 5%
AMP
92% 3%
84% 4%
70% + 6%
68% + 6%
96% + 3%
PPO
99% + 2%
93% + 4%
89% + 5%
82% + 4%
94% + 3%
(a) Without J.C.
(b) With J.C.
Figure 5: Multi-modal prompting. When trained on the direction task, the policy often learns to walk backwards. Task Tokens enable adding human-defined priors through additional tokens. By combining orientation priors, the BFM is instructed to face the movement direction.
Table 2 describes the percentage of times Task Tokens outperformed each alternative. First, we see that Task Tokens outperforms MaskedMimic (J.C. only) and MaskedMimic Fine-Tune. This suggests that the user-designed conditions and fine-tuning a less effective way to adapt compared to Task Tokens terms of motion quality. In addition, we observe that despite Task Tokens resulting in faster convergence, fewer parameters, and better performance, PULSE scores higher in terms of human-likenn due to the fact that PULSE constrains the high-level representation stay closer to the prior, whereas MiskedMimic doesn't t. Incorporating such constraint with Task Tokens might improve the motion quality even further. We leave this investigation future work.
outto--distributtin for the base MaskedMimic model,
of the motion. is likely
From the results above, we infer that Task Tokens offer a good balance in the tradeoff' between efficiency, motion quality, and robustness.
4.4 MULTI-MODAL PROMPTING
While some objectives are easy to define through target goals, others easier to define through rewards. Here, we show how Task Tokens can be trained alongside humann-constructed priors (tokens) achieve more desirable behaviors. We showcase two scenarios, one in the Direction task and another in Strike.
The Direction task provides a reward for moving in the right direction but does not consider the humanoid's 's orientation. As a result, the policy may converge to walking backwards. While this behavior achieves high reward and success metrics, it an unwanted behavior. Figure we show that by combining human-designed priors, providing target height and orientation for the head, the training converges to an upright forward-moving motion.
An additional challenge is Strike. In this task, the agent needs to hit a target. An emergent behavior is walking backwards toward the target and then performing a "whirlwind" motion, where the agent swirls in circles to hit the target with its hand. In Figure 6 we showcase a combination of 2 prior modalities. First, conditioning on the orientation (similar to the Direction task) the agent is instructed to face the target during the locomotion phase. Then, once close to the target, the agent is guided to strike the target with its foot using a textual objective "a person performs a kick". more detailed explanation is readily available in the appendix Section
Notably, we observed that fine-tuning the entire model leads to the well-known catastrophic forgetting phenomenon (Gooffellow et al. 2015), impairing its ability retain and integrate such multi-modal
8

---

Published as a conference paper at ICLR 2026
Figure 6: Incorporating user priors: Joint and text-based goals can be provided alongside the task-specific task tokens. Here, the user-defined objectives guide the motion style, ensuring the character faces the object while walking upright, and strikes using kick.
prompts. In contrast, Task Tokens preserve the pre-trained prompting capabilities by keeping the foundation model frozen, enabling coherent integration of learned and human-specified behaviors. PULSE, in comparison, does not support multi-modal prompts at all, as it lacks the mechanism to condition on diverse inputs beyond goal embeddings.
LIMITATIONS AND FUTURE WORK
While Task Tokens offer an efficient and effective mechanism for adapting Behavior Foundation Models (BFMs) to downstream tasks, this work also illuminates several promising avenues for future exploration. The performance of Task Tokens inherently linked to the expressivity and coverage of the underlying pretrained BFM. Future work could thus investigate methods to identify mitigate gaps in BFM knowledge for target tasks. To enhance the naturalness generated behaviors, especially for humanoids, future work could explore incorporating methods that better enforce human-likeness prior within the BFM, such as discriminative methods.
Our current experimental validation has primarily demonstrated the efficacy of Task Tokens with the MiskedMimic architecture. Further research is therefore needed to empirically verify and adapt the methodology across broader spectrum of GC-BFM architectures, which would solidify the approach's 's generality, Additionally, while the design of task-specific reward functions and observation spaces currently requires domain expertise, future research could explore methods (semi-)autonn) )attomee this process, potentially lowering the barrier to entry.
crucial next step involves investigating the transferability of Task Token-adapte policies to real- world robotic systems, process that will require addressing inherent sim-to-real challenges. Another key future direction involves extending validation to genuinely robotic tasks beyond simple animation, encompassing complex agentic behaviors that require high-level decision-makinn in umpredictable environments, thus rigorously testing Task Token efficacy for robust real-world interaction.
The current approach trains separate, lightweight task encoder for each task. Scaling this to multi- task or lifelong learning scenarios by exploring shared, compositional, or continually learned Task Token encoders presents interesting challenge. Investigating whether the agent correctly utilizes task information, for instance by examining attention maps associated with the Task Token, could offer valuable insights. such analysis would help verify the mechanism's effectiveness and potentially guide improvements in task encoder design.
Finally, investigating more sophisticated architectures for the Task Encoder itself, beyond the current feed-forward network, could unlock further performance gains. Addressing these areas will further enhance the Task Tokens framework and advance the development of more versatile, adaptable, and capable humanoid agents.
SUMMARY
This work introduces Task Tokens, novel approach for enhancing MaskedMimic as Goal- Conditioned Behavior Foundation Model (GC-BFM). Our method enables a hybrid control paradigm: provide high-level behavioral priors, which are augmented by task-specific embeddings learned via a Task Encoder trained with reinforcement learning. The encoder maps observations to goal tokens, optimizing dense rewards while keeping the BFM's robustness and multi-modal capabilities.

---

Published as a conference paper at ICLR 2026
Experimental results demonstrate that Task Tokens effectively balance efficiency, task precision, and robustness. They achieve rapid convergence and high success rates, surpassing existing methods with superior generalization on OOD cases. This is accomplished while maintaining prompting capabilities and exhibiting unmatched parameter efficiency. Human studies further confirm the human-likeness of the generated movements. However, despite these benefits, the approach relies on the quality of the underlying BFM and is currently restricted to simulated environments.
By enabling more flexible and parameter-cfficient adaptation of BFMs to specific tasks, this work presents a promising avenue for creating nuanced, responsive, and controllable humanoid agents. This could lead to more realistic character animations and capable humanoid robots capable of tailored behaviors across wide range of tasks, balancing prompt engineering with reward design.
REFERENCES
Siddhant Agarwal, Harshit Sikchi, Peter Stone, and Amy Zhang. Proto successor measure: Representing the space of all possible solutions of reinforcement learning. arXiv preprint 11X1:22411.19188. 2024.
Lili Chen, Kevin Lu. Aravind Rajeswaran, Kimin Lec, Aditya Grover, Misha Laskin, Picter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling, Advances in neural information processing systems, 34:15084 15097, 2021.
Jiawci Gao, Ziqin Wang, Zeqi Xiao, Jingbo Wang. Tai Wang. Jinkun Cao, Xiaolin Hu, Si Liu, Jifeng Dai, and Jiangmiao Pang. Coohoi: Learning cooperative human-object interaction with manipulated object dynamics. Advances in Neural Information Processing Systems, 37:79741-79763, 2025.
Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks, 2015. URL
abs/
1312. 6211
Neil Houlsby, Andrei Giurgiu. Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. 2019. URL Iat:ps
7/arxiv. org/ abs/
Edward J. Hu. Yelong Shen. Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang. Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, 2021. UtL./ttpss///rxxii..rrg////
2106-09685
Mazeyu Ji, Xuanbin Peng, Fangchen Liu, Jialong Li, Ge Yang. Xuxin Cheng, and Xiaolong Wang. Exbody2: Advanced expressive humanoid whole-body control. arXiv preprint arXiv:2412.13196, 2024.
Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation, 2021. URL attts:///arii>: wttp/oo/////////////////////// httess//aaxxv.orrr///////0101000 20111111110000000
Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia), (116))248:122485 16, October 2015.
Chenhao Lu, Xuxin Cheng, Jialong Li. Shiqi Yang, Mazeyu Ji, Chengjing Yuan, Ge Yang, Sha Yi, and Xiaolong Wang. Mobile-television: Predictive motion priors for humanoid whole-body control. arXiv preprint arXiv:2412.07773, 2024.
Zhengyi Luo, Jinkun Cao, Kris Kitani, Weipeng Xu, et al. Perpetual humanoid control for real-time simulated avatars. In Procecdings of the IEEE/CVF International Conference on Computer Vision, pages 10895-10904, 2023.
Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris M. Kitani, and Weipeng Xu. Uni- versal humanoid motion representations for physics-hased control. In The Twelfth International Conference on Learning Representations, 2024a. URL http::://pppseeeeeeeeeeee// htt.://www.e..ommmoooooroooaaaa. waw.mmmmddd000000000000000
net/ /orr:
Zhengyi Luo, Jiashun Wang, Kangni Liu, Haotian Zhang, Chen Tessler, Jingho Wang, Ye Yuan, Jinkun Cao, Zihui Lin, Fengyi Wang. et al. Smplolympics: Sports environments for physically simulated humanoids. arXiv preprint arXiv:2407 0018000187.000 2024b.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, and Gavriel State. Isaac gym: High performance GPU based physics simulation for robot learning. In Thirty- fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. wtt.hhtps:///oooom inmppeerrviieee few. net forum??
1-
10

---

Published as a conference paper at ICLR 2026
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example-guided deep reinforcement learning physies-based character skills, ACM Transactions On Graphics (TOG), 37(4 2018.
Xue Bin Peng. 7e Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa. Amp: Adversarial motion priors for stylized physics-based character control. ACM Transactions on Graphics (TOG), 40(4):1-20, 2021.
Xue Bin Peng. Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Trans. Graph., 41(4), July 2022.
Martin L Puterman. Markov decision processes: discrete stochastic dynamic programming. John Wiley & Sons, 2014.
Stéphane Ross, Geoffrey Gordon, and Drew Bagnell. A reduction of imitation learning and structured pre- diction to no-regret online learning. In Proceedings of the fourteenth international conference artificial intelligence and statisties, pages 627-635. JMLR Workshop and Conference Proceedings, 2011.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1ew:1700 707.06347, 2017.
Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: A framework for temporal abstraction in reinforcement learning. Artificial intelligence, 112(1-2):111211) 1999.
Chen Tessler. Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng. Calm: Conditional ad- versarial latent models for directable virtual characters. In ACM SIGGRAPH 2023 Conference Proceedings, pages 1-9, 2023.
Chen Tessler, Yunrong Guo, Ofir Nahati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Unified physics-beee character control through masked motion inpainting, 2024. URL httpp:s://azv....r////aaaa/ 14393
Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. Advances in Neural Information Processing Systems, 34:13 23.11323,200 2021.
Paolo Maria Viceconte, Raffaello Camoriano, Giulio Romualdi, Diego Ferigo, Stefano Dafarra, Silvio Traversaro,
Giuseppe Oriolo, Lorenzo Rosasco, and Daniele Pucci. Adherent: Learning human-like trajectory generators for whole-body control humanoid robots. IEEE Roboties and Automation Letters, 7()):7799-2786, 2022.
Jungdam Won, Deepak Gopinath, and Jessica Hodgins. Physics-based character controllers using conditional vaes. ACM Transactions Graphics (TOG), (44):111,, 2022.
Philipp Wu, Arjun Majumdar, Kevin Stone, Yixin Lin, Igor Mordatch, Pieter Abbeel, and Aravind Rajeswaran. Masked trajectory models for prediction, representation, and control. In International Conference on Machine Learning, pages 37607-37623, PMLR, 2023.
Brianna Zitkovich, Tianhe Yu, Sichun Xu, Peng Xu, Ted Xiao, Fei Xia, Jialin Wu, Paul Wohlhart, Stefan Welker, Ayzaan Wahid, Rt-2: langungganaugtttttitt models transfer web knowledge robotic control. In Conference on Robot Learning, pages 1165-2183 PMLR, 2023.
A TASK ENCODER ARCHITECTURE DETAILS
The Task Encoder is intentionally designed as lightweight and generic module. It is implemented as a feed-forward multilayer perceptron (MLP) with ReLU activations, operating on continuous task- goal observations expressed in the agent's 's egocentric frame. Prior to being processed by the network, all input features are normalized using a running mean-standard deviation estimator computed online during training.
The MLP consists of a small number of fully connected layers and produces a fixed-dimensional output, yielding Task Token ri E 1g512 We do not employ residual connections, attention mechanisms, or recurrent components in the Task Encoder. Preliminary experiments indicated that such additions were unnecessary for stable optimization or final performance in this setting, and that a simple architecture sufficed given the expressiveness of the underlying behavior foundation model.
This design choice emphasizes that task adaptation in our framework is primarily mediated through the token interface with the frozen BFM, rather than through architectural complexity or task-spe engineering within the encoder itself.
11

---

Published as a conference paper at ICLR 2026
B ENVIRONMENTS TECHNICAL DETAILS
The controllers operate at 30 Hz, and the simulation runs at 120 Hz.
The tasks are designed the versatility and adaptability the models across a range of real-worid scenarios, each adding layers of complexity to the control problem.
(a) Direction
(b) Steering
(c) Reach
(d) Strike
(c) Long Jump
Figure 7: Visual samples from the tasks.
Direction: This task involves directing the character move in a specific direction. We test the model's ability to control basic locomotion and alignment with a target direction. We measure success by the humanoid's 's speed in the target direction deviating from the target speed by more than 20% in the measurement period.
Steering: This task requires the humanoid to move specific direction while also ensuring that it faces some orientation with its pelvis. This tests more nuanced proceeding motion of the model, creating more diverse scenarios. Success is defined by the character not deviating from the target direction speed by more than 20% while also not deviating from the direction of facing by sum greater than 45%.
For this scenario, we task the humanoid with reaching specified coordinate with the right hand. This requires precision movement to achieve the specific target. The success is measured by reaching distance within 20cm between the right hand's position and the target position.
Reach:
Strike: Here we challenge the model to make the character walk toward a target and, once within range, perform action to knock down the target. This task tests the model's ability to handle both locomotion and more intricate, task-oriented behaviors, involving precise timing and spatial awareness. Success is then defined by the target falling to its side in some orientation and not deviating from it by more than -78
Long Jump The character is tasked with committing over a line after 20 meters, not touching the ground after crossing the jump start line. Success is defined by achieving jumping distance greater than 1.5 meters.
in meter-wide corridor, then jumping
C TRAINING AND EVALUATION DETAILS
Each experiment seed was trained for 4000 epochs on 1024 parallel environments, totaling approxi- mately 120 million frames. This results roughly 1-2 GPU-days per seed. PULSE was also trained on 120 million frames, but using only 128 parallel environments. All experiments were run on single NVIDIA A100 V100 GPU per seed.
In the main Task Tokens results, we used a Task Encoder implemented as an MLP with hidden layers of size [512, 512, 512), and a critic MLP with layers size [1024, 1024, 1024]. We additionally concatenated the current positions of the head and pelvis joints to the Task Encoder input, which led to slightly more human-like motion.
All reported results reflect the mean and standard deviation of the success rate across the 5 random training seeds. In performance plots, the trend line shows the mean, and shaded areas represent standard deviation across seeds. Note that the J.C. Only baseline does not report variance it corresponds to the performance of the single, re-used MaskedMimic base model evaluated without task-specific retraining.
Each reported result corresponds to the final model checkpoint after 4000 training epochs. Note that success rates in training and evaluation may differ due differences in episode termination criteria.
12

---

Published as a conference paper at ICLR 2026
For training, used the original published hyperparameters of PULSE. All other methods were trained using MaskedMimic's published hyperparameters. Please refer to the respective publications for additional implementation details.
For AMP, we use chosen to remain stylistically aligned with the original MaskedMimic corpus. While AMP can achieve stronger performance when provided with task-specific motion data tailored to each skill, such curation is not assumed in our setting. This choice reflects our goal of evaluating task adapta- tion without per-task motion dataset design, thereby emphasizing the benefit of reusing a generic, pretrained model.
coanootionfoousseee motion dataset (c.g.. walking, running, and transitions),
D MULTI-PHASE PROMPTING VIA A MINITE-STATE MACHINE
For tasks that naturally decompose into sequential phases (e.g.. locomotion followed by interaction), we employ multi- phase prompting using multiple prior tokens in conjunction with single Task Token. The selection of the active prior token is governed by simple finite-state machine (FSM). This mechanism enables structured, phase-dependent behavior without introducing multiple task-specific encoders.
Concretely, in the Strike task, the agent first locomotes toward the target while being conditioned on an orientation prior that encourages facing the target. Once the agent reaches a predefined striking distance, the FSM transitions second state, activating prior token that encodes kicking motion (specified via a textual objective). This transition is deterministic and based solely on geometric proximity, allowing smooth phase switching during single rollout.
distance 2 threshold
Locomotion:
distance < threshold
Striking: Kicking Prior
start
TaskToken
Orientation Prior
Figure 8: Finite-State Machine for the Strike task. The agent transitions from locomotion phase a striking phase based on geometric proximity to the target.
ABLATION STUDY
We experimented with several variables, constructing the Task Encoder. The results are shown in Table 3
E. MASKEDMIMIC ADAPTATION PARADIGM
Table Algorithm training scheme ablations. While fine-tuning the whole MaskedMimic model can produce performing results, we' we've shown it lacks the human-like abilities of Task Tokens.
Method
Reach
Direction
Steering
Long Jump
Strike
Task Tokens (ours)
95.37 + 1.80
96.89 + 4.33
83.66 + 5,66
99.75 0.57
76.61 + 3.49
Task Tokens (ours) + J.C.
94.88 + 1.99
99.26 # 0.79
88.69 + 4.04
MaskedMimic (I.C. only)
24.77
2.19
3.83
MaskedMimic F.T.
93.70 4.59
99.10 1.29
87.44 6.79
47.36
83.07 5.71
MaskedMimic FT. + J.C.
92.88 3,42
98.86 0.32
96.41 4.94
13

---

Published as a conference paper at ICLR 2026
Results in Table demonstrate superior performance when using J.C. when available. It is worth noting that F.T. +.C.. means fine-tuning while using the joint priors and does not imply preserving the multi-modal prompting capabilities.
E.2 TASK ENCODER ARCHITECTURE
We further present some architectural changes made to the Tak Encoder and their effect output performance, listed in 4 Bigger MLP denotes using [512, 512, 512] size MLP encoder versus (256, 256] and Using Current Pose denotes whether the current positions of the head and pelvis concatenated to the input alongside the task goal. When using joint conditioning, the performance stays high for every choice except when using smaller encoder with current pose information. This result replicates when not using joint conditioning.
Table 4: Task Encoder architectural ablations.
Method
Bigger MLP
Using Current Pose
Steering Success Rate
Task Tokens (ours) J.C.
True
True
87.77 7.14
Task Tokens (ours) J.C.
True
False
87.58 + 7.02
Task Tokens (ours) J.C.
False
False
86.88 6.65
Task Tokens (ours)
False
False
84.28 : 7.72
Task Tokens (ours)
True
False
83.30 10.06
Task Tokens (ours) +.C..
False
True
79.47 = 4.71
Task Tokens (ours)
True
True
78.59 8.93
Task Tokens (ours)
False
True
66.31 13.11
F HUMAN STUDY TECHNICAL DETAILS
To assess the quality of the motions generated by our method we conducted human study. The participants were met with this description before filling out the form:
In this study, you will watch three short videos side by side each time. These videos show different ways character moves to complete task. Your job is to decide which movement looks the most human-like. Each video is labeled A, B, or C - the labels are shuffled every time. Just pick the one you think does the best job.
Don't tt worry - there's no right or wrong answer! We just want your opinion.
used Google Forms to create forms, each containing 40 questions 8 questions for each the tasks listed in Section B In each question, showed videos side-by-side of 3 randomly sampled sequences generated by the algorithms. We ensured that Task Tokens was presented every time. Joint conditioning used when applicable, i.e. for Direction, Steering and Reach tasks. The participants were asked to choose which algorithm looks most human-like for the task described the question. An example
14

---

Published as a conference paper at ICLR 2026
Q 1: Which example looks more human-like for task *hwalking in red direction**?
A
B
O c
To ensure bias, shuffled the order of algorithms in every question and captured the videos from similar angles. The number of participants was 96: 20, 24, 52 for the forms re- spectively. We analyzed the winning rate for each environment as winning percentage 4 =
Task Tokems chosen
None of the participants received any compensation for par-
#Task Tokens chosen + # Algorithm A chosen ticipation.
15


Reviews:
[{"review_id": "KQF7lJ7Tvf", "reviewer": "Reviewer_Vh4Q", "summary": "In this work, the authors propose Task Tokens, where a pre-trained behavior foundation model is then adapted for a new task by using RL to learn a conditioning latent. In this method, the authors first train a behavior foundation model that takes a \"goal\" through some task encoder as a conditioning. Then, given a new task specified through a reward function, the task token itself is optimized with RL. The authors evaluate this method through task performance and efficiency. In addition, they analyze the stability and the human preference of the results. In these performance analyses, the authors show favorable results for their method.", "questions": "1. What are the major challenges in adapting this method to a manipulation setup?\n2. How well can you teach this method tasks that were not in the original dataset? I imagine full fine-tuning can learn any new tasks, vs. without fine-tuning you will not be able to learn something sufficiently off-distribution. But how can we evaluate it?\n3. What's the reason behind this method taking equivalent amount of environment steps to reach success as full-fine tuning? My intuition says you should be able to reach comparable performance much faster since this is RL in a much smaller space.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The demonstrated performance of the model in the videos is quite impressive.\n2. The idea itself is simple and leads to impressive results.\n3. The authors evaluate it on a diverse set of tasks and on good set of robustness evaluation.", "weaknesses": "1. The method is only evaluated in one setup with one behavior foundation model, MaskedMimic. Evaluation in other settings, such as manipulation, would go a long way to establish the generality of this method.\n2. It is hard to evaluate how \"new\" each of the elicited behaviors are. I.e. how well can this method elicit behaviors that were and were not in the training set of the original training dataset of the BFM? Especially tasks like directions should be in the dataset, and yet the Joint Conditioning method fails to elicit this behavior, which makes it seem the JC method is not great at eliciting pre-trained behavior.\n3. The authors make the \"human-like\" behavior of the prompting method a big part of the study, but devote very little space to understanding the difference between this and PULSE which outperforms task tokens. A further understanding of this difference would be appreciated.\n4. In figure 3, the Y axis label is in % but the labels are between (0-1).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gT6sddl8U0", "reviewer": "Reviewer_up9Q", "summary": "This paper presents \"Task Tokens,\" a novel framework for adapting large-scale Behavior Foundation Models (BFMs), such as MaskedMimic, to specific downstream tasks. They propose to use a parameter-efficient approach that trains a new, lightweight \"Task Encoder\" for each specific task using reinforcement learning (PPO) while freezing the pretrained network to avoid catastrophic forgetting. This small encoder maps current task-specific goals (e.g., target coordinates) to a \"Task Token.\" This token is then concatenated with the BFM's standard inputs (like state tokens and user-defined \"prior tokens\") to adapt the BFM's behavior\n\nThe proposed method allows the PPO gradient to flow through the frozen BFM, updating only the small Task Encoder. This encoder learns to \"steer\" the general-purpose BFM to solve the specific RL task while keeping the BFM's learned prior of natural human motion. The method is up to 125x more parameter-efficient and converges 6x faster compared to finetuning the whole BFM network. It also preserves the BFM's robustness in out-of-distribution (OOD) scenarios where standard fine-tuning fails.", "questions": "Besides some questions raised in the weakness section, I have some additional questions:\n\nQ1. It's not very clear to me what exactly the inputs to the task encoder are. Are they the same observation as the BFM? Is some goal information concatenated to the task encoder?\n\nQ2. Is the BFM sensitive to the exact position of the task tokens in the input sequence?\n\nQ3. Could there be some task tokens that represent some more \"generation\" modifications to the model? This reminds me of a paper altering the behavior of VLA models via some \"activation steering\" [1]. For example, one might train a task encoder that makes the humanoid move faster or move with lower torso height, and these encoders may be useful to alter the model's behavior in different tasks, along with more task-specific encoders.\n\nReference: \n[1] Häon, B., Stocking, K.C., Chuang, I., & Tomlin, C.J. (2025). Mechanistic interpretability for steering vision-language-action models. ArXiv, abs/2509.00328.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "**Elegant and Novel Solution to Important Problem** The main contribution is the application of a light-weight task encoder to steer large-scale behavior models. The method, by design, prevents catastrophic forgetting by freezing the core BFM. This is an elegant and, to my knowledge, novel solution (at least in the field of RL and robot learning) to address the balance of general policy and specific task performance. The effectiveness of such an approach opens up the opportunity to better control the behavior of robot foundation models.\n\n**Solid Empirical Validation** The experiments show the proposed method is highly parameter-efficient, and it also preserves the foundation model's robustness in some cases compared to finetuning (see Figure 4). The human study (Table 2) confirms that the resulting motions are perceived as more human-like than those from baselines like PPO, AMP, or naive fine-tuning, demonstrating that the BFM's prior is indeed preserved. The framework (Sec 4.4, Fig 5/6) also allows for a seamless combination of two control modes: user-defined \"Prior Tokens\" (like text or joint conditions) provide high-level style guidance, while the learned \"Task Token\" optimizes for the low-level, reward-specific objective. This hybrid approach is highly flexible and a step forward in usability.", "weaknesses": "**Limited Task Complexity:** This is the main weakness. The tasks evaluated (Direction, Steering, Reach, Strike, Long Jump) are primarily reactive motor skills or goal-reaching tasks. While diverse, they do not appear to require complex, long-horizon composition of tasks. It is unclear if the \"Task Token\" approach, which optimizes a single, continuous token, is sufficient to adapt the BFM to tasks requiring multi-stage sequential logic (e.g., \"find the block, pick it up, and place it in the correct box\").\n\n**Reward and Human-likeness Trade-off:** The human study (Table 2) reveals that Task Tokens were consistently rated as less human-like than PULSE. The authors suggest PULSE \"constrains the high-level representation to stay closer to the prior.\" This implies that the RL-driven Task Token, in its pursuit of maximizing the reward, may be finding solutions that are on the edge of the BFM's natural motion manifold—less human-like, but more optimal for the task.\n\n**Scalability:** The current approach trains one separate Task Encoder for each new task. The paper's scalability claim (Contribution #2) refers to *per-task* efficiency (a small ~200k encoder vs. a 25M model). It also highly depends on the reward definition to adapt to certain tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "rGAcKbN4oZ", "reviewer": "Reviewer_MgVR", "summary": "The paper proposes an approach to adapt pre-trained Behavior Foundation Models (BFMs) to downstream tasks by training \"task tokens\" -- special tokens which are added to the prompt of BFM, which represent the task-specific information. These tokens are trained via RL from environment rewards. They are modeled as neural networks which map task goals to a fixed tokens dimension.\n\nThe authors conduct multiple experiments to demonstrate the efficiency of their approach compared to baselines -- finetuning BFMs directly, pure RL without BFMs and other approaches which make use of motion capture data.\n\nOverall, the proposed approach is simple, concise and efficient.", "questions": "* What are the design choices which are important for the task encoder? Can you do a more careful scientific study here so that readers understand what things matter?\n\n* Can you explain discrepancies in Table 1? Why baselines numbers are inconsistent?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "* The approach is sound, makes sense, and simple\n* The experimental evidence suggests that it works well in practice, despite being much cheaper than alternatives\n* The approach essentially makes use of \"prompt engineering\" by making it automatic and training \"tokens\" from data", "weaknesses": "* I think the technical presentation of the paper could be improved:\n\n** The \"task tokens\" is the major contribution of the paper, but it does not spend enough time discussing different design choices regarding \"Task encoder\" architecture. It feels that authors just tried a few simple things and it \"just worked\". However, I think the research community will benefit greatly from a proper analysis of the different impacts of design choices in such architecture. I.e., do they use some form of normalization ? What are activation functions? Are there residual connections and if not, why not? I think also trying to conduct a more careful study here will end up in a method which eventually performs even better than what the authors have\n\n** Table 1 looks a bit suspicious to me. The results of the baselines are inconsistent. You see that PPO performs well on Strike, Reach and Direction but achieves significantly lower performance on Steering and Long Jump. Similar can be said about other methods. Why is that? What it looks to me could be that the authors did not spend enough effort trying to make sure that the baselines are optimized carefully.\n\nMinor: Since the proposed method is much cheaper than the baselines, it would be very helpful to have a figure which shows total FLOPs spent compared to baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OWyyIWMgdK", "reviewer": "Reviewer_zBs9", "summary": "This paper introduces Task Tokens, a parameter‑efficient adaptation mechanism for goal‑conditioned behavior foundation models (GC‑BFMs) such as MaskedMimic. A lightweight task encoder maps task observations to a learned token that is appended to the BFM’s input sequence alongside optional human priors; the BFM is kept frozen, and PPO updates only the task encoder via gradients flowing through the frozen model. Across several humanoid control tasks, Task Tokens achieve competitive success rates to full fine‑tuning with far fewer trainable parameters, improved robustness to friction/gravity perturbations, and preserved multi‑modal prompting. A human study indicates motions are generally more human‑like than PPO/AMP and fine‑tuned MaskedMimic.", "questions": "- Figure 3 shows several saw-tooth-like drop‑offs for MaskedMimic fine‑tune and Task Token. What causes these instabilities?\n- AMP often trails PPO and even fails on Direction/Steering. Why is that when it’s specifically designed for these scenarios? Did you retune AMP per task?\n- Efficiency: please report wall‑clock training time (GPU‑hours) and inference throughput for Task Tokens vs. full fine‑tune and PPO/AMP. Does the 6× speedup in steps translate to real time?\n- Could you compare against PEFT baselines applied inside the BFM (LoRA, adapters) with comparable parameter budgets? Where does Task Tokens win/lose relative to these?\n- Section 4.4 mentions two phases (direction prior then semantics “kick”). Are you using multiple task token adapters here or just multiple prior tokens with a single task token? If multiple task tokens are used, how are they scheduled/combined?\n- Token analysis: do task tokens cluster across runs/rollouts and across tasks? Visualizing the resulting token embeddings across training runs for different tasks could reveal whether clustering suggests scalability.\n\nOverall, this is a well‑motivated and seemingly effective adaptation mechanism that preserves the strengths of GC‑BFMs. Addressing the missing PEFT baselines, clarifying efficiency in wall‑clock terms, explaining AMP’s underperformance, and adjusting the framing of some sections would strengthen it further.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "The paper is well‑motivated. It clearly articulates the gap between prompt‑only GC‑BFMs and reward‑driven task optimization, and proposes a neat integration that leverages the transformer token interface without touching the backbone. Freezing the foundation model to avoid collapse/forgetting while steering behavior through a small encoder is a sound and practical design choice to preserve the motion manifold. Empirically, the method is consistently competitive with full fine‑tuning while using ~200k parameters per task, and it outperforms standard RL and AMP/PULSE on several tasks. The study on combining learned tokens with human priors (orientation and text prompts) convincingly demonstrates directability that full fine‑tuning appears to erode. The experimental section is generally decent with five seeds, ablations on the task‑encoder architecture, OOD tests, and a user study. I especially appreciate the explicit discussion of limitations around human‑likeness and reliance on the pretrained BFM compared to PULSE.", "weaknesses": "My main issue is the overstatement of some contributions. The method is a logical adoption of ideas from NLP to BFMs, RL training for these tasks is a sensible way to tune the performance of the model, and the results are decent. There is no need to exaggerate other parts of the paper.\n\nStatements on the criticality of using up to “×125 fewer parameters” seem misplaced when the baseline methods train only approximately 25M parameters. This is especially true when training on humanoid simulations, where the sim is also likely a major bottleneck. If the efficiency gains are truly massive, it would be great to have a comparison of GPU‑hours, memory, and inference latency. In addition, it seems like the baseline fine-tuning does not employ standard parameter‑efficient fine‑tuning methods inside the BFM (e.g., LoRA/adapters), which would be the real baseline when talking about efficiency.\n\nThe robustness claims are also inflated. “Order of magnitude” gains at very low friction values are only caused by the baseline going slightly faster towards zero. At that point, any small increase in success rates (i.e. going from 0.01% to 1%) is going to be orders of magnitude larger. Given the huge standard deviations, it is even questionable if the result is statistically significant.\n\nFinally, scalability remains unclear: training one encoder per task is simple but offers no multi‑task, compositional, or continual learning path, and the claim that fine‑tuning catastrophically forgets multi‑modal prompting is largely anecdotal without a quantitative retention test on the original prompts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)