Vintix II: Decision Pre-Trained Transformer is a Scalable In-Context Reinforcement Learner
Authors: Andrei Polubarov
Lyubaykin Nikita
Alexander Derevyagin
Artyom Grishin
Igor Saprygin
Aleksandr Serkov
Mark Averchenko
Daniil Tikhonov
Maksim Zhdanov
Alexander Nikulin
Ilya Zisman
Albina Klepach
Alexey Zemtsov
Vladislav Kurenkov
Venue: ICLR 2026 Poster (2026)
Keywords: in-context reinforcement learning
Primary area: reinforcement learning

Abstract: Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at inference. Algorithm Distillation (AD) pioneered this paradigm and was subsequently scaled to multi-domain settings, although its ability to generalize to unseen tasks remained limited. The Decision Pre-Trained Transformer (DPT) was introduced as an alternative, showing stronger in-context reinforcement learning abilities in simplified domains, but its scalability had not been established. In this work, we extend DPT to diverse multi-domain environments, applying Flow Matching as a natural training choice that preserves its interpretation as Bayesian posterior sampling. As a result, we obtain an agent trained across hundreds of diverse tasks that achieves clear gains in generalization to the held-out test set. This agent improves upon prior AD scaling and demonstrates stronger performance in both online and offline inference, reinforcing ICRL as a viable alternative to expert distillation for training generalist agents.

Published as a conference paper at ICLR 2026
VINTIX II: DECISION PRE-TRAINED TRANSFORMER IS A SCALABLE IN-CONTEXT REINEORCEMENT LEARNER
Andrei Polubarov AXXX, Applied AI Institute
Nikita Lyubaykin AXXX, Innopolis University
Alexander Derevyagin AXXX, HSE
Artyom Grishin Innopolis University
Igor Saprygin HSE
Aleksandr Serkov HSE
Mark Averchenko ITMO University
Danii Tikhonov HSE
Maksim Zhdanov AXXX, NUST MISIS
Alexander Nikulin AXXX, MSU
Ilya Zisman
Humanoid
Albina Klepach AXXX
Alexey Zemtsov AXXX, NUST MISIS
Vladislav Kurenkov AXXX, Innopolis University
ABSTRACT
Recent progress in in-context reinforcement learning (ICRL) has demonstrated its potential for training generalist agents that can acquire new tasks directly at infer- ence. Algorithm Distillation (AD) pioneered this paradigm and was subsequently scaled to multi-domain settings, although its ability to generalize to unseen tasks remained limited. The Decision Pre-Traines Transformer (DPT) was introduced as an alternative, showing stronger in-context reinforcement learning abilities in simplified domains, but its scalability had not been established. In this work, we extend DPT to diverse multi-domale environments, applying Flow Matching as a natural training choice that preserves its interpretation as Bayesian posterior sam- pling. As result, obtain agent trained across hundreds of diverse tasks that achieves clear gains in generalization to the held-out test set. This agent improves upon prior AD scaling and demonstrates stronger performance in both online and offline inference, reinforcing ICRL as a viable alternative to expert distillation for training generalist agents.
INTRODUCTION
Building general-purposs agents remains a central goal for the AI community. Significant progress has been made in NLP and CV (OpenAT] 2024 Team 2024 Bai et al.) 2025 Wang et al. (024a)) with recent models capable of solving wide range of tasks. contrast, reinforcement learning lags behind in developing Large Action Models that can handle broad task diversity. The prevailing ap- proach involves training Transformer-basee architectures (Vaswaní al. 2023) offline datasets et al. 2024), and enhanc-
in manner analogous to LLM pretraining (Reed et al. ing inference through retrieval -aureeee-- generation (Sridhar et 2025). Despite these advances, key limitation persists: such systems typically do not leverage rewards as an external signal for real-time policy correction and adaptation.
In-context learning (ICL) has made adaptation appear routine for large language models: short prompt can alter model's behavior without parameter updates (Brown et al. 2020). Transferring in-context learning capabilities to reinforcement learning aims to enable agents to achieve reward-
optimizing behavior on new environments without task-specific finetuning. Algorithm Distillation
(AD) (Laskin al. (022) and the Decision Pretrained Transformer (DPT) (Lee al. 2023)
flagship approaches in this area. with several recent extensions (Zisman et al., 2024] 2025 Sinii
al. 2024: Nikulin et al. 2025 Dong al. 2025). While these methods demonstrate in-context

---

Published as a conference paper at ICLR 2026
behavior when deployed in single-domain and grid-like environments, their scalability to diverse, continuous-contro settings remains underexplored.
Polubarov et al. (2025) scaled in-context RL to the multi-domain setting by training AD large cross-domain dataset. Despite strong in-context capabilities on the training tasks, performance on unseen tasks remained limited, leaving room for improvement. At the same time, the ability of DPT to approximate posterior sampling over actions positions as a strong backbone alternative for scaling in-context RL.
As the task diversity of the data increases, more expressive policy classes (Mandlekar et al. 2021) required distill increasingly multi-modal behaviors. Vanilla DPT and its extensions primar- ily target discrete-action environments (Lee et al. 2023 Dong et al., 2025), where inference-timg sampling is straightforward. To fully unlock the potential of DPT in continuous, multi-modal action spaces, such expressive policy classes with native inference-time sampling must be adopted.
In this work, we scale DPT the cross-domain setting and address the challenges posed by com- plex, multi-modal continuous action distributions by leveraging a flow-based policy head, thereby enabling in-context adaptation to unseen tasks and parametric variations across diverse environ- ments.
Our key contributions are:
1. Scaling DPT to the cross-domain setting with a wide range of tasks, including robotic locomotion and manipulation, HVAC control, PDE optimization, autonomous driving, and other applications (see Section 3)
2. Substantially improving test- timt performance on unseen tasks relative to prior Large Ac- tion Models (see Figure B).
3. Building on Polubarov et al. 2025), we collect and open-source a large cross-domain dataset containing over 700M transitions (a 3.2x increase) across 209 training tasks span- ning 10 domains, with 46 additional tasks (compared to previously) reserved for evalu- ation support future research in the field (see Table
RELATED WORK
In-Context Reinforcement Learning. The term in-context learning refers to the ability of large language models to adapt to new tasks when prompted with a few demonstrations at inference time (Brown et al. 2020. Liu et al.] 202T). Transferring this adaptability to the meta-reinffrrcemnnt learning (Meta- RL) setting has led to the development of wide range of approaches (Beck et al, 2025 Moeini et al. 2025). The first family of methods is derived from RL (Duan et al. 2016), where task-related information is encoded with sequence models updated through joint online policy iteration. Notable successors include AMAGO-1,2 (Grigsby et al.) 2024a by and RELIC (Elawady et al.] 2024). The second line of work focuses model-based approaches that either infer task- related belief state [Dorfman et al. 2021 Wang et al. 2023a) or construct world models for planning (Rimon al. 2024| Son al.] 2025). The final category frames Meta-RL from the perspective of context- based imitation learning, including Algorithm Distillation (AD) (Laskin et al., (022) and its
extensions (Zisman al. 2024 Sinii et al. [2024] Tarasov et al. (025). which aim to distill policy improvement from collected learning trajectories, as well as the Decision Pretrained Transformer (DPT) (Lee et al. 2023) and its variants (Dong al., 2025] Chen et al., (025). which perform posterior sampling by training on demonstrations relabeled with optimal actions. Our model builds upon the DPT framework and extends it to the multi-domain continuous control setting.
Generalist Agents and Large Action Models. Generalist agent systems are designed to oper- ate across diverse task domains with varying MDP structures. The first category of approaches develops such agents through multi-domain training, either from scratch (Reed et al., 2022 Gal- louédec et al. 2024). with potential expansion via retrieval-augmenntr generation at inference time
(Srihhrr et al. 2025), or by augmenting lasion-language models (VLMs) with action experts. The latter strategies include channel-wise action discretization (Brohan al. 2023 Team et al.) 2024 Kim et al. 2024). compression-bast tokenization schemes (Pertsch et al. 2025). and based generative controllers that capture high-dimensional, multi-modal action distributions (Wen al.
2

---

Published as a conference paper at ICLR 2026
Training
Inference
]
Domains
Flow Matching
hgg
A
hr
Decision Pretrained Transformer (DPT)
Collection Sxpert
Agurry
Og, 4o.
44 9g. Ti
OL, <2,
Cross-Domain Dataset
Figure 1: Approach Overview. Stage I: A dataset collected using the noise-distillation technique Zisman et al., 2024), which covers suboptimal regions of the state space. is relabeled with the demonstrator's s optimal actions. Stage 2: During training, the flow-machhing (OT) head is condi- tioned on hidden representations from the Decision Pretrained Transformer (Lee et al. 2023). Stage 3: At inference, actions decoded using Heun's method.
Black et al. 2024] Shukor al. 2025 Intelligence et al. 2025). The second line of re- scarch locuses on scaling offline, memory- based meta-reinforcemeent learning, most notably Al- gorithm Distillation (AD) (Laskin et al., 2022] and the Decision Pretrained Transformer (DPT) (Lee et al. 2023), to cross-domain settings (Polubarov et al. (025). Our work follows the latter direction, enriching cross-domain datasets and leveraging inference-time self-correction through in- context reinforcement learning and prompting with expert demonstrations, rather than conditioning on lasiuaelanguage model representations.
2025
Following the success of denoising diffusion 5 Ho et al. 2020 Dhariwal & Nichol 2021 and flow matching approaches (Lipman et al. 20023 Esser et al. (2024) in modeling complex distributions, researchers have sought to leverage their expressive power and mode-preseering properties for continuous con- trol applications (Zhu et al. 2024). The main research directions include: augmenting Tabula Rasa reinforcement learning algorithms with diffusion and flow matching methods (Wang al. 2023b Chi et al., 2024, Wang et al., [224b) Zhang et al.) 2025 Park et al. 2023); applying generative flow- based models as planners and world models (Janner et al. 2022 He et al.) [023 Farebrother et al.) 2025]; using flow matching an action expert for generalist agents (Ni et al. 2023) and vision- language-action (VLA) models (Black et al., 2024] Shukor et al.) 2025) Intelligence et al. 2025).
Flow Based Methods in Reinforcement Learning. models (Sohi-Dickstein et al. 2015
Our work is most closely aligned with the last direction, focusing on the application of flow-based policies offline, memory-based Meta-RL setting.
APPROACH
Our approach extends DPT (Lee et al. 2023) to the multi-domain continuous control setting with complex multi-modal action distributions. Previous DPT variants that operated continuous ac- tions (Dong et al. 2025) yield only moderate results, presumably because Gaussian heads fail to capture multi-modality effectively, creating a likelihood mismatch for complex action posteriors. To address these limitations and enable native sampling from action distribution during online in- ference, our approach integrates rectified-fii single model to represent richer action distributions. To scale up training. a diverse dataset covering 10 domains with distinct MDP structures was collected. The following sections describe the dataset, the architecture of the flow-based head for DPT, and our training and inference procedures.
objective with in-context conditioning, allowing
3.1 DATASET
The training dataset is collected using the Continuous Noise Distillation (CND) method described in the Vintix paper (Polubarov et al. 2025). Progressive action noising alters demonstrator policy quality, thereby increasing the coverage of the visited {s, a, r) tuples distribution (see Appendix G
3

---

Published as a conference paper at ICLR 2026
for task-level reward progression graphs), which typically improves test-time performance (Brown ee al. 2019). The collected acte-nction transitions are then relabeled with demonstrator actions as required by the DPT pipeline (Lee et al., 2023). Our training dataset spans 10 diverse domains, which we describe in detail in Appendix C.1 Higher-level dataset statistics are listed Table I and task-level dataset summary is available in Appendix [ To test the model's adaptation to seen environments, we reserved 46 tasks across all domains for validation and excluded them from training. description of the train-test split and its motivation is provided in Appendix C.2
Domain
Tasks
Episodes
Timesteps
Sample Weight
-entrrial-BBcchmark
16
288k
72M
10.1%
Bi-DexHands
15
216.24 6.
31.7M
4.5%
Meta-Worke
45
670k
67M
9.4%
Kinetix
42
1.1M
62.8M
8.9%
CetyLeaan
20
146.4k
106.7M
15.0%
ControlGym
9
230k
100M
14.1%
HumEnv
12
120k
36M
5.1%
MuJoCo
11
665. 1k
100M
14.1%
SinerGym
22
42.3k
30.9M
4.4%
Meta-Drive
17
271.9k
102.6M
14.4%
Overall
209
3.8M
709.7M
100 %
Table 1: Training Dataset Summary. Training dataset covers 209 distinct tasks across 10 domains
ranging from robotic manipulation to HVAC control and autonomous driving
3.2 MODEL ARCHITECTURE
The model consists of three blocks: encoder that maps input actions, observations, and rewards into fixed-size embedding vectors; a transformer backbone responsible for the main representational capacity; and a flow-based decoder head conditioned on the transformer's hidden states, which out- puts actions in the original action space (see Figure D..
To address variability in action and observation spaces, partition all tasks into non-overlapping groups sharing the same oction-observation structure. Each group is processed by its own encoder and decoder (both represented MI.s),, making the model task-agnostic within each group and enforcing reliance on contextual information.
As the backbone, used the TinyLLaMA (Zhang al. 2024a implementation of the transformer. All positional encodings were removed, as they are not required in DPT (Lee et al. (023). The final model consists of 16 layers, 24 heads, an embedding size of 1536, and post-attention feed-forward hidden size of 6144, resulting in a total of 928 million parameters. Training was performed on 8 H100 GPUs with batch size of 64 and input sequences of length 4096. A full description of the training setup is provided in Appendix
Given the transformer output h E IRd, we parametrize the context-dependent vector field u(t, h, xx) [0. 1] x Rd R - Re using a time encoder 7 [0, 1] + dR and an MLP (r))),),, h, h, td,+ + R
The vector field defines a context- deeeedett flow (t, h, æo) [0, 1] Rd R - E as the solution to it = vtt, h, 2.) with initial condition Zo. The flow at terminal time t = 1 is defined as the policy
x( (1,
so that sampling a- ( h)) corresponds to drawing Po and integrating the ODE to = 1.
3.3 TRAINING
The data-loading pipeline follows the vanilla DPT setup (Lee et al. 2023 and samples (og. C, a") tuples from the multi-task cross-domain dataset U, Dr. Here, Og is the query observation, a" a"~ (: o,)) is the corresponding demonstrator action, and C {(o,, r.) denotes a task- specific context of length L. Each context element consists of the observation, the applied action,
4

---

Published as a conference paper at ICLR 2026
and the resulting reward. Unlike the vanilla DPT implementation, which also includes the next observation o', we omit o' as our experiments indicate that does not affect model performance.
Each imput sequence consists of BOS token, query token, and L context tokens that ran- domly permuted. A causal Transformer encodes the sequence and produces hidden states {hy}) + A context--onditioned sectr--fildd head supervises all positions {1..... L 1) predict a" by minimizing the rectified-flow matching objective (Liu et al. 2022):
Teg LRF (0,, (/ Teg 7(1)) - (a' - Ftj = (1-1)) vo.j a*
We encode ij E [0, 1] using a sinusoidal time embedding r(4,) = [sin((, cos(t; E Bdd, The frequency vector f E Rd,/2 /2 learnable and initialized on a logarithmic scale over [fmin fmax]:
fx
fmin (Smaxx) fmnn
2,72
k 0....... d 1.
Pseudo-code formalizing the training step our model provided in Appendix B.1
3.4 INFERENCE
Policy evaluation is conducted in two regimes: online and offline, following the DPT paper. In the online setting, the model begins with empty context C = {.) and incrementally appends observed (oy, a, r) interactions during deployment. When |C] exceeds the maximum length L, the oldest transition is removed. In the offline setting, the context C = {(o, a,, r.)) with Lc S L fixed and remains unchanged throughout inference.
Given a selected task-group g with action dimension Pa. Io (0, Io_.) is sampled from the base distribution and integrated through the learned vector field V1 from which is then taken as the output action a. Heun's 's method (second-order Runge- Kutta) with M uniform steps and step size At 1/M applied for numerical ODE integration. The vector field is conditioned on the hidden state of the last Transformer token h Appendix B.2 provides the full inference procedure.
1. yielding 21,
4 EXPERIMENTAL EVALUATION
4.1 EVALUATION DETAILS
Metrics The main evaluation metric for all experiments in this section is the total episode return, normalized with respect to random and expert policy scores: scoremormalsee
anoredeere www.dmemntttttrroooooooo scorerandom following Gallouédec et al. 2024)) Sridhar al. (2025); ; Polubarov
Score" now random
al. 2025). Although demonstrators had be retrained, the normalization scores align with those
reported in Polubarov et al. (2025) within one standard deviation for all overlapping domains. Task- level normalization scores for all domains considered in the study are provided in Appendix H To
compute aggregated scores per domain, we used the inter-quartile mean (IQM) implementation of Agarwal al. (202T), as it yields more robust performance estimates.
Baselines We compare Vintix II against two prior action models: Vint (Polubarov et al., 2025) and REGENT (Sridhar et al., 2025). We leave out the comparison with JAT (Gallouédec al. 2024) as it was shown (Polubarov et 2025) to under-perform Vintix by a substantial margin on all of the domains considered. Comparisons with Vintix ix are conducted in both online and offline settings, as both modes are supported, whereas comparisons with REGENT focus on prompted (offline) evaluation, since was designed specifically for this deployment scenario. Scaled returns for REGENT are taken from the original paper. of improved demonstrators for World in both our work and Vintix, their absolute returns are equal to or higher than those reported in JAT and REGENT (see appendix D.I). Although this lowers our normalized scores relative to those reported in REGENT, we retain the comparison, as focus is on measuring the ability to match demonstrator performance rather than on absolute score differences.
important to note that, due to the use
5

---

Published as a conference paper at ICLR 2026
Industrial Benchmark
MujoCCo
0.9310
1.0614
1,0455
1,0709
1,0500
0.9987
Normalized Score IQM (95%
0.8122
0.9975
1.0067
1,0000
0,7092
0,9590
0.8239
0.9781
0.9833
0.5983
20
2
2
2
0.9078-0
2
2
2
2a
0.6986.0
2
2
2
0.9317
2
2
2
0,9500
2
2
2
2
HumEnv
MetaDrive
SinerGym
ControlGym
CityLearn
0.6681
1.0307
10047
1.0200
0.9886
0.9274
0,9300
0.4579
0.9709
0.3607
0,8474
0.7724
0.7377
0.9535
0.207020
20
2
2
2
0.7619
2
21
2
2
0.6533
20
2
2
2
0.5892
21
23 0.9192
20
21
2
2
Number of shots
Figure 2: Online inference on training tasks Although the model starts with context, it infers relevant task-specific information for self-correction. All runs are conducted with four different random seeds, and the IQM of normalized scores reported.
We do not compare our model with RI.2 Duan et al. 2016), Amago-(1,2) (Grigsby et al. 2024a or RELIC (Elawady et al. (202), as these Meta-RL algorithms operate in an online setting, whereas work focuses on training from static datasets.
Evaluation Protocols We conducted all experiments in both online and offline setups as de- scribed in Section 3.4 Online inference uses FIFO-style memory filling, effectively implementing sliding attention window over past interaction history, while offline inference is performed with fixed set of demonstrator episodes serving as the task-specific context.
It should be noted that the evaluation setup for Meta-Worke was modified. In both Vintix and REGENT, due to inheritance from JAT (Gallouédec et al. (022),, goal positions are not fixed during dataset collection and evaluation, thereby violating the ML1 protocol (Yu et al,, (22T]) for measuring goal adaptation within a single task. To address this, we fixed the goal position across environment resets for all dataset collection and evaluation runs. Consequently, our approach exposed to orders of magnitude fewer goal positions during pre-training and was therefore evaluated under a more challenging setup.
4.2 PERFORMANCE ON TRAINING TASKS
Online Deployment Firstly, we evaluate the ability of our model to perform iterative self- correction on training tasks with no context provided. To verify this, the model was deployed on 209 training tasks using the online inference procedure.
Figure 2 shows normalized returns per episode during online inference. Most domains exhibit con- sistent improvement over the course deployment, highlighting the model's test-time adaptive behavior. On Kinetix, ControlGym, MuJoCo, and MetaDrive, near-demonstrator performance is reached from the very first episode, indicating that DPT infers a strong prior for training tasks and requires only few episodes for correction. Individual task-level scores for training environments are provided in Appendix H.T
Offline Deployment To examine how shifting to the offline evaluation scenario affects perfor- mance training tasks, re-ran the model with 2500 task-specific transitions provided.
As shown in Figure 3 additional task-specific prompts improve performance across all domains, yielding average gain of +4.1 %. This represents considerable improvement given that online performance on training tasks is already demonsseont..toor Scaled performance for offline runs is reported in Appendix H.2

---

Published as a conference paper at ICLR 2026
Test Split
Train Split
ControlGym
ControlGym
MetaDrive
MujoCo
MetaDrive
MujoCo
0.75
0.5
0.5
0.25
Industrial Benchmark
0.25
Industrial Benchmark
CityLearn
CityLearn
0
-0.25
-0.25
0.5
-0.5
SinerGym
Bi- DexHandd
SinerGym
Bi Dexllands
HunEav
Meta World
HumEny
Meta: World
Kinetix
Kinetix
Vintix (Offline)
Vintix (Online)
REGENT (25 demos)
Vintix (Online)
Vintix (Offline)
RIGENT (100 demos)
Figure 3: Domain-level normalized scores (Left) Evaluation results for tasks unseen during training. (Right) Evaluation results for 209 training tasks. Offline runs of our model were conducted with a prompt size 2500 transitions, compared to 5000 transitions for Vint ix, and 25 and 100 episodes for REGENT on testing and training tasks, respectively. Domain-level aggregation is performed using IQM.
4.3 PERFORMANCE ON UNSEEN TASKS
Offline Deployment Next, we benchmark the model's ability to infer demonstrator-Ievel poli- cies on entirely unseen tasks from limited set of expert demonstrations (offline scenario). For this purpose, we evaluated our model on all 46 held-out tasks across 10 domains. The prompt size was capped at 2500 transitions to ensure a fair comparison with REGENT (25 demonstrations) on the Meta-Wor ML45 split, as our model's context length is restricted to 4096 tokens (see Section 3.2) and thus cannot accommodate more than 40 episodes of demonstrations on Meta-World, Vintix was evaluated on four overlapping domains (Industrial-Benchmaaf Bi Bi-DexHands, MuJoCo, and Meta- Keta-World) with demonstrator prompts provided.
The results are summarized in Figure 3 In the offline setting, model achieves over 75% of demonstrator performance on entirely unseen tasks in MetaDrive, CityLearn, SinerGym, and Con- trolGym domains (102%, 78%, 92%, and 100% normalized scores, respectively). When compared with the prompted version of Vintix, it yields improvements DexHands, +4% on MuJoCo with parametric variations, and +63% the Meta- Woold ML45 split. Furthermore, comparison with REGENT reveals that our approach reaches an 8.2% higher normal- ized return on unseen tasks in the Mera-World ML.45 benchmark. More detailed, environment- level scaled returns are reported in Appendix H.2
scaled return +17% on Bi-
These results suggest that the flowe DPT architecture provides a strong inductive bias, sup- porting the emergence of fully parametric in-context imitation, which in turn leads to enhanced performance compared to prior action models.
Online Deployment Further experiments aim to evaluate the ability of our model to exhibit adaptive behavior in entirely unseen environments and their task variations, without any prior infor- mation provided. To test this, we rerun model on 46 unseen tasks, starting with empty initial context, which is referred to as online evaluation in Section 3.2
Figure shows that cold-start evaluation of our model performs par with the prompted version for 8 out of 10 domains, while it lags behind on the Meta- World ML45 and Bi-DexHands ML20 benchmarks. However, several studies (Anand et al., 2021: Grigsby et al. 2024b) argue that task adaptation on the Meta-World World ML45 split is unrealistic without additional information, which may explain moderate results in this domain. Bi-DexHands MI.20 is more challenging due to its

---

Published as a conference paper at ICLR 2026
higher control dimensionality, variable observation and action space structures, and smaller num- ber of training tasks (Chen et al. 2022). Task-level granularity is available in Appendix H.1
Since data collection and evaluation protocols for Mleta-Worl fix the end-effector goal state be- tween resets and ensure that training and evaluation goal sets are non-overlapping, the training tasks in Meta- Meta-World effectively implement the ML1 benchmark for held-out goal variations, making them suitable for assessing in-context capabilities. Accordingly, Figure P (right radar plot) reports online evaluation results for the Meta-World ML1 split. The results show that demonstration-less version of our model achieves an 85% normalized score, outperforming REGENT, which was provided with 100 expert-level episodes covering richer end-effector goal distribution, by 3%.
The observed advantage may be attributed to deployment time self-corrective behavior, which is presumably induced by DPT's potential ability effectively implement context-based Bayesian posterior sampling (Lee et al. 2023).
4.4 ANALYSIS AND ABLATIONS
Metaworld door-open-v2
Sinergym warchouse-mixeed_dees feb feb
MetaDrive cones_5
CityLeam 2022_phase Lataase___tss____
Action (Truncated Y-componen))
1.5
0.055
1.5
1.0
0.0
1.0
0.5
2.23
0.5
0.000
0.0
-0.3
-05
Action (Truncated SVD X-component)
Emply Context
10 Transitiios
100 Transitions
500 Transitions
Figure 4: Action beliefs over context sizes Action distributions are shown for different prompt sizes during offline evaluation, ranging from no context 10, 100, and 500 transitions of task-specific demonstrator data. Projections into 2D space are obtained using Truncated SVD. The gradual de- crease in distributions' entropy indicates that model exhibits behavior consistent with posterior sampling.
Progressive Concentration of Action Beliefs One of the main theoretical results of Lee et al. 2023) was show that DPT implements in-context posterior sampling (PS), generalization of Thompson sampling for RL in MDPs. To test whether our trained agent exhibits this behavior, analyze its action distributions as function of context length L. For each task in Figure we fix query observation 0g from evaluation rollouts and vary the context from empty to 500 (og. a, r) demonstrator tuples. each we draw 100 action samples for every Og. The KDEs in Figure Tigureecconsie consistently show posterior-like contraction: with short contexts the distributions wide and uncertain, while longer contexts sharpen into narrow peaks, accompanied by a monotone decrease in entropy, evidence consistent with in-context PS. More action belief graphs can be found in Appendix F
Effect of Number of Demonstrations To conduct an ablation study assessing how the size of the demonstration prompt influences performance on unseen tasks during offline evaluation, we re- deployed our model with task-specific prompts ranging from 500 to 4000 (og, a, tuples. Context size is reported in transitions rather than episodes, as DPT operates on permuted, episode-agnostic context dataset. Results for REGENT Meta- Meta-World ML45 compared
like-for-like basis, since both models cap episode length at 100 timesteps this domain. Vint ix was evaluated with 4000-transition prompt across all shared domains.
Figure [5] agggregatess the results of the experiment, showing that performance improves with prompt size for World, Industrial-Benddn Benchmark, and SinerGym, while remaining stable for all other
8

---

Published as a conference paper at ICLR 2026
Vintix II
REGENT
Vintix (Offline)
Industrial Benchmark
Meta-World
Bi-DexHands
MujoCo
Kinttix
0.75
1.00
0.15
1.05
0.35
Normalised Score IQM (95% C1)
0.60
0.07
0.31
1.02
0.30
0.93
0.98
0.23
0.15
-0.17
0.19
0.00
0.90
0.25
0.95
0.15
0.5
2.5 45
0.5
2.5
0.5
0.5
25 45
0.5
2.5 45
HumEnv
MetaDrive
SinerGym
ControlGym
CityLearn
0.15
1.05
1.05
1.05
0.80
0.12
1.01
0.09
1.02-
0.97
1.02
0.78
0.06
0,98
0.93
0.98
0.77
0.03
0.89
0.00
0,95
0.85
0.95
0.75
0.5
2.5
45
0,5
2.5
45
0.5
2.5
0.5
2.5
45
0.5
25
45
Number of expert transitions (x10²)
Figure 5: Normalized returns vs. number of demonstrations Offline evaluation is conducted on 46 held-out tasks with task-specific prompts varying size. Results for Vint and REGENT are reported the corresponding domains. IQM aggregation is applied across random seeds.
domains. On Meta-World, our model scales slightly faster than REGENT; however, it constrained by a context length of 4096, which limits comparisons under larger demonstration budgets.
Overall, observed results suggest that augmenting the context with additional data boosts model's performance, or at least does not degrade it in cases where only a few demonstrations are sufficient for successful task completion.
5 CONCLUSION
In this work, demonstrated that the Decision Pretrained Transformer paired with flow-based generative policy head provides a strong framework for scaling In-Context Reinforcement Learning across multiple diverse domains. Our model exhibits substantial capabilities both in zero-shot evalu- ation and when provided with few demonstrations, and to our knowledge, it the first Large Action Model that successfully operates in both regimes simultaneously. Furthermore, in contrast to Srid- har et al. (2025), which relies on a semi- cemi-parametric approach with additional retrieval modules, our agent is deployed in fully parametric setting with fewer design choices the inference stage. The 3.2x expansion of the dataset collected by Polubarov et al. (2025) enhances large-scale cross-domain training while keeping it compatible with major ICRL approaches, bringing research commu- nity closer to creating generalist systems that support cross-domain knowledge transfer (Beck al.)
2025
Moeini et al. 2025).
While current results promising. it should be noted that with the expanded dataset, model trained with a token-to-paramete ratio of less than one. Recent work on scaling laws for large foundation models reports an optimal ratio of around 20 tokens per parameter (Hoffmann al.
2022 Besiroglu et al. 2024). This underscores the importance of further scaling training datasets and highlights the need for comprehensive studies of scaling laws for large action models. Another limitation lies in demonstration-les. evaluation, which still lags behind prompted runs, suggesting that although ICRL models strong in exploitation, their test-time exploration capabilities remain limited. In addition, the challenge of developing action models that are agnostic to the input-output dimension remains open, restricting current models from transferring to entirely unseen domains and limiting their applicability in practical scenarios.
REPRODUCIBILITY AND LLM STATEMENT
To ensure reproducibility our results, we provide implementation details in Appendix B and Appendix E Additionally, source code and datasets will be made available to reviewers during the

---

Published as a conference paper at ICLR 2026
rebuttal process, they require polishing for improved accessibility. LLMs (specifically ChatGPT) were used solely to refine the manuscript.
ACKNOWLEDGMENT
This work was supported by the Ministry of Economic Development of the Russian Fed- eration (Agreement No. 139-10-2025-034 dated June 19, 2025, unique identifier IGK 00000C3313923PDD000).
REFERENCES
Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron Courville, and Mare G Bellemare. Deep reinforcement learning at the edge of the statistical precipice. Advances in Neural Informa- tion Processing Systems, 2021.
Ankesh Anand, Jacob Walker, Yazhe Li. Eszter Vértes, Julian Schrittwieser, Sherjil Ozair, Théophane Weber, and Jessica B. Hamrick. Procedural generalization by planning with self- supervised world models, 2021. URL http:::t/ttxii/t/ee
abs///////// /2111. 01587
Shuai Bai, Keqin Chen. Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaobai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL https://arxixv org. /ttp/b////00..... 13922
Jacob Beck, Risto Vuorio, Evan Zheran Liu, Zheng Xiong, Luisa Zintgraf, Chelsea Finn, and Shi- mon Whiteson. A tutorial meta-reinforcemmnt learning. Foundations and Trendy@ in Ma- chine Learning,
2025, ISSN 1935-8245.
doi: 10.1561/22000000800 URL
http: doi. org.
/ 10t..111162220006660000
Tamay Besiroglu, Ege Erdil, Matthew Barnett, and Josh You. Chinchilla scaling: A replication attempt, 2024. URL
hatp://b///400000 10102
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Lucy Xiaoyang Shi, James Tanner, Quan Vuong, Anna Walling, Haohuan Wang, and Ury Zhilinsky. mo: A vision- language-action flow model for general robot control, 2024. URL httpt::t///// httpps:///arxivi// org/
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym, 2016.
01540
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choro- manski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, Pete Florence, Chuyuan Fu, Montse Gonzalez Arenas, Keerthana Gopalakrishnan, Kehang Han, Karol Hausman, Alexan- der Herzog, Jasmine Hsu, Brian Ichter, Alex Irpan, Nikhil Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Lisa Lee, Tsang-Wa Weng Edward Lee, Sergey Levine, Yao Lu, Henryk Michalewski, Igor Mordatch, Karl Pertsch, Kanishka Rao, Krista Reymann, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Pierre Sermanet, Jaspiar Singh, Anikait Singh, Radu Soricut, Huong Tran, Vincent Vanhoucke, Quan Vuong. Ayzaan Wahid, Stefan Welker, Paul Wohlhart, Jialin Wu, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-2: Vision- Laiion-languag-acttion models transfer web knowledge to robotic control, 2023. URL https: /ttp://aaxivvvv/ wtwp:/v.....wbba.aaeeeeeee 15818
Daniel S. Brown, Wonjoon Goo, and Scott Niekum. Better-than-demonse imitation learning via automatically-rankee demonstrations, 2019. URL httppp///aaxxwwoo///bbb////////
03:76
10

---

a Published as a conference paper at ICLR 2026
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sustry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Herbert-Voss, Gretchen Krueger, Tom Hemighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Bemnr, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners, 2020. URL https://arxiv. org. abs 2005 14165
Alejandro Campoy-Nieves, Antonio Manjavacas, Javier Jiménez-Rabsso, Miguel Molina-Solana, and Juan Gómez-Romero. Sinergym virtual testbed for building energy optimization with reinforcement learning. Energy and Buildings, 327:115075, January 2025. ISSN 0378-7788. doi: 1tt.0///..ebuild22000 115075. URL URLhttp: htwp:ttp//// doi http:////...........n...iiiiiii. 2024. 115075
Weiqin Chen, Xinjie Zhang, Dharmashankar Subramanian, and Santiago Patemnin. Filtering learn ing histories enhances in-context reinforcement learning, 2025. URL hRttht::s///axxi
abs 2t5//2555..111
Yuanpei Chen, Tianhao Wu, Shengjie Wang. Xidong Feng, Jiechuang Jiang, Stephen Marcus McAleer, Yiran Geng, Hao Dong, Zongqing Lu, Song-Chun Zhu, and Yaodong Yang. To- wards human-level bimanual dexterous manipulation with reinforcement learning, 2022. URL
httts::/aaxiv/ org/ abs 220. 08686
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion, 2024. URL
https: wwtp///////..... 2303
Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis, 2021. URL
https://arxin org/ abs/ 05233
Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, and Vahid Tarokh. In-context re-
inforcement learning from suboptimal historical data, 2025. URL
1707
Ron Dorfman, Idan Shenfeld, and Aviv Tamar. Offline meta reinforcement learning identifiability challenges and effective data collection strategies. In A. Beygelzimer, Dauphin, Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, 2021. URL httts:://ppenree
Yan Duan, John Schulman, Xi Chen, Peter L. Bartlett, Ilya Sutskever, and Pieter Abbeel. RI:: Fast reinforcement learning via slow reinforcement learning, 2016. URL htttt:://axxiv..t/
abs/1611 02779
Ahmad Elawady, Gunjan Chhablani, Ram Ramrakhya, Karmesh Yadav, Dhruv Batra, Zsolt Kira, and Andrew Szot. Relic: A recipe for 64k steps of in-context reinforcement learning for embodied ai, 2024. URL https:/ //tp://aaivvvv/./////i/.../.a.va org http//bb///22.22222/
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion En- glish, Kyle Lacey, Alex Goodwin, Yannik Marek, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis, 2024. URL
2403. 03206
Jesse Farebrother, Matteo Pirotta, Andrea Tirinzoni, Rémi Munos, Alessandro Lazaric, and Ahmed Touati. Temporal difference flows, 2025. URL httphhttt////tttaae http:/awwwi.oogga////aaaaaa /2503. 09817
Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation emror in actor-critic methods, 2018. URL
Quentin Gallouédec, Edward Beeching, Clément Romac, and Emmanuel Dellandréa. Jack of all trades, master of some, a multi-purpose transformer agent, 2024. URL https: brgg/bbs/240 240.b 09844
11

---

Published as a conference paper at ICLR 2026
Jake Grigsby, Linxi Fan, and Yuke Zhu. Amago: Scalable in-context reinforcement learning for adaptive agents, 2024a. URL http://///www..oooo////////////
Jake Grigsby, Justin Sasek, Samyak Parajuli. Daniel Adebi, Amy Zhang, and Yuke Zhu. Amago-2 Breaking the multi-task barrier in meta-reinforcement learning with transformers, 2024b. URL
httpp::/arae www./aaass....oo. al// /21 11 11188
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforeement learning with a stochastic actor, 2018. URL https:
//arxiv. waaxiv.crg/absa....... http:/abs/11100000
801 01290
Haoran He, Chenjia Bai, Kang Xu, Zhuoran Yang, Weinan Zhang. Dong Wang, Bin Zhao, and Xue- long Li. Diffusion model is an effective planner and data synthesizer for multi-task reinforcement learning, 2023. URL httttss://axxx....////p
2305. 18459
Daniel Hein, Stefan Depeweg. Michel Tokic, Steffen Udluft, Alexander Hentschel, Thomas A. Run- kler, and Volkmar Sterzing. A benchmark environment motivated by industrial control problems. In 2017 IEEE Symposium Series Computational Intelligence (SSCI), pp. 1-8. IEEE, Novem- ber 2017. doi: 10.11099sci2221178888999999 URL /ttp://////////......o.oo////./.
201 8280935
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models, 2020. URL
https://arae //rxivv www./or.///ooooooooooooooooo.sss 2200 11239
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hen- nigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, Erich Elsen, Jack W. Rae, Oriol Vinyals, and Laurent Sifre. Training compute-optimal large language models, 2022. URL 2203. 15556
org/bb//
Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Manuel Y. Galliker, Dibya Ghosh, Lachy Groom, Karol Hausman, Brian Ichter, Szymon Jakubezak, Tim Jones, Liyiming Ke. Devin LeBlanc, Sergey Levine, Adrian Li-Bell, Mohith Mothukuri, Suraj Nair, Karl Pertsch, Allen Z. Ren, Lucy Xiaoyang Shi, Laura Smith, Jost Tobias Springenberg, Kyle Stachowicz, James Tanner, Quan Vuong, Homer Walke, Anna Walling. Haohuan Wang, Lili Yu, and Ury Zhilinsky. To.s: Dision-language-actee action model with open-world generalization, 2025. URL URL htttt::///aart brgg/as// 2504. 16054
Michael Janner, Yilun Du, Joshua B. Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis, 2022. URL httpp::://aaxiv.or///a//////////
Vanshaj Khattar and Ming Jin. Winning the cityleam challenge: Adaptive optimization with evo- lutionary search under trajectory-bast -based guidance, 2022. URL
abs/
2212. 01 939
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, Quan Vuong. Thomas Kollar, Ben- jamin Burchfiel, Russ Tedrake, Dorsa Sadigh, Sergey Levine, Percy Liang, and Chelsea Finn. Openvla: An open- source assionllaguaageacttio action model, 2024. hRtpstt/://axiii abs/2406. 09246
Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, and Volodymyr Mnih. In-context reinforcement learning with algorithm distilla- tion, 2022. URL
http://awwxx...coa////////// 14215
Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Supervised pretraiingg can learn in-context reinforcement learning, 2023. URL
https::/arxiv.com http:///wxivv..oor///////// 14892
12

---

Published as a conference paper at ICLR 2026
Quanyi Li, Zhenghao Peng, Lan Feng, Qihang Zhang, Zhenghai Xue, and Bolei Zhou. Metadrive: Composing diverse driving scenarios for generalizable reinforcement learning, 2022. URL
Jacky Liang. Viktor Makoviychuk, Ankur Handa, Nuttapong Chentanez, Miles Macklin, and Di- eter Fox. Gpu-accelerated robotic simulation for distributed reinforcement learning, 2018, URL
https://arxiv.orr////a org. /1100.111.. 05762
Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling, 2023, URL httpt:tttts/aiiiteeeet org/ htch:/bs//2/ 2210 -2747
Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre- train, prompt, and predict: A systematic survey of prompting methods in natural language pro- cessing, 2021. httthtts:://arrt httpa////aawii..../aaa//////////
13586
Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning generate and transfer data with rectified flow, 2022.
abs ottp//bb///99..03333
Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang, Rohun Kulkarni, Li Fei- Fei, Silvio Savarese, Yuke Zhu, and Roberto Martín-Martín. What matters in learning from of- fline human demonstrations for robot manipulation, 2021. URLhttps: //tp://axivvvvvvv org/ abs/
2108. 03298
Antonio Manjavacas, Alejandro Campoy Nieves, Javier Jiménez-Raboso, Miguel Molina-Solana, and Juan Gómez-Romero. An experimental evaluation of deep reinforcement learning algorithms for hvac control. Artificial Intelligence Review, 57, 06 2024. doi: 10.110077110622-100 11000--022--08992--
CPLEX User's Manual. Ibm ilog cplex optimization studio. Version, 12(1987-2018):1, 1987.
Michael Matthews, Michael Beukman, Chris Lu, and Jakob Foerster. Kinetix: Investigating the training of general agents through open-ended physics- based control tasks, 2025. URL https:
23208
Walter Mayor, Johan Seaano-Ceroon, Aaron Courville, and Pablo Samuel Castro. The impact of on- policy parallelized data collection on deep reinforcement learning networks, 2025. URL https:
//rxiiv.. org abs/ 2206 03404
Amir Moeini, Jiuqi Wang, Jacob Beck, Ethan Blaser, Shimon Whiteson, Rohan Chandra, and Shang- tong Zhang. A survey of in-context reinforcement learning, 2025. URL https://arxiv.
org/abs 2502. 07978
Fei Ni, Jianye Hao, Yao Mu, Yifu Yuan, Yan Zheng, Bin Wang, and Zhixuan Liang. Metadiffuser: Diffuusio model as conditional planner for offline meta-rl, 2023. URL hollhttts://arxi.ooom abs/ /200. 9923
Alexander Nikulin, Ilya Zisman, Alexey Zemtsov, and Vladislav Kurenkov. Xland-100b: A large- scale multi-task dataset for in-context reinforcement learning, 2025. URL /ttps::/a/xxxv org/ 2406 0..066.00 08973
Kingsley Nweye, Zoltan Nagy, Sharada Mobanty, Dipam Chakraborty, Siva Samkaranarayanan, Tianzhen Hong, Sourav Dey, Gregor Henze, Jan Drgona, Fangquan Lin, Wei Jiang, Hanwei Zhang, Zhongkai Yi, Jihai Zhang, Cheng Yang. Matthew Motoki, Sorapong Khongnawang, Michael Ibrahim, Abilmansur Zhumabekov, Daniel May, Zhihu Yang, Xiaozhuang Song, Han Zhang, Xiaoning Dong, Shun Zheng, and Jiang Bian. The citylearn challenge 2022: Overview, re- sults, and lessons learned. Marco Ciccone, Gustavo Stolovitzky, and Jacob Albrecht (eds.), Pro- ceedings of the NeurIPS 2022 Competitions Track, volume 220 of Proceedings of Machine Learn- ing Research, pp. 85 103. PMLR, 28 Nov-09 Dec 2022. URL https: http:a/:///coceeenngss... m Tlr.press/w223. wwwye2/e html
Kingsley Nweye, Kathryn Kaspar, Giacomo Buscemi, Tiago Fonseca, Giuseppe Pinto, Dipanjan Ghose, Satvik Duddukuru, Pavani Pratapa, Han Li, Javad Mohammadi, Luis Lino Ferreira, Tianzhen Hong, Mohamed Ouf, Alfonso Capozzoli, and Zoltan Nagy. Cityleam v2: energy- flexible, resilient, occupant-centric, and carbon-aware management of inteeiite commu- nities. Journal of Building Performance Simulation, 18(1):17-38, November 2024. ISSN
13

---

Published as a conference paper at ICLR 2026
1940-1507. doi: 10.1080/19400149..02224111111111 19401493 202.44449222222248888881
OpenAl. Gpt-4 technical report, 2024. http://ttss//aaa.....raaa/////..
Seohong Park, Qiyang Li, and Sergey Levine. Flow q-learning, 2025. URL https://arxiv.
erg/abs/2n 2502. 02538
Karl Pertsch, Kyle Stachowicz, Brian Ichter, Danny Driess, Suraj Nair, Quan Vuong, Oier Mees, Chelsea Finn, and Sergey Levine. Fast: Efficient action tokenization for vision-language-action models, 2025.
09747
Andrey Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Ilya Zisman, Denis Tarasov, Alexan- der Nikulin, and Vladislav Kurenkov. Vintix: Action model via in-context reinforcement learning, 2025. URL htt.s:t:: btttps://ariive www.ii.worg//aao /abrg////////// /2001 19400
Antonin Raffin, Ashley Hill, Adam Gleave, Anssi Kanervisto, Maximilian Ernestus, and Noah Dormann. Stable-basclines:: Reliable reinforcement learning implementations. Journal of Machine Learning Research, 22(268):1 2021. URL
13m4
Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Had- sell, Oriol Vinyals, Mahyar Bordbar, and Nando de Freitas. A generalist agent, 2022. URL
https://arxx otg/ abs 2205 06175
Zohar Rimon, Tom Jurgenson, Orr Krupnik, Gilad Adler, and Aviv Tamar. Mamba: an effective world model approach for meta-reinforcement learning, 2024. URL https::/arxiv..r/
05s/2403.09855
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https::/arxii......////a//aa wttpaaawwwwi..ra///aaaaaaaaaaa/e
06347
Mustafa Shukor, Dana Aubakirova, Francesco Capuano, Pepijn Kooijmans, Steven Palma, Adil Zouitine, Michel Aractingi. Caroline Pascal, Martino Russi, Andres Marafioti, Simon Alibert, Matthieu Cord, Thomas Wolf, and Remi Cadene. Smolvla: A vision-language-actionn model for affordable and efficient robotics, 2025. URL /tl.tttp:://rri..oorr///////////
01844
Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, and Sergey Kolesnikov. In- context reinforcement learning for variable action spaces, 2024. URL BRRLhttss:///xxivv
abs fbbs/2312.r 13327
Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics, 2015. URL https:://aariiiii org/
abs/1500 03585
Jachyeon Son, Soochan Lee, and Gunhee Kim. Distilling reinforcement learning algorithms for in-context model-basee planning, 2025. httpass:://axxii org abs///bs//// .10.590...9999 19009
Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. Regent: A retrieval- augmented generalist agent that can act in-context in new environments, 2025. URL https:
//arxiv. //axxiv.org/abs// haap://aa/aaa 2412. 04759
Richard Sutton and Andrew G Barto. Reinforcement learning: An introduction. 1998.
Denis Tarasov, Alexander Nikulin, Ilya Zisman, Albina Klepach, Andrei Polubarov, Nikita Lyubaykin, Alexander Derevyagin, Igor Kiselev, and Vladislav Kurenkov. Yes. q-learnin helps offline in-context rl, 2025. /tt.nttpp:://arvv..rrr/////
17666
Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens con- 05530
text, 2024. /
14

---

Published as a conference paper at ICLR 2026
Octo Model Team, Dibya Ghosh, Homer Walke, Karl Pertsch, Kevin Black, Oier Mees, Sudeep Dasari, Joey Hejna, Tobias Kreiman, Charles Xu, Jianlan Luo, You Liang Tan, Lawrence Yun- liang Chen, Pannag Sanketi, Quan Vuong, Ted Xiao, Dorsa Sadigh, Chelsea Finn, and Sergey Levine. Octo: An open-source generalist robot policy, 2024. URL htttps://aaxiv..o abs/2405 12213
Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother, Mateusz Guzek, Anssi Kanervisto, Yingchen Xu, Alessandro Lazaric, and Matteo Pirotta. Zero-shot whole-body humanoid control via behav- ioral foundation models, 2025. URL. httttttst:////rrxt. http:/awwwv.rr...aa.aaooooo abs// 2504 11054
Emanuel Todorov, Tom Erez, and Yuval Tassa. Mujoco: A physics engine for model-based control.
In 2012 IEEE/RSJ International Conference Intelligent Robots and Systems, pp. 5026-5033, 2012. doi: 10100//R0..2022..3888
Mark Towers, Ariel Kwiatkowski, Jordan Terry John U. Balis, Gianluca De Cola, Tristan Deleu, Manuel Goulão, Andreas Kallinteris, Markus Krimmel, Arjun KG, Rodrigo VicentVe An- drea Pierré, Sander Schulhoff, Jun Jet Tai, Hannah Tan, and Omar G. Younis. Gymnasium: A standard interface for reinforcement learning environments, 2024. URL https://arxtt
org/ 2407. 17032
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones. Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need, 2023. URL https:
org/abs/ 1706. 03762
Jianhao Wang, Jin Zhang, Haozhe Jiang, Junyu Zhang, Liwei Wang, and Chongjie Zhang. Offline meta reinforcement learning with in-distribution online adaptation, 2023a. URL https://
orxiv.org/abs/2220 19529
Junyang Wang, Haiyang Xu, Haitao Jia, Xi Zhang, Ming Yan, Weizhou Shen, Ji Zhang, Fei Huang. and Jitao Sang. Mobile-agent-v2: Mobile device operation assistant with effective navigation via multi-agent collaboration, 2024a. URL htttt::: org/ abs 2406. 01014
Yinuo Wang, Likun Wang, Yuxuan Jiang, Wenjun Zou, Tong Liu, Xujie Song, Wenxuan Wang, Liming Xiao, Jiang Wu, Jingliang Duan, and Shengbo Eben Li. Diffusion actor-critic with entropy regulator, 2024b. httphhttt///axxi/t/
15177
Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies as expressive policy class for offline reinforcement learning, 2023b. URL
061 93
Junjie Wen, Yichen Zhu, Jinming Li. Minjie Zhu, Kun Wu, Zhiyuan Xu, Ning Liu, Ran Cheng, Chaomin Shen, Yaxin Peng, Feifei Feng, and Jian Tang. Tinyvla: Towards fast, data-efficient vision-language-action models for robotic manipulation, 2025. httpshtts///axxi..oom
abs/2409.125
Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Avnish Narayan, Hayden Shively, Adithya Bellathur, Karol Hausman, Chelsea Finn, and Sergey Levine. Meta-world: A benchmark and evaluation for multi-tast task and meta reinforcement learning, 2021. htthhttps://axii.. org/
abss1990.. 10897
Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model, 2024a.
Shiyuan Zhang, Weitong Zhang, and Quanquan Gu. Energy-weighted flow matching for offline reinforcement learning, 2025.
Xiangyuan Zhang, Weichao Mao, Saviz Mowlavi, Mouhacine Benosman, and Tamer Başar. Con- trolgym: Large-scale control environments for benchmarking reinforcement learning algorithms, 2024b.
http///wwvvvvog..eee///// abs/ 2311. 8736
Zhengbang Zhu, Hanye Zhao, Haoran He, Yichao Zhong, Shenyu Zhang, Haoquan Guo, Tingting Chen, and Weinan Zhang. Diffusion models for reinforcement learning: A survey, 2024. URL ittts:://arxte otg/abs//////ooooooooooooooo 211233
15

---

Published as a conference paper at ICLR 2026
Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, and Sergey Kolesnikov. Emergence of in-context reinforcement learning from noise distillation, 2024. URL https:
watxiv.crg/abs/23330.. 12275
Iyy Zisman, Alexander Nikulin, Viacheslav Sinii, Denis Tarasov, Nikita Lyubaykin, Andrei Pol- ubarov, Igor Kiselev, and Vladislav Kurenkov. N-gram induction heads for in-context rl: Im- proving stability and reducing data needs, 2025. URL https: /ttess//axxivvi org/ bbbrgg/bb//4111
01958
A BACKGROUND
Reinforcement Learning Reinforcement learning (RL) (Sutton & Barto (1998) formalizes quential decision making as a Markov Decision Process (MDP) (S. p. r. Po. 7). Here S and A denote the state and action spaces, p(s' ( | Po is the initial-state distribution, and y E [0, 1) is a discount factor. A policy π (a
is the transition kernel, r(s, a) is the reward function,
induces trajectories (so, do: F1; 81,-.-) with so Po- actions sampled from .t.- st). and next states drawn from p(- Sr, at). The standard objective is to find a policy that maximizes the expected discounted return
(
7 7T++1
Classical RL methods estimate value functions and improve policies through repeated interaction with fixed environment. In this setting a separate policy is typically optimized for a single MDP, so even moderate changes in the reward function transition dynamics tend to degrade performance and often require retraining, rather than yielding an agent that robustly operates across different dynamics.
In-Context Reinforcement Learning Unlike standard RL, ICLR refers to agents that are capable of learning during inference. Such agents are conditioned not only on the current state 8,, but also on some context C, which helps the model understand the dynamics of the environment and maximize reward through a trial-and-error paradigm. Prominent representatives
such models are AD and DPT (Laskin et al. (2022): Lee al. (2023)). ) Both models are capable not only of solving training tasks, but also of adapting to unseen dynamics.
DPT's main advantage is that its behaviour during inference can be interpreted Bayesian posterior sampling. To achieve this, the authors pretrain a transformer Mp Squery; Dý) on an offline dataset of trajectories using a simple supervised objective: given query state Squery and in-context dataset of past transitions D; = {(1,, a1, 81, ri), (8j, aj aj, ry,,,))
the model is trained to predict the optimal action a":
(()) ED
[(Mo(-
Squary, a)),
jEIm
where 1-Ma(-) Squerry Dj), a²) mize C(0).
log Ma(a" Squery: Dj). Parameters are optimized to mini-
The authors evaluate the model in two regimes: offline and online. In the offline regime the context D, is provided in advance, while in the online setting D; is extended with the interaction history. Sampling actions from the output distribution Mo(- Squery D,) leads to behaviour that combines efficient online exploration with conservative offline performance. This is why we focus on these algorithms in our work. A sparse training dataset that consists of many different tasks from different domains may further improve performance and help to build a strong large action model within the ICRL framework.
Flow Matching Flow matching (Lipman et al. (2023) Liu et al. (2022) is a widely used frame-
work for building generative models. It offers simple training and fast inference, while being capable of modeling complex distributions. These properties have led to the popularity of flow matching in various continuous-control applications (Park et al. (2025): Zhang et al. (2025): Black et al. (2024).
16

---

Published as a conference paper at ICLR 2026
Suppose target distribution p(x:) the d-dimensional space I is given. Consider ODE de (ar, E) with initial condition
zo. where the velocity field is Rd [0, RRd The flow db : R2d x [0.1] + _d is the collection of solutions of this ODE such that, for any 2o, we have (r 0) = zo and 4
z vb 1) = (x 1), 1). The goal of flow matching is to fit a velocity field (a(2, t) with the corresponding flow (2, t) that transports an initial simple Gaussian distribution N(0, Ta) at = 0 to the target distribution p(a)) at tt
In this work we consider rectified flow matching (Liu et al. 2022) ) as it provides a particularly simple training objective. Given a pair (Zo. with Zo NO, Ia) and ~ p(z), and time t UO, 1), the linear interpolation is defined by 2, = (1 t) 2o + tzz The loss is
(()) P= 0i pi-p(a)) tounf(()))
4) (a1 - -
Once is trained, samples from p(a) can be obtained by first sampling zo from (0, La) and then numerically solving the ODE up
B ALGORITHMS
B.
TRAIN STEP
Algorithm Training step
Require: causal transformer T3; encoders do Pas Ør: flow head Uyi learnable BOS embedding 3;
learnable frequency vector f
1: Sample a minibatch {(og') c(6), )
1..B do 2: for (.(0) T0.L++1 < [8, (a ).)). 0),
3:
4: end for
5: 1o.1+1 &- (T
6: for b = 1..B do
7: Sample Fo, ~ (0. Ia), + (1) for 1..L
8: It + (1 t) 1
(b) (b)
+ 4
(h)
(b)
9:
of (t,c,x²) t un (tt 1, 2iz)
10: end for
CRP Eb ||b(0) (b)) ( )
B.2 INFERENCE STEP
Algorithm Inference step
Require: causal transformer To: encoders do. da. òr: flow head vy; BOS embedding b Require: C' (might be empty), query observation og
1: Fo.L + [b, cat(óo(o,, O, 0), {cat(do(o;) (ri))))
2: ho:L < (o(:).))
3: Sample Zo ~ (0, Ia)
4: x + Tn
5: Al +- 1/M
6: for m = O..M 1 do
7:
t+ mát
8:
kl+ +
h1, x)
9:
12 D (n((tt/M At), RL, &. + k1 At.)
f+ (k1+k2)
10:
11: end for
12: return ( <-
17

---

Published as a conference paper at ICLR 2026
C DATASET DETAILS
C.I GENERAL INFORMATION
MuJoCo. MuJoCo Todorov et al. 2012) is physics engine developed for multi-joint continuous control tasks. For this study, we selected standard continuous control environments from OpenAI Gym (Brockman et al. 2016) and Gymnasium (Towers al. 2024).
Meta-Worid. Meta-World (Yu 2021) is an open- open-souree robotic manipulation benchmark de- signed for meta-reinfore meta-reinforrement learning and multitask learning, containing 50 distinct tasks. Unlike AMAGO-2 (Grigssyy et al. 2024b), which uses full 500-transition rollouts, we limited the maximum episode length to 100 timesteps, as in JAT (Gallouédec et al. 2024). This adjustment increases the context size, measured in terms of the number of episodes, without compromising task completion, as the shorter horizon remains sufficient to solve all tasks in the benchmark. Additionally, we fixed the goal state between environment resais-changing it only after re-initialization therefore ensuring compatibility with traditional memory-based Meta-RL formulations.
Bi-DexHands. Bi-DexHands (Chen et al. 2022 provides set of bimanual manipulation tasks specifically designed for experiments in RL, MARL, offline RL, multi-task task RL, and Meta-RL. The benchmark includes 20 complex high-dimensional continuous control tasks. Some subsets of tasks share the same action and observation space structures, making the environment suite compatible with the Meta-RL -L framework.
-erhhnark-Bnchmark (Hein et al. 2017) is synthetic RL benchmark designed to simulate real-world industrial applications, such as controlling gas and wind turbines. The environment's 's transition dynamics and stochasticity can be adjusted using the setpoint param- eter p. By increasing this parameter from to 100 in steps of 5, we obtained 21 tasks sharing a unified state and action structure. We selected the standard reward, which was subsequently scaled down by a factor of 100.
Industrial-Benchmarkk
Kinetix. Kinetix (Matthews et al. 2025) is an open-ended 2D physics-based control enviromment built the harelarre-cceelerattedd Jax2D engine. It integrates wide range of procedurally generated tasks including locomotion, manipulation, and video video-game -aoe--aame--ee like cemaariss-ithhe a single frame- work. Tasks are defined by variations in objects, goals, and dynamics, and the authors distinguish three task scales-a m, and 1-to categorize different levels of complexity. The reward structure standardized: agents are incentivized to bring a "green shape" into contact with a "blue shape" and penalized for collisions with "red shapes," with auxiliary dense reward based on inter-shape distance. To ensure that the model infers the reward function implicitly from the provided context, color information omitted from the observation.
CearLearn. CityLearn Nweye et al., 2024 is an open-source reinforcement learning framework that provides multiple environments for long-tem term management of building energy resources and demand response urban settings. includes 24 tasks, each defined by continuous action and observation spaces with dimensionalities that vary according to the simulation configuration and the number of buildings. In this study, however, only the Phase and Phase 2 tasks from the CityLearn 2022 Challenge (Khattar & Jin (022) were considered. Both tasks simulate year of operational electricity demand and photovoltaic (PV) generation data from five single-family buildings in the Sierra Crest housing development in Fontana. To expand the of environments and enable the model to process multiple episodes within its context, these tasks were further divided into 24 monthly tasks.
HumEnv. HumEnv (Tirinzoni et al. 2025) is a humanoid locomotion environment built on Mu- JoCo (Todorov et al.) 2012). It was originally developed for training and evaluating Meta's Meta- Motivo model. The benchmark includes three categories Tracking tasks require following a sequence of target observations, Goal tasks involve reaching designated state (e.g., standing with hands raised), and Reward tasks consist of performing a spe- cific activity (e.g., running, lying down). Although the tate-action space remains identical across
tasks: Tracking, Goal, and Reward.
18

---

Published as a conference paper at ICLR 2026
all tasks, the reward functions differ. The environment offers broad set of predefined tasks in high-dimensional setting, making well-suited for our study.
Sinergym. Sinergym (Campoy-Nieves et al. 2025) a flexible benchmark designed for training reinforcement learning (RL) algorithms to reduce energy costs while maintaining indoor tempera- tures within a fixed range by managing energy-consuming devices such as fans, coolers, and heaters. By default, the framework provides an interface to simulate energy consumption across eight model buildings, each evaluated under three different weather profiles. The observation and action spaces defined within a stochastic continuous domain. To shorten episode length, the default annual timeline is partitioned into months, which are grouped such that neighboring months are included task. In some cases, however, months were separated because energy consumption
within the depended more strongly on temporal factors than on model actions. Furthermore, certain combina- tions of building models, weather profiles, and months generated excessive noise and were therefore excluded.
MetaDrive. MetaDrive (Li et al. 2022) is a reinforcement learning (RL) suite of environments designed simulate autonomous driving. In this setting, roads, obstacles, and other vehicles generated randomly. MetaDrive provides a wide range of tasks, as its road segments can be rear- ranged in numerous configurations, substantially altering both the road layout and the difficulty level for the agent. However, the state and action spaces remain fixed.
ControlGym. ControlGyy (Zhang et al. 2024b an open-source gym providing environments that range from linear systems chaotic, acaee systems governed by partial differential equa- tions (PDEs). It offers extensive parameterization to facilitate evaluation under conditions that ap- proximate real-world applications. The benchmark includes 46 continuous-control tasks, each with continuous action spaces and latent (hidden) states and observations of varying dimensionaliiy. In this study, we focus on a subset tasks: two linear tasks -Aircraft (i){ and Cable (cm and two linear PDE tasks convection diffusion reaction and schrodinger.
C.2 TRAIN vs. TEST TASKS SPLIT
To evaluate model's capabilities for inference-time optimization, divided the full set of 255 tasks into two disjoint subsets. The validation subset contained 46 tasks, which were separated from the training dataset of 209 tasks. Below, we provide the details of the split for each domain.
C.2.1 MuJoCo
Performance in the MuJoCo domain was evaluated on locomotion tasks with modified physical parameters altered through the provided XML API. For each task, we adjusted viscosity from 0 to 0.05 and 0.1, while gravity was varied by #10 percent.
C.2.2 WORLDWORLD
We adopted the commonly used ML45 train-test test split for Meta-RL setting, which includes 45 train- ing tasks and 5 tasks reserved for validation: bin-picking, box-close, door-lock, door-unlock, and hand-insert.
C.2.3 BI-DEXHANDS
The ML20 benchmark, proposed in the original paper (Chen et al., 2022). was selected as the train-test split. It assigns 15 tasks to the training set, with the remaining reserved for validation: door-close-outward, door-open-inward door-open-outward, hand-kettle, and hand-over. However, the hand-over task has a unique state-action space dimensionality that is not represented in the training set, making it incompatible with the multi- head encoder architecture. To ensure correct cal- culation of quality metrics for this domain, we conservatively report a performance (random) for this task.
19

---

Published as a conference paper at ICLR 2026
C.2.4 INDUSTRIAL- DOUSTRIAL-BENCHHARKE BENCHMARK
Training and testing tasks for Industrial- Benhhmarrk were obtained by dividing the environments into two non-overlapping subsets based on the setpoint parameter: values from 0 to 75 defined the training set, while setpoints from to 100 were assigned to the validation set.
C.2.5 KINETIX
Since we obtained generalist agents only for tasks from the Kinetix test split (whereas the Kinetix training set consists of large collection of randomly generated tasks), we could not adopt the train-test split described in the original paper (Matthews et al. 2025). Instead, we designated six tasks for evaluation: h8 h8_ www.uuccieeellleee balance (S), arm_hard and h7 4hruuststblock (M), and mjc_walker, hard_lunar Jander, and pinball Lard (L). The remaining predefined tasks were assigned to the train- ing set. The test split was selected to enable meaningful evaluation across scales: S includes a non- trivial balancing task, M involves environments requiring precise control and adaptability without color information, and I. consists of tasks demanding advanced movement strategies and obstacle navigation. Overall, this selection emphasizes tasks that challenging, non-trivial, and well-suited for assessing generalization and transfer.
C.2.6 CITYLEARN
Since predefined split exists for individual CityLearn tasks (Nweye et al. 2024). the last two monthly tasks from both Phase and Phase were assigned to the validation set, yielding train- ing and validation sets of 20 and tasks, respectively. This splitting strategy appropriate for time-series data, as it enables effective evaluation of the ability of in-context reinforcement learning algorithms to generalize varying dynamics.
C.2.7 HUMENV
In the original study (Tirinzoni et al. 2025). only Reward and Goal tasks were used for evaluation, while training was conducted exclusively Tracking tasks, line with MetaMotivo's architecture, which was designed to learn state representations. Based on task diversity and expert convergence, we selected 10 Reward tasks and Goal tasks for training and reserved tasks for testing: t-pose (Goal), rotate-x-5-0.8 (Reward), and split-0.5 (Reward).
C.2.8 SINERGYM
Following the approach proposed by (Majavacas al. 2024), training and testing tasks were de- fined using different weather conditions for the same building types. Specifically, tasks with hot or cool weather profiles were assigned the training set, while mixed-weathn tasks were allocated to the test set. The final split consisted of 22 training tasks and 11 testing tasks.
C.2.9 METADRIVE
The original paper LLi al. 2022) did not specify train/test split; therefore, we defined one. In total, we selected 21 tasks: 16 for training and 5 for testing. The MetaDrive environment generates maps using various types of road segments, including straight sections, curves, circles, and sharp turns. Since the exact configuration of each segment (e.g., curvature placement) changes with ev- ery reset, we applied seeding to ensure that each task remained deterministic. However, the random selection of the lane in which the agent spawns preserved.
C.2.10 CONTROLGYM
ControlGym (Zhang et al.) 2024b) does not provide predefined train-test split by default. We therefore propose split based on (I) state complexity (dimensionality) and (2) the robustness of demonstrator performance in terms of total reward. Specifically, three of the 12 environments were selected for validation: acl, cm5, and schrodinger. When constructing the test set, prioritized en- vironments that have counterparts with the same obtion-observattin space remaining in the training set, enabling meaningful assessments of transfer.
20

---

Published as a conference paper at ICLR 2026
D DEMONSTRATORS
D.1 TRAINING
MuJoCo. For MuloCo (Todorov et al.) 2012), demonstrators were trained using behavioral cloning on the dataset provided by JAT (Gallouédec et al.) 2024). The resulting demonstrators achieved performance levels consistent with those reported in the JAT paper.
Meta-Worid. We used trained agents open-soureed by Gallouédec et (2024) as demonstrators for the Meta-World (Yu et al., (02T) benchmark. However, Turther analysis revealed that some agents achieved unsatisfactory success rates. To address this issue, we retrained demonstrators for the following tasks: disassemble, coffee-pull, coffee soccer, push-back, peg-insern sise-rs-r---de, and pick-out-of-hole For efficient fine-tuning and hyperparameter search, we used the training scripts provided by JAT (Gallouédec et al. 2024). Re-training improved performance on the aforemen- tioned tasks, although some demonstrators continued to exhibit unstable results.
Bi-DexHandd. PPO (Schulman al., 2017) implementation provided by the authors (Chen al. 2022) was used to obtain demonstrators. We increased the number of parallel IsaacGym (Liang al. 2018) environments from 128 2048 improve convergence, following recent empirical evidence (Mayor et al. 2025). For certain tasks, namely Re-Orientation and Swing-Cup, we were able to substantially exceed the performance reported the original work. However, for some environments, the RL agents continued to exhibit stochastic performance even after training for over 1.5 billion timesteps.
Industrial-Benehhe Benchmark. The implementation of PPO from was used to train demonstrators. For all runs, we applied advantage normalization, set a KL- divergence limit of 0.2, used a discount factor of 0.97, 2500 environment steps, batch size of 50, and a training duration of 1 million timesteps. To increase episodic context size and facilitate score comparability, we limited the episode length to 250 transitions. Additionally, we used delta- rewards for training our RL agents, this reward formulation demonstrated superior performance compared to agents trained with scaled classic rewards, based on validation results for both types of returns.
(Raffin et al., 2021)
Kinetix, For each task scale (s, m, and I). we first trained generalist agent using PPO (Schulman 2017) with entity-type observations. The agents for scales S, M, and L were trained for IB, 2B, and 3B timesteps, respectively, with a batch size of 12288 and 512 rollout steps. Hyperparameters were further tuned to increase success rates on procedurally generated random morphologies during training. Each generalist agent was then fine-tuned to solve individual tasks from the Kinetix test split. This process yielded reliable demonstrators for 48 test tasks across all scales. During data collection, observations were stored as real-valued vectors, resulting in three groups corresponding to scales S, M, and L.
CityLearn. Demonstrators were trained using the Linear Programming (LP) reduction method proposed by the winning team of the CityLeean 2022 Challenge, Team Together (Nweye et al.) 2022]. The CPLEX solver from IBM ILOG (Manual) 1987) was used to compute the optimal monthly dispatch, which converged rapidly for each task. For both the Phase and Phase 2 task sets, the average performance the demonstrators matched or exceeded the best scores reported on the challenge leaderboard. When evaluated against the demonstrators monthly basis, the top three challenge solutions achieved comparable or lower scores.
HumEnv. All demonstrators were trained using TD3 (Fujimoto et al. 2018), with MLPs serving as the Actor and Critic in accordance with the original configuration (with one fewer layer for certain tasks). Each network was trained for 30 million steps, consistent with the original paper. The resulting rewards were generally close to those reported in (Tirinzoni et al. 2025) across different tasks. The humanoid's ' initial state was determined by a set of motions and a fall probability, with fall positions sampled defined in the authors' repository. Consequently, the initial state was either a fall or a random frame taken from a randomly selected motion. In all models, the original test motion set was used the initial one, with fall probability of 0.2.
21

---

Published as a conference paper at ICLR 2026
Sinergym. As baseline algorithms, SAC (Haanoja al. 2018 TD3 (Fujimoto et al.) 2018). and PPO (Schulman et al., 2017) were taken from the and trained on each task using the global parameters reported by (Campoy-Nieves et al. 2025 Each model was trained for approximately 0.5 million time steps per task. Overall, the three algorithms demonstrated comparable performance, but PPO achieved higher rewards greater number of tasks. Owing to its stability and consistent results, PPO was selected as the primary demonstrator for the Sinergym tasks.
framework (Rafiin et al. 2021
MetaDrive. For our demonstrators, we used the expert policy provided by the authors (Li et al., 2022], which was trained with PPO (Schulman et al. 2017) on the generalization environment that procedurally generates road layouts in MetaDrive. This policy was subsequently fine-tuned on circle maps to improve performance. Since the PPO policy could not reliably avoid static obstacles, we adopted the authors' rule-based algorithm for maps containing such obstacles. Accordingly, three demonstrators were used: (1) PPO policy fine-tuned for 10 million steps on circle maps, achieving 95% success rate; (2) PPO policy fine-tuned for 10 million steps maps without circles obsta- cles, achieving a 98% success rate; and (3) the rule-based algorithm, which achieved 98% success rate maps with obstacles.
ControlGym. For each task, we selected demonstrator from three built-in heuristic controllers. These controllers access the environment's unobservable internal state to select actions, and the choice for each task was based on empirical performance. For most tasks, we used the H2 Ho controller derived from the block generalized algebraic Riccati equation (GARE) (Zhang et al.) 2024b). Exceptions include the linear PDEs, for which used LQR because H ible with these settings; acl and ac6, for which we used LQR and LQG, respectively, both based on discrete-time state-space models (Zhang et al. 2024b). We modified Hz/H>c to Moore-Penrose Penrose pseudoinverses when matrix inverses were singular or ill-conditioned. Nevertheless, none of the controllers achieved stable performance on ac3, ac7, ac8, ac9, and ac10; these environments were therefore excluded from the target task set. We also trained PPO (Schulman et al.) (017) and SAC (Haarooj al. 2018) on these tasks, but neither achieved stable performance comparable to LQR or LQG.
H incompat-
H
E HYPERPARAMETERS
Hyperparameter
Value
Action Decoder Steps
32
Learning Rate
0.00005
Gradient Clipping Norm
2.5
Optimizer
Adam
Beta 1
0.9
Beta 2
0.99
Batch Size
64
Gradient Accumulation Steps
1
Transformer Layers
16
Transformer Heads
24
Context Length
4096
Transformer Hidden Dim
1536
FF Hidden Size
6144
MI.P Type Normalization Type
GptNeoxMI.P
LayerNorm
Training Precision
bfl6
Parameters
928053513
Table 2: Hyperparameter configuration
22

---

Published as a conference paper at ICLR 2026
F ACTION BELIEFS
Guce--pen--2
0:50
0.00
-050
hartnor V2
landle press siće v2
plate slide book 92
pash wall v2
Action (Truncated componmnt)
-1
23
3.5
235
0.8
255
-25
D0
2.5
50
stelf place
stick push
bis picking
box close 92
0.
0.
-2
doarladew2
dearunlscce2
-05
-4
Action (Truncated X-compmeett
Empty Context
Transitions
Transitions
Figure 6: Action beliefs context sizes for other tasks in Merld
23

---

Published as a conference paper at ICLR 2026
janjja
jooljat, jul
1.0
1a
10
10
05
0.5
63
85
80
0.
80
BD
45
100
Sconer-oool mp
Soomte-miodd jan. feb
Soone-miond jun jinjal aug
Sonnemtter mar ApT myy
80
Action (Y component)
Sonne-mtleed oct now
Soome-mtood sep
wandhoxsemmted ape may
worehoune--rtboo dec Jan feb
1.8
-10
warehouss snoeed jul.
wamthanas-maasdjjun
wardhouse-mn mac.uaae--aaaaraaaaa
warthsus-maol_.rr
Action componeent
Empty
Transitions
Figure 7: Action beliefs context sizes for other tasks in SinerGym
24

---

Published as a conference paper at ICLR 2026
dcL0,088
04
64
03
46
005
666
846
0050
6905
0/00
0/25
6425
6100
0A25
6068
000
006
6.16
deE
componontt
0.2
0.45
040
8.1
03
0.8
645
620
00
2.55
0.00
8/b
010
8I5
-0.50
1003
BICD
0
0J52
0005
0050
0,23
0300
DJRS
0.50
4.1
Action componeet)
Empty Context
Transitions
100 Transitions
500 Transitions
Figure 8: Action beliefs over context sizes for other tasks in MetalDiiv
phose
0.10
6:15
6.10
0.10
6/6
Action ("runcated 5VD (-comporon)
0.00
000
6/00
4.55
4.08
0:10
500
9/6b
8.10
0:35
0.2
035
0.2
8.4
0.
0.00
0.55
030
1/5
phose
615
+3
4055
0.00
0.00
0.5
030
04
56
2.8
0.4
0.
06
0008
.003
0000
0250
Action (Truncated SVD X-compunent)
Empty Context
Transitions
100 Transitions
Transitions
Figure 9: Action beliefs context sizes for other tasks in CityLearn
25

---

Published as a conference paper at ICLR 2026
G TASK-LEVEL DATASET VISUALIZATION
amerbly-v
bulton pres tapdonmo2
press butten butten topdove will-v2
1.
60
5rttnn--wee-22
buthosgreewallw
bifieebutter22
crffne-plll12
1.3
00
asl-sravv
dumenile-i
cloe-clae-d
1.3
00
dnwendssew
drswetopeees2
Swowh-lloe-s?
1.2
0.
Suacehopeow
hammest
handlegrmusci
1.3
00
Sandde-pllae ader2
handi-ppll.2
love-pull-v2
pex: pxy-trart--tdev
1.3
0.
pick-plaews
Moars Nortahee
09
plate-dide badev2
plan-dsle-ssbevt
plateelie-dd
09
gich-buribs?
pwb-v2
podl-wellvv
09
recchaalld
placeri
etécepaloy2
1.3
09
rikc pu
owtey-istov2
sweepes
winddw -loseve
69
Nr.ppinnges
boeelow-22
éner-bol-vd
00
doar unixkey
land iocd v2
0.5
09
Trojectory Share
Figure 10: Mean normalized noise-distilled trajectories for Merl-World domain
26

---

Published as a conference paper at ICLR 2026
1/3
0.
who-hhub-stoww
1.3
Ausele
ungry Minde-q
1.0
0.
m-MF-andippe
1/3
worrd veiilev
vi
hec Sitoolaccck
cor derustvd
1.0
thhut--hhoe--- right eag
Moan Normsizzd (99k
1/9
us. oxcile v2
cr kl
10 L.T up +1
Brattall right
lunderst
Frejo chottcc
1 two-swue-1-33
1.0
Jas Iefo
Bund picbullve
1.3
69
Fnjo bulf chectab oayy vi
1xjo wilker ouay wi
1.9
waje-waler-v
marph 2et
Las marph drectien-ct
h aanp- denaatioe
1.0
Hat Myriiiig May Aay moreere
60
1landd la
hupper otay
reje huppes
gynapp cauy
1.9
Iragectory Share
Figure 11: Mean normalized noise-distilled trajectories for Kinetix domain
27

---

Published as a conference paper at ICLR 2026
1/3
0.
www.
Shaddww www.owaaaddit..oo arolears
0.3
Normaltod Return
shalonhundgen
skalenhundscioon
1.
0a
Judowehanawttggu
shstewhunäwwcch
1.3
0.
0.5
69
900
0.25
375
129
0.9
0.28
1.00
3.00
0.25
1.0
0.25
180
Trajectory Share
Figure 12: Mean normalized noise-distilled trajectories for Bi-DexHands domain
detS
0.
del 0.85
del 3W5
del.E. 309
def 0..35
1.3
0.5
Je..O.
JeL.2.911
Moan Normelleed (19X
1.0
1.3
0.0
coree_C C_30
commuc_233
0.5
0.9
COTAL C.30
0.5
0.55
8.25
2.50
075
120
Trojectory Share
Figure 13: Mean normalized nisseedistilled trajectories for MetaDrive domain
28

---

Published as a conference paper at ICLR 2026
1/3
Indurnal 20
Induttaal www.-aannnnaaaaao
borchhmuni--
tnduitaa Serelmaak ap
Indaitial benchesack
indeactall berddumani v1
induttial Berahhnakk 05.v
0.
HUN
Normal
Induntal berchmark--b--
Industrall loenciemarkeeee
indurttaal Awencheamii-tt-ooo
Indhuttra weraal---erreee www.mmmree----emm
05
Renalanac
bencismark
bendlmank
1.0
1.0
00
Imgectory Share
Figure 14: Mean normalized noise-distilled trajectories for domain
atcemf.4
0.5
Moan Kormalzad (915cc1)
1.0
69
plccanssion akcemneaaeddilaanmo.oo Jsatinn
plc.ware
schnddnggr
1.0
Trojectory Share
Figure 15: Mean normalized noise-distilled trajectories for ControlGym domain
29

---

Published as a conference paper at ICLR 2026
1/3
Moon Normall and Itturn
Hoppar-e
Acthvv wM
Swimaaned
1.
Readher-v vi
Fusher-ve
Halthhrtus wi
0.5
0.0
2.25
2.50
0.5
1.0
0.00
0.35
8.75
1.00
9.29
025
0.00
0.75
1.90
Trijectry Share
Figure 16: Mean normalized noise-distilled trajectories for MuJoCo domain
booce cool Jab
ooll
1.0
Jrn
hww.trttt.r.oo.oo.. Jrl. sog
spr. may
1.3
Some Snl, J9
beatee Jol
bauce miiol Jab
bouce mieel jan
1.0
Sa0ce mieed yaa. may
Scace mied scc
Score mined_aep
verhhuue coel uppuray
1.2
Normslzzed feturr (onN
0.
wanvintoe-crol dec Jm
wonehocee-cool.co mur
wwwkhoee--coo..ro
www.boue-cee..t
warebuse-tstt apr my
wardbose fot dec jac. leb
wanebo use fot, Jur, J.l ag
wir mar
60
wurdous ha
warclouic oct
warchous loL
vur closs mised Lest
0.3
00
warchicuse cliodd der jnn ltb
warehouse mixed JaLagg
warehouse stord jas
wurdhosse stned sur
0.5
warthaus mieolas
1.0
0.5
6.0
025
0.50
3.55
120
Trojectory Share
Figure 17: Mean normalized nosse-distilled trajectories for SinerGym domain
30

---

Published as a conference paper at ICLR 2026
plases
plase.
ghase
plane.
1/3
063
0.9
0.3
Norallzon
2M2 p Mas.o...sssss Jusil
gtiigtase. Linki
1822 3uo2,
0.3
ati_staie_Aisk.e
1.0
3822 www.ct Kuakk
2024 plat 1
ptace 1
1.0
0.
Trajectory Share
Figure 18: Mean normalized noise-distilled trajectories for CityLeern domain
RDW.ED-mmw--g--6b
SEWARD
kZWARD-crmol..4 5od
10
REw.d.Drcmocchh
A2WARDFFOLSS--55 S.P
REMAAG-sphS 4,3
Normaltaed Return (PrS
Rww.ilD-headtteo
GONi-t post
GOAL-CNoch, nelluct
3
0.5
1.0
00
0.3
1.50
Trojectory Share
Figure 19: Mean normalized noise-distilled trajectories for HumEnv domain
31

---

Published as a conference paper at ICLR 2026
32

---

Published as a conference paper at ICLR 2026
H TASK-LEVEL PERFORMANCE
H.1
COLD-START ONLINE
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
bin-picking-v2
test
1.9
410.0 64.44
0.0
box-close-v2
test
82.0
583,9 + 13.3
0.0
door-lock-v2
test
119.3
0.65
door-unlock-v2
test
94.7
8.0.0
0.03
hand-insert-v2
test
2.5
5.4.
0.14
assembly-v2
train
44.7
283.3 27.63
1.05
basketball-v2
train
2.6
591.7 # 19.84
1.0
button-press-toodoww--2
train
28.6
31.20
0.01
outionnpeess-tooooowwwalla2
train
30.1
0.12
button-press-v2
train
31.5
646.8 11.04
0.9
button- buttoon-pess--alll--22
train
8.0
6.7.
0.7
2uffee-button-v2
train
33.0
29.68
1.14
coffee-pull-v2
train
4.2
349.8 104.95
1.03
coffee-push-v2
train
4.2
570.1 105.82
0.0
dial-turn-ve v2
train
29.8
794.2 13.12
1.0
disassemble-v2
train
33.9
486.8 88.9
0.14
door-close-v2
train
5.0
1.0
door-open-v2
train
54.2
1.0
drawer-close-v2
train
107.6
866.2
1.0
drawer-open-v2
train
134.0
2.1
1.0
faucet-close-v2
train
247.5
767.0
1.0
faucet-open-v2
train
249.5
757.1 + 8.85
1.0
hammer-v2
train
90.3
683.8 23.29
1.0
handle-press-side--2
train
53.6
843.9 11.33
1.0
hansle-press-v2
train
54.0
809.1 #
1.16
handlc-pulll-siee-22 v2
train
2.3
494.4 49.06
0.7
handle-pull-v2
train
14.1
7.5.7
0.09
lever-pull-v2
train
61.4
608.6 40.05
0.68
peg-insert-side-w
train
1.7
3
0.0 0.31
peg-unplug-side-we
train
5.8
474.3 89.02
pick-out-of-hole-v2
train
1.5
381.2 28.75
0.0
pick-place-v2
train
1.1
473.7 122.34
0.1
pick-place-wall-v2
train
0.0
540.6 86.62
0.23
bate-slide-bacc-siiee--
train
33.7
743.9 60.52
0.23
plate-slide-back
train
34.2
679.4 78.11
1.0
plate-slide-side-22
train
23.4
672.9 130.16
1.0
plate-slide-v2
train
72.6
525.5 154.19
0.9
push-back-v2
train
92.37
0.0
push-v2
train
5.3
762.0
1.09
push-wall-v2
train
6.2
746.4 16.4
0.0
reach-v2
train
140.0
639.2 142.86
0.8
reach-wall-v2
train
157.0
601.9 140.25
1.2
sheff-place-v2
train
0.0
261.1 22.94
0.1
soccer-v2
train
4.9
483.6 + 48.56
0.1
stick-pull-v2
train
2.7
519.2
0.9
stick-push-v2
train
2.7
6.88
1.0
sweep-into-v2
train
16.3
795.2 11.04
-0.0
sweep-v2
train
12.5
499.6 22.37
0.7
window-close-v2
train
56.9
594.2+388 38.3
0.29
window-open-v2
train
43.2
618.4 + 46.16
0.79
Table Online Evaluation Results for Morld
33

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
Shand-lunar-laneer-v1
test
-0.4
1,0 = 0.0
0.3 0.24
--hard-pinbal--il
test
-0.6
1.0 = 0.0
0.1 0.28
I-mje-walker-vl
test
-0.3
1.0
0.2 # 0.0
m-arm-hard--l
test
0.0
1.0
1.0 0.0
hhh-1-thruutblock-vv
-1.0
1.0
0.0
s- s-h8-unicycle balance-vl
test
0.4
1.0
0.5
l-chain-lander-vl
train
-0.3
1.0
1.0
I-chain-thrus--vl
train
0.1
1.0
1.0
l-grasp-easy-vl
train
-0.4
1.0
1.0
l-hl-car-left-vl
train
0.0
1.0
0.2
1h10-mmorph-dirccttoon--v
train
0.0
1.0
1.0
1h14-simple-thrusters.
train
-0.4
1.0
1.0
I-h3-car-right-vl
train
0.2
1.0
1.0
1-17-maze---vl
train
-0.5
1.0
0.6
1-19-morph-directionn-1
train
0.0
1.0
0.2
-mjj-half--hhoeta--easyy-
train
0.0
1.0
1.0
I-jjehhulf-cheetab-vl
train
-0.3
0.0
0.3
--jjehhopper-casy-vll
train
0.3
1.0
1.0
I-mje-hopper-vl
train
-0.3
1.0
0.1
I-mje-swimmer-vl
train
0.2
0.9
0.8
I-mje-walker-easy-vl
train
0.1
1.0
0.09
1-pr-v1
train
0.6
1.0
1.0
m-arm-left-vl
train
0.0
1.0
1.0
m-arm-right-vl
train
0.1
1.0
1.0
m-arm-up-vl
train
-0.5
1.0
1.0
m-ho-unicycle-vvl
train
-0.4
1.0
1.0
n--hl0-thnisterrhh-eaass-v/
train
0.0
1.0
0.8
m-hll-ttnus-eef--aayyy-
train
0.0
1.0
1.0
m-hll2thruusfällleffttt
train
0.0
1.0
1.0
n--hl3-tbrustfall-riihh-
train
0.1
1.0
1.0
n--hi--thrust-right-
train
0.0
1.0
1.0
train
0.0
1.0
1.0
m-h22car-right-vl
train
0.0
1.0
1.0
m-h3-car-thrust-vl
train
-0.5
1.0
1.0
m-bh-thrust-the-needle
train
-0.7
1.0
1.0
hi-h5-angry-birds--
train
0.4
1.0
0.6
m-h6-thrust-over-v
train
-0.5
1.0
1.0
m-h7-car-flip-v
train
-0.4
1.0
1.0
m-h8
nn-h--weird-vehicle--v
train
0.2
1.0
1.0
mn-B9-spin-the-rih---wya-
train
-0.3
1.0
1.0
s- w-hh-weaakthrust-vll
train
-0.5
1.0
1.0
s-hl-trrust-over-blll-w
train
-0.3
1.0
1.0
s-hh-ome-whheel-car-y
train
0.0
1.0
1.0
s-hb-poinn-thh-thuuseeryy
train
0.0
1.0
1.0
s- s-h4-thrustaim--il
train
0.3
1.0
1.0
s-h5-rotate-fall-vi
train
-0.6
1.0
1.0
s-h6-unicycle-right--/
train
0.0
1.0
1.0
-h7-unicycle-left-v1
train
0.1
1.0
1.0
Table 4: Online Evaluation Results for Kinetix
34

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized
Split
Name
Return IQM
Ant-v4
param shift
-00299
4989.7 2058.0 0.
1.0 = 0.01
HalfCheetah-v4
param_shift
-264.2
7331.8 + 420.5
1.0 = 0.01
Hopper-v4
param
18.2
2264.3 = 229.33
1.0 0.47
Humanoid-y4
param-shift
121.9
7373.8 # 132.4
1.0
HumanoidStandup- v4
param
33027.7
303552.9 + 19279.0
1.0
param shift 58.5
9256.8 + 293.62
1.0
InvertedPendulum-yy
param shift 5.2
222.2 + 9.37
0.8
Pusher-v4
param_shift
-14.88
-3.88 # 0.57
1.0
Reacher-v4
param_shift
41.9
-5.8 + 1.06
1.0
Swimmer-v4
param_shift
-0.3
95.3 1.28
1.0
Walker2d-v4
param.shin shift
3.3
5169.3 106.04
1.0
Ant-v4
train
-8.00
6420.5 + 17.13
1.0
HalfCheetah-v4
train
267.2
7786.1 16.1
1.0
Hopper-v4
train
14.0
3390.4 # 17.78 17.788
1.0
Humanoid-y4
train
117.9
7521.7 # 12.92
1.0
HumanoidStandup-v44
train
34034.7
299534.8 + 2187.55
1.1
WwertedDooubleeeeduuumm
train
61.7
9359.5 0.11
1.0
InvertedPendulum-yy
train
5.7
1000.0
1.0
Pusher-v4
train
-149.8
24..7
1.0
Reacher-v4
train
-43.0
-5.6
1.0
Swimmer-v4
train
1.7
96.3 # 0.96
1.0
Walker2d-v4
train
2.7
5357.3 + 13.14
1.0 0.02
Table 5: Online Evaluation Results for MuloCo
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
test
930.0
1380.0 0.0
-1.3 # 0.01
shadowhanddonn hww.doooooponnnnwwrrrr
test
-15.0
410.0 0.0
0.3 0.03
shadowhanddooropee hwwpw//wwnnddooooooottaaarr
test
5.0
610.0 0.0
0.1
hhadowhandkettle
test
200.0
53.0
-1.0
shadow www.whhannhssttaceooooooooooo
train
90.0
285.0
1.09
train
100.0
4.0
1.1
aandowwanndctchaareeess
train
0.5
6..0
0.0
shadow shwdowhanddattoeee hww.deaachhoerrenddeeam
train
4.0
3.00
1.3
train
0.8
2..0
0.2
shadow wawddwwhanddooollosenieeeee
train
1.2
8.8
1.01
shadowhande hwwdwwaaaannagaplaccccc
train
5.0
500.0
1.16
shadow wandliftunderamm
train
-1.5
4.0
1.0
Shadowhandpen
train
0.0
190.0
0.0
shadow shadowhandpusheee handpushblock
train
230.0
4.0.0
0.5
shadowhandreorff handreorientation
train
50.00
31.0.0
0.34
shadow handscissors
train
40.0
725.0
1.0
Shadowwhndswinge
train 490.0
4000.0
1.0
train
45.0
280.0 # 0.0
1.0
shadow haw.oo/hwanwwooatchhunderrmm
train
1.5
24.0 + 0.0
0.7 0.5
Table 6: Online Evaluation Results for Bi-DexHands
35

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
industrial-benchmakk--
test
-998.6
-561.5 = 0.0
1.0 + 0.02
industrialbenchmark-8000
test
-779.9
485.44 0.0
1.0 0.0
industrial-bennchmar--------
test
-813.8
-507.7
1.0 0.0
industrial-Denccmark-90000
test -883.4
-521.9
1.0
industriall-enccmmark-
test
922.88
5.00.0
1.0
industrial-benchmark.
train
-35..4
-180.8
1.0
industrial-benchmark-IMM
train
-40.44
2.0.3
1.0
industrial-benchmark-155554
train
-44..4
23..0
1.0
Industrial-bencbmaar-20000
train
475.9
2209.7
0.9
Industrial-beccchmark--54 train
-46..1
-2.2.9
0.8
industrial-benchmakk-30000
train
480.1
-2.7.8
1.0
industrial-bencchaak-333.
train
501.7
+..4.4
1.0
industrial-benchmark440000
train
-28.8
-3.7.4
0.9
industrialbeenchmarkk4455v1
train
-547.8
-3.1.0
1.0
industrial-benchmakk-5557
train
-377.1
-1.4.0
1.0
indastrial-eenccmaakk-000011
train
-574.4
-3.8.3
1.0
industrial-benchmark-5555
train
660.3
401.7
1.0
industrial-benccmaak--00
train -6363
-3.0.1
0.9
train -666.0
4.0.1
0.9
train -69..5
470.8
1.1
train -718.8
4.0.4
1.0
Table 7: Online Evaluation Results for Industrial-Bi -eatstiallBecchmarkk
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
GOAL-I_pon
test
-3200.0
-605.0 + 0.0
0.2 + 0.01
REWARD-roaatcxx5-00
test
8.0
211.0 0.0
0.1
REWARD-split-0.5
test
9.0
258.0 0.0
0.0
GOAL-crouch.meddim
train
3120.0
-341.0
0.3
GOAL-lie front
train
3300.0
-570.0
1.0
GOAL-sit hand behind
train
320000
533.00
0.9
GOAL-zombie
train
-3200.0
410.0
0.4
REWARD-craw)-0.5-00
train
10.0
140.0
0.9
REWARD-crouchh-
train
0.2
2.0.0
1.0+0003
REWARD-headsttand
train
0.0
247.0
0.0
REWARD-mmoveegoo-0
train
12.0
261.0
0.2
REWARD-moov-ego-022
train
10.0
$65.0
0.1
REWARD-move-go-00.
train
5.0
257.0
0.3
REWARD-raisarrn--hh
train
0.5
240.0 10000000 0.0
0.0
train
9.0
220.0 + 0.0
0.05
Table 8: Online Evaluation Results for HumEny
36

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
5zone-mixed dec jan feb
test
-320.0
-173.0 0.0
0.4 + 0.16
$zone-mixed_jum jun_jul_aug
test
-240.0
-110.0 0.0
1.0 + 0.02
Szone-mixed _mar _aar_pprrppr _ap._appaaa. Szone-mixed _oc_nov coone-mixe_.se
-330.0
-90.0 0.0
0.13
test
-300.0
-80.0
0.9 0.14
-250.0
0..0.0
-0.1
warehouse-misn apr may
test -290.0
1.0.00
1.2
warehouse-mixed dec jan feb
test -800.0
-.0
0.7
parehouse-miied_jull
test -650.0
+.00
1.1
warehouse-miied__um
test -370.0
-1.0.0
0.9
test -66.00
-2.0.0
0,06
warchouse-mixet mixed sep
test -340.0
-..0.0
1.0
Szone-cool_dee jan_.eb
train
-30.00
-1.0.0
0.11
5zone-cool jun_jul aug
train
-300.0
-1.0.0
0.38
Szone-cool war apr may
train
-300.0
-.0
1.07
Szone-cool_oct_noe
train
-300.0
0.0
1.07
Szone-coolssep
train
-320.0
1.000
-0.0
5zone-hot.c how-hoo-tddejaa fja
train
-300.0
0.0
0.0
5zone-hot jun jul aug
train
-230.0
0.0
1.1
5zone-hot hott-ootttaa-aaeeeeeeeeeeo mar apr may
train
-80.0
-.000
0.19
Exone-hot_oct.ng
train
-350.0
0.0
0.16
Szone-hot_sep
train
-240.0
-1.0.0
0.1
warehouse-cool.apr.
train
-290.0
0.00.0
0.9
warchhous--colldec..iii
train -70..0
-3.0.0
0.1
warchouse-cool_maa
train
665.0
2.00
0.77
warehouse-cool nov
train 550.0
-1.0.0
0.7 0.09
warehouse-cool oct
train
250.0
0.0
0.9 + 0.05
warchouse-hot..pr_mma
train
-1300.0
400.0
0.7 0.05
warchouse-hot.dccc.aa.de
train -15.00
0.0
0.13
warehouse-hot.jjn. julllaa
train
-1000.0
-.00
0.9
warehouse-hot
train -230.0
-.0
0.03
warehouse-hh saaeeehhtoooooooooooeeeeeee nov warehouse-hot oct warehouse-hot_sep
train -530.0
0.0
0.93
train -1100.0
-300.0
0.9 0.04
-860.0
4...0
1.0
Table 9: Online Evaluation Results for SinerGym
37

---

Published as a conference paper at ICLR 2026
Task Name
Random Score
Expert Score
Normalized Return IQM
Split
cones C 69
test
9.1
535.0 0.0
0.9 0.22
def_C_574
test
9.4
$11.40000
1.0 = 0.01
def.O_783
test
8.8
58.
1.0 0.03
def.T_971
test
6.7
454.7 0.0
1.0 0.0
cones_C.520
train 10.0
8.0
0.15
cones -ones_C.54 C.574
train
11.4
6.0
0
cones C 785
train
10.0
6.5
0.3
cones_C_80
train
11.4
7.9.2
0.6 0.37
cones_C_859
train
8.2
5.7.4
0.8
cones_S
train
9.9
4.0
0.12
def.C.520
train
7.9
9.0
1.0
def.C.69
train
6.0
5...1
1.0 0.01
C_785
train
5.7
64..4
1.0
Ce85
train
9.9
730.5
1.0
def_C_859
train
7.8
58..1
1.01
def_0.291
train
8.4
7.0
1.03
def-0.64
train
8.2
56,09
1.03
def O 971
train
10.3
7.0
1.04
def S
train
9.6
4.6
1.0 0.04
def 855
train
10.7
485.3
1.0 0.0
def_T-98
train 9.0
480.1 = 0.0
1.0 0.01
Table 10: Online Evaluation Results for MetaDrive
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
2022 phase 1 task 10
-1833.1
-702.2 + 0.0
0.8 0.0
2022 phase 1 task 11
test -236.11
-1262.1 0.0
0.8 # 0.0
2022_phh 2022_phase 2at____aaskkkl 2tt.spaa.taak_______
10
test -1724.5
-611.3
0.8 0.0
11122_phaee_2.ce
test -2143.7
-.07.4
0.8 # 0.0
2022_phasse wttppp_lttask..0.000
train
-2212.6
-1048.3
1.0 0.0
20022_phaee... 1_tass___task011
train 2017.0
-.31.4000
1.0 0.0
2022 phase 1 task 02
train 1600.7
-4..44
1.0
2022 phase 1 task 03
train 1602.2
-506.8
1.0
2022-phase__ 1tta__tt_k0000
train 2097.8
1054.44
1.0
1
train 2254.5
12.8.11
1.0 # 0.0
1022-phase__l_takk
train 1162.44
-3.3.6
1.0 0.0
2022. 2022.phase. wwtss.lltass.........pppappaaaaa 07
train 1511.0
-33.7
1.0 0.0
2022.phase_11.ne ..tasee_l.tak....
train 1325.6
22.011
1.0 0.0
2022. 2002_ppase__/ tast 09
train 1518.1
45..1
1.0 0.0
2022 phase 2.taskk00 .00
3022.phase
train 2365.0
-1.02.1
1.0
2022_ph htt___paaaaaaakeeeeeep
train 2086.1
-9.79
1.0
2022.phase.2.taas 22tiusse__ttsk00222
train 1678.5
590,0
1.0
2022 htt_-pphaa__asstte
train 1780.2
-7.5.1
1.0
2022 2022.phase 2.task 04
train 2408.6
-1389.3
1.0
2022. -pasephaaeeeee 2 task 05
train 2604.0
-1698.2+0.0
1.0
2022__phase_2 2tttkk00006666
train 1965.77
-934.7
1.0
2022. 11022_phae__2.
train 1568.9
-3.0.4
1.0
2022_phase._
wtwpppaaeaaataa...... 2__aask008
train 1270..2
-2.1.3
1.0
2022.phase_2.tta 1atasse2.task_00
train 1504.55
-423.5 + 0.0
1.0
Table 11: Online Evaluation Results for CityLeam
38

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
aireraft
test
-67000.0
-0.5 0.0
1.0 0.0
cable_5
test
-56000.0
-9.7 = 0.0
1.0 0.0
pde_schrodinger
-3300000.0
-2146.7
-4.5 1.49
aircraft
train
-67000.0
0.0
1.0
aireraft 44
train
10000..0
-0.3
1.0
aircraft
train
12000..0
-1.5
0.9
aireraft
train
-720000.0
-1.3.4
-17.5
cable_1
train
42000.0
-3..3
1.0
cable_2 2
train
49000.0
-20.6
1.0
cable_3
train -55000.0
-15.2
1,0
cable 4
train -55000.0
-12.5
1.0
dile.convectio.diifee diffusion.reaction reaction
train -58000.0
3.0
0.3
Table 12: Online Evaluation Results for ControlGym
39

---

Published as a conference paper at ICLR 2026
H.2 PROMPTED OFFLINE
Task
Random Score
Expert Score
Normalized
Split
Name
Return IQM
bin-picking-v2
test
1.9
410.0 = 64.37
0.0
box-close-v2
test
82.0
583.9 13.28
0.0
door-lock-v2
test
119.3
831.9 9.21
1.0
door-unlock-v2
test
94.7
813.0 + 2.07
0.9
hand-insert-v2
test
2.5
5.466
1.0
assembly-v2
train
44.7
283.3 27.63
1.0
basketball-v2
train
2.6
591.7 19.84
1.0
button
putton-press-toooowwn-22
train
28.6
499.0
1.0
button-pres-ttooooow-wlll--
train
30.1
35.04
1.0
button-press-v2
train
31.5
646.8 2 11.04
1.0
button-press-wall--22
train
8.0
6.7.6
1.0
coffee-button--2
train
33.0
733.0 29.68
1.0
coffce-pull-v2
train
4.2
349.8 104.95
0.0
coffee-push-v2
train
4.2
570.1 105,82
1.0
dial-turn-v2 v2
train
29.8
1.0
disassemble-v2
train
33.9
486.8 88.9
1.02
door-close-v2
train
5.0
543.7 - 34.04
0.01
door-open-v2
train
54.2
5
1.01
drawcr-close-v2
train
107.6
4.77
1.0
drawer-open-v2
train
134.0
493.9
1.0
faucet-close-v2
train
247.5
767.0 + 8.36
1.0
faucet-open-v2
train
249.5
757.1 8.85
1.0
hammer-v2
train
90.3
683.8 23.29
1.0
handle-press-side--2
train
53.6
843.9 11.33
1.0
handle-press-v2
train
54.0
809.1
1.04
handlc-pulll-ide--22
train
2.3
49.00
1.0
handle-pull-v2
train
14.1
5.557
1.0
lever-pull-v2
train
61.4
608.6
1.0
peg-insert-side-v2
train
1.7
314.5 153.35
1.02
peg-unplug-side-v2
train
5.8
474.3 89.02
0.07
pick-out-of-hole-v2
train
1.5
0.01
pick-place-v2
train
1.1
122.34
0.05
pick
pack-place-wall-oe
train
0.0
540.6
1.03
Mlteesllidebback-sidee
train
33.7
1.0
plate-slide-hack-22
train
34.2
68.14
1.0
plate-slide-side-22
train
23.4
672.9 130.16
1.0
plate-slide-22
train
72.6
LO 0.03
push-back-v2
train
1.2
464.4
push-v2
train
5.3
7
1.0
push-wall-v2
train
6.2
746.4 16.4
1.0
reach-v2
train
140.0
639.2 142.86
1.0
reach-wall-v2
train
157.0
601.9 140.25
0.35
shelf-place-v2
train
0.0
261.1
0.0
soccer-v2
train
4.9
483.6 48.56
1.0
stick-pull-v2
train
2.7
419.2
1.0
stick-push-v2
train
2.7
635.9 7.86
1.0
sweep-into-v2
train
16.3
795.2 11.04
1.0
sweep-v2
train
12.5
499.6 22.37
1.0
window-close-v2
train
56.9
594.2 + 38.3
1.0
window-open-v2
train
43.2
618.4 46.16
1.0
Table 13: Offline Evaluation Results for
40

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
Shand-lunar-laneer-vi
test
-0.4
1,0 = 0.0
-0.1 0.36
--hard-pinbal--il
test
-0.6
1.0 = 0.0
0.4 + 0.22
I-mje-walker-vl
test
-0.3
1.0
0.2 0.0
m-arm-hard--l
test
0.0
1.0
1.0 0.0
hhh-4-thruutblockvvv
-1.0
1.0
0.0
s- s-h8-unicycle balance-vl
test
0.4
1.0
0.2
l-chain-lander-vl
train
-0.3
1.0
1.0
I-chain-thrust-vl
train
0.1
1.0
1.0
l-grasp-easy-vl
train
-0.4
1.0
1.0
l-hl-car-left-vl
train
0.0
1.0
0.24
1h10-mmorph-dirccttoon--v
train
0.0
1.0
1.0
1h14-simple-thrusters.
train
-0.4
1.0
1.0
I-h3-car-right-vl
train
0.2
1.0
1.0
1-17-maze---vl
train
-0.5
1.0
0.8
1-19-morph-directionn-1
train
0.0
1.0
1.0
-mjj-half--hhoeta--easyy-
train
0.0
1.0
0.06
I-jjehhulf-cheetab-vl
train
-0.3
0.0
1.08
--jjehhopper-casy-vll
train
0.3
1.0
1.0
I-mje-hopper-vl
train
-0.3
1.0
1.05
I-mje-swimmer-vl
train
0.2
0.9
1.1
I-mje-walker-easy-vl
train
0.1
1.0
0.1
1-pr-v1
train
0.6
1.0
1.0
m-arm-left-vl
train
0.0
1.0
1.0
m-arm-right-vl
train
0.1
1.0
1.0
m-arm-up-vl
train
-0.5
1.0
1.0
m-ho-unicycle-vvl
train
-0.4
1.0
1.0
n--hl0-thnistrrrhh-eaasy-v/
train
0.0
1.0
1.0
m-hll-ttnus-eef--aayy--
train
0.0
1.0
1.0
m-hll2thruusfällleffttt
train
0.0
1.0
0.0
n--hl3-tbrustfall-riihh-
train
0.1
1.0
1.0
n--hi--thrust-right-
train
0.0
1.0
1.0
train
0.0
1.0
1.0
m-h22car-right-vl
train
0.0
1.0
1.0
m-h3-car-thrust-vl
train
-0.5
1.0
1.0
m-bh-thrust-the-needle
train
-0.7
1.0
1.0
hi-h5-angry-birds--
train
0.4
1.0
0.9
m-h6-thrust-over-v
train
-0.5
1.0
1.0
m-h7-car-flip-v
train
-0.4
1.0
1.0
m-h8
nn-h--weird-vehicle--v
train
0.2
1.0
1.0
mn-B9-spin-the-rih---wya-
train
-0.3
1.0
1.0
s- t-hh-weaakthrust-vll
train
-0.5
1.0
1.0
s-hl-trrustoover-blll-w
train
-0.3
1.0
1.0
s-h2-ome-wheeelccar-y
train
0.0
1.0
1.0
s-hb-poinn-thh-thuuseeryy
train
0.0
1.0
1.0
s- s-h4-thrustaim--il
train
0.3
1.0
1.0
s-h5-rotate-fall-vi
train
-0.6
1.0
1.0
s-h6-unicycle-right--/
train
0.0
1.0
1.0
--17-nnccylle-lftt-v1
train
0.1
1.0
1.0
Table 14: Offline Evaluation Results for Kinetix
41

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert
Normalized
Split
Name
Score
Return IQM
Ant-v4
param shift
-103.1
4988.8 2057.47
1.0 = 0.01
HalfCheetah-v4
param_shift
-66.22
7331.8 + 420.5
1.0 = 0.01
Hopper-v4
param
18.9
2139.9 = 286.52
0.8 = 0.26
Humanoid-y4
param_shift
121.9
7373.8 = 132.4
1.0
HumanoidStandup-te v4
param
33027.7
303552.9 = 19279.0
1.0
param shift 58.5
9256.8 + 293.62
1.0
InvertedPendulum-yy
param shift 5.5
222.1 + 7.62
0.6
Pusher-v4
param_shift
-16.88
-2388 + 0.57
1.0
Reacher-v4
param_shift
41.9
-5.8 1.06
1.0
Swimmer-v4
param_shift
-0.3
95.3 1.28
1.0
Walker2d-v4
param.shin shift
3.1
5176.1 98.82
1.0
Ant-v4
train
-55.2
1.0
HalfCheetah-v4
train
267.2
7
1.0
Hopper-v4
train
13.8
3385.3 # 16.47
1.1
Humanoid-y4
train
117.9
7521.7 # 12.92
1.0
HumanoidStandup-v44
train
34034.7
299534.8 # 2187.55
1.1
WwertedDooubleeeeduuumm
train
61.7
9359.5 0.11
1.0
InvertedPendulum-vy
train
5.7
1000.0
1.0
Pusher-v4
train
-149.8
24..7
1.0
Reacher-v4
train
-43.0
-5.6
1.0
Swimmer-v4
train
1.7
96.3 # 0.96
1.0
Walker2d-v4
train
2.7
5357.3 + 13.14
1.0
Table 15: Offline Evaluation Results for MuJoCo
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
test
930.0
1380.0 + 0.0
-1.2 # 0.01
shadowhanddonn hww.doooooponnnnwwrrr
test
-15.0
410.0 0.0
0.3 0.03
shadowhanddooroon hwwpw//wwnwddooooooottaaarr
test
5.0
610.0 0.0
0.1 # 0.0
hhadowhandkettee
test
200.0
53.0
+0.0
shadow www.whhannhssttaceooooooooooo
train
90.0
285.0
1.0
train
100.0
4.0
1.1
aandowwanndctchaareeess
train
0.5
6.00
0.1
shadow shwdowwanddattoeee hww.deaachhoerreuddeeam
train
4.0
3.00
0.9
train
0.8
2..0
0.9
shadow wawddwwhanddooollosenieeeee
train
1.2
8.8
0.01
shadowhande hwwwwwaaaangaggalacccccc
train
5.0
500.0
1.01
shadow wandliftunderamm
train
-1.5
4.0.0
1.0
Shadowhandpen
train
0.0
190.0
1.0
shadow chadowhandpussie handpushblock
train
230.0
4.0.0
1.0
shadowhandreorrf handreorientation
train
50.00
31.0.0
0.6
shadow handscissors
train
40.0
725.0
1.0
hhadowwandswwngg
train 490.0
4000.0
1.0
harddoowhadswwieel
train
45.0
280.0 # 0.0
1.0
shadow haw.ooohhanwwwoachhhudderrm
train
1.5
24.0 + 0.0
0.4
Table 16: Offline Evaluation Results for Bi-DexHands
42

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
industrial-benchmakk--
test
-998.6
-561.5 0.0
1.0 0.0
industrialbenchmark-8000
test
-779.9
485.4 0.0
0.8 + 0.0
industrial-bennchmar--------
test
-813.8
-507.7
1.0 0.01
industrial-Denccmark-90000
test -883.4
-521.9
1.0 0.0
industriall-enccmmark-
test
922.88
-..0.0
0.0
industrial-benchmark.
train
-35..4
-1.0.8
1.0
industrial-benchmark-IMM
train
-40.44
210.33
1.0
industrial-benchmark-155554
train
-44..4
23..0
1.0
Industrial-bencbmaar-20000
train
475.9
22.9.7
1.0
Industrial-beccchmark-254 train
-46..1
-2.2.9
1.0
industrial-benchmakk-30000
train
480.1
-2.7.8
1.0
industrial-bencchaak-333.
train
501.7
+..4.4
1.0
industrial-benchmark440000
train
-28.8
-3.7.4
1.0
industrial-eenchmnakk445551
train
-547.8
-3.1.0
1.0
industrial-benchmakk-5557
train
-377.1
-1.4.0
1.0
indastrial-eenccmaakk-000011
train
-574.4
-3.8.3
1.0
industrial-benchmark-5555
train
660.3
401.7
1.0
industrall-ercchmaat---00--
train
-6363
43.0.1
1.0
Industrial-bencharrkee
train -666.0
4.0.1
1.0
train -69..5
4.0
1.1
industrial- 1lausrral-Denccmmar-5555-///
train -718.8
4.0
1.01
Table 17: Offline Evaluation Results for Industrial BeaehiallBBnchmark
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
GOAL-I_pon
test
-3200.0
-605.0 + 0.0
0.2 + 0.01
REWARD-roaatcx-5-00
test
8.0
211.0 0.0
0.02
REWARD-split-0.5
test
9.0
258.0 0.0
0.1
GOAL-crouch.meddim
train
3120.0
-341.0
0.4
GOAL-lie front
train
3300.0
-570.0
1.0
GOAL-sit hand behind
train
320000
-535.0 +
0.05
GOAL-zombie
train
-3200.0
410.0
0.48
REWARD-craw)-0.5-00
train
10.0
140.0
1.0
REWARD-crouchh-
train
0.2
2.0.0
1.0
REWARD-headsttand
train
0.0
2.7.0
0.1
REWARD-mmoveegoo-0
train
12.0
261.0
0.2
REWARD-moov-ego-022
train
10.0
265.0 0.0
0.0
REWARD-move-go-00.
train
5.0
257.0
0.3
REWARD-raisarrn--ht
train
0.5
240.0#00 0.0
0.1
train
9.0
220.0 - 0.0
0.06
Table 18: Offline Evaluation Results for HumEny
43

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
5zone-mixed dec jan feb
test
-320.0
-173.0 0.0
0.9 0.09
$zone-mixed_jum jun_jul_aug
test
-240.0
-110.0 0.0
1.0 + 0.03
Szone-mixed _mar _aar_pprrppr _ap._appaaa. Szone-mixed _oc_nov coone-mixe_.se
-330.0
-90.0 0.0
1.2 0.08
test
-300.0
-80.0
1.1 0.07
-250.0
0..0.0
0.7 0.02
warehouse-misn apr may
test -290.0
1.0.00
1.15
warehouse-mixed dec jan feb
test -800.0
+.0
0.03
warehouse-mixxdd_uul
test -650.0
+3.0.0
1.1
warehouse-miied__um
test -370.0
-1.0.0
0.9
test -66.00
-2.0.0
1.0
warchouse-mixet mixed sep
test -340.0
-..0.0
1.12
Szone-cool_dee jan_.eb
train
-30.00
-1.0.0
0.11
5zone-cool jun_jul aug
train
-300.0
-1.000
1.0 0.05
Szone-cool war apr may
train
-300.0
-.0
1.09
Szone-cool_oct_no
train
-300.0
0.0
1.09
Szone-coolssep
train
-320.0
0.000
0.9
5zone-hot.c how-hoo--ddejaa fja
train
-300.0
0.0
1.05
5zone-hot jun jul aug
train
-230.0
0.0
1.03
5zone-hot hott-ootttaa-aaeeeeeeeeeeo mar apr may
train
-80.0
0.0
1.0
Exone-hot_oct.ng
train
-350.0
0.0
1.03
Szone-hot_sep
train
-240.0
-1.0.0
0.9
warehouse-cool.apr.
train
-290.0
+..0.0
1.01
warchhous--colldec..iie
train -70..0
-3.0.0
1.05
warchouse-cool_maa
train
665.0
2..0
1.0
warehouse-cool nov
train 550.0
-1.0.0
1.01
warehouse-cool oct
train
250.0
0.0
1.02
warchouse-hot..pr_mma
train
-1300.0
400.0
1.03
warchouse-hot.dccc.aa.de
train -15.00
0.0
0.15
warehouse-hot.jjn. jullaau
train
-1000.0
-.00.0
0.2
warehouse-hot
train
-230.0
-.0
1.0
warehouse-hh wateeeehhtooooooooooeeeeeee nov warehouse-hot oct warehouse-hot_sep
train -530.0
0.00.0
1.0
train -1100.0
-300.0
1.0
train -860.0
41..0
1.0
Table 19: Offline Evaluation Results for SinerGym
44

---

Published as a conference paper at ICLR 2026
Task Name
Random Score
Expert Score
Normalized Return IQM
Split
cones C 69
test
9.1
535.0 0.0
0.7 0.31
def_C_574
test
9.4
611.4 = 0.0
1.0 = 0.01
def.O_783
test
8.8
58.
1.1 0.03
def.T_971
test
6.7
154.7=00
1.0 0.01
cones_C.520
train 10.0
8.0
1.0 0.06
cones -ones_C.54 C.574
train
11.4
6.0
0.8 # 0.17
cones C 785
train
10.0
6.5
0.9
cones_C_80
train
11.4
7.9.2
0.6 0.33
cones_C_859
train
8.2
5.7.4
1.2
cones_S
train
9.9
4.0
0.79
def.C.520
train
7.9
9.0
1.0
def.C.69
train
6.0
55..1
1.0 0.01
C_785
train
5.7
64..4
1.0
Ce85
train
9.9
730.5
1.0
def_C_859
train
7.8
58..1
1.01
def.O.291
train
8.4
7.0
1.0
def-0.64
train
8.2
56,09
1.02
def O 971
train
10.3
7.0
1.0
def S
train
9.6
4.6
1.0 0.05
def 855
train
10.7
485.3
1.0
def.T-98
train 9.0
480. # 0.0
1.0 0.01
Table 20: Offline Evaluation Results for MetaDrive
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
2022 phase 1 task 10
-1833.1
-702.2 + 0.0
0.8 0.0
2022 phase 1 task 11
test -236.11
-1262.1 0.0
0.8 # 0.0
2022_phh 2022_phase 2at____aaskkkl ttt.spaa.taak_______
10
test -1724.5
-611.3
0.8 0.0
11122_phaee_2.ce
test -2143.7
0.07.4
0.8 # 0.0
2022_phasse wttppp_lttask..0.000
train
-2212.6
-1048.3
1.0 0.0
20022_phaee... 1_tass___task011
train 2017.0
-.31.4000
1.0 0.0
2022 phase 1 task 02
train 1600.7
-4..44
1.0
2022 phase 1 task 03
train 1602.2
-506.8
1.0
2022-phase__ 1tta__tt_k0000
train 2097.8
1054.44
1.0
1
train 2254.5
12.8.11
1.0 # 0.0
1022-phase__l_takk
train 1162.44
-3.3.6
1.0 0.0
2022. 2022.phase. wwtss.lltass.........pppappaaaaa 07
train 1511.0
-33.7
1.0 0.0
2022.phase_11.ne ..tasee_l.tak....
train 1325.6
22.011
1.0 0.0
2022. 2002_ppase__/ tast 09
train 1518.1
45..1
1.0 0.0
2022 phase 2.taskk00 .00
3022.phase
train 2365.0
-1.02.1
1.0
2022 htt___paaaaaaakeeeeeep
train 2086.1
-9.79
1.0
2022.phase.2.taas 22tiusse__ttsk00222
train 1678.5
590,0
1.0
2022 htt_-pphaa__asstte
train 1780.2
-7.5.1
1.0
2022 2022.phase 2.task 04
train 2408.6
-1389.3
1.0
2022. -pasephaaeeeee 2. task 05
train 2604.0
-1698.2+0.0
1.0
2022__phase_2 2ttkkk00006666
train 1965.77
-934.7
1.0
2022. 11022_phae__2. wttee...tsskkfi
train 1568.9
-300.4
1.0
2022_phase._2
wtw.ppaaaaatta......1 22_aask008
train 1270..2
-2.1.3
1.0
2022.phase.2.ttar Satasee2.task.09
train 1504.55
-423.5 + 0.0
1.0
Table 21: Offline Evaluation Results for CityLeam
45

---

Published as a conference paper at ICLR 2026
Task
Random Score
Expert Score
Normalized Return IQM
Split
Name
aireraft
test
-67000.0
-0.5 0.0
1.0 0.0
cable_5
test
-56000.0
-9.7 = 0.0
1.0 + 0.0
pde_schrodinger
-3300000.0
-2146.7 0.0
1.0 0.0
aircraft.2
train
-67000.0
0.0
1.0
aireraft 44
train
-10000.0
-0.3
1.0
aircraft
train
12000.00
-..5
1.0
aireraft
train
-720000.0
-1.3.4
2.78
cable_1
train
42000.0
-3..3
1.0
cable_2 2
train
-9900.0
-20.6
1.0
cable_3
train -55000.0
-15.2
1,0
cable.4
train -55000.0
-12.5
1.0
pde_convection.c diffusion
diffusion.reactioon reaction
train -58000.0
3.0
1.0
Table 22: Offline Evaluation Results for ControlGym
DATASET SIZE AND METADATA
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action Shape
Reward Scale
Split
Name
Trajectories
Shape
InvertedPendulum--4
1000488
10
68.2
(4,)
(1,)
0.1
train
Invered/Daulleeedduuuu.--
1000480
10
55.7
(11.)
(1.)
0.1
train
Humanoid-v4
1000488
15
207.8
(376.)
(17.)
0.1
train
Walker2d-v4
1000586
10
350.4
(17,)
(6,)
0.1
train
Hopper-v4
1000462
10
227.7
(11.)
(3.)
0.1
train
Ant-v4
1000653
10
524.7
(27.)
(8.)
0.1
train
HumanoidStandup.v4
1000000
15
1000.0
(376.)
(17.)
0.01
train
Swimmer-v4
1000000
5
1000.0
(8,)
(2.)
1.0
train
Reacher-v4
1000000
50.0
(11.)
(2.)
0.1
train
Pusher-v4
1000000
5
100.0
(23.)
(7.)
0.1
train
HalfCheetah-v4
1000000
1000.0
(17.)
(6.)
0.1
train
Table 23: Dataset Metadata for MuJoCo
46

---

Published as a conference paper at ICLR 2026
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Name
Trajectories
Shape
Shape
Shadowhandblocksttck
250140
10
248.6
(428.)
(52.)
train
shadowhandbottlecap
135065
10
123.7
(420.)
(52.)
train
shadowhhandcttchabbresttt
165078
10
98.5
(422.)
(52.)
1
train
ahadoovhandcacchveerunddaam
100034
10
55.6
(422,)
(52.)
I
train
shadowhanndcuchunddeezm
100029
10
64.9
(422,)
(52.)
train
shadowwhanddoorrlsseenaatt
100014
10
23.8
(417.)
(52.)
train
shadowhanddgaaspandplaac
500250
332.8
(425.)
(52.)
train
shadowhandlifundeeee
500301
10
455.0
(417.)
(52.)
train
shadowhandpen
135077
10
120.5
(417.)
(52.)
train
hhadowhandpushbllee
135065
10
123.2
(428.)
(52.)
train
shadowhanddeerrccnatiion
600228
463.8
(422,)
(40.)
train
shadowhandseisssors
175075
10
149.0
(417.)
(52.)
train
shadowhandswingeup
320225
10
299.0
(417.)
(52.)
train
whadowhandswitch
135036
10
124.0
(417.)
(52.)
train
hhaddowhaandwoocccccuddeer
100027
10
65.1
(446.)
(52.)
train
shadowwbanddorrrlossotwwddd
265185
10
249.0
(417.)
(52.)
test
hhudlowbaandoorrpennnvvarrd
265164
10
249.0
(417.)
(52.)
test
hhndooowanddorropeooutvaddd
250242
10
249.0
(417.)
(52.)
test
shadowhandkente
135033
10
124.0
(417.)
(52.)
test
shadowhandover
100040
10
64.4
(398.)
(40,)
test
Table 24: Dataset Metadata for Bi-DexHands
47

---

Published as a conference paper at ICLR 2026
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Name
Trajectories
Shape
Shape
assembly-v2
100000
15
100.0
(39.)
(4.)
train
basketball-v2
100000
100.0
(39.)
(4.)
train
button-press-toodoww--22
100000
15
100.0
(39.)
(4.)
train
butlon-press-toodoowwwwlyyy
100000
15
100.0
(39.)
(4.)
train
button-press--2
100000
100.0
(39.)
(4.)
train
-utton--ress--all--22
100000
15
100.0
(39.)
(4.)
train
coffee-button-v2
100000
100.0
(39.)
train
coffee-pull-v2
100000
17
100.0
(39.)
train
coffee-pushe puuse-22
100000
100.0
(39.)
train
dial-turn-v2
100000
15
100.0
(39.)
train
disassemble-v2
100000
100.0
(39.)
train
door-close-v2
100000
15
100.0
(39.)
train
door-open-v2
100000
100.0
(39.)
train
drawer-close-v2
100000
100.0
(39.)
train
drawer-open-v2
100000
15
100.0
(39.)
train
faucet-close-v2
100000
100.0
(39.)
train
faucet-open-v2
100000
15
100.0
(39.)
train
hammer-v2
100000
15
100.0
(39.)
train
handle-press-side-22
100000
15
100.0
(39.)
train
handle-press-v2
100000
100.0
(39.)
train
handle-pull-side-v22
100000
15
100.0
(39.)
train
handle-pull-v2
100000
100.0
(39.)
train
lever-pull-v2
100000
13
100.0
(39.)
train
peg-insert-side-22
100000
12
100.0
(39.)
train
peg-unplug-side-ve
100000
15
100.0
(39.)
train
pick-out-of-hole-v2
100000
15
100.0
(39.)
train
pick-place-v2
100000
15
100.0
(39.)
train
pick-place-wall-v2
100000
15
100.0
(39,)
train
plate-slide-back-sid--
100000
15
100.0
(39.)
train
plate-slide-
slide-back-v2
100000
15
100.0
(39.)
train
Hate-slide-side-22
100000
100.0
(39.)
train
plate-slide-v2
100000
15
100.0
(39.)
train
push-back-v2
100000
100.0
(39.)
train
push-v2
100000
15
100.0
(39.)
train
push-wall-v2
100000
100.0
(39.)
train
reach-v2
100000
15
100.0
(39.)
train
reach-wall-v2
100000
100.0
(39.)
train
shelf-place-v?
100000
15
100.0
(39.)
train
soccer-v2
100000
13
100.0
(39.)
train
stick-pull-v2
100000
15
100.0
(39.)
train
stick-push-v2
100000
15
100.0
(39.)
train
sweep-into-v2
100000
100.0
(39.)
train
sweep-v2
100000
15
100.0
(39.)
train
window-close-v2
100000
100.0
(39.)
train
window-open-v2
100000
15
100.0
(39.)
train
bin-picking-v2
100000
100.0
(39.)
test
box-close-v2
100000
15
100.0
(39.)
(4.)
test
door-lock-v2
100000
100.0
(39.)
(4.)
test
door-unlock-v2
100000
15
100.0
(39.)
(4.)
test
hand-insert-v2
100000
15
100.0
(39.)
(4.)
test
Table 25: Dataset Metadata for Meta-World
48

---

Published as a conference paper at ICLR 2026
Task Name
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward
Split
Trajectories
Shape
Shape
Scale
def
300184
10
232.9
(275.)
(2.)
train
def eef_C_69
400192
10
301.1
(275.)
(2.)
train
def_C.859
450235
10
283.1
(275.)
(2.)
train
def.C.574
500242
10
263.3
(275.)
(2,)
test
def.C.85
500302
10
286.1
(275.)
(2.)
train
def C.785
500188
10
293.8
(275.)
(2.)
train
def 520
600292
10
258.4
(275.)
(2.)
train
def_O_785
600207
10
364.8
(275.)
(2.)
test
def_0_291
700325
10
357.4
(275.)
(2.)
train
def_0_647
600323
10
350.7
(275.)
(2.)
train
def.O 0.971
750438
10
355.5
(275.)
(2.)
train
400184
10
365.6
(275.)
(2.)
test
def_T _
400192
10
320.2
(275.)
(2.)
train
def 98
400188
10
363.7
(275.)
(2.)
train
cones_S
500224
10
451.8
(275.)
(2,)
train
cones_C.69
700312
10
458.0
(275.)
(2.)
cones_C_859
700433
10
461.5
(275.)
(2.)
train
cones C 574
800384
10
518.5
(275.)
(2.)
train
cones C.85
850386
10
532.8
(275.)
(2.)
train
Jones_C_780
800457
10
495.7
(275.)
(2.)
train
cones cones_C_520
1000367
10
587.3
(275.)
(2.)
train
Table 26: Dataset Metadata for MetaDrive
Task
Trajectory Length
Number of Trajectories
Mean Eplen.
Obs.
Action Shape
Reward Scale
Split
Name
Shape
2022 phase
task 00
533628
10
729.0
(44.)
(5.)
0.25
train
2022 phase task 01
533628
10
729.0
(44.)
(5.)
0.25
train
2022 2022_phase_ _.taskk 02
533628
10
729.0
(44,)
(5.)
0.35
train
6022_pphaee. l_tase____ss____e_eeee 00
533628
10
729.0
(44,)
(5.)
0.35
train
2022.phase__1
__ta____s_______ 04
533628
10
729.0
(44.)
(5.)
0.3
train
2022 10222_phase_1 __taskkkkak .05
533628
10
729.0
(44.)
(5.)
0.25
train
2022 phase task 06
533628
10
729.0
(44.)
(5.)
0.3
train
2022 phase task 07
533628
10
729.0
(44.)
(5.)
0.35
train
2022_phase___ras _aall_aak__00
533628
10
729.0
(44.)
(5.)
0.4
train
2022.phase_ __tas__0PP
533628
10
729.0
(44.)
(5.)
0.35
train
533628
10
718.7
(44.)
(5.)
0.3
test
2022 1022-ppaae___tass
533628
10
729.0
(44.)
(5.)
0.25
test
2022. 2 t00
533628
10
729.0
(44.)
(5.)
0.25
train
2022 1022_ph 2 task 01
533628
10
729.0
(44,)
(5.)
0.25
train
2022. 4022_phase task 02
533628
10
729.0
(44.)
(5.)
0.35
train
2022-phase __ttsss___ksk
533628
10
729.0
(44.)
(5.)
0.35
train
2022-phase.2.task -4
533628
10
729.0
(44.)
(5.)
0.2
train
2022-phase _2takk-05
533628
10
729.0
(44.)
(5.)
0.2
train
2022 phase 2. task 06
533628
10
729.0
(44.)
(5.)
0.25
train
2022 phase 2 task 07
533628
10
729.0
(44.)
(5.)
0.35
train
2022. -phse 2_ 2_taskk_000. 08
533628
10
729.0
(44.)
(5.)
0.4
train
2022 phase 2022.phase 2.ta2k_00000
533628
10
729.0
(44.)
(5.)
0.35
train
2022-phase 10
533628
10
729.0
(44.)
(5.)
0.3
test
2022 2022.phase_22.1 11
533628
10
729.0
(44.)
(5.)
0.25
test
Table 27: Dataset Metadata for CityLearn
49

---

Published as a conference paper at ICLR 2026
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Name
Trajectories
Shape
Shape
s-h0-weak-thrust-vl
100012
15
25.0
(140.)
(6.)
train
s-hl-bhrust-over-balllyy
100014
15
22.3
(140.)
(6.)
train
s-h2-one-wheedlcarrv/
100104
15
209.3
(140.)
(6.)
train
-hh--pointthhe-thrusteee
100019
15
90.1
(140.)
(6.)
train
t--b--trrus-aimmyy
100012
15
42.5
(140.)
(6.)
train
s-h5-rotate-fallly
100013
15
27.0
(140.)
(6.)
train
s-h6--niicycle-right--7
100084
232.0
(140.)
(6.)
train
s-h7-amiicycle-left-v1
100074
14
191.9
(140.)
(6.)
train
<-h8-unicyccee-allanee-w
100012
15
25.1
(140.)
(6.)
test
m-bhll-brruuttllficcassvvvv
100014
15
54.6
(247.)
(6.)
train
m-h19-ttruuttdfff-aar---yyy-
100082
15
138.0
(247.)
(6.)
train
m-h5-anggy-birdds-l
100015
15
72.7
(247.)
train
m-arm-right-vl
100007
15
46.7
(247.)
(6.)
train
n-hhl2thrrusffalllefftt-
100012
15
41.6
(247.)
train
m-h7-car-flip-vl
100013
15
31.7
(247.)
train
m-h2-car-right-vl
100072
15
206.2
(247.)
(6.)
train
m-b8-weirr--vehille--
100072
15
215.2
(247.)
(6.)
train
m-h0-umicycle-vl
100021
15
29.4
(247.)
train
m-h144thrussblock-v/
100017
15
27.3
(247.)
test
m-h3-car-thrust-vl
100019
15
54.2
(247.)
(6.)
train
n-th-ssin--hhe-rihtt-aayy-
100011
15
36.7
(247.)
(6.)
train
m-arm-hard-vl
100009
15
46.4
(247.)
(6.)
test
m-h10-tbrust-right-eass
100014
15
49.6
(247.)
train
100047
15
175.6
(247.)
(6.)
train
n-hd-thrusstth--neellleww
100011
15
25.3
(247.)
train
m-arm-left-vl
100012
15
80.0
(247.)
train
m-arm-up-vl
100015
15
54.5
(247.)
train
m-hhl3ttrrusfiallrrihhny
100009
15
36.5
(247.)
train
m-h6-thrrustoveer-vl
100011
15
17.8
(247.)
train
I-chain-lander-vl
100018
15
33.8
(679.)
train
Lmjjc-half-cheeaahhe
100095
15
194.1
(679.)
train
1-h7-maze---vl
100082
15
116.1
(679,)
train
1-pr-v1
100020
67.5
(679.)
train
l-hl-car-left-v/
100094
15
186.2
(679.)
train
l-hard-pinball-vl
100038
15
106.5
(679.)
test
Imje-swimmer-vl
100095
15
230.8
(679.)
train
1-h3-car-right-vl
100067
15
179.5
(679.)
train
-mjjo-balf-chhetah-essyya
100101
15
234.2
(679.)
train
Immjc-walker-easyy-i
100042
15
169.6
(679.)
train
I-chain-truut-vi
100007
15
58.4
(679.)
train
Imjc-walker-vl
100038
15
106.6
(679.)
test
-h1l--morph-dircctoon-22
100031
15
107.1
(679.)
train
-h9--morph-dircction--
100075
15
90.4
(679.)
(6.)
train
-h14-siimlle-hrusstee--v
100014
15
28.4
(679.)
(6.)
train
-hard-lnaa-llnderr-v
100039
15
42.4
(679,)
(6.)
test
I-mjc-hopper-easy--11
100043
15
172.2
(679.)
(6.)
train
1-mje-hopper-vl
100018
15
65.3
(679,)
(6.)
train
l-grasp-easy-vl
100003
15
30.4
(679.)
(6.)
train
Table 28: Dataset Metadata for Kinetix
50

---

Published as a conference paper at ICLR 2026
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Name
Trajectories
Shape
Shape
300000
15
250.0
(6.)
(3.)
train
industrial-enncharkke
300000
15
250.0
(6.)
(3.)
train
industrial-bent -e-----eenmmmm--o11
300000
15
250.0
(6.)
(3.)
train
patustrial-benchmarr--a.
300000
15
250.0
(6.)
(3.)
train
industrial-bente benchmark-20-v1
300000
15
250.0
(6.)
(3.)
train
300000
15
250.0
(6.)
(3,)
train
300000
15
250.0
(6.)
(3.)
300000
15
250.0
(6.)
(3.)
train
300000
15
250.0
(6.)
(3.)
train
300000
15
250.0
(6.)
(3.)
train
industrial- -bt.haaa--bbbncaaa------V
300000
15
250.0
(6.)
(3.)
train
industrial- bew.hmann-nsm--vl
300000
15
250.0
(6.)
train
industrial
300000
15
250.0
(6.)
(3.)
train
300000
15
250.0
(6.)
train
300000
15
250.0
(6.)
(3.)
train
industrial- benchmark-75-vl
300000
15
250.0
(6.)
(3.)
train
300000
15
250.0
(6.)
(3.)
test
industrial-bend benchmark-85-v1
300000
15
250.0
(6.)
(3.)
300000
15
250.0
(6.)
(3.)
test
industrial-benchmaaee benchmark-95-v1
industrial-
300000
15
250.0
(6.)
(3.)
test
industrial-bencee benchmark-100-1
300000
15
250.0
(6.)
(3.)
test
Table 29: Dataset Metadata for Industrial IndustrallBenchhmatt B--BcBnhhmk
Task
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Name
Trajectories
Shape
Shape
REWARD--move-go-00
300000
10
300.0
(358.)
(69.)
train
LEWARD-move-egoo0-22
300000
10
300.0
(358.)
(69.)
train
REWARD-raisearm-hh
300000
10
300.0
(358.)
(69.)
train
REWARD-craw1-0.5.00
300000
10
300.0
(358,)
(69.)
train
REWARD-crouch-0
300000
10
300.0
(358,)
(69,)
train
REWARD-rotatexx55-0
8
300000
10
300.0
(358.)
(69.)
train
REWARD-rrtate-x550...
300000
10
300.0
(358.)
(69.)
test
REWARD-split-0.5
300000
10
300.0
(358,)
(69.)
test
REWARD-headstand
300000
10
300.0
(358.)
(69.)
train
DoWARDD-moe-eggo-0--
300000
10
300.0
(358.)
(69.)
train
GOAL-t pose
300000
10
300.0
(358.)
(69.)
test
GOAL-crouch cectichmmeiimmmmm
300000
10
300.0
(358,)
(69,)
train
GOAL-zombie
300000
10
300.0
(358.)
(69.)
train
GOAL-lie
300000
10
300.0
(358.)
(69,)
train
LOAL-sit.hand.beet behind
300000
10
300.0
(358,)
(69,)
train
Table 30: Dataset Metadata for HumEnv
51

---

Published as a conference paper at ICLR 2026
Task Name
Trajectory Length
Number of
Mean Eplen.
Obs.
Action
Reward Scale
Split
Trajectories
Shape
Shape
aircraft aircraft_2
1000000
10
1000.0
(3.)
(6.)
0.005
test
1000000
10
1000.0
(3.)
(6.) 0.005
train
aircraft_4
1000000
10
1000.0
(2.)
(5.)
0.02
train
aircraft_5
1000000
10
1000.0
(5.)
(11.)
0.02
train
aircraft.6
1000000
1000.0
(5.)
(14.)
0.0005
train
cable_1 1
1000000
10
1000.0
(2.)
(2.)
0.01
train
cable 2
1000000
1000.0
(2.)
(2.)
0.01
train
cable_3
1000000
10
1000.0
(2.)
(2.)
0.01
train
cable_4
1000000
1000.0
(2.)
(2.)
0.01
train
cable_5
1000000
10
1000.0
(2.)
(2.)
0.01
test
pde_ cileccovvectin...m diffusion reaction
1000000
100.0
(10.)
(1.)
0.0077
train
yde_sccrodinngr
1000000
10
500.0
(10.)
(1.) 0.0001
test
Table 31: Dataset Metadata for ControlGyy
Task Name
Trajectory Length
Number of Trajectories
Mean Eplen.
Obs.
Action
Reward Scale
Split
Shape
Shape
Szone-cool_dec feb
140340
10
721.2
(17.)
(2.)
train
Szone-cool, -on--oonoo_uujuuuu
1 140380
10
736.1
(17.)
(2.)
train
5zone-cool mar apr may
140438
10
736.4
(17.)
(2.)
train
5zone-cool nov
I 140515
10
731.8
(17,)
(2.)
train
5zone-cool_see
140400
720.0
(17.)
(2.)
train
5zone-hot saane-hht_dee._jjee se__ddec_aaaa feb
| 140136
10
720.5
(17,)
(2.)
train
Szone-hot.jun.jul Lug
140419
736.0
(17,)
(2.)
train
5zone-hot aap.-aaraappmaaaaa
140527
10
736.1
(17.)
(2.)
train
#zome-hotLoct.nov
140392
732.4
(17.)
train
5zone-hot sep
140400
10
720.0
(17.)
train
5zone-mixed dec jan feb
140388
10
720.7
(17.)
test
5zone-mixed_jin jun_jul jul_ju aug
140373
10
735.7
(17.)
test
5zone-mixed mar_apr may 5zone-mixed_oct.. _nv Szone-mixed
I 140469
10
735.9
(17.)
test
140448
10
731.5
(17.)
test
140400
10
720.0
(17,)
test
warehouse-cool apr may warehouse-cool dec jan feb
140474
732.4
(22.)
train
I 40215
10
718.8
(22.)
train
warehouse-cool.me
140616
744.0
(22.)
train
warehouse-cool
1 140400
10
720.0
(22.)
train
warehouse-cool.
I 140616
744.0
(22.)
train
parehouse-hot.ant
140364
10
732.2
(22.)
train
warehouse- www.eeoooo.eee....oeeeee
1 140323
10
719.2
(22.)
train
warehouse- hot.eun..njj.i..... aal.jjuu_uuugg aug
140412
10
735.9
(22.)
train
warehouse-hot. mar
1 140616
10
744.0
(22.)
train
warehouse-hot_nov
140400
10
720.0
(22.)
train
sarehouse-hot-ooo
140616
10
744.0
(22.)
train
sarchouse-hotsssee
140400
10
720.0
(22.)
(2.)
train
warehouse- mixedd apr may
140296
10
731.8
(22.)
(2.)
test
warehouse-mixet dec jan feb
140428
720.2
(22.)
(2.)
test
warehouse-mixx _ul_auu Lug
140616
10
744.0
(22.)
(2.)
test
prrehouse-mixed. warehouse-minn ciaiseemixed__mmee _mar
140400
720.0
(22.)
(2.)
test
140616
10
744.0
(22.)
(2.)
test
warehouse mixedddssieddeep
140400
10
720.0
(22.)
(2.)
test
Table 32: Dataset Metadata for SinerGym
52

---

Published as a conference paper at ICLR 2026
J DOES MULTI-DOMAIN HELP GENERALIZATION
To understand if mmulti-domalie setting helps reach the better performance, we trained separate single- domain models for several representative domains under exactly the same conditions as the 10- domain model (identical architecture size, hyperparameters, and number of seen tokens per domain).
The results in the Table 33 show that the multi multi-domain model either outperforms or matches the single-domain models on both online and offline metrics, with particularly clear gains on Meta- World and Industrial Benchmark, and no systematic degradation elsewhere. This suggests that, de- spite using group-specific encoders, the shared backbone does exploit cross-domain data and yields measurable benefits over training separate models.
Domain
Online Single
Online Full
Offline Single
Offline Full
Industrial Benchmark (Test)
0.950 + 0.004
0.980 + 0.007
0.950 + 0.007
0.960 + 0.009
Meta-World (Test)
-0.000 + 0.001
0.140 + 0.023
0.460 + 0.029
0.690 + 0.036
SinerGym (Test)
0.900 + 0.026
0.910 + 0.020
0.970 + 0.010
0.960 + 0.017
Industrial Benchmark (Train)
0.940 + 0.009
0.990 + 0.006
1.010 + 0.001
1.020 + 0.001
Meta-World (Train)
0.650 0.015
0.850 0.080
0.990 + 0.001
1.000 0.000
SinerGym (Train)
0.960 + 0.031
1.000 + 0.026
1.010 + 0.004
0.990 + 0.017
Table 33: Single-domain and multi domain models comparison
K COMPARISON WITH GAUSSIAN HEADS
To ensure that Flow matching head is suitable choice, we conducted an additional ablation compar- ing standard Gaussian policy heads (GH) with flow-matching heads (FM). We trained single-domain models with GH and with FM, keeping the number of parameters, batch size, optimization hyperpa- rameters, and number of update steps identical for each pair of models. The results are summarized in the Table 3
On unseen test tasks, FM heads perform at least as well as GH in three out of four domains, and substantially better in several cases: on SinerGym the offline test score improves from 0.790 to 0.970, on HumEnv from 0.070 to 0.130 and on Meta-Woo from 0.390 to 0.460. The only exception Industrial Benchmark, where GH achieves slightly higher test performance than FM.
On training tasks, the picture is mixed: GH achieves higher scores in some cases (e.g., Industrial Benchmark and Meta- Borad,,rld), while FM is comparable or better in others (e.g.. HumEny, Siner- Gym). This suggests that GH can sometimes fit the training tasks more tightly, whereas FM heads tend to provide better or more robust generalization on held-out test tasks in several domains.
Domain
Online GH
Online FM
Offline GH
Offline FM
HumEnv (Test)
0.070 + 0.023
0.070 + 0.030
0.070 + 0.020
0.130 + 0.020
Industrial Benchmark (Test)
0.990 + 0.003
0.950 + 0.004
0.990 + 0.007
0.950 + 0.007
Meta-World (Test)
0.000 + 0.001
-0.000 + 0.001
0.390 + 0.029
0.460 + 0.034
SinerGym (Test)
0.400 + 0.087
0.900 + 0.025
0.790 + 0.046
0.970 + 0.010
HumEnv (Train)
0.440 + 0.025
0.440 + 0.044
0.440 + 0.023
0.440 + 0.024
Industrial Benchmark (Train)
1.020 0.001
0.940 + 0.008
1.020 0.001
1.010 - 0.001
(Train)
0.960 + 0.002
0.650 + 0.014
0.960 + 0.002
0.990
+ 0.001
SinerGym (Train)
0.920 + 0.009
0.960 + 0.030
0.920 + 0.009
0.970 0.010
Table 34: Gaussian and Flow Matching Heads single-domain models comparison
53

---

Published as a conference paper at ICLR 2026
L COMPARISON WITH VINTIX ON ALL DOMAINS
While Vintix was trained and evaluated on only domains, we were interested in full comparison all 10 domains test set. The per-domain results are presented in Table (5| To ensure fair comparison, we matched Vintix's model size to our setup. As shown, Vintix performs substantially worse than our model.
Domain
Vintix Online
Vintix II Online
Vintix Offline
Vintix II Offlie
Bi-DexHands
0.01 + 0.02 -0.43 0.08
-0.45 + 0.08
-0.42 0.08
0.07 0.05
CityLearn
0.78
0.77 = 0.00
0.78 + 0.00
ControlGym
0.97 = 0.03
0.31
1.00 = 0.00
1.00 0.00
HumEnv
0.04 + 0.06
0.02
0.06 0.02
0.07 + 0.01
Industrial Benchmark
0.42 + 0.05
0.08
0.86 0.05
0.96 0.01
Kinetix
0.20 0.05
0.11
0.17
0.23 0.05
Meta-Wood World
0.05 0.01
0.14
0.09
0.69
MetaDrive
1.02 + 0.00
1.02 0.21
1.00
1.02 - 0.00
MuJoCo
0.98 + 0.01
1.00 - 0.00
1.00 + 0.00
1.00 = 0.00
SinerGym
0.04 + 0.06
0.86 + 0.07
0.08 + 0.02
0.92 + 0.02
Table 35: Comparison between Vintix and Vintix II in online and offline settings across domains on
test
54


Reviews:
[{"review_id": "6qg1hy97OM", "reviewer": "Reviewer_rRdx", "summary": "This paper extends pretrained Decision Transformer models to diverse multi-domain environments and integrates a flow-matching objective for action generation. The authors evaluate on 209 tasks across 10 domains and report that their approach (DPT with flow matching) outperforms baseline methods.", "questions": "- What observation modalities are used in each domain? Are the inputs entirely proprioceptive, or do any domains include image-based observations?\n - How does the performance of the multi-domain model compare to models trained separately for each domain? Is there measurable improvement from cross-domain data sharing?\n - Could the authors provide experiments or discussion on applying the model to high-dimensional visual inputs?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is clearly written and easy to follow.\n - The topic of leveraging past interaction data for generalizable policy learning is important and relevant to the community.", "weaknesses": "- The primary contribution appears to be the incorporation of a flow-matching objective into a Decision Transformer framework. However, flow-matching and related diffusion-style generation objectives have been extensively explored in prior work, which makes the contribution feel incremental.\n - The paper does not clearly specify the types of observations used in each domain. Since the tasks span diverse settings, it is important to clarify whether inputs are proprioceptive states, images, or mixed modalities. If the experiments rely solely on low-dimensional proprioceptive inputs, the significance and applicability to high-dimensional tasks may be limited.\n - Although the motivation emphasizes multi-domain generalization, the architecture groups tasks and encodes them using group-specific encoders. This raises the question of how much knowledge is actually shared across domains. A comparison with models trained on single-domain data would help clarify whether multi-domain training provides measurable benefit.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NeUGYS2JD5", "reviewer": "Reviewer_3WLW", "summary": "The paper presents the Decision Pre-Trained Transformer (DPT) with a flow-matching generative policy head as a scalable agent for cross-domain, in-context reinforcement learning. The authors create a new large, cross-domain dataset featuring over 700 million transitions across 10 domains, and successfully train the DPT agent to achieve high performance on unseen tasks in several domains. \n\n**Recommendation:**\\\nI recommend to reject. The paper is lacking in several fundamental areas, including the following: comparison with baselines, clarity of research methodology, and unconvincing results.", "questions": "- Which contributions are novel, and in what way exactly?\n- How was hyperparameter tuning performed? And how was contamination between validation and test sets avoided? \n- Several potential baselines were mentioned in the related works section, why did you not compare to them? Why are the existing baselines only evaluated on very few of the domains?\n- How did you test for significance of the results? \n- Figure 3 shows mostly overlapping performance between the online and offline setting, and Figure 4 shows mostly no change in performance as context size increases. Do these two things suggest your agent is not actually learning in-context? \n- Figure 4 shows concentration of the action distribution for longer contexts, but how does this actually relate to performance? \n\n\n**Things to improve that did not impact decision:**\n- Figure 2: What do the red lines and shaded regions indicate?\n- Figure 3: How is the performance for the online setting defined? \n- Line 437: Performance on the Industrial-Benchmark domain does not improve significantly with prompt size as the shaded regions are completely overlapping. \n- If each domain requires its own pre-trained encoder, is it really a cross-domain setting?", "limitations": "", "rating": 2, "confidence": 3, "soundness": 1, "presentation": 2, "contribution": 2, "strengths": "- The analysis in Section 4.4 is potentially interesting. \n- A sizeable new cross-domain benchmark and dataset is created. \n- High performance on unseen tasks in 6 out of 10 domains.", "weaknesses": "- Very little comparison with baselines\n- No mention of the significance of the results. Not mentioning number of seeds, the standard deviation or confidence intervals for any methods, makes it impossible to judge any level of significance. \n- Unclear research methodology. There is no mention of hyperparameter tuning or strict validation - test set splits.\n- Reproducability is lacking. The experimental details are very lacklustre. \n- Results not that strong. Only on a single domain (Meta-World) out of 10 does the performance on unseen tasks appear to be better than one of the baselines (Vintix), whilst appearing to be the same as the other baseline (REGENT). Furthermore, performance in the online and offline setting are overlapping in most domains, suggesting it doesn't actually learn in-context at all. This is corroborated by the mostly flat curves in Figure 5. \n- Overclaiming in the introduction. \n- Lack of background section makes it difficult to judge novelty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UioWRdM4tt", "reviewer": "Reviewer_FmEz", "summary": "This work presents an in-context reinforcement learning (ICRL) method that extends the Decision Pre-trained Transformer (DPT) architecture with flow matching to model complex output distributions. An empirical study on a large cross-domain dataset demonstrates the improved performance of this design compared to baselines.", "questions": "- What is the source of the optimal actions used for training?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The writing is clear and detailed.\n- The empirical study is comprehensive and fair.\n- The method improves upon the original DPT without too much overhead or modification.", "weaknesses": "- The paper lacks a background section to formalize RL, ICRL, and flow matching, which are the focus of this work. \n- I don't think the removal of the next observation $o'$ is well-justified, as it is crucial for capturing the dynamics of the environment, especially when the context is randomly permutated. Without $o'$, RL algorithms that optimize the policy based on the reward signal would not work unless it's a bandit setting. In this case, the DPT is likely merely learning by imitation and using the contextual information for task identification. Even in imitation learning, I doubt removing the next observation will leave the learner intact because, as I pointed out previously, $o'$ is crucial for capturing the environment dynamics. I suspect that the authors' claim that removing it will not impact the performance is due to that the dynamics of the testbeds are somewhat consistent across tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Bmg9VYHBAq", "reviewer": "Reviewer_KAK3", "summary": "This paper extends the Decision Pre-Trained Transformer (DPT), a large-scale transformer model, to diverse reinforcement learning domains using a decision-transformer-style objective. DPT integrates a flow-matching policy head to better model multi-modal continuous-action distributions and is trained on a large multi-domain dataset. The authors evaluate DPT both in offline settings (fixed demonstration contexts) and online settings (autoregressive conditioning on recent transitions). The authors publish the used large-scale dataset with over 700m transitions. The paper also draws a connection, empirically, to the posterior-sampling interpretation of in-context learning, suggesting that DPT behaves as a Bayesian in-context learner following the reasoning of Lee et al. (2023). Empirically, DPT achieves strong performance across most domains and outperforms existing generalist RL models such as Vintix and REGENT on several benchmarks.", "questions": "- Could you provide ablations isolating the contribution of the flow-matching policy head relative to standard Gaussian heads?\n- Could you clarify the experimental protocol for the baselines? Were baselines retrained on your dataset or reproduced from prior work?\n- Do you plan to release the code publicly to support reproducibility and benchmarking?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is well-written, well-structured, and easy to follow. The dataset composition, architecture, and training setup are described clearly, and the figures are of high quality.\n- The scale of the dataset and experiments is high. The construction of the model, as well as the dataset, covering 10 domains is a significant engineering effort. The resulting benchmark could be a valuable community resource.\n- Replacing the Gaussian policy head with a flow-matching head for continuous actions is a natural and elegant design choice. \n- DPT performs consistently well on both seen and unseen tasks, with significant gains in the more complex domains.", "weaknesses": "**Novelty**:\n\nThe paper would benefit from a clearer articulation of what exactly differentiates DPT from prior large-scale decision-transformer architectures such as REGENT and Vintix. Currently, the novelty seems to lie mainly in the use of the flow-matching policy head and the dataset scale. A comparison table summarizing architectural and algorithmic differences (policy head, encoding choice, inference procedure, etc.) would make the contributions clearer.\n\n**Ablation studies**:\n\nI would appreciate more detailed ablations that would allow a more finegrained assessment of what architectural choices contribute most to performance. Primarily the choice of using a flow-matching algorithm for policy heads comes to mind here, but also the choice of embeddings and context ordering, or context length.\n\n**Experiments**:\n\n- it seems that the selection of experiments makes it somewhat hard to draw quantitative conclusions from the chosen implementation choices, because most of the included domains appear to be solves by all tested models. Only few (Meta-World and Bi-DexHands) provide sufficient difficulty to distinguish algorithms. \n\n- Although the results are strong, I could not infer whether all baselines were re-tuned on the same dataset and training budget. In particular, Vintix and REGENT may have been evaluated under different protocols. Clarifying this would improve the fairness of the comparison.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)