Latent Adaptation of Foundation Policies for Sim-to-Real Transfer
Authors: Longchao Da
T Pranav Kutralingam
Lirong Xiang
Hua Wei
Venue: ICLR 2026 Poster (2026)
Keywords: Sim-to-Real
Domain Adaptation
Foundation Policy
Primary area: transfer learning, meta learning, and lifelong learning
TL;DR: This paper introduces a foundation policy with latent adaptation that enables efficient and robust sim-to-real transfer without costly retraining.

Abstract: The sim-to-real problem remains a critical challenge in the real-world application of reinforcement learning (RL). The conventional sim-to-real methods heavily rely on resource-intensive re-training of the policy network to adapt to new domains, which limits the flexibility of the deployment of RL policies in ever-changing environments. Inspired by human locomotion, where individuals adjust their gait to new surface conditions without relearning the skill of walking, we introduce Latent Adaptation of Foundation Policies (Found-adapt), a framework that decouples this problem into skill acquisition and environment adaptation. Our method first pretrains a foundation policy on unlabeled offline trajectories from the source simulator, capturing diverse long-horizon behaviors as reusable skills. At deployment, instead of retraining the policy, we perform efficient latent space adaptation: a small amount of target-domain data is collected to refine a latent representation through an adapter network that incorporates parameter efficient alignment, which produces a task-ready controller under various system dynamics. This adaptation occurs entirely in the latent space, avoiding costly policy optimization while enabling robust transfer. Empirical results across multiple locomotion tasks and dynamic variations demonstrate that our method significantly reduces the sim-to-real gap. Further sensitivity analysis provides interesting insights into the requirements for data quality and applicable situations. These findings highlight how foundation policies with latent adaptation could serve as a general and efficient paradigm for real-world RL deployment.

Published as a conference paper at ICLR 2026
LATENT ADAPTATION OF FOUNDATION POLICIES FOR SIM-TO-REAL TRANSFER
Longchao Da', Thirulogasankar Pranav Kutralingam Lirong Xiang² & Hua Wei Arizona State University. "Comeell University [Longchao, tknolast, hua. wei waw.eea...eeeeeeeee edu, Lxiangeeornel..dd
ABSTRACT
The sim-to-real problem remains a critical challenge in the real-world application of reinforcement learning (RL). The conventional sim-to-real methods heavily rely on resource-intensive re-training of the policy network to adapt to new domains, which limits the flexibility of the deployment of RL policies ever-changing en- vironments. Inspired by human locomotion, where individuals adjust their gait to new surface conditions without relearning skill walking, we introduce La- tent Adaptation of Foundation Policies (Found-adapt) framework that decou- ples this problem into skill acquisition and environment adaptation. Our method first pretrains foundation policy on unlabeled offline trajectories from the source simulator, capturing diverse Long-hrizzon behaviors as reusable skills. At deploy- ment, instead of retraining the policy, we perform efficient latent space adaptation: a small amount of target-domain data is collected to refine a latent representation through an adapter network that incorporates parameter efficient alignment, which produces a task-ready controller under various system dynamics. This adaptation occurs entirely in the latent space, avoiding costly policy optimization while en- abling robust transfer. Empirical results across multiple locomotion tasks and dy- namic variations demonstrate that our method significantly reduces the sim-to-real gap. Further sensitivity analysis provides interesting insights into the requirements for data quality and applicable situations. These findings highlight how founda- tion policies with latent adaptation could serve general and efficient paradigm for real-world RL deployment. Implementation and experiment are available here
INTRODUCTION
Deep Reinforcement Learning (DRL) has enhanced intelligent decision decision-making by trial-and-eme fashion policy optimization. It can learn complex multi-tasks from video games (Shao et af.) 2019) and sophisticated robotie manipulators (Nguyen & La 2019) directly based raw observations, which demonstrates superhuman performance in some environments with high-dimensional state and action spaces (Wang al., 2016). However, real-world deployment of RL policies still lags behind the simulator successes (Da et al. 2024a); mainly because the policies trained in the simu- lator can rarely generalize across domains with different system dynamics. disturbances, and sensor noises, etc. (Da et al., 2025a), i.c., known as the Sim-to-Real Challenge (Wagenmaker et al. 2024).
Simulator
Tasks
Adapt.
Real World
Simulator
Flexible Adapt.
Real World
Figure 1: The difference between traditional sim-to-real adaptation (left) and the proposed method (right). Traditional sim-to-real involves two steps: task policy network training and task network adaptation, both of which are expensive and tightly bound. Proposed foundation policy æ enables flexible adaptation at an arbitrary task in any environment dynamics.
"Corresponding Author.

---

Published as a conference paper at ICLR 2026
Bridging such sim-to-real gaps for RL policies remains challenge in robotics, autonomous driving, and other safety-critical domains (James et al. 2019 Daza et al. 2023: Da et al. (023a). Traditional
approaches, such domain randomization (Huber et al] 2024) and domain adaptation (Farhadi et al. (024), typically rely on extensive online training for enough dynamics distribution coverage in the target domain: by either iteratively collecting real-world rollouts deploying costly sys- tem identification procedures (Song et al.) (022)) etc. This generally require substantial training resources to mitigate the gap. Thus, two fundamental challenges persist: (1) addressing new tasks requires retraining for task-specific policy prior to adaptation. (2) Adapting to new system dynamics necessitates re-training of the sim-to-real process, which incurs significant extra efforts and costs.
Inspired by (Sterelny 2012), humans typically rely on previously acquired skills and adapt them to new environments without relearning from scratch, Instead of re-deriving motor primitives every time, people reuse shared repertoire of skills and flexibly adjust them according to context and feedback. This analogy motivates the use of offline RL (Chen et al. 2024). where policies can be trained on diverse prior experience and later specialized to novel domains without costly online training. It serves as a foundation policy paradigm, which has offered a more generalizable policy acquisition by decoupling representation learning from task-specific policy training. By capturing generalist 'skillsets' from massive offline datasets, these methods can later instantiate near-optimal behaviors in zero- few-shot fashion to even unseen tasks (Luo et al. 2025), simply by condition- ing on small amounts of prompts (Park et al.] 2024b). Given the generalist potential, för challenge (1), we proposed to leverage the foundation policies as the basic architecture for sim-to-real tasks to alleviate the re-training burden for policy networks. Then, for challenge (2), to better tackle the sim-to-real gap, we propose Found-adapt, latent space adaptation method that efficiently provides online adaptation with three steps: Pross-domain initial alignment, Dynamics signature extraction, and eventual Dynamics signature guided adaptation.
With only a small amount of target-domain data, our lightweight adapter estimates an appropriate latent representation that bridges the source and target dynamics. This enables effective sim-to- real adaptation without heavy retraining, while maintaining the ability to generalize across novel tasks. We have verified the effectiveness of the proposed method, which reveals great potential in real-world usage of RL policies. In conclusion, our contributions are threefold:
Inspired by evolved apprentice, we propose to solve the sim-to-real challenge by leveraging the foundation policies with generalizability and get rid of task-specific retraining.
We propose a novel and efficient adaptation method that leverages foundation policy rep- resentations and higher-order (meta) dynamics to derive the policy that bridges the source and target distributions during execution, improving the policies' sim-to-real ability.
Broad Evaluation and Real- World Applicability. We demonstrate Found-adapt on multi- ple zero-shot RL tasks and environments, showcasing its strong transfer performance and practical potential by easily adapting to both new tasks and new environments.
2 RELATED WORK
Sim-to-Real Methods in RL There have been explorations to tackle the sim-to-real problems by interventions at different stages of the MDP process. In observation, there are practices like domain randomizations (Tobin et al. 2017 Tiboni et al. 2023), domain adaptation (Hu et al. 2022 Ho et al.) 2021), and sensor fusions (Mahajan al. 2024 Bohez et al. 2017). which intend cover broader distribution during the policy training to overcome the potential shift. In action, there are works tackling the action latency Amold al. 2019 Dezfouli & Balleine| 2012) and quantifying the action uncertainties (Ilhan et al.] 2021 Da Silva et al. 2020). Apart from the above, recent work intends to bridge the transition dynamics (Da et al. (025a), a popular branch of work leverages grounded action to better calibrate the dynamics shift (Hanna & Stone 2017 Da et al. 2024b), but all of these methods require extra training to shrink the domain gaps of the learned policies, which require extensive training leverage the pre-tedi foundation models with existing and reusable skillsets, and design a way to efficiently prompt them to solve tasks in specific domain with efficient computing overhead.
mass of data. Different from these traditional methods,
Unsupervised Policy Pre-training Previous studies have introduced a variety of unsupervised, task-agnostic objectives for pre-training diverse set of policies, thereby speeding up subsequent
2

---

Published as a conference paper at ICLR 2026
task learning. These methods pre-train policies with either exploration (athakk et al.] 2017 2019
Mendonca et al. 2021 Rajeswarr et al. (023) or skill discovery objectives (Gregor et al. 2016 Ey-
senbach et al. 20188 Sharma et al. a 2019 Klissarov & Machado 2023 Park et al. 20233, Ditferenr
from this direction, we focus on the offine setting, where we could leam and evaluate diverse poli-
cies purely from an offline dataset with unlabeled trajectories (Da et al. (024c)
Adaptive Policy Networks Another line of work focuses on motor adaptation through end-to-end policy networks, such as Rapid Motor Adaptation (RMA) (Kumar et al. (202]) and Universal Policy with Online System Identification (UP-OSI) (Yu et al. 2017). A line of follow-up research inspired by RMA in real-world robotic systems includes bipedal locomotion (Kumar et al. 2022). J in-hand
object manipulation (Qi et al. 2023), manipulator control (Liang et al., 20024), and humanoid real- world policy adaptation (Hu et al. (025), they perform adaptation by inferring latent dynamics from recent observations, specifically, they adapt policies to varying dynamics by jointly learning task objectives and online adaptation mechanisms, but they remain task specific and require extensive online training. which limits their generalizability. Different from their method, we propose a gen- cralizable sim-to-real foundation policy that can adapt to various tasks and system dynamics more flexibly and without per-task retraining.
3 PRELIMINARIES
The Sim-to-Rest Problem in RL Following the classic definition (Sutton et al. 1998 Ding et al., 2020 Da et al. 2024b Turnau et al. 2025), we formulate reinforcement learning in the standard Markov decision process setting (MDP): M (S, A, Pr., r, μ4, 7), where S is the state space, A the action space, P S x A- + ASS) refers to the transition probability. The reward function can be represented as r SxxxxxxSSSS - R. μ the initial state distribution, and the discount factor is y E (0, 1]. A policy in RL, such as T(lss), defines a distribution over actions given state 8, and its learning objective is to maximize the expected cumulative discounted reward E, [E,- 7'r(se, ae, $1+1)]
In practice, due to the cost of real-world exploration consequences, RL policies in a simulator Esim and then exccuted in a real environment Ereat for testing or deployment (Salvato let al. 2021 Zhao al. 2020). But since Esim always holds differences in dynamics compared to Ereal. L.e,, Psim Preal thus, the policy often suffers a performance drop in Eyeal, which is denoted as the Sim-to-Reul gap.
usually trained
Foundation Policies. Inspired by foundation models in vision and language (Da al. 2025b),
foundation policy is generalist control policy trained from diverse offline trajectories, intended to capture broad repertoire of reusable skills without specializing single downstream task. Given offline trajectories 'D collected from a source environment Exim: we define foundation policy as a pair (1), x). where Φ : S -> is a state encoder mapping states into latent space and (alss, 2) is a latent-condittoned policy with E Z. The encoder & extracts reusable behavioral primitives, while varying. spans a family of skills that can be composed to address different tasks. In this sense, foundation policies naturally support task prompting, where new rewards or goals are realized by selecting an appropriate latent. Moreover, their condiicond structure also offers the potential for system adaptation, since adjusting provides mechanism to align policy behavior with changes in environment dynamics. Our work develops this perspective by introducing latent adaptation techniques that unify task and system adaptation within the same framework.
4 METHODOLOOGY
In this section, inspired by existing work (Park al. 2024b), will first introduce the learning of the policy model in Hilbert Space, as an instantiation for the foundation policies, and then we introduce details how adapt the latent space solve the sim-to-real challenge while adapting towards various tasks based on the foundation policy's 's structure.
In this paper, source env. Feim and target env. Freel might be used interchangeably.
3

---

Published as a conference paper at ICLR 2026
Step1. Unlabeled Offline Data
Step2. Learn Encoder: ø
Step3. Foundation Palicy
Hilbert
Ø State Encoder
Space
pt
Trajectories
Esim
Support
Apply
Direct Direct Deploy
g
d' (*.g)) zo (10(2) ا())ه
2(8,2,8) (o(n') ((t), #)
A. Cross Domain
Initial Alignment
Step4. Found-adapt
Zymc
C. Signature-Guided Latent Adaptation
Adapter
Ffinal
<[aas, Ffimal)
Dreal
8"
arg min
Lurg, 96 (Fare:
r
Ereat
B. MetoDynamic (permutation-ivvaiint signature)
Figure 2: Overview of the proposed method. Offline trajectories from the simulator Esim train state encoder & and latent-conditioned policy π [aass, using intrinsic rewards. Direct deploy- ment degrades under dynamic gaps. We therefore perform latent adaptation with small batch of target-domain data Dtar: (i) a weighted joint least-squares fit yields an initial latent "ere (ii) a Meta- Dynamic network extracts refines & into Zfnal. The refined latent conditions π for robust execution in the target environment Frar without retraining the policy.
ionuutaaiioinivriill distributional features yy. (iii) an adapter network
4.1 FOUNDATION MODEL IN HILBERT SPACE
Based on the definition in Section one concrete instantiation of foundation policy is given by embedding states into a Hilbert space. The key idea, introduced by (Park al. 2024b), is that distances in this space can be aligned with temporal relations in trajectories, which makes the representation suitable for skill composition and task adaptation. Formally, an encoder maps states into latent space Z = R² with Euclidean inner product. The encoder is trained so that [|d(s)) - ((g)) || approximates the true temporal distance between (certain state) and g (goal), thus capturing the long- horizoo structure of trajectories.
+ Z
Given such an encoder, latent conditiend policy (a|s, 2) with 2 2 is trained using an intrinsic reward rss, 2, s') = (ó(s') (0s), z). The intrinsic reward aligns policy transitions with the latent vector z, ensuring that for any direction = E Z the policy can induce state changes consistent with of directional primitives), and the collection {T(-|s,
(thereby forming be viewed as a skill family that spans the embedding. For downstream tasks, one can solve for an appropriate =" by aligning the latent prediction with the task reward, which reduces to least-squares regression on offline samples. In the special case of goal reaching, =" simplifies to the normalized vector between state (is) and goal @(g). yielding closed-form solution.
2} can
This enables task-level adaptation without retraining the policy parameters. However, the solution implicitly assumes that the transition statistics used for representation learning Esim remain con- sistent at deployment in Ereat- In practice domain shift in dynamics often invalidates this assump- tion. leading to degraded performance. To address this sim-to-real gap. we next introduce latent adaptation mechanism that augments the above regression with additional components for robust cross-dynamics deployment.
4.2 LATENT ADAPTATION FOR SIM-TO-REAL TRANSFER
Baseline latent vector 2 for foundation policy. To deploy Hilbert foundation policy new task, one solves for the optimal latent vector as follows:
arg
E (s =||=11
min
(B(s, a, (0(s') - o(s), =))²
(I)
where R(s, a, s') is the task reward and @(s) is the Hilbert embedding from Section 4.1 In practice,
given samples (( ai r.)) 1 let + (0($)) d(81), ... d(5x)) ((sn)) R Nx x
and

---

Published as a conference paper at ICLR 2026
fri...,,, rN] E ERR. Eq. reduces to the standard least-squares problem:
2 arg min || r12 ZE
which yields the closed form solution: 1 ( 4) - derivation is shown in Appendix C
After normalization to unit norm, we obtain x" 2/ 2/1 For goal reaching tasks with target g. this further simplifies to the normalized vector z" = ( - (6)) (+))/|(((g) o(s)]--
The performance gap introduced by the dynamics gap. The baseline inference in Eq. Tasumme that the transition statistics used during training remain valid at deployment. Formally, let the simu- lator dynamics be Psim (s' s/|s, a) and the real dynamics be (s a). We denote their discrepancy as As(s (s, a) (s (s a) - Psim (s' (s' a). This difference propagates into the feature matrices, since each row is defined by the embedding difference ((s') ((s). Thus, if 4 Psim the feature matrix computed under Psim. then under real dynamics we have 4 Praal Dsim + A
where As
collects the deviations induced by Ap.
The least- estimate under simulator data is Esim ( ssim) 1
sim ?, whereas the true optimal solution under real dynamics is &real = (Q. Preal) (eea)))1ggh - real". Substituting Preal Dsim + Dg reveals that &real * Sii and the difference As 2real siim shows the dynamics gap the level of the latent solution. Without explicitly correcting for As, deploying <(ass, îsim) in Ereat will generally lead degraded performance.
Initial Alignment. To mitigate this gap, One possible way to address this issue is to re-train the state encoder & using additional trajectories from the target environment Ereat, i.e., constructing a new dataset Dear {(s}, at, r and updating so that aligns with P Preal- However, such re- re-training is computationally intensive and reduces the modularity of foundation policies by entangling representation learning with deployment. Instead, we propose keep fixed as learned from Feim and directly adapt the latent variable toward Freat by leveraging D Concretely, this requires only solving regression problems of the form min, || Tsim l!2 + 2|| Ttar 2 and performing lightweight parameter updates for adapter at inference time. This procedure avoids costly re-optimization of ø, yet yields žfinal that conditions π ( Zfinll for robust deployment in Ereal-
A.
Since we tend to consider samples from both the simulator and the real conditions, we extend the least-squares step. Suppose we have Dare {(s,, a,, & r.),, from the simulator and D {(5), āj. 8), F5 from the real environment. We form the feature matrices P.im [o(s]) ( E NRDD and Ptar (d(3,)
ر E MxD with corresponding reward vectors
Tsim E RN and Ttar E RM We then solve the weighted regression as follows:
Zsrc arg min || | sim Fsim 11 + + 2||
Ttar
ZERD
where > biases the fit toward real data. This admits the closed form Zare = (ATA)-1AT
with A
t and
sim
Artaa followed by normalization. This weighted joint regression
rsim
integrates information from both domains, allowing the latent solution to provide an initial, partially corrected dynamics gap while preserving the efficiency of closed-form inference.
B. MetaDynamic A Permutation-Invarii domain alignment addresses first-order differences in transition features, it may fail to capture higher-order variations such distributional shifts in state visitation structural patterns in dy- namics. To address this, we compute permutation-invalte summary of the target-domain:
Network for Dynamics Signature. While cross-
1 Metaa)ynamic((o(3)))
g
where MetaDynamic is a set-encoding function trained on simulator data and frozen at deployment. The permutation invariance serves a critical component, since ( is an unordered collec- tion of states sampled from trajectories rather than a sequence with fixed ordering; the representation
Here & denotes feature matrix derived from the encoder á; is additional Icarnable parameter.
5

---

Published as a conference paper at ICLR 2026
must depend only the empirical distribution of states, not sample order. We interpret dynamics signature of the target domain, ie., a compact descriptor encoding statistical regularities of Ereat beyond mean feature alignment. This signature serves auxiliary input to refine latent adaptation, enabling the policy to exploit structural information about the target dynamics without retraining the encoder ş). The details of the MetaDynamic structure and training configurations provided in the Appendix.
C. Signature-Guided Latent Adaptation. Although cross-domain alignment partially reduces the dynamics gap, the latent solution Eare may still drift when directly applied in Eiar To further adapt, we introduce a parametric adapter : R2 x E + R2 that leverages both the initial solution Eare and the dynamics signature 7). The adapter is initialized to approximate the identity mapping, i.e., go(2are- 17) R 2ere. ensuring stability at the start of adaptation. We then update 0 using the target-domain dataset Darr updating the parameters for a small number of gradient steps by minimizing below, the loss fj conditions on the solution Zere and the dynamics signature ry:
M
( = M
1
Σ ==1
Pur. 90 (Zrae: 11 - 53 1
(5)
()(0)
where Ptar.j is the j-th row of the target-domain feature matrix and řÿ is the corresponding reward. This produces an adapted parameter set 0* and yields refined latent vector:
90- (Zare: 17)
Zinal
D
1|g8- (Zare> 7?]l
The resulting 2tinal incorporates both cross-domain alignment and distributional structure, and con- ditions the foundation policy π (a|s, Zfmaat) for robust execution in Etar without retraining the policy.
In summary, the proposed latent adaptation provides a parameter-efficient approach to sim-to-real transfer. From the perspective of conventional RI. policies, which typically require costly policy æ retraining on target-domain data, our method is lightweight and avoids modifying the policy network altogether. From the perspective of foundation policies, which were originally conceived for task prompting under fixed dynamics, our method repositions them as a versatile paradigm that can adapt not only across tasks but also across environments with differing dynamics, thus transforming their role from intra-domain generalization to a unified framework for sim-to-real transfer.
4.3 OVERALL FRAMEWORK
As shown in Fig. our framework reframes foundation policies to address the challenging sim-to- real transfer problem through prompting and parameter-eeffiiiet domain adaptation. We begin with offline trajectories collected from the simulator Fsim. which are used to train a state encoder & and a latent-conditioned policy [[ass, 2). This training stage produces foundation policy that encapsu- lates a repertoire of reusable skills indexed by latent variables z, supporting flexible task prompting without modifying the policy parameters. However, direct deployment in a target environment Etar is hindered by dynamics gaps that render simulator-derive latents suboptimal. To overcome this, our latent adaptation procedure adjusts while keeping fixed: we propose (i) cross-domain regres- MetaDynamic encodes a dynamics signature y), and (iii) a signature-guided adapter refines Zere into Ztimall The resulting žfinal conditions the frozen foundation policy for robust execution in Etar- This parameter-efffcient design redefines the scope of foundation policies, elevating them from intra-domain task generalization to a unified paradigm for sim-to-reat deployment.
sion yields an initial 2xre- (ii) permutation-invaiiae
EXPERIMENTS
In this section, we experiment on environments from a widely adopted bench- mark in offline RL (Park et al. 2024a). We test different methods' sim-to-real transferability using the sim-to-sim setting, which is a common practice to verify the methodology for better reproducibil- ity (Da al. 2025a Zhao al. 2020 James al. 2019) treat the default environment setting
Dataset and Tasks:

---

Published as a conference paper at ICLR 2026
Setting
Methods
Task's 's Average Return (Sim to Real Gup)
Average Time Cost
Hilbert Space Embeddinng
Stand(A t)
Walk(A
(J)
4(s) under dynamics perturbs
Excos
Foundation Policy
764.99
a.73
Transfer
494.24 (-393.37)
318.93 (-446.06) (-446.066)g177.
0.05
Vanilla-AAT
376.27 (-511.34) _
185.66 (:579,33) 19122
7,59 = 1.70
UGAT
326.49 (-438.50 -4)3850))
9.15 2.50
PAD
$57.03
(-330.58)
448.50 (-316.49)
Found-adapt
562.72 (4202)2))))
472.25 4-292.74)
6.22s 0:12
Direct- Transfer
222.49 (-6.5 (-665.12)mmm
150.98 (-614.01)
5,36 1: 0.11
Vanilla-GAT
60.57 (-827.04) (1487
(-693.03)_s4..0
7,56 2.30
G2
UGAT
175.42 (-712.199999)
163.25 (-601.74) 240
9.10s + 2.90
273.16 (-614.45))
189.17 (-575.82)
Found-adapt
231,75 (-655.86)
182.46 (-582.53)
6.11 + 0.07
Transfer
213.15 (167,466--664
105. (-659. g
5.14s
Vanilla-AAT
32.28 (-855.33) g12m
28.80 (-736. 1933
7.71 = 2.70
UGAT
(-815, 34425
16.45 (-748.54)
9.10 1.70
PAD
262.21 (-625.40)
127.96 1-637.03)
Found-adapt
322.06 (-565.55)aas..a
120.87 (-644, 12)444
6.08 0.11
Direct- Transfer
63.81 (-823. 80)
33.03 (1733.66)_44.
5.28 0.09
Vanilla-GAT
57.85 (-829. a20027
25.42 (-739.57 (7).57)) (1)
7,61 2.50
G4
UGAT
80.33 (-807) 228) 144.76
14.51 (-750.48) 4844
8.63 + 2.40
PAD
78.69 (-808.9 (-808.92) ww
37.13 (-727) a9.a
Found-adapt
71.70 (-815.9 1111
42.57 (-722/42)
6.12 + 0.11
Table 1: Performance under Gravity variations comparing baseline methods, and Found-adapt. Each cell shows the average return with the sim-to-real gap relative to Ésim and the standard de- viation over runs. Higher is better for return, and lower is better for time cost. The best average per column is shown in bold, and the second best is underlined. PAD runtime is marked as '-,, indicating not directly comparable (hour-scale). The right column shows 2D t-SNE of the encoder embeddings ((s) under gravity perturbations (Go, G1, and G4). As the dynamics perturbation in- creases, the manifold structure observed in Go progressively distorts, revealing representation drift under perturbed environments, more comprehensive results shown in Appendix, Fig.
as Esi and control various Ereal by designing high fidelity system dynamics following the configu- rations in Table 3 In brief, there two sets of variables of settings: friction and gravity, which are applied at each joint and affect degree of freedom, to reflect high-fidelity system dynamics.
Baselines: We compared with several classic and advanced baselines provide empirical insights. The first baseline is Di Dirt--Traas fer, which is the foundation policy learned in Evim and di- rectly deployed to Ereat- Then, we compared with the grounded action transformation (GAT) (Hanna & Stone 2017). and uncertainty-based GAT (UGAT) (Da et al. 2023b). which is developed by post-training grounding module for the foundation policy to ground the actions provided by the foundation policy, and UGAT integrates an evidential layer to derive the uncertainty and use it to re- ject low low-quality actions. Besides, we provide four versions of Found-adapt for the ablation study: F (init) (closed form solution T [aas, =x)) by cross-domain initial alignment from Sec. 4.2 (dyna) (= directly calculated from MetaDynamic network in Sec. 4.2 F (init, dyna) (make use of both initial alignment and meta dynamics signature, but without online adaptation.) and F-all (i.c., full model of Found-adapt). There are methods in deployment adaptation branch, however, this branch of methods not adaptive to different tasks, thus, it needs be pretrained on specific tasks then perform deployment adaptation individually, to represent this research direction, we implemented classic baseline: Policy Adaptation during Deployment (PAD) by (Hansen al.) given the high time cost, we only provide magnitude in evaluation.
The primary goal of this work is to mitigate the performance gap of the derived policy Ta in the simulation environment Exim and in the real-world environment Ereat, thus we calculate the performance difference A for the episode return of each task. We denote their differences as Ageum For given metric 'Episode Return' in two domains. Normally, Ryeal Rgim- thus the gap AReuM is negative, and the higher, the smaller the gap (f), applied in Table.
Evaluation Metrics:
! Aetum Rreal Raim
7

---

Published as a conference paper at ICLR 2026
942.6
Stand
Walk
Flip
Run
Averege Return
Retum
Direct
628.6
Difluity
Dificdyy
Diflalyy R-W
netrrbyy
CAT
Dura
UGAT
PGU
e
47
The performance on the friction-based settings (F1 F6). In this image, from left to right, the blue bar shows the performance drop in the original system dynamies (Esim) along with the increase of the difficulty of the system (reflected by the friction severeness as in Table. 3 The foundation policy can adapt to various tasks, and our method shows best sim-to-real gap mitigation.
Figure 3:
5.1 EXPERIMENT ANALYSIS
(1) The performance comparison among the baselines. In this section, we tend to verify the performance of our method in comparison to baselines. In Table [ we compared Found-adapt with four baselines under various gravity changes, including three types: Direct Transfer, grounded action transformation (GAT, UGAT), and deployment adaptation (PAD). It is worth noting that de- ployment adaptation methods require pretraining on task-specific data collections; thus, this branch of method can not provide absolute fair comparison to the task: 'Adapt policy various tasks and multiple system dynamics at mean time without further training". More details please visit Appendix E (3). We evaluate 4 different tasks in the Walker environment, comparing different methods' performance (return) and sim-to-rea gap in brackets (the higher the better), meanwhile, we also compare the time cost during online adaptation. Our method consistently ranked the top 2 performing methods. Sometimes it is slightly worse than the PAD; however, PAD requires task- specific pretraining and updates are coupled downstream behavior, whereas our method leverages a task-agnostic foundation representation shared across tasks: stand, flip, etc. We also perform comprehensive evaluation on friction settings, as shown in Fig. 3 from stand to run. As the dif- ficulty increases, the overall performance drops; however, as shown in the red line, our method successfully makes an improvement from the blue bar (which is the direct transfer performance in Ereal). The adaptation process with the latent representation changes is shown in the Fig. (b))
800
660
walk
Performance
600
Autoptation
Improvement
400
300
Return
400
200
400
300
200
20
200
Dirett Transler
15 Fridton
200
40 Groviyy
30
10
110
5
init
dyna
init+dyna
full
(a) The ablation study of Found-adapt on the friction Fl setting. Dotted box shows full model improves across tasks, and adaptation is necessary, as in init tuttttd t-dyna (w/o. adapt). Figure 4: The partial results of case studies on the ablation and sensitivity of proposed Found-adapt.
(b) landscape on Walk of both factors analyze the difficulty of the optimization task, our method provides consistent improvement.
(2) The ablation study of the proposed Found-adapt. The proposed method mainly has three components: Initial alignment solution Exre, MetaDynamies network with signature 7) and Latent adapter go- (Sec A-C). We perform the ablation study to understand the contribution of different components in the model's result. As shown in Fig. (a), F(init) uses only Zarcs can score well on the easy stand task but exhibits high variance other tasks because it fails to capture higher-order dynamics features; F(dyna) uses only dynamic signature 77. underperforms since the sole signature struggles to produce a valid latent 2; interestingly. the F(init-dyna) performs worst, since it applies simple merge with žere and 17, as the latent mixes uncalibrated cues from alignment and higher-order dynamics, it fails to exploit either;
add the adapter go (Fq. 5). F(all) ours: yields consistent
8

---

Published as a conference paper at ICLR 2026
gains on all tasks (dotted box) by reconciling eere with 77, reducing bias and variance, and turning the dynamics signature into extra corrections, more results are in Fig. g
(3) Study of the system variable sen- sitiveness and the model's ability. We
Return (1s2)
Loss (1e-3)
analyze how performance changes with friction and gravity jointly on Walk task. The results in Fig. (b) show two- factor response surface where the adapted policy (gradient red surface) consistently lies above direct transfer (blue), indicat- ing non-negative lift across the domain. The gap (improvement) is most visible harder ranges: strong gravity shifts (-34 to -44) combined with moderate friction, suggesting that adaptation receives larger gains where the sim-to-real gap is wide. Moving toward the nominal dynamics (gravity ~ -9.81, friction s 1), the two surfaces converge, but ours remains slightly higher, evidencing reliable improvement even in mild conditions. Overall, despite the non- linear and rugged landscape, the adapted model maintains robustness and yields the largest benefits in adverse regimes.
3D
Adapter Loss Meen Verience
R(FoulRar) 283.5
drop
mask
noise
[e] Comuiption (86)
(b) Adaptation Steps
Figure 5: The (1) data quality analysis under three
modes, and (2) latent vector evolution 2 - Zrefined
(4) The analysis of the requirement of target-domain data. In collected target-domain data, Diar ([sj.Ty, s of) We corrupt transitions with rate p% that selects an index set J c (L...., We study three modes that mimic common data issues: (i) drop: remove the selected tuples r.. s bee from Diar (data sparsity); (ii) mask: keep length but set 8j 0, 8/ for j J (bad imputation / perturbations scaled by the empirical per-dimension standard deviation to the raw signals, i.e., sj s sy t1, sj s 2 +2, F with c1, E2, E3 ~ (0,, o²), simulate the label noise scenarios. We evaluate on the consistent Walker environment, stand task under gravity GI in Table. B As shown in the Fig. 5 All three modes only suffer slightly with drops when the corruption is less than 50%, showing our method relatively robust to data quality. Then, Performance degrades most gracefully under drop, indicating method could capture the dynamics signature from little data with high efficiency. mask fails early: means zero-filled values inject systematic bias, and hinders the adaptation, pulling the latent toward degenerate solutions once Z 50 - 60% of data are masked. The noise hurts consistently under heavy corruption (2 50%). In practice, when data are suspect, it is suggested to discard rather than zero-fill pero-filling when applying method, keep noise low, and consider denoise objectives noisy data are unavoidable.
0, and r,
-as-zero' bias); (iii) noise: add Gaussian
(5) The relationship of the adaptation loss and the sim-to- real ability. This study aims to understand how the adapta- tion process advances the sim-to-real ability of method. We treat the 'Stand' task as an example. the process, we leveraged the pre-trained foundation policy, and applied our method; then, we recorded the loss changes (step-wise) during the adaptation of network go as in Eq. []] by this, we derive the n. i E [1, 60) from 60 steps. We test each n, 's perfor- mance and calculate the improvement as in Eq. 7 To better illustrate the correlation, we take negative value of the loss along the x-axis and then normalize both dimensions. On left side of Fig. 6. the grey area shows the 95% confidence interval around the fitted regression line, on the right side, we show the KDE of the joint distribution to move beyond linear analysis, it shows a high-density ridge aligned with the regression (darker + denser), confirming monotonicity and the scarcity of cases where lower loss yields better performance ( indicates p < 0.001). This study reveals that: as 'oloss' increases (i.e., the better adaptation results), the sim-to-real per- formance consistently grows
1.0
0.73 **
Improve
0.5
0.0
0.8
1.0
0.8
1.0
-Loss
Figure 6: Correlation of the adap- tation loss and the target domain Ereat performance improvement.

---

Published as a conference paper at ICLR 2026
CONCLUSION
This work presents the study of leveraging foundation policies for adaptive sim-to-real transfer. The framework pre-trains versatile foundation policy from offline simulation data and then employs a lightweight latent adapter, inferred from limited target-domain samples, to align with real-world dynamics efficiently. This design enables rapid adaptation without costly retraining, mirroring how humans reuse previously acquired skills and flexibly adjust them to novel environments.
Empirical evaluations across diverse locomotion tasks and varying dynamics factors, such gravity and friction, demonstrate that our method mitigates domain gaps and achieves robust transfer com- pared to conventional approaches. Mcanwhile, we acknowledge two limitations: (i) the degree of mitigation varies across environments, and (ii) sensitivity to hyperparameters may require careful tuning for different tasks; the prompt quality also affects generalization performance in the target domain. These show opportunities for refinement through adaptive weighting, uncertainty-awee objectives, and broader real-world extensions.
ACKNOWLEDGEMEET
The work partially supported by NSF award #2442477. We thank Amazon Research Awards, Cisco Research Awards, Google, and OpenAl for providing us with API credits. The authors ac- knowledge Research Computing at Arizona State University for providing computing resources. The views and conclusions in this paper should not be interpreted as representing any funding agencies.
ETHICS STATEMENT
This research focuses on the sim-to-real adaptation challenge in reinforcement learning, which is fundamental and critical to the real-world application of RL. By solving the sim-to-real challenge, this work will be useful for mitigating potential real-world policies' risks, such autonomous driv- ing, etc. In this research itself, there is no ethical concern.
REPRODUCIBILITY STATEMENT
The implementation details and instructions needed reproduce the main experimental results are included in the supplementary material. Implementation and experiment are available at here
REFERENCES
Steven Bohez, Tim Verbelen, Elias De Coninck, Bert Vankeirsbilck, Pieter Simoens, and Bart Dhoedt. Sensor fusion for robot control through deep reinforcement learning. In 2017 IEEE/RS/ International Conference on Intelligent Robots and Systems (IROS), pp. 2365-2370. Icce, 2017.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PmLR, 2020.
Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, and Lina Yao. On the opportunities and challenges of offline reinforcement learning for recommender systems. ACM Transactions Information Systems, 42(6):1-26, 2024.
Longchao Da Hao Mei, Romir Sharma, and Hua Wei. Sim2real transfer for traffic signal control. In 2023 IEEE 19th International Conference on Automation Science and Engineering (CASE), pp- 1-2. IEEE, 2023a.
Longehao Da, Hao Mei, Romir Sharma, and Hua Wei. Uncertainty- grounded action transfor- mation towards sim-to-real transfer for traffic signal control. In 2023 62nd IEEE Conference on Decision and Control (CDC), pp. 1124-1129. IEEE, 2023b.
Longchao Da, Chen Chu, Weinan Zhang, and Hua Wei. Cityflower: An efficient and realistic traffic simulator with embedded machine learning models. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 368-373. Springer, 2024a.
10

---

Published as a conference paper at ICLR 2026
Longchao Da, Minquan Gao, Hao Mei, and Hua Wei. Prompt to transfer: Sim-to-real transfer for traffic signal control with prompt learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 82-90, 2024b.
Longchao Da, Porter Jenkins, Trevor Schwantes, Jeffrey Dotson, and Hua Wei. Probabilistic offline policy ranking with approximate bayesian computation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 20370-20378, 2024c.
Longchao Da, Justin Turnau, Thirulogasankar Pranav Kutralingam, Alvaro Velasquez, Paulo Shakarian, and Hua Wei. survey of sim-to-real methods in rl: Progress, prospects and chal- lenges with foundation models. arXiv preprint arXiv:2502.13187 2025a.
Longchao Da, Rui Wang, Xiaojian Xu, Parminder Bhatia, Taha Kass-Hout, Hua Wei, and Cao Xiao. Flans: A foundation model for free-form language- based segmentation in medical images. In Proceedings of the 31st ACM SIGKDD Conference Knowledge Discovery and Data Mining V. 2, pp. 404-414, 2025b.
Felipe Leno Da Silva, Pablo Hernandez-Leal, Bilal Kartal, and Matthew E Taylor. Uncertainty- aware action advising for deep reinforcement learning agents. In Proceedings of the AAAI con- ference artificial intelligence, volume 34, pp. 5792-5799, 2020.
Iván Gareía Daza, Rubén Izquierdo, Luis Miguel Martínez, Ola Benderius, and David Fernández Llorca. Sim-to-real transfer and reality gap modeling in model predictive control for autonomous 2023.
driving. Applied Intelligence, 53( (3(10):12799-22)
Amir Dezfouli and Bernard Balleine. Habits, action sequences and reinforcement learning. Eu- ropean Journal of Neuroscience, 15(7):103-1005) 2012.
Zihan Ding. Yanhua Huang, Hang Yuan, and Hao Dong. Introduction reinforcement learning- In Deep reinforcement learning: fundamentals, research and applications, pp. 47-123. Springer, 2020.
Gabriel Dulac-Amold, Daniel Mankowitz, and Todd Hester. Challenges of real-world reinforcement learning. arXiv preprint arXiv: 12901, 2019.
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity all you need: Learning skills without a reward function. arXiv preprint arXiv: 180.00000, 2018.
Amirfarhad Farhadi, Mitra Mirzarezaee, Arash Sharifi, and Mohammad Teshnehlab. Domain adap- tation in reinforcement learning: a comprehensive and systematic study. Frontiers of Information Technology & Electronic Engineering, 25(1 (5(11))1446-146) 2024.
Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611.0733 2016.
Josiah Hanna and Peter Stone. Grounded action transformation for robot learning in simulation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 31, 2017.
Nicklas Hansen, Rishabh Jangir, Yu Sun, Guillem Alenyå, Pieter Abbeel, Alexei A Efros, Lerrel Pinto, and Xiaolong Wang. Self-supervised policy adaptation during deployment. In International Conference Learning Representations.
Daniel Ho, Kanishka Rao, Zhuo Xu, Eric Jang, Mohi Khansari, and Yunfei Bai. Retinagan: object-aware approach to sim-to-real transfer. In 202 IEEE International Conference on Robotics and Automation (ICRA), pp. 10920-10926. IEEE, 2021.
Chuqing Hu, Sinclair Hudson, Martin Ethier, Mohammad Al-Sharman, Derek Rayside, and William Melek. Sim-to-real domain adaptation for lane detection and classification in autonomous driving. In 2022 IEEE Intelligent Vehicles Symposium (IV), pp. 457-463. IEEE, 2022.
Kaizhe Hu, Haochen Shi, Yao He, Weizhuo Wang, C Karen Liu, and Shuran Song. Robot trains robot: Automatic real-world policy adaptation and learning for humanoids. arXiv preprint ar0iv:2508. 12252, 2025.
11

---

Published as a conference paper at ICLR 2026
Johann Huber, François Hélénon, Hippolyte Watrelot, Faiz Ben Amar, and Stéphane Doncieux. Domain randomization for sim2real transfer of automatically generated grasping datasets. In 2024 IEEE international conference robotics and automation (ICRA), pp. 1-12-4118. IEEE, 2024.
Ercument Ilhan, Jeremy Gow, and Diego Perez-Liebana. Action advising with advice imitation in deep reinforcement learning. arXiv preprint arXiv:2104.08441, 2021.
Stephen James, Paul Wohlhart, Mrinal Kalakrishnan, Dmitry Kalashnikov, Alex Irpan, Julian Ibarz, Sergey Levine, Raia Hadsell, and Konstantinos Bousmalis. Sim-to-real via sim-to-sim: Data- adaptation networks. Proceedings of
efficient robotic grasping via the IEEE/CVF conference computer vision and pailern recognition, pp. 12627-12637, 2019.
Martin Klissarov and Marlos C Machado. Deep laplacian-based options for mmmporally-extendee exploration. arXiv preprint vrXiv:2301.11181, 2023.
Ashish Kumar, Zipeng Fu, Deepak Pathak, and Jitendra Malik. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.0403: 2021.
Ashish Kumar, Zhongyu Li, Jun Zeng. Deepak Pathak, Koushil Sreenath, and Jitendra Malik. Adapt- ing rapid motor adaptation for bipedal robots. In 2022 IEEF/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1161-1168. IEFE, 2022.
Yichao Liang, Kevin Ellis, and Joao Henriques. Rapid motor adaptation for robotic manipulator arms. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 16404-16413, 2024.
Jianlan Luo, Charles Xu, Fangchen Liu, Liam Tan, Zipeng Lin, Jeffrey Wu, Pieter Abbeel, and Sergey Levine Fmb: functional manipulation benchmark for generalizable robotic learning. The International Journal of Robotics Research, 44(4):592-606, 2025.
Ishaan Mahajan, Huzaifa Unjhawala, Harry Zhang, Zhenhao Zhou, Aaron Young, Alexis Ruiz, Stefan Caldararu, Nevindu Batagoda, Sriram Ashokkumar, and Dan Negrut. Quantifying the sim2real gap for gps and imu sensors. arXiv preprint arXiv:2403. 11000 2024.
Russell Mendonca, Oleh Rybkin, Kostas Daniilidis, Danijar Hafner, and Deepak Pathak. Discover- ing and achieving goals via world models. Advances in Neural Information Processing Systems, 34:24379 243919 2021.
Hai Nguyen and Hung La. Review of deep reinforcement learning for robot manipulation. In 2019 Third IEEE international conference on robotic computing (IRC), pp. 590-595. IEEE, 2019.
Seohong Park, Oleh Rybkin, and Sergey Levine. Metra: Scalable unsupervised rl with metric-aware abstraction. arXiv preprint arXiv:2310.088: 22000.887.... 2023.
Seohong Park, Kevin Frans, Benjamin Eysenbach, and Sergey Levine. Ogbench: Benchmarking offline goal-conditioned rl. arXiv preprint 10.20092,, 2024a.
Seohong Park, Tobias Kreiman, and Sergey Levine. Foundation policies with hilbert representations. In International Conference on Machine Learning, pp. 39737-39761. PMLR, 2024b.
Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp. 2778-2787. PMLR, 2017.
Deepak Pathak, Dhiraj Gandhi, and Abhinav Gupta. Self-supervised exploration via disagreement. In International conference on machine learning, pp. 5062-5071. PMLR, 2019.
Haozhi Qi, Ashish Kumar, Roberto Calandra, Yi Ma, and Jitendra Malik. In-hand object rotation via rapid motor adaptation. In Conference on Robot Learning, pp. 1722-1732. PMLR, 2023.
Sai Rajeswar, Pietro Mazzaglia, Tim Verbelen, Alexandre Piché, Bart Dhoedt, Aaron Courville, and Alexandre Lacoste. Mastering the unsupervised reinforcement learning benchmark from pixels. In International Conference on Machine Learning, pp. 28598-28611 PMLR, 2023.
12

---

Published as a conference paper at ICLR 2026
Erica Salvato, Gianfranco Fenu, Eric Medvet, and Felice Andrea Pellegrino. Crossing the reality gap: A survey on sim-to-real transferability of robot controllers in reinforcement learning. IEEE Access, 9:153171-153187. 2021.
Kun Shao, Zhentao Tang, Yuanheng Zhu, Nannan Li, and Dongbin Zhao. A survey of deep rein- forcement learning in video games. arXiv preprint arXiv: 10944, 2019.
Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. Dynamics-aware unsupervised discovery of skills. arXiv preprint N.KVv:/907.0655// 2019.
Rui Song, Senhui Gao, and Yao Li. Sim-to-real in unmanned surface vehicle control: A system identification-bant based approach for enhanced training environments. In 2024 9th International Con- ference on Electronic Technology and Information Science (ICETIS), pp. 563-570. IEEE, 2024.
Kim Sterelny. The evolved apprentice. MIT press, 2012.
Richard S Sutton, Andrew G Barto, et al. Introduction to reinforcement learning, volume 135. MIT press Cambridge, 1998.
Gabriele Tiboni, Karol Arndt, and Ville Kyrki. Dropo: Sim-to-real transfer with offline domain randomization. Robotics and Autonomous Systems, 166:104432, 2023.
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojeiech Zaremba, and Pieter Abbeel. Do- main randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 23-30. IEEE, 2017.
Justin Turnau, Longchao Da, Khoa Vo, Ferdous AI Rafi, Shreyas Bachiraju, Tiejin Chen, and Hua Wei. Joint-local grounded action transformation for sim-to-real transfer in multi-agent traffic control. arXiv preprint arXiv:2507.15177 2025.
Andrew Wagenmaker, Kevin Huang, Liyiming Ke, Kevin Jamieson, and Abhishek Gupta. Over- coming the sim-to-real gap: Leveraging simulation to learn to explore for real-world rl. Advances in Neural Information Processing Systems, 37:78715-78765, 2024.
Yui Wang. Jun Jason Zhang. Xinhu Zheng, Xiao Wang, Yong Yuan, Xiaoxiao Dai, Jie Zhang, and Liuqing Yang. Where does alphago go: From church-turing thesis to alphago thesis and beyond. IEEE/CAA Journal of Automatica Sinica, 3(2): 13--100, 2016.
Omry Yadan. Hydra: framework for elegantly configuring complex applications. 2025. Accessed: 2025-09-23.
https://
crrdraacc//ocs/inngg
Wenhao Yu, Jie Tan, C Karen Liu, and Greg Turk. Preparing for the unknown: Learning universal policy with online system identification. arXiv preprint 1rXivv/702.02444 2017.
Wenshuai Zhao, Jorge Peña Queralta, and Tomi Westerlund. Sim-to-real transfer in deep rein- forcement learning for roboties: survey. In 2020 IEEE symposium series on computational intelligence (SSCI), pp. 737-744, IEEE, 2020.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A LLM USAGE STATEMENT
In accordance with the ICLR 2026 submission guidelines, disclose that large language models (LLMs) were used only for language editing and grammar checking. No LLMs were employed for generating research ideas, designing methodologies, producing experimental results, creating data. All scientific content, analysis, and conclusions were developed and verified by the authors.
Egre
Eyar
Figure 7: The 2D t-SNE visualization of Hilbert-space embeddings for Walk environment across default and 4 gravity settings: ((s) under increasing dynamics perturbations. From go (default standard gravity) 94 (strongly perturbed dynamics), the embedding structure gradually collapses: the clean manifold geometry go becomes increasingly distorted and contracted as dynamics shift grows. This illustrates that the state-visitation distribution under perturbed dynamics diverges sig- nificantly from the simulator distribution, producing noticeable drift in the fixed encoder's ' feature space, which our adaptation module is designed to correct the policy on that environment.
B HILBERT SPACE PRELIMINARIES
Definition B.1 (Real Hilbert Space). A real Hilbert space is real vector space 2 equipped with an inner product as (+--) Z Z - R, such that the induced norm ||||| V(z, =) makes Z into a complete metric space under the distance d(z, z') || z'Il. Completeness means that every Cauchy sequence in Z converges (with respect d) point Z.
In our approach, we rely on the geometry real Hilbert space (3)). Below, we briefly introduce the key definitions and properties:
B. INNER PRODUCT
For any 2, w Z, the inner product (z, w) EP 2; ws, which satisfies:
Symmetry: (z, w) (w: z).
Linearity: (az + 2', w) = (z, w) + (,', w).
Positive definiteness: (2,z) with equality iff 0.
B.2 NORM
The norm induced by the inner product measures the "length" of a vector: ||ž|| = V 2), with the following properties:
| || 2 O, and || iff =
la 2|| = ||] ||^||| for any scalar a.
|
w|ll ||=||| || (triangle inequality).
14

---

Published as a conference paper at ICLR 2026
B.3 METRIC
The inner-product norm induces a metric d on 2: d(z,=') = z'1l = v(z 2', - =²), which satisfies:
d(2,2/) 2 0, and d(z, 2/)
= 2=
d(2,2/) = 2() (symmetry).
d(z,z') S =") d(=", x²) (triangle inequality).
Introducing d is necessary because First, it provides a principled way to compare embeddings: d(o(s), ((g)) quantifies the similarity of states. Second, many representation-laamiig objectives (e.g. contrastive or temporal losses) are formulated in terms of distances. Besides, Hilhert-space policies use inner products or distances define intrinsic rewards, ensuring temporally coherent be- haviors. This geometric structure underpins both zero-shot inference and goal-conditioned planning in our foundation policy.
C DERIVATION OF CLOSED FORM SOLUTIONS
In order to solve for the optimal vector = in the latent space Σ to get the near-optimal policy so- lutions. The objective is minimize the difference between the observed reward and the linear combination of the feature matrix and 2. Formally, the optimization problem is:
z² = arg min ED (1(s,,,,8) (0(8, =)) sE2
where the observed reward for transition (s, a, 8') is denoted r(s, a, 8'), the feature vector de- scribing the transition is @(s, a, E Rd, the task-specific latent vector to optimize is E R4, the expectation over the dataset D is represented by Ep, and the inner product in Rd is denoted by (s-).
In order to get the closed-form solution for this optimization problem, we have the derivation as follows. We can simplify the goal in matrix form for n samples in the dataset:
((2) = (r(s, a, s') - (d(s, a, s'),
2))2 = -|rr 1=
where the vector observed rewards r = [m1, .2..... Fn] with dimensions n X 1, the feature matrix is & = (oi, d2, ... on] with dimensions x d, and the latent vector to optimize is z E IRd
Expanding the loss:
|| 1= 2r TT + -T TTz.
(10)
And then differentiating with respect to 2:
OC(z) O=
24 r 2Φ T2.
(11)
Because the unique global minimum exists if the function is differentiable and the second derivative is positive semidefinite, and the critical point where the derivative equals zero corresponds to the global minimum, we next set the derivative to zero:
-2 r 2 +=
=0,
(12)
T oz
+
+
Then we rearrange the result, which gives the closed-form solution:
(Φ ( -
(13)
15

---

Published as a conference paper at ICLR 2026
where & & is the covariance matrix of the features, + represents the correlation between features and rewards, x" minimizes the squared error between the observed rewards and the predictions given by d2.
D NOTATION SUMMARY
In this section, we provide table of detailed notation and explanations that appear in the paper. provides a comprehensive summarization the terms used in the preliminary and methodology sections.
Symbol
Deseription
M = (S, P.r,u,n)
Markov decision process.
a(a[s)
Policy distribution over actions.
ppss)
State encoder to latent space.
Z = RD
Hilbert latent space.
d(s,g) [|{(s) - +(6))
Latent distance between states.
a(als,))
calent-conditioned foundation policy.
2, 8 (p(8) +)
Intrinsic reward from latent direction.
Task-specific optimal latent vector. Feature matrix of transitions. Reward vector.
① RAXD
RM
( TT
Least-squares latent solution.
AP Pres Pam
Dynamics discrepancy.
T real Piim A4
Real feature matrix.
Latent from joint regression.
X
Weight for real-domain samples. Permulation-invarii dynamics signature.
7 MetaDynamic (aaanDDynaio((((())))) (8x()))
90 Faxe: 17)
Adapter network for latent refinement.
Finaal
Refined latent for deployment.
Table 2: Notation summary for the proposed latent adaptation framework.
Setting Name
Parameter Changes
Description
Default
Default
default setting of Esm
G1
Gravity -9.8 - 15
Gruvity-level Exai easiest Gravity-level Exul middle easy
Gravity -9.8 - 24
Gravity -9.8 +
Gravity-level Esca middle hard
G4
Gravity -9.8 - -44
Gravity-level 4 Emci hardest
F1
Friction [4, 0.4, 0.4]
Friction-level E casiest
F2
Friction
+ |5, 0.5, 0.5]
Friction-level 2 Fieal slight easy
Friction [6, 0.6, 0.6]
Friction-level Frea middle casy
F4
Friction [7, 0.7, 0.7]
Friction-level E.Ca- slight hard
F5
Friction + [8, 0.8, 0.8]
Friction-level 5 Ercal harder
F6
Friction + 118, 1.8, 1.8]
Friction-level Emnal hardest
Table 3: Simulator-to-Real Configurations for E
E MORE EXPERIMENTAL RESULTS
E.1 DETAILS OF EXPERIMENTAL SETUPS
(1). Computing Resources: All case studies are conducted on a local workstation equipped with an NVIDIA GeForce RTX 4090 GPU (24 GB memory), CUDA 12.2, and driver version 183.01,
16

---

Published as a conference paper at ICLR 2026
800
800
stand
Stand
run
walk
fli
walk
600
600
Return
Return
400
200
200
dyna Friction Test Set
init--dyna
Init
dyna
dyna
full
Gravity Test Set
Figure 8: The full ablation study results. The left side shows the experiment performed the friction test set, and the right side shows the experiment result on the gravity- test cases. As shown in the dotted line box, the full model performs the best among the other three ablated model structures.
running under Ubuntu Linux. The main experiments reported in the tables were conducted with high-performance computing cluster equipped with NVIDIA A100 GPUs (80 memory).
(2). Training and Data Collection Configs: All the pre-training of the foundation models was executed with 2,000,000 episodes, we adopt a unified configuration across all environments follow- ing byydra-based setup (Yadan) 2025). We employ the DDPG agent pre- pre-training and base-based observations (with optional pixel-based experiments like Xit Kitchen using frame stacking and action repeat). In the total episodes, buffer capacity is fixed at 1,00 1,000,000 transitions, serve as the source- domain dataset D addi- tion, we also construct target-domain dataset Dtar by executing the pre-trained policy in the target environment for 5, 000 rounds of interaction.
set the first 1, 1,000 frames for seeding the replay buffer. Replay
(3). Baseline Implementation Details: In this section, we provide details of the baseline models' training. For PAD (Hansen et al.) model, we follow the official repository as released the project website Besides, since the grounded action transformation methods could provide adaptation on the policy network in real time by training a forward model and inverse model, and ground the ac- tion based on the estimated real-world system dynamics, thus, we adapt this branch of method and conduct online adaptation, the implementation mainly follow the release code base at public repos- itory g To make fair comparison the online adaptation time, we fixed the epochs of adaptation as 200 across the main table's experiments. In uncertainty-awar GAT (UGAT), it integrates the ev- idential deep learning module for uncertainty prediction in the inverse module, and the uncertainty will be used to decide whether to accept proposed action (low-uncetaainty take the original action (high-unee snigh-uncertaitty). this method generally provides consistent improvement than Vanilla-GAT model based on experiment.
E.2 INTERPRETATION OF THE DATA QUALITY RESULTS.
Following the discussion in Sec. 5.1 ()), Fig. (four panels) extends the analysis stand to walk, run, and flip under three corruption modes (drop, mask, noise). Two patterns are consistent across tasks. First, drop exhibits graceful degradation: at less than 50% removal rate, stand and walk decline only mildly, indicating that reducing target-domain sample size might weaken the adapter's evidence, but method could still perform relatively robustly. In contrast, deteriorate more sharply at the same rate; these skills start from a lower baseline because we are zero-shol prompted
and flip
lask while adapting new environments, making them more sensitive when losing the target domain data and amplifying variance the latent refinement.
Second, mask fails early in four tasks, which shows that, converting selected transitions zeros injects biased signals into the squared-loss objective of Eq. 5 pulling the refined latent toward de- generate regions and causing rapid collapse once moderate fraction is masked. By comparison, noise shows a nonlinear response: light corruption can act like mild regularizer (occasionally
htnns://wwernicclsssasncc.oommp
17

---

Published as a conference paper at ICLR 2026
drop
mask
naise
stand
walk
Nutur
100
200
run
Rip
300
200
10
30
50
10
80
90
19
10
20
30
40
60
70
20
90
comuption (9))
comruttion 1961
Figure 9: More comprehensive data quality analysis on four different tasks with three modes of corruptions and varied corruption rates.
producing small bumps), but heavier noise degrades performance, particularly for the dynamics- sensitive runiflip, because disrupts the consistency between (s, s') which the adapter relies.
Overall, the curves suggest task- sensitivity hierarchy: stand = walk are comparatively robust sparsity, whereas run and slip fragile both to reduced evidence (50% drop) and signal corruption (mask/noise)) Taken together, these results highlight that Found-adapt is surprisingly resilient: delivers stable performance under substantial sparsity (e.g.. 550% drop in stand/walk) and degrades gracefully relative to more brittle perturbations. However, on the more demanding skills: run and flip, we observe that, the curves exhibit noticeably larger variance under comparable corruption. We attribute this to the compounded difficulty of zero-shot adaptation to new task while adapting: contact- and timing-sensitive dynamics amplify small inconsistencies between (s,s'), s'), and the induced latent landscape can be more rugged. In conclusion, Found-adapy still delivers consistent sim-to-real lift on unseen tasks by efficient parameter tuning, without retraining the whole policy, which goes beyond conventional pipelines.
F FEASIBILITY IN GOAL-CONDD RL SETTING.
We show that Found-adapt applies to goal-conditioned RL (GCRL) with no change to the foun- dation policy training loop, only to how the task latent is instantiated from goal. Let g denote a goal (e.g., a target state or target representation). As in the main text, the foundation policy is (ala (s, 2). the transition feature is ((s, a, 8') E R.D. and rewards modeled linearly in the latent: ((s,a,s)) = (é(s,a,5) 2)-
For GCRL we simply index the latent by the goal and write žg- Many standard goal rewards (e.g., indicator of reaching or shaped distance to g) admit linear fit in the same feature space; we denote their labels for transition, j by Tg..-
Goal-to-latent mapping in the simulator. Given set of simulator transitions {(s,, a; (,), N and goal form
R NxD
with rows
T sim, Φ(s;, ag s/),
R
sim
g.ssi
The goal-specific source latent 2 obtained exactly as in the main method (veighnedreegularized least-squares in the same feature basis):
aFc arg min D
|| Fg.s 11 + 2|ailk (4 Paim + A1)
4 Tg.sim.
When partial target data are also available, the same stacked/weighted normal equations used the main method can be applied to include them.
18

---

Published as a conference paper at ICLR 2026
Zyrc
30
Adapter Loss Mean
Veriance
0 6
12
18
42
2.5
20
0.0
-2.5
10
StRuall 55386
0
6 12
Adapter-Induced Latent Shill
Adeptation Steps
Figure 10: The visualization of the Eere and Frefined (after normalization). The left part, from 2ure Zrefined, shows the latent space changes during the adaptation process. Together with the loss function decrease, the latent representation is changing, which leads to the final better solution in the Ereat domain dynamics, yielding better performance in the Ereal rewards. This visualization also confirms the refinement process is smooth and stable, demonstrating that the lightweight adapter does not introduce instability. Due the simple network architecture, has low risk of overfitting.
Tarrettddmain refinement via the latent adapter. At inference, Found-adapt already computes an environment signature from target transitions (DynamicNet over é-features) and refines the latent via the adapter go (-))). For given goal g. define the target design matrix and labels
tar E MMD fg,tar RR.
constructed from the same target transitions but goal-relabelled rewards (e.g., success indicator for g. shaped distance g in your representation). The adapter objective used throughout the paper directly applies:
M Σ M
1
9o- <2ac
(0; g)
Prar Do (= ,1))
fnnal Ag
vD
22g
5-1
The resulting 24
.final conditions the same foundation policy i ralss, 2) for the goal g. i.c., . (a zinal). without retraining the policy.
Because g only changes the scalar labels Fg.- while keeping the feature design matrices sim and Ptar unchanged, Found-adapt is compatible with standard GCRL data strategies: (i) hindsight relabelling. for any stored transition (s, a, s'), reuse it for many g by recomputing fg: and (ii) on-the- fly goal queries, compute aige (closed form) and its one-step refinement 2 No changes to é, or æ are needed.
ffinal per goal inference.
Special case: representation-shapee goals. tation used by é, e.g.,
If the goal reward is distance in the same represen-
Fg(s, a, -1lp(ss') و
then, under the linear reward model already assumed by Found-adapt, the induced 2y coincides with the leaat-squares projection of this goal signal onto the feature span of o(s, s'). Hence the same identifiability and conditioning arguments the main method carry over, and the adapter uses to correct residual simulation-to-reat mismatch per goal.
Thus, GCRL in Found-adapt amounts to swapping task IDs for goals when forming the reward la- bels Fg. all estimators and refinements, least-squares for sre DynamicNet environment signature m, and the adapter ge remain unchanged. Thus, foundation policy ग (a s, 2) can be potentially de- _final at inference, preserving the inference-time adaptation
ployed across many goals by computing =g advantages demonstrated in the main (task-conditioned) setting.
G DETAILED MODEL. CONFIGURATIONS
The MetaDynamic network used in Sec. 4.2.B is cermutation-invariii encoder, Given set it produces a dynamics signature E IR through an
, of target-domain encoded states (0(3,) )) ==1 /= elementwise transformation, pooling, and global projector such below:
N
1
Dnne (o(3)))
(14)
N
19

---

Published as a conference paper at ICLR 2026
Architecture: We use lightweight DeepSet-style module below:
elementwise encoder Penc: MLP(dd + 128 + 128, ReLU),
permutation-invarian mean pooling,
projector p: MLP((188 + 128 + K, ReLU) with 64.
Pre-training: The encoder is trained offline using simulator trajectories collected under 12 dynam- ies variations (gravity, friction, and actuator-strength perturbations). Training uses combined loss:
L LcB + a LNTXent:
a = 0.1
(15)
The cross-entropy term LCg encourages the encoder separate different dynamics regimes, while the NT-Xent loss LNTXem (Chen et al. 2020) pulls together sets from the same dynamics and pushes apart sets from different ones, producing smooth and discriminative dynamics signature.
Deployment: After pre-training, the parameters of MetaDynamic are frozen. During deployment, receives small set of real-domain state encodings and outputs the corresponding dynamics sig- nature 77 without further updates.
H DISCUSSION ON FUTURE WORK
Regularization for stable latent refinement. The refinement of the latent vector through the adapter de is supervised by the projection | (o(Za 7) T'rry 1| which constrains the solution only through the row space of Pung- In theory, multiple latent vectors may achieve similar projection error, and this limited when the latent dimension is small, and the adapter is conditioned on Aaee which serves as strong prior and restricts the refinement a meaningful neighborhood of the initial solution. More- over, the adapter is a low-capacity MLP that is optimized for only few hundred steps, providing strong form of implicit regularization. Adding an appropriate regularization term toward Axe or incorporating geometric constraints in the latent space may further enhance the refinement, which eventually provides better identifiability and improved performance under stronger perturbations.
potentially introduce certain degree of overfitting. We find this effect to be
Extending to real-world sim-to-real adaptation. Our experiments follow the widely adopted sim-to-sim evaluation protocol, which provides controlled and repeatable dynamics variations for isolating the effect latent-space adaptation. This setting is both standard and practical for analyz- ing dynamic generalization. An important direction for future work is deploy the proposed method in real hardware environments to more directly assess its sim-to-real capability. Additionally, replac- ing state-based features with image observations or vision-based encoders would make the pipeline more applicable to realistic robotic systems, where dynamics shifts often manifest through visual changes. Such extensions would further validate the flexibility of foundation-polle policyy adaptation and broaden its applicability to perceptual and real-world domains.
20


Reviews:
[{"review_id": "5K2YOpIdJl", "reviewer": "Reviewer_QoG5", "summary": "This paper addresses the critical challenge of sim-to-real transfer for reinforcement learning policies. The authors propose \"Found-adapt,\" a framework that builds upon foundation policies (specifically, the HILP[1] framework) to enable efficient adaptation from a source simulator to a target domain with different dynamics. The core idea is to decouple skill acquisition (pre-trained in sim) from environment adaptation via a task vector $z$ in the Hilbert task space.\n\nThe proposed adaptation mechanism consists of three stages. First, a **Cross-Domain Initial Alignment** step computes an initial latent vector $z_{src}$ via a weighted least-squares regression from both sim and real data. Second, a **Dynamics Signature** is generated using a pre-trained, permutation-invariant network to encode a \"dynamics signature\" $\\eta$ from a small batch of target-domain data. Finally, in the **Signature-Guided Adaptation** stage, a small adapter network, $g_{\\theta}$, takes both $z_{src}$ and $\\eta$ as input is then fine-tuned for a small number of steps on the target-domain to get the final, adapted latent vector $z_{final}$, which is subsequently used by the frozen foundation policy.\n\nThe authors evaluate their method on several \"sim-to-sim\" locomotion tasks (e.g., Walker, Cheetah) with varying dynamics (friction, gravity). The results show that Found-adapt can mitigate the performance gap and offers a parameter-efficient alternative to retraining the entire policy.", "questions": "The main questions are elaborated in the weakness part, here are some minor questions:\n\n* In the ablation study (Fig 4a), the `F(init+dyna)` variant performs very poorly. This implies the adapter $g_{\\theta}$ is not just combining $z_{src}$ and $\\eta$ but reconciling them. Could the authors explain why this simple combination fails so badly and what this reveals about the learned representations of $z_{src}$ and $\\eta$?\n* How was the hyperparameter $\\lambda$ (the weight for target-domain data in Sec 4.2.A) selected, and how sensitive is the performance of $z_{src}$ (and $z_{final}$) to this choice?\n* How sensitive is the framework to the amount of target-domain data ($M$)? What is the minimum number of target samples required for the `MetaDynamic` network to extract a useful signature $\\eta$ and for the adapter $g_{\\theta}$ to be effectively tuned?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "* Important Problem: The paper tackles the sim-to-real gap, which remains a significant and high-impact obstacle for the practical application of reinforcement learning in robotics and the real world.\n* Promising Framework Idea: The core concept of adapting the *latent space* of a foundation policy, rather than fine-tuning the policy network itself, is appealing. It suggests a modular, parameter-efficient, and fast adaptation method.\n* Conceptually Sound Decomposition: The idea of decomposing the adaptation problem into an initial task-aligned latent ($z_{src}$) and a separate dynamics signature ($\\eta$) is interesting. It attempts to distinguish \"what to do\" from \"how the environment behaves.\"", "weaknesses": "* Clarity of Contribution (vs. HILP): **(major)** The paper dedicates a very large portion of its methodology (Sec 4.1) to a detailed recapitulation of the HILP framework [1], which is existing prior work. This positioning obscures the paper's own, new contributions. The actual novel parts are confined to Sec 4.2. Of these, Sec 4.2.A (weighted regression) is a straightforward and minor extension of HILP's original least-squares solver. This leaves Sec 4.2.B/C as the main contribution, however, the `MetaDynamic` network (Sec 4.2.B), is not clearly explained. The paper states it is \"trained on simulator data and frozen at deployment\" but provides no details on its training objective, loss function, architecture, or the data used for its pre-training. This is a major omission that makes the contribution difficult to assess and the work impossible to reproduce.\n\n* Weak Experimental Validation (Sim-to-Sim): The paper's abstract and introduction frame the problem as \"real-world application\" and \"sim-to-real transfer.\" However, all experiments are conducted in a \"sim-to-sim\" setting (transferring between different MuJoCo physics parameters). While this is a common practice, the lack of a single real-world hardware experiment significantly weakens the paper's central claims. \n\n* Marginal Empirical Results: The empirical results are not strongly convincing. In Table 1 and Figure 3, the proposed method (\"Ours\") shows only marginal improvements over the baselines. In some cases (e.g., Stand and Flip tasks under G3/G4 gravity), the \"Ours\" method appears to be outperformed by the PAD baseline, suggesting it may not be uniformly more robust. \n* Missing Related Work (RMA): The \"Adaptive Policy Networks\" related work section is notably incomplete. It omits the highly relevant and successful line of work on Rapid Motor Adaptation (RMA) like[2-5], which also uses a latent-conditioned policy ($z$) to adapt to different dynamics.\n\n* Missing Critical Ablation (Direct Latent Optimization): The paper's primary claim is that its ($z_{src}$, $\\eta$) $\\rightarrow$ $g_{\\theta}$ $\\rightarrow$ $z_{final}$ pipeline is an effective adaptation mechanism. However, it fails to compare against a much simpler and highly relevant baseline: directly optimizing the latent vector $z$ itself using data from the target environment. This approach (treating $z$ as a small set of learnable parameters at deployment) has shown great success in related work (e.g., RTR [5]), which also adapts a latent-conditioned policy for sim-to-real). It is unclear if the proposed `MetaDynamic` network and adapter $g_{\\theta}$ are necessary at all, or if simply fine-tuning $z_{src}$ would achieve the same or better results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dPjw1lVsmk", "reviewer": "Reviewer_141r", "summary": "This paper proposes Found-Adapt, a latent adaptation framework for sim-to-real transfer of pre-trained foundation policies, extending the Hilbert-space foundation model (HILP; Park et al., 2024). The approach performs latent-space adaptation rather than full policy retraining. It first applies a cross-domain latent alignment via weighted regression to initialize a latent vector, updating the pre-trained model using a batch of target-domain data. Because this linear update approximates only a local warping of the latent space, the authors introduce a permutation-invariant MetaDynamic summary of target-domain states to encode dynamics differences, followed by a nonlinear adapter optimized for latent reward consistency. Experiments on locomotion tasks under simulated dynamics perturbations show improved transfer relative to direct deployment and domain randomization.", "questions": "1. What is the exact architecture, representation, dimensionality and training procedure of the MetaDynamic model? Is there prior work motivating this structure for cross-domain transfer?\n\n2. How is the weighting parameter λ defined and tuned? Does its optimal value depend on target batch size?\n\n3. What is the performance without the initial alignment ($\\lambda = 0$, i.e. step 1 using only sim data)?\n\n4. Could the proposed framework support online adaptation similar to PAD?\n\n5. How do perturbations affect the Hilbert-space embeddings? Is there any analysis of representation drift between simulator and target domains?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles the important problem of efficient sim-to-real adaptation for foundation policies, a critical and emerging area in RL.\n- The method adapts policies through latent-space updates and avoids costly retraining.\n- The paper is generally well organized and builds upon the prior HILP framework in a coherent way.\n- The two modules (cross-domain alignment and nonlinear adapter) play distinct roles—one theoretically motivated, the other more practical and performance-driven.\n- The paper compares against relevant baselines including direct deployment, domain randomization, and PAD.\n- Results show consistent improvements over naive transfer and domain randomization, reaching similar performance to PAD.", "weaknesses": "While the paper is technically sound and addresses a timely topic, several issues limit its clarity and impact:\n\n- Comparison to PAD: \n  - The paper reports comparable performance to PAD but frames PAD as incomparably inefficient. However, PAD’s longer pre-training time is not directly comparable to Found-Adapt’s adaptation cost, since that cost corresponds to task pre-training rather than the adaptation method itself. PAD achieves similar reward recovery with only a few hundred online samples, compared to the 5k target samples used for Found-Adapt. The comparison would be more balanced if adaptation sample efficiency (rather than pre-training) were highlighted.\n  - Although the architectures differ, both methods effectively correct representation drift between domains by learning mappings that preserve task-relevant latent consistency. PAD adapts the feature extractor online via self-supervision, while Found-Adapt adds a learned adapter that refines the latent vector offline. The similarity in effective mechanism could explain their comparable empirical performance.\n\n- The paper does not analyze how the Hilbert embeddings change under dynamics perturbations and after adaptation. Since part of HILP’s original appeal lies in the interpretable geometry of the latent space, understanding how perturbations distort these embeddings—and whether adaptation restores that structure—would be a great addition. Without this analysis, it is unclear whether the adaptation preserves the interpretable properties of the original foundation policy framework.\n\n- Unclear role of key components: \n  - The linear realignment step largely reuses the HILP latent regression with added target data, but details such as the weighting parameter λ are not explained or analyzed. \n  - The nonlinear MetaDynamic adapter is presented as a black box; its architecture, representation, and training details are missing, making it difficult to assess generality or reproducibility.\n  - It is unclear whether, after a nonlinear adapter is added, the target alignment is still needed at all, or one could use the original (i.e. $\\lambda = 0$) with similar performance.\n\n- The number of gradient steps and sample efficiency of the adaptation process are not well described. Questions about overfitting or stability with different training and architecture remain open.\n- The work builds heavily on Park et al. (2024), reusing the Hilbert-space policy and regression-based task prompting. The main novelty lies in the addition of the adapter, which, though reasonable, feels more heuristic than conceptually grounded.\n- All tests are sim-to-sim under controlled perturbations; no real-robot experiments are shown, despite strong emphasis on sim-to-real transfer. This limits the external validity of the claims.\n\n- Presentation issues: \n  - Figure 5B is unreferenced.\n  - Line 439 refers to a missing table (\"Table ???\").\n  - The files in anonymous repository link are currently inaccessible, with message: “The requested file is not found.”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "d2lz6LBk3m", "reviewer": "Reviewer_NFbW", "summary": "This paper introduces Found-adapt, a novel framework for addressing the sim-to-real transfer challenge in reinforcement learning. The key contribution is a latent space adaptation method that enables foundation policies to transfer across domains without expensive policy retraining. The approach consists of three main components: (1) cross-domain initial alignment via weighted least-squares regression, (2) a permutation-invariant MetaDynamic network that extracts dynamics signatures, and (3) a signature-guided adapter that refines latent representations. The method is evaluated on multiple locomotion tasks under varying system dynamics (gravity and friction variations), demonstrating effective sim-to-real gap mitigation while maintaining computational efficiency.", "questions": "1. Scalability to high-dimensional action spaces: All tested environments have relatively low-dimensional continuous action spaces - how does the method scale to high-dimensional or discrete action spaces, and are there theoretical or practical limits to the dimensionality it can handle?\n\n2. Comparison of adaptation mechanisms: How does your latent space adaptation compare conceptually and empirically to other lightweight adaptation strategies such as context-based meta-learning, fine-tuning only the final policy layers, or dynamic rescaling of actions? What makes latent adaptation specifically advantageous for the sim-to-real problem?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. Clear problem formulation: The paper addresses a critical deployment challenge where traditional sim-to-real methods require expensive retraining for each new environment, and motivates the solution through an intuitive biological analogy of how humans adapt existing motor skills to new conditions without relearning from scratch.\n\n2. Rigorous theoretical foundation: The approach is built on solid mathematical principles with formal analysis of how environment changes affect policy performance, closed-form solutions with clear assumptions, and well-justified design choices where each component addresses specific aspects of the adaptation problem.\n\n3. Exceptional computational efficiency: The method achieves adaptation in seconds compared to hours required by baseline approaches, representing orders of magnitude speedup by freezing most network parameters and performing lightweight updates only in the latent space.", "weaknesses": "1. Limited domain coverage: The experimental validation is restricted exclusively to locomotion tasks, limiting the claimed applicability as a general foundation policy framework.\n\n2. Sim-to-sim evaluation only: All experiments use simulated environments with modified physics parameters rather than actual real-world robots, raising concerns about whether the method can handle the full complexity of real-world dynamics.\n\n3. Limited baseline comparisons: The paper primarily compares against Direct-Transfer, GAT/UGAT, and PAD while missing comparisons with recent domain randomization methods, meta-RL approaches designed for adaptation, and other foundation model techniques, which weakens the empirical validation of the claimed advantages.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hnwCBelqIN", "reviewer": "Reviewer_A9Lz", "summary": "This paper introduces Found-adapt, a framework for efficient sim-to-real transfer in reinforcement learning that adapts pretrained foundation policies through latent-space alignment rather than policy retraining. The method first trains a foundation policy on offline simulator trajectories in a Hilbert-space embedding that encodes reusable behavioral primitives. At deployment, adaptation is performed in three lightweight stages: a cross-domain least-squares latent alignment that integrates simulator and limited real data, a MetaDynamic network that extracts a permutation-invariant dynamics signature from the target domain, and a signature-guided adapter that refines the latent vector to align behavior with new dynamics. This parameter-efficient approach enables adaptation without modifying policy weights. Extensive experiments on locomotion tasks (Walker, Cheetah, Quadruped, AntMaze, Kitchen) under varying gravity and friction demonstrate that Found-adapt substantially narrows the sim-to-real performance gap compared to strong baselines (GAT, UGAT, PAD), achieving similar performance with orders-of-magnitude lower adaptation time. Detailed ablations, sensitivity analyses on data corruption, and studies linking adaptation loss to performance further support the framework’s robustness and practical potential for real-world RL deployment.", "questions": "please address the concerns above", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper reframes sim-to-real transfer as latent alignment rather than policy re-optimization. This perspective—adapting a frozen foundation policy by operating in its latent space—is conceptually fresh and could generalize across domains and tasks. The Hilbert-space foundation policy formulation (Sec. 4.1) gives a principled way to tie skill structure to temporal dynamics, aligning well with recent trends in foundation models for control.\n\n2. The method is well-structured and mathematically grounded. Equations (1–6) and Figure 2 outline a coherent pipeline linking the least-squares latent solution, MetaDynamic signature, and adapter update. The combination of permutation-invariant dynamics encoding and parameter-efficient adaptation is elegant and interpretable.\n\n3. Experiments span multiple environments and perturbations (gravity G1–G4, friction F1–F6), with comparisons to both task-specific (PAD) and dynamics-specific (GAT/UGAT) baselines (Table 1, Figures 3–9). Found-adapt consistently ranks among the top methods, often within 5-10% of PAD but with orders-of-magnitude faster adaptation (seconds vs. hours).\n\n4. The ablation study (Figure 4a) carefully isolates the contribution of the initial alignment, dynamics signature, and adapter. The sensitivity analyses on data corruption (Figures 10–11) and the strong correlation between adaptation loss and performance improvement (Figure 6) demonstrate a rare level of diagnostic rigor for sim-to-real studies.\n\n5. The approach avoids retraining the policy or encoder entirely, performing adaptation through small-scale optimization over a latent vector. This is appealing for real-world robotics, where data collection is costly. The inclusion of analysis on robustness to noisy and sparse target-domain data (Sec. 5.1 (4)) adds credibility to practical deployability.", "weaknesses": "1. Dynamics signature lacks true transition information\n\nThe MetaDynamic encoder (Sec. 4.2B) computes η solely from unordered states, ignoring actions and successor-state relations. As a result, η summarizes state visitation statistics rather than transition dynamics P(s′|s,a). This makes η non-identifiable and may cause adaptation to correct policy-induced distribution shifts rather than genuine system-dynamics changes.\n\n2. Latent update relies on unstable closed-form inversion\n\nThe least-squares solutions (Eqs. 2-3) require inverting ΦᵀΦ and AᵀA without any regularization or conditioning safeguards. Appendix C provides no analysis of rank deficiency or noise sensitivity. In low-data or ill-conditioned regimes, this can yield unstable latent estimates, undermining the method’s reliability under realistic sim-to-real conditions.\n\n3. Adapter optimization is underconstrained\n\nThe adapter gθ is trained with a simple projection loss (Eq. 5), which constrains the latent only within the row space of Φ_tar. Without explicit regularization or geometric constraints toward z_src, multiple gθ can fit the same projections, causing overfitting and inconsistent behavior in closed-loop execution.\n\n4. Target reward requirement contradicts the “unlabeled” claim\n\nAlthough the paper emphasizes adaptation from limited unlabeled data, both Eq. 3 and Eq. 5 require real-domain reward labels. This assumption is not addressed or justified, and in many real-world settings such rewards are unavailable or sparse. The omission makes the “label-efficient” narrative misleading and limits practical applicability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)