From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning
Authors: Honglin He
Yukai Ma
Brad Squicciarini
Wayne Wu
Bolei Zhou
Venue: ICLR 2026 Poster (2026)
Keywords: Urban Navigation
Foundation Models
Reinforcement Learning
Primary area: applications to robotics, autonomy, planning

Abstract: Navigation foundation models trained on massive web-scale data enable agents to generalize across diverse environments and embodiments. However, these models, which are trained solely on offline data, often lack the capacity to reason about the consequences of their actions or adapt through counterfactual understanding. They thus face significant limitations in the real-world urban navigation where interactive and safe behaviors, such as avoiding obstacles and moving pedestrians, are critical. To tackle these challenges, we introduce the Seeing-to-Experiencing (S2E) learning framework to scale the capability of navigation foundation models with reinforcement learning. S2E combines the strengths of pre-training on offline videos and post-training through reinforcement learning. It maintains the model's generalizability acquired from large-scale real-world videos while enhancing its interactivity through reinforcement learning in simulation environments. Specifically, we introduce two innovations:
1) an Anchor-Guided Distribution Matching strategy for offline pretraining, which stabilizes learning and models diverse motion patterns through anchor-based supervision; and
2) a Residual-Attention Module for reinforcement learning, which obtains reactive behaviors from simulation environments without erasing the model’s pretrained knowledge.
Moreover, we establish a comprehensive end-to-end evaluation benchmark, NavBench-GS, built on photorealistic 3D Gaussian Splatting reconstructions of real-world scenes that incorporate physical interactions. It can systematically assess the generalizability and safety of navigation foundation models.

Published as a conference paper at ICLR 2026
FROM SEEING TO EXPERIENCING: SCALING NAVIGATION FOUNDATION MODELS WITH REINFORCEMENT LEARNING
Honglin Hel-*, Yukai Ma".", Brad Squicciarini" Wayne Wu', Bolei Zhou
University of California, Los Angeles
"oco Robotics https: https//vallllucce g thub. 1o/ 52E
ABSTRACT
Navigation foundation models trained on massive web-scale data enable agents generalize across diverse environments and embodiments. However, these models, which are trained solely offline data, often lack the capacity to reason about the consequences of their actions or adapt through counterfactual understanding. They thus face significant limitations in the weal-world urban navigation where interactive and safe behaviors, such as avoiding obstacles and moving pedestrians, are critical. To tackle these challenges, (S2E) learning framework to scale the capability of navigation foundation models with reinforcement learning. S2E combines the strengths of pre-training on offline videos and post-training through reinforcement learning. It maintains the model's s generalizability acquired from large-scale real-world videos while enhancing in- teractivity through reinforcement learning in simulation environments. Specifically, we introduce two technical innovations: I) an Anchor-Guides Distribution Match- ing strategy for offline pretraining, which stabilizes learning and models diverse motion patterns through anchor-based supervision; and 2) a Residual-Attention Module for reinforcement learning, which obtains reactive behaviors from simula- tion environments without erasing the model's pretrained knowledge. Moreover, establish comprehensive end-to-end evaluation benchmark, NavBench-Cis built on photorealistic 3D Gaussian Splatting reconstructions of real-world scenes that incorporate physical interactions. can systematically assess the generaliz- ability and safety of navigation models. Extensive experiments show that S2E mitigates the diminishing returns often seen when scaling with offline data alone. We perform a thorough analysis of the benefits of Reinforcement Learning (RL) compared to Supervised Fine-Tuning (SFT) in the context of post-training for robot learning. Our findings emphasize the crucial role of integrating interactive online experiences to scale foundation models in Robotics. Code will be made available.
introduce the Beeing-to-Experienccing
COCO
Real-world deployments of S2E wheeled and quadruped robots
Figure 1: Real-world deployments S2E. S2E achieves zero-shot generalization across environments and
embodiments. We demonstrate its effectiveness on wheeled and quadruped robots in diverse urban scenarios.
Equal contribution.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Foundation models have demonstrated transformative capabilities across various domains, including language understanding (Touvron et al., 2023; Bai et al., 2023), generation (Rombach al., 2021; Lin et al., 2024), and visual recognition (Wang et al., 2023; Kirillov et al., 2023; Yang et al., 2024). Through training on massive data, these models significantly enhance the generalizability and adaptability of downstream tasks. However, applying foundation models to robot navigation presents unique challenges (Firoozi et al., 2023) due to the complex nature of sequential decision-making in dynamic real-world environments. For example, in bustling urban space, navigation foundation models must make real-time decisions to avoid collisions with obstacles, such as trash bins, and safely maneuver through ever-changing crowds.
Recent work on navigation foundation models has harnessed large-scale web videos and human demonstrations for pretraining (Shah et al., 2023a;b; Sridhar et al., 2024). These approaches primarily rely on passive visual learning (Kim et al., 2024; Brohan et al., 2022), where models are trained to imitate behaviors in massive video data. Such data captures diverse visual observations of the real world; yet, it lacks explicit information on physics and cause-and-effec relations, which are crucial for decision-
As a result, navigation policies trained solely on offline data often exhibit limited reactivity to the surroundings and struggle to adapt to diverse objects and motions.
While offline video data helps build proper perceptual prior for the model, it only captures statistical correlations, not grounded causality (Silver& & Sutton, 2025). Visual imitation (Dai et al., 2025; Ren et al., 2025) teaches an agent what actions look like, but not how to adapt, recover, or reason about counterfactual outcomes when the environment changes. Thus, navigation foundation models must move from seeing to experiencing: actively interact with the world, receive feedback. and refine behaviors through trial and error. As shown in Figure 2, sim- ilar to humans learning skatehoarding, where experience with balance, falling, and correction is irreplaceable, agents must interact with the world to acquire true adaptability. Reinforce- ment Learning (RL) enables agents to bridge the gap between observations and actions, offering a scalable interactive learning paradigm that enriches model capabilitiess beyond the behavior cloning of static datasets. However, RI. alone has shown lim- ited success in building generalizable navigation models. Prior approaches (Shen et al., 2019; Putta et al., 2024; Truong et al., 2021; Lin & Yu, 2025; Xie et al., 2025) have trained agents in narrow synthetic environments using RL; however, due to poor sampling efficiency and the lack of inductive priors, models struggle to achieve scalable and generalizable navigation capabilities in the real world.
How do humans learn tn ride skateboard?
inittale
practise
Interactive leaming
Wibh videos
How do learn to navigate?
Offiie
peetraining
Figure 2: Motivation. Like humans, Al agents must also go through interactive practices and Iearn from feedback to obtain actionable skills.
To this end, we propose a new learning framework, Sceing-to- teattt---peeeeiiii (S2E), to scale navigation foundation models with reinforcement learning in simulated environments while maintaining the generalizable visual representation acquired through pretraining on offline videos. This framework comprises two crucial technical components. First, we introduce Anchor- ing (AGDM). a strategy for pre-training on real-world video data. It is designed as an anchor-guided model architecture learn complex multimodal distributions in normalized motion trajectory space, thereby enabling the efficient representation of diverse behaviors across various scenarios. This design significantly mitigates learning uncertainty and provides a more reliable backbone to ease subsequent online adaptation. Moreover, the anchor-based architecture naturally supports the cross-embodiment Module (RAM) for
Distribution Match-
deployment of the foundation model. Second, we propose a Residual-Attenin RL post-training in simulation environments. It designed as a residual architecture by copying the pretrained attention block and learning a residual component that selectively captures knowledge acquired from online interactions. This design enables agents to acquire novel capabilities, such as obstacle avoidance and movement anticipation, through reinforcement learning while preserving the generalizable visuomotor representations learned from offline pre-training:
We further introduce NayBench-Gi BeenchhGS, a comprehensive end-to-end evaluation benchmark built on realistic 3D Gaussian Splatting environments with accurate physics and interactive dynamics. Unlike
2

---

Published as a conference paper at ICLR 2026
prior evaluations that rely offline 2D iideo-based testing (Shah et al., 2023a;b; Sridhar al., 2024), NavBench-GS enables reactive simulation to facilitate closed-loop policy assessment in photo- realistic 3D scenes. These scenes, combining realistic visual appearance and physical interaction with reproducibility, address a longstanding challenge in robotics: the difficulty of replicating real- world environments for end-to-end evaluation. It enables standardized, reproducible evaluation of navigation foundation models in terms of generalization and safety in unseen settings.
Extensive experiments show that reinforcement learning substantially enhances the policy perfor- mance and alleviates the diminishing returns associated with scaling solely on offline data. We analyze the effectivvness of Reinforcement Learning (RL) versus Supervised Fine-Tuming (SFT) in post-training for robot learning. Although both methods are widely discussed in relation to large language models (L..Ms) (Kumar et al., 2025), they remain underexplored the field of scaling robot learning. Additionally, we demonstrate the generalizability of the proposed S2E framework through world evaluations in challenging scenarios.
RELATED WORK
Goal conditioned navigation. Goal conditioned navigation is the most common setting for robotic navigation. Prior works have developed diverse approaches represent navigation goals, which can be categorized into three main paradigms: 1) image-goal navigation, where target images serve as visual references (Mezghan et al., 2022; Ramakrishnan al., 2022; Zhu et al., 2017); 2) position-gon goal navigation, which directly encodes destination coordinates (Chaplot et al., 2020a; Chattopadhyay al., 2021; Gordon et al., 2019); and 3) object-goal navigation, where targets are specified through object categories (AI-Halah et al., 2022; Chang et al., 2020; Chaplot et al., 2020b).
Deep reinforcement learning (Mirowski al., 2016) has demonstrated promising results in mapless navigation by eliminating dependency maps. However, these methods often suffer from limited generalization, particularly in visual navigation (Shen et al., 2019; Putta et al., 2024; Truong et al., 2021), due to the constrained diversity of training scenarios. The synthetic nature and restricted variation in simulated training worlds inherently limit the policy's 's ability to adapt to real-world complexity and unseen scenarios.
Navigation foundation models. Many recent works have proposed various vision-based navigation foundation models (Shah et al., 2023a;b; Sridhar et al., 2024), leveraging advantages cross- sensor capabilities and rich vision data for improved generalizability across different robot platforms and camera configurations. City Walker (Liu et al., 2024b) and NWM (Bar et al., 2024) further enhance the environmental comprehension of policy by incorporating future state prediction, enabling more informed navigation decisions. However, a key limitation such approaches is the lack of environmental interactions in the training data, which, as demonstrate in Section 4.2, results in poor performance in obstacle and pedestrian avoidance. To address this, it is essential to develop policies that are generalizable and capable of high-quality local planning, rather than relying solely on path-following capabilities.
Hybrid learning with pretraining and finetuning. The paradigm of offline pretraining followed by RL fine-tuning has emerged as powerful framework for training robust control policies, bridging the gap between data efficiency and real-world adaptability. Early successes in playing games, such AlphaGo (Silver al., 2016) and AlphaStar (Arulkumaran al., 2019), demonstrated the potential of combining large-scale pretraining (e.g., supervised learning from expert trajectories) with RL fine-tuning to achieve superhuman performance. This approach has since been extended to train foundation models, where pre-trained LLMs (Touvron et al., 2023; Halterman & Keith, 2025) and VLMs (Chen et al., 2024) are fine-tuned via Reinforcement Learning from Human Feedback (RLHF) to align with human values and preferences.
3 S2E LEANING FRAMEWORK
To address the challenge of training generalizable and interactive foundation navigation models, propose S2E, a hybrid learning framework that combines pretraining on videos and reinforcement learning in simulated environments. Figure 3 illustrates the overall framework of the proposed S2E.
3

---

Published as a conference paper at ICLR 2026
Pretraining loss
Imago
Encoder
Slef NeffAtteenton
K, VV
Score Pa
Call Lieg
RAM Blocks
RGB observations
04-b-1:1
Trajectories (H).
Target
N
PPO loss RAM
Reg.
position
MLP
image
min Cram
Pu
Anchors
MLP
Velocity
Ps
scale
Figure 3: Illustration of S2E framework. The model receives continuous RGB frames context information.
goal point or goal image guidanee, and uses spatial anchors as queries for prediction. First, context embeddings are fused via a self-attention module. The outputs are then used as keys (K) and values (V). Mcanwhile, the anchor features fr serve as queries (Q). Subsequently. RAM blocks compute weighted fcatures from K and V based on the anchor queries Q. and produce refined anchor features. A classification and regression head decode the anchor features to predict scores and normalized trajectories with velocity scale. In the pretraining stage, model is trained end-to-end with NLL regression loss (Equation 2). the fine-tuning stage, only the parameters within the RAM blocks are optimized using the policy gradient from Cam-
Categorical
Gausiim
III
Diffusion
Anchor
(a) Anchor-guided Gaussian. Misture Behaviors
(b) Comparisons among Action Represeciations
Figure 4: Illustration of anchor-guided distribution matching, (a) Illustration of anchor-guided Gaussian Mixture behaviors in a sidewalk scenario, where anchors guide diverse behavior generation. (b) Comparison of predicted action distributions between different representations.
It aims to learn a visual navigation policy π that enables robot to navigate from the source waypoint coordinate ps to the target waypoint coordinate Pa- This task focuses on local navigation (Zhu et al., 2017; Liu et al., 2024b) between consecutive waypoints, which can be easily extended to long-distance tasks by chaining waypoints obtained from path planning (Thrun, 2002) or GPS. At each timestep I, the observations are RGB frames 0:- spanning the past k: frames, and the target coordinates Pa or target image Ig- The output locomotion models
short-term relative waypoint as its action a, then exccute it to move the robot incrementally toward the goal. Our framework consists of two key technical components: 1) Anchor- Anchor-guided guided distribution matching for pretraining generalizable backbone representation (Section 3.1), and 2) Residual-Attention Module for injecting obstacle avoidance capability with RL (Section 3.2).
3.1 PRETRAINING WITH -UIDDDD DISTRIBUTION MATCHING
backbone for extracting meaningful visual features is essential for generalization across diverse scenarios. We pretrain our model on 100 hours of navigation videos collected from various robots and platforms (Shah et al., 2021; Karnan al., 2022b;a; Liu et al., 2024a; 2025; Hirose et al., 2025), covering wide range of environments. However, modeling multi-mode data distributions still presents significant challenges (Sridhar et al., 2024), as the model must generate diverse predictions under the same conditions. To address this, introduce the anchor-guided distribution matching.
Anchor-guided distribution matching. Robot navigation trajectories are inherently multimodal given the same observation, multiple distinct actions may be valid. Effectively modeling this multimodality is crucial for generalizable policies. However, common representations like discrete actions (Shafiullah al., 2022) unimodal Gaussians (Shah et al., 2023a;b; Liu et al., 2024b)
4

---

Published as a conference paper at ICLR 2026
lack expressiveness, limiting the model capture the information from partial observation, the accumulation of prediction errors over time (Codevilla et al., 2019), etc. While diffusion models (Chi et al., 2023; Sridhar et al., 2024), though expressive, overly flexible and difficult to control in navigation, often yielding fragmented trajectories that compromise safety and stability.
To address this, we propose anchor-guider Gaussian Mixture Model (GMM) (Gu et al., 2021; Shi et al., 2022) to represent robot actions in urban navigation, as illustrated in Figure 4 (a). This formulation offers both multimodality and structure. Anchors, uniformly sampled in the robot's forward direction, serve as interpretable high-level intentions. Each anchor corresponds to a Gaussian mode in the mixture, with learned scores reflecting likelihoods. The policy learns to generate and choose trajectories based on these anchors, enabling diverse yet goal-ee behaviors. This approach combines expressiveness with training stability and also well-suited for fine-tuning with RI..
We present the distributions of actions using various representations in Figure 4 (b), including categorical, unimodal Gaussians, diffusion policy, and our anchor-gut suchor--uiied Gaussian Mixture Model (GMM). The categorical representation learns actions in discretized bins, which restricts its ability to capture nuanced or multimodal behavior. The unimodal Gaussian representation tends towards a single modal distribution, making it ineffective in capturing the multiple valid actions in trajectories. The diffusion policy learns a smooth and comprehensive distribution, but is too flexible, which complicates the guidance and control of behaviors. Our anchor-guides GMM provides structured, multi-modal distribution, where different anchors specialize in different high-level intentions (e.g., going straight, turning. yielding). Also, the model retains mote- uncertainty by allowing moderate variance within each anchor's predicted distribution with a learned standard deviation.
The model architecture for distribution matching is illustrated in Figure 3. Specifically, we generate M representative intention points Pa E IR M x2 by K-Means (Lloyd, 1982) on the unified dataset, which serves as an additional input beyond RGB frames and target position. The distribution of the action We at timestep f under observation Or-k+1:: is a Gaussian Mixture Model (GMM) defined as:
M
q( fwwlor auilor-k41:)
= Σ
9m Nu - M C Wy pgg : p"),,
(1)
m=1
where the score distribution Om denotes the probability of each intention point pa being selected, we (Wz, wyy) denotes the position input of the locomotion generate action a. M//y" "2/y" and nT denote the mean, standard deviation, and correlation predicted by the m-th trajectory head, respectively. Additionally, we predict a scale v E Rt per mode, allowing the policy to model absolute trajectory magnitude while preserving directionality.
Training objective. The model is trained end-to-end with two dense training losses Latt, and Lreg, on each prediction head after each RAM block i. The first the Negative Lik-Liielioood (NLL) loss as shown in Equation 2 used to supervise both the classification and trajectory heads. Inspired by Shi al. (2022), we employ an assignment strategy that selects the mode whose predicted direction best aligns with the ground-truth trajectory for optimization. The second loss is an 1.2 regression loss used to optimize the velocity scale,
Calli log Ny ub, o2: thy o"y p²) log(gn))
(2)
Cregit ||0 =11.
(3)
where the selected mode h is the one whose anchor is closest to the ground truth and chosen for optimization. More details about pretraining video data are provided in the App. E.2 and App. E.3.
3.2 REINFORCEMEN' LEARNING WITH RESIDUAL ATTENTION MODULE
To enhance the specific interaction capabilities of a pre-trained navigation model, a closed-loop RL phase is essential. While imitation learning provides a strong initialization, it inherently lacks the mechanism to correct covariate shift, i.e., the divergence between the training distribution and induced trajectory from the policy. When encountering out-of-distriben -istiiiiibtttinn (OOD) states, prior from offline data become unreliable. RL addresses this failure mode by providing online, corrective feedback, allowing the policy to learn inductive biases for recovery and fine-gne -ge-inaaidd manipulation that absent in static datasets. To achieve this goal, a naive approach is to fine-tune all parameters using RL. However, such strategies introduce two fundamental problems.
5

---

Published as a conference paper at ICLR 2026
Forgetting of pre-trained capabilities (FPC). Previous works (Wotczyk al., 2024; Schmied et al., 2023) show that full-parameter fine-tuning in RL can cause a pretrained policy to lose behaviors previously performed well. This degradation arises from interference in the function approximator during adaptation and is particularly problematic in transfer RL. When the distribution of visited states shifts, the pretrained capabilities in the under-visited regions deteriorate significantly.
Domain shift. Full- Full-parameter RL fine-tuning also exposes the model to a severe observation level domain shift. Let Drea( ()) denote the real-world observation distribution and Dsim (o) the simulator observation distribution. These distributions differ significantly in texture, lighting, object appearance, and sensor noise, i.e. Dreal() * (ii (o). When parameters of the observation encoder Og are updated only using observations o ~ Dsim (o), the learned feature extractor J is optimized on the simulator domain (o) However, when deployment, the policy receives images from Deaa (o). There would be a feature difference between the feature from the pre-trane pre-trained feature extractor F and the fine-tuned one F measured by
Acatt
Dro [Fee (rim(() (o)] Eow [ (o)]
Since Deim and Dreat differ the pixel level, full-model RL fine-tuning yields ||rear| || that grows rapidly, reflecting that the encoder overfits to simulated RGB statistics and no longer produces the pretrained representation, significantly degrading real-world performance.
Residual attention module. To address these chal- lenges, more selective form of fine-tuning is re- quired: that is, we fine-tune only the modules that can enhance task-specific adaptation while avoiding interference with the pretrained visual representa- tions and preventing degradation of previously ac- quired capabilities. In navigation task, we aim to fine- tune components that are tightly coupled with agent- environment interaction yet robust to sim-to-real gaps. We identify
K.V
K.V
zeto initialuedd linvar layer
Attention
Attention Block $
Trainable copy
Block
zero initialived lineur loyer
Z
layers as the ideal target. Unlike visual encoders dv self-attention layers, which process raw scene textures and are highly sen- sitive to domain shifts (Tobin et al., 2017; Sridhar et al., 2024), cross gross-attention explicitly models the relationship between the agent and the environment. The architectural role of coossaatenttion is computing
pre-uzmming
(1) fine-tuming
Figure 5: Residual attention module.
QK
Attn(Q, K, V) softmax
V.
(5)
Vd
where Q encodes the agent state (e.g., trajectory tokens) and (K, V) encodes observation features. This operation explicitly binds agent behavior to environmental context and primarily captures relational structure, which is far more stable under appearance changes than raw visual features.
To fine-tune this mechanism effectively, we introduce the Residual-Attention Module (RAM), as illustrated in Figure 5. Inspired by previous works (Zhang et al., 2023; Wu et al., 2024; Alayrac et al., 2022; Li et al., 2023), we freeze the original pre-trained parameters Op of the cross-attention layer dn to preserve generalized capabilities and introduce parallel, trainable copy O;, This copy is gated by zero-initialized linear layers 2. Formally, the adapted output Q' is computed as
Q' (bl:::KVVee) + Z (+D(Z(Q): V:Oi)).
(6)
The irooiiitiializatinn of Z guaranties that at the start fine-tuning. the contribution of the residual
branch is null, ensuring Q' = uD (Q; K, VDD This mechanism creates structural curriculum via the gradient flow. Since Z is a linear projection initialized zero, the backpropagated gradient to the adapter parameters,
OC
Ve, L. 02
Wz.

---

Published as a conference paper at ICLR 2026
initially vanishes. Consequently, the adaptation branch remains dormant during the high- varianar phase of early RL exploration and only becomes active as the gate weights We gradually move from zero, allowing for a controlled, progressive injection of interaction dynamics.
As an additional advantage, this design is substantially more parameter- and computation-effiiien than full parameter fine-tuning. In practice, the full-sized model with parameters Ou (Hu et al., 2022) contains millions to billions of parameters, learning a full-sized update AO from the simulator is computationally expensive. Each iteration requires full forward-backwart passes through the entire encoder and decoder, implemented by many transformer blocks, dramatically increasing memory usage. With our approach, i.e., instead of updating the full model, we train lightweight plugin module whose parameter O2 satisfies A < encoder (and thereby avoiding the domain-shiff issue) and still improving the policy performance.
[ool, while omitting costly updates to the vision
Reward function design. In reinforcement learning, the reward function provides the fundamental learning signal that shapes agent behaviors by reinforcing desirable outcomes and penalizing unde- sirable ones. We design the reward progressively, moving from essential objectives to higher-level refinements, = Rg + RR + Rx, where
Global goal Rg encourages the agent to efficiently reach the destination while ensuring basic safety. To achieve this goal, we employ four terms terms- dennnsspea/// goal-reaching Rg.d, Rgg and dense/sparse collision penalties Re.d, Re,s together.
Rule regularization R& enforces general navigation rules, such as sidewalk centering and social compliance.
Human likeness Rx encourages smooth, natural, and interpretable behaviors that align with human navigation patterns.
The details on each reward item can be found in App. E.S.
Training objective. The pretrained model outputs waypoint trajectory wi E 10x0, which is robot- agnostic. To enable training with dynamics-aware robots the simulator, we employ a differentiable controller Fa that takes uy as input and generates velocity commands for the locomotion module Fi. We only finetune the parameters +, of additional branches from RAM blocks, so the gradients from the context features V,, (ncoder(Ot-k-a employ a PPO-based objective with entropy regularization:
and fr are truncated. To optimize the policy, we
min Lyo = -policy + aLvalue , Hax
(8)
Epllicy =
E, [min(rA,, clip(ri. 1 c. c.1+c)A.))
(9)
Lallue
Ee (V (V
R=)
(10)
M Σ
Hr
Σ 9m - log 2 m2 oy m2
4m log qm:
(11)
=(o.l=)) where r, T Toala, (s.) the probability ratio, Ae is the estimated advantage, and e is the clipping threshold, V. is the value network that takes the context feature as input and use an MLP to output the value prediction. H. is simplified approximation of the entropy of the GMM that does not admit closed-fon analytic solution for the KL divergence. Additional details are provided in the App. E.4.
EXPERIMENTS
In this section, we provide our experimental results. Section 4.1 validates the effectiveness of RL in further scaling navigation foundation models. Section 4.2 benchmarks sfate-of-the-art navigation foundation models in realistic 3DGS scenes. Section 4.3 presents real-world evaluation results and demonstrates the generalizability of S2E. For more details, please refer to the App. C and App. D.
4.1 SCALING UP MODEL PERFORMANCE VIA REINFORCEMENT LEARNING
We first validate our motivation: while large-scale offline pre-training yields broad generalization, investigate if RL can improve the performance after pretraining by enabling the model to learn
7

---

Published as a conference paper at ICLR 2026
(a) Data from real-worid
(b) Data from simulator
(c) 3D Ganssian Splating testing environments
Figure 6: Overview of training environments and evaluation benchmark. (a) Real-world data has realistic appearances but lacks interactions. (b) Syathetic data from simulator supplements rich physical interactions. (c) Scene in NavBench-GS offers realistic visual appearances and physical interactions for E2E evaluation.
70
URBAN-SIM
NavBench-GGS-Ohsme
XBBDA
60
25
Coaa livw
-50
Success Rate (PG)
220
.50
Suooss rate
40
Rate
30
45
20
40
10
SET
SFT
35
R1
50k
100k
1.50k
200k
250k
750k
Training Frnmes
Additionil LOPS
le6
Additioal GFLOPS
le6
(a) Effectiiveness of RL
Comparison of S2C-SFT and S2E-RL URBAN-SIM and NavBennhh-S
Figure 7: Effectiveness of reinforcement learning. (a) Success rates of policies trained with varying amounts of data, showing gain from RL fine-tume tine-tuning over only supervised learning. Dotted lines indicate the performance of prior methods. (b) Performance comparison between SFT and RL policies under increasing training cost.
from embodied interactions in simulation. Recent studies on scaling laws (Kaplan al., 2020) suggest that model performance improves predictably with increasing data volume, model size, and compute budget. However, the diminishing returns are exhibited the system approaches the scaling frontier (Kaplan et al., 2020). As for the embodied navigation, this frontier arrives even earlier due to the relatively low-dimensional space of the action, the model quickly saturates its capacity to benefit from simply scaling in supervised learning. In large language models (LLMss,, there have been numerous comparisons (Kumar et al., 2025) between two different paradigms for post-training: Reinforcement Learning (RL) and Supervised Fine- TinniTuning (SFT). However, it remains unclear how post-training should be approached in the field of robot learning, especially as more scalable training methods are being developed in this domain (Black al.).
To systematically study this phenomenon, we first evaluate: 1) S2E-BC: A pure behavior cloning model trained solely on offline pretraining data. 2) S2E-SFT: An agent fine-tuned with data from URBAN-SIM (Wu et al., 2025) after pretraining. 3) S2E-Full: Our full approach combining pre- training followed by RL fine-tuning. Figure (a) shows that increasing the data scale for S2E-BC yields only marginal improvements beyond a certain point. Expanding the dataset from 250k to 750k samples results in mere 2% increase in success rate, suggesting that more offline data is insufficient to further enhance navigation capability. In contrast, RL significantly enhances the model's 's performance by leveraging interactions in simulation, achieving 15% improvement in success rate over the pretrained model, with no additional offline data used. These results provide compelling evidence that RI. can overcome the fundamental limitations of traditional scaling laws.
Additionally, we compare the performance scaling of supervised fine-tuning (SFT) and reinforcement learning (RL) across two benchmarks in Figure (b), including the in-distribution test URBAN- test in NavBench-GS. The data used for SFT is collected via
SIM and the out-of-distributie pretrained policy interacting within the environment, and episodes involving collisions are removed to ensure training stability. As additional training cost increases, RL maintains or improves success rates, while SFT suffers from severe overfitting. These results demonstrate that RL not only more sample-efficient but also more robust under increased training costs.
8

---

Published as a conference paper at ICLR 2026
Emply
Obstacle
Podestrian
Obstacle + Pedestrian
Video
Method
Data
SR f
RC f
CT
SR
RC
CT
SR
RC
CT
SR f
RC1
CT
GNM
70h
0.23
0.51
0.72
0.16
0.49
0.90
0.09
0.53
1.28
0.07
0.44
2.31
VINT
80h
0.28
0.51
0.60
0.13
0.46
1.21
0.07
0.48
1.22
0.08
0.39
1.99
NoMal
100h
0.15
0.46
0.35
0.11
0.44
0.89
0.09
0.48
0.68
0.08
0.42
1.83
MBRA
700b
0.61
0.75
0.35
0.51
0.77
0.53
0.71
0.84
1.01
0.51
0.69
2.09
Walyer
2000h
0.66
0.72
0.42
0.43
0.63
0.74
0.56
0.66
1.04
0.37
0.62
2.25
Walker
100h
0.67
0.90
0.15
0.52
0.79
1.00
0.63
0.66
2.34
0.47
0.51
2.52
S2E
100h
0.82
0.92
0.00
0.57
0.78
0.69
0.74
0.78
1.50
0.51
0.73
1.58
Table I: NavBench-GS Benchmark. Comparison of navigation foundation models across four tasks.
4.2
NAVBENCH-ON GS BENCHMARK
Although training durations and offline data sources vary between foundation models, our NavBench- GS benchmark remains fair and meaningful. We follow standard evaluation practices where foun- dation models are frequently trained on different corpora due to concerns about data privacy and distribution bias. We keep all environments unseen and distributionally distant from the training data.
Benchmark. The core and foundational function for robot in urban scenarios is to navigate from point A to point B. In this task, the main commands beyond reaching the goal include: 1) not collide with static objects, and 2) not crash with moving objects. To this end, we design a benchmark in photo-realistic and physically interactive scenarios from Vid2Sim (Xie et al., 2025), spanning 26 scenarios, each instantiated with four tasks, i.e., 1) empty environments, 2) environments with random static obstacles, 3) environments with moving pedestrians, and 4) environments with obstacles and pedestrians. We use success rate (SR), route completion (RC) and collision times (CT) to measure the model performance. An episode is deemed successful if the robot reaches the destination with remaining distance of less than meter and has fewer than 3 collisions with obstacles or pedestrians.
Baselines. To thoroughly evaluate our method's advantages, we compare against several state-of- the-art navigation foundation models: 1) image-based approaches, including GNM (Shah et al., 2023a), ViNT (Shah al., 2023b), NoMaD (Sridhar et al., 2024), and 2) point-based approaches MBRA (Hirose et al., 2025), CityWalker (Liu et al., 2024b), City Walker* (re-trained with the same dataset used in the current work). Several S2E variants are also evaluated in Table 4.
Quantitative Results. As demonstrated in Table I, S2E consistently outperforms all bascline methods in both SR and RC across all test scenarios, validating the effectiveness of our S2E framework. Compared with point-based approaches, our results show that scaling performance with RL is significantly more effective than simply scaling up the amount compared with City Walker, our model trained with only 100h of data already surpasses video-based methods trained with over 2000h of data, underscoring the superior efficiency of reinforcement leamning in scaling performance.
training data. For example,
4.3 REAL-WORLD EVALUATIONS
For real-world evaluation, we use 25 real-world scenarios, each repeated times to test the model's performance. We consider two types of environments: Empty, where only structural boundaries such as walls are present except for the ground; Obstacle, where static objects are randomly placed between the agent's starting point and the destination. We validate our approach through a comprehensive real- world evaluation using two distinct robotic platforms: 1) the Unitree GO2 quadrupedal robot, and 2) a wheeled robot. Figure provides a qualitative comparison between S2E-Fuul and baseline methods, clearly demonstrating $2E-Full'ss superior collision avoidance capability in complex scenarios where other methods fail. Quantitative results Table 2 further confirm these observations, with S2E-Full achieving the highest performance in both success rate and collision avoidance metrics. These results clearly illustrate that the interactive capabilities learned through RL training in simulation transfer effectively to the real world in zero-shot manner. And illustrated in Figure 9, the model demonstrates effective road keeping and collision avoidance when navigating where there are obstacles and pedestrians in the environment.
9

---

Published as a conference paper at ICLR 2026
Wheeled robot
SZE-Fal
Method
SR 1
RC |
CT
NoMaD
0.25
0.55
0.76
CityWalker Walker
0.28
0.44
0.78
S2E-BC
0.32
0.51
0.78
Goad
Giaa Gandlicaate
S2E-Full
0.51
0.64
0.60
candiiddes
Quadruped robot
SRRR
Method
SR 1
RC't
CT J.
NoMaD
0.26
0.52
0.75
City W
0.31
0.54
0.79
S2E-BC
0.34
0.63
0.91
S2E-Full
0.55
0.69
0.62
Stating
Table 2: Real-world results.
Figure 8: Visualization of real-world results.
Road following
Collision avoidance
Visualization inference results on urban scenarios
Figure 9: Deployment of S2E on real-world sidewalks. deploy S2E real-world urban scenarios and visualize model predictions. Red lines denote the left and right sides the wheeled robot at this moment, while the green bar indicates the predicted future trajectory. Note that the predicted future trajectories exhibit the capability of our model to follow sidewalks and avoid obstacles.
4.4 ABLATION STUDY
In this section, we conduct ablation studies to evaluate the effectiveness of key components of our method, particularly the AGDM and RAM for scaling with RL. More experiments are in the App. D.
Effectiveness of anchor guidance. As shown in App. D.5, S2E-BC-Single trained with single- mode matching. while S2E-BC adopts anchor-guided distribution matching to model multi-modal distribution. Under the same setting, S2E-BC significantly outperforms S2E-BC-Sing in both success rate (+11%) and collision (-0.64) in scenarios with obstacles, demonstrating that anchor- guided distribution matching improves the model's ability to capture complex distributions.
Effectiveness of residual attention module. To evaluate the pro- posed learning under the limited-modee setting, we conduct ablation studies different fine-tuning strategies, where DecFT-RL indicates fine-tuning on action decoder layers from pretrained initialization. As shown in Table 9, our approach achieves the highest success rate and lowest collision times on NavBench GG- Obstacle, demonstrat- ing the effectiveness of our finetuning strategy under limited-modun adaptation. More results are provided in App. D.3.
Methods
SR t
CT
PPO SFT
0.02
2.37
0.49
0.77
DecFT-RL
0.39
0.91
Ours
0.57
0.69
Table 3: Effectiveness of RAM.
CONCLUSION
We propose novel Beeing-to-Experienece (S2E) framework for learning navigation foundation models. It integrates an Anchor-Guider Distribution Matching strategy to adapt to diverse real-world conditions and Residual-Attene Module (RAM) for incremental improvement in interactive learning. Extensive experiments demonstrate that the models trained from our framework achieve zero-shot transfer to unseen scenarios and can be seamlessly deployed across different robots.
Limitations and Future Work. Since current systems lack 3D perception, even S2E sometimes fails avoid collisions, which remains persistent challenge for vision-only navigation approaches. One potential solution is integrating depth or occupancy prediction to infer 3D structural cues.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENT
The project was supported by the NSF Grants CNS-2235012, IIS-2339769, and T1-2346267. Honglin He is supported by the Amazon Trainium Fellowship. We thank Coco Robotics for the generous equipment donation.
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. All datasets used in this study are publicly available or derived from open-source simulation platforms; private or sensitive data are involved. No sensitive data were collected during the experimental process. The research does not involve human or animal subjects. For real-world deployment, the robots operated under strict safety constraints: their maximum velocity and acceleration were limited by both hardware and software, and all tests were conducted in controlled environments. Researchers were present during experiments, with the ability to immediately intervene and stop the robot to ensure safety. The methods are designed to improve navigation safety and reliability, and are not intended for harmful applications. All experiments comply with institutional safety regulations and legal requirements. We are committed to research integrity, transparency, and reproducibility, and plan release relevant code and model weights to facilitate verification by the community.
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure reproducibility of our work. The main paper clearly specifies the model architecture App. E.2), training objectives (Section 3), and evaluation metrics (Section 4.2). Additional implementation details hyperparameter configurations, and environment settings provided in the App. E). We plan to release them after double-blin review to enable full verification by the community. All datasets used in this study are publicly available, and preprocessing steps described in (App. E).
REFERENCES
Ziad Al-Halah, Santhosh Kumar Ramakrishnan, and Kristen Grauman. Zero experience required:
Plug & play modular transfer learning for semantic visual navigation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 17031-17041, 2022. 3
lean-Baptisse Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican. Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23775 2022. 6
Kai Arulkumaran, Antoine Cully, and Julian Togelius. Alphastar: An evolutionary computation perspective. In Proceedings of the genetic and evolutionary computation conference companion, pp. 314-315, 2019.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun. Navigation world models. arXiv preprint arXhv:2412.03572, 2024. 3
Kevin Black, Noah Brown, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, Lachy Groom, Karol Hausman, Brian Ichter, et al. æ0: vision-language-action flow model for general robot control. arXiv preprini ARXIV.2410.24164. 8
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, et al. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint arXiv:2212.06811 2022. 2
Matthew Chang, Arjun Gupta, and Saurabh Gupta. Semantic visual navigation by watching youtube videos. Advances in Neural Information Processing Systems, 33:4283 2020.

---

Published as a conference paper at ICLR 2026
Devendra Singh Chaplot, Dhiraj Gandhi, Saurabh Gupta, Abhinav Gupta, and Ruslan Salakhutdinov. Learning to explore using active neural slam. arXiv preprint arXiv:2004.05155, 2020a.
Devendra Singh Chaplot, Dhiraj Prakashchand Gandhi, Abhinav Gupta, and Russ R Salakhutdinov. Object goal navigation using goal-oriented semantic exploration. Advances in Neural Information Processing Systems, 23:4247-42555
2020b.
Prithvijit Chattopadhyay, Judy Hoffman, Roozbeh Mottaghi, and Aniruddha Kembhavi. Robustnav:
Towards benchmarking robustness in embodied navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15691-15700, 2021.
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generie visual- ssual--inguistic tasks. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 24185-24198, 2024. 3
Cheng Chi, Zhenjia Xu, Siyuan Feng, Eric Cousineau, Yilun Du, Benjamin Burchfiel, Russ Tedrake, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. The International Journal of Robotics Research, pp. 02783649241273668. 2023. 5
Felipe Codevilla, Eder Santana, Antonio M López, and Adrien Gaidon. Exploring the limitations of behavior cloning for autonomous driving. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3329-9338, 2019. 5
Yinlong Dai, Robert Ramirez Sanchez, Ryan Jeronimus, Shahabedin Sagheb, Cara M Nunez, Heramb Nemlekar, and Dylan Losey. Civil: Causal and intuitive visual imitation learning. arXiv preprint arXiv:2504.17959 17959, 2025. 2
Vishnu Sashank Dorbala, James Mullen, and Dinesh Manocha. Can an embodied agent find your "cat-shaped mug"? lim-based zero-shot object navigation. IEEE Robotics and Automation Letters, 9(5):4083 4090, 2023. 26
Roya Firoozi, Johnathan Tucker, Stephen Tian, Anirudha Majumdar, Jiankai Sun, Weiyu Liu, Yuke Zhu, Shuran Song, Ashish Kapoor, Karol Hausman, et al. Foundation models in robotics: Applications, challenges, and the future. The International Journal of Robotics Research, pp. 027836492412815008 2023. 2
Daniel Gordon, Abhishek Kadian, Devi Parikh, Judy Hoffman, and Dhruv Batra. Splitnet: Sim2sim and task2task transfer for embodied visual navigation. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 1022-1031, 2019,
Junru Gu, Chen Sun, and Hang Zhao. Densetnt: End-to-end trajectory prediction from dense goal sets. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 15303-15312, 2021. 5
Andrew Halterman and Katherine A. Keith. Codebook Ilms: Evaluating Ilms as measurement tools for political science concepts, 2025. URL
Noriaki Hirose, Lydia Ignatova, Kyle Stachowiez, Catherine Glossop, Sergey Levine, and Dhruv Shah. Learning to drive anywhere with model-based reannotation. arXiv preprint urXiv:2505.0559) 2025. 4, 9, 24
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022. 7
Marco Huber, Tim Bailey, Hugh Durrant- Whyte, and Uwe Hanebeck. On entropy approximation for gaussian mixture random vectors. In 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 181-188. IEBE, 2008. 25
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint ar ar0iv:2011.0600 (03361,0000801.000000000101111 2020. 8
12

---

Published as a conference paper at ICLR 2026
Haresh Karnan, Anirudh Nair, Xuesu Xiao, Garrett Warnell, Soeren Pirk, Alexander Toshev, Justin Hart, Joydeep Biswas, and Peter Stone. Socially Compliant Navigation Dataset (SCAND), 2022a. URL http:::/ddoi.ooe 1rt.......1........... 18738/ T8/ /PPRRRRRH 4
Haresh Karnan, Anirudh Nair, Xuesu Xiao, Garrett Warnell, Sören Pirk, Alexander Toshev, Justin Hart, Joydeep Biswas, and Peter Stone. Socially compliant navigation dataset (scand); A large- scale dataset of demonstrations for social navigation. IEEE Robotics and Automation Letters, 2022b. 4, 23, 24
Bernhard Kerbl, Georgios Kopanas, Thomas Leimküihler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139-1, 2023. 18
Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-language-action model. arXiv preprint (rXiv:2406009246 2024.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et Segment anything. In Proceedings of the IEEE/CVF international conference computer vision, pp. 4015 2023. 2
Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming-Han Hssan Yang. Phillip H. S. Torr, Fahad Shahbaz Khan, and Salman Khan. Llm post-training: A deep dive into reasoning large language models, 2025. URL https:
//arxiv. org/ abs 250022502200 2 1 3,8
Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jianwei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded ext-to-image generation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.
2023. 6
Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, al. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12037-12047, 2025. 18
Bin Lin, Yunyang Ge, Xinhua Cheng, Zongjian Li, Bin Zhu, Shaodong Wang. Xianyi He, Yang Ye, Shenghai Yuan, Liuhan Chen, al. Open-sore sora plan: Open- large video generation model. arXiv preprint arXiv:2412.00133 2024. 2
Kwan Yee Lin and Stella X Yu. Let humanoids hike! integrative skill development on complex trails. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 22498 225507, 2025. 2
Wei Liu, Huihua Zhao, Chenran Li, Joydeep Biswas, Billy Okal, Pulkit Goyal, Yan Chang, and Soha Pouya. X-mobility: End-to-end generalizable navigation via world modeling. arXiv preprint arXiv:2410.1749. 2024a. 23, 24
Wei Liu, Huihua Zhao, Chenran Li, Joydeep Biswas, Soha Pouya, and Yan Chang. Compass: Cross- embodiment mobility policy via residual rl and skill synthesis. arXiv preprint arXiv:2502.16372. 2025. 4, 23, 24
Xinhao Liu, Jintong Li, Yicheng Jiang, Niranjan Sujay, Zhicheng Yang, Juexiao Zhang, John Abanes, Jing Zhang, and Chen Feng. Citywalker: Learning embodied urban navigation from web-scale videos. arXiv preprint arXiv:2411.17820 2024b. 3, 9,19 19
Stuart Lloyd. Least squares quantization in pem. IEEE transactions on information theory, 28(2): 129-137, 1982.
Lina Mezghan, Sainbayar Sukhbaatar, Thibaut Lavril, Oleksandr Maksymets, Dhruv Batra, Piotr Bojanowski, and Karteek Alahari. dleorry-augment reinforcement learning for image- goal navigation. In 2022 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3316-3323. IEBE, 2022.
13

---

Published as a conference paper at ICLR 2026
Piotr Mirowski, Razvan Pascanu, Fabio Viola, Hubert Soyer, Andrew J Ballard, Andrea Banino, Misha Denil, Ross Goroshin, Laurent Sifre, Koray Kavukcuoglu, et al. Learning to navigate in complex environments. arXiv preprint arXiv: 2016. 3
Philip Polack, Florent Altché, Brigitte d' Andréa Novel, and Arnaud de La Fortelle. The kinematic bicycle model: A consistent model for planning feasible trajectories for autonomous vehicles? In 2017 IEEE intelligent vehicles symposium (IV), pp. 812-818. IEEE, 2017. 26
Pranav Putta, Gunjan Aggarwal, Roozbeh Mottaghi, Dhruv Batra, Naoki Yokoyama. Joanne Truong, and Arjun Majumdar. Embodiment randomization for cross embodiment navigation. 2024 TEEF/RSJ International Conference Intelligent Robots and Systems (IROS), pp. 5527-5534.
IEEE, 2024.7 2,3
Santhosh Kumar Ramakrishnan, Devendra Singh Chaplot, Ziad Al-Halah, Jitendra Malik, and Kristen Grauman. Poni: Potential functions for objectgoal navigation with interaction-fee learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp.
18890-18900, 2022. 3
Hao Ren, Yiming Zeng, Zetong Bi, Zhaoliang Wan, Junlong Huang, and Hui Cheng. Prior does matter: Visual navigation via denoising diffusion bridge models. arXiv preprint UrXiv:2504.000 2025. 2
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models, 2021.
Thomas Schmied, Markus Hofmarcher, Fabian Paischer, Razvan Pascanu, and Sepp Hochreiter. Learning to modulate pre-trained models in rl. Advances in Neural Information Processing Systems, 36:38211-38255, 2023. 6
Nur Muhammad Shafiullah, Zichen Cui, Ariuntuya Arty Altanzaya, and Lerrel Pinto. Behavior transformers: Cloning & modes with one stone. Advances in neural information processing systems, 35:22955-22968, 2022. 4
Dhruv Shah, Benjamin Eysenbach, Gregory Kahn, Nicholas Rhinehart, and Sergey Levine. Rapid exploration for open- world navigation with latent goal models. arXiv preprint arXiv:2104.05859 2021. 4, 21, 23, 24
Dhruv Shah, Ajay Sridhar, Arjun Bhorkar, Noriaki Hirose, and Sergey Levine. Gnm: A general navigation model to drive any robot. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp. 7226-7233. IEEE, 2023a. 3,4,9
Dhruv Shah, Ajay Sridhar, Nitish Dashora, Kyle Stachowicz, Kevin Black, Noriaki Hirose, and Sergey Levine. Vint: A foundation model for visual navigation. arXiv preprini arXiv:2306. 14846, 2023b. 2, 3, 4, 9, 18, 20
William B Shen, Danfei Xu, Yuke Zhu, Leonidas J Guibas, Li Fei-Fei, and Silvio Savarese. Situational fusion of visual representation for visual navigation. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2881-2888 2019. 2,
Shaoshuai Shi, Li Jiang, Dengxin Dai, and Bernt Schiele. Motion transformer with global intention localization and local movement refinement. Advances in Neural Information Processing Systems, 35:6531-6543,, 2022. 5
David Silver and Richard S Sutton. Welcome to the era of experience. Google AI, 2025.
David Silver, Aja Huang, Chris Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, loannis Antonoglou, Veda Panncershelvam, Mare Lanctot, et al. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):4844489 2016. 3
Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Seungeun Yi. Michaël Ramamonjisou, et al. Dinov3. arXiv preprint ar 2508 10104, 2025. 24
14

---

Published as a conference paper at ICLR 2026
Ajay Sridhar, Dhruv Shah, Catherine Glossop, and Sergey Levine. Nomad: Goal masked diffusion policies for navigation and exploration. In 2024 JEEE International Conference on Robotics and Automation (ICRA), pp. 63-70. IEEE, 2024. 2, 4, 6, 26
FrodoBots Team. Frodobots-2k dataset.
https: / frodobots s h/root/ss/roososseeoooooooooooes 2KK, 2024. Hugging Face. 24
Sebastian Thrun. Probabilistic robotics. Communications of the ACM, 45(3):52-57, 2002.
Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 JEEE/RSJ international conference on intelligent robots and systems (IROS), pp. 23-30. IEFE, 2017. 6
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint urXiv:2302.13977 2023. 2.
Joanne Truong, Denis Yarats, Tianyu Li, Franziska Meier, Sonia Chernova, Dhruv Batra, and Akshara Rai. Learning navigation skills for legged robots with learned robot embeddings. In 202/ JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 484 491. IEEE, 2021. 2,
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. 24
Yinuo Wang, Likun Wang, Yuxuan Jiang, Wenjun Zou, Tong Liu, Xujie Song, Wenxuan Wang, Liming Xiao, Jiang Wu, Jingliang Duan, et al. Diffusion actor-critic with entropy regulator. Advances in Neural Information Processing Systems, 37:54183-54204, 2024. 25
Zhenyu Wang, Yali Li, Xi Chen, Ner-Nam Lim, Antonio Torralba, Hengshuang Zhao, and Shengjin Wang. Detecting everything in the open world: Towards universal object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1 11433-11443, 2023. 2
Maciej Wolczyk, Bartlomiej Cupiat, Mateusz Ostaszewski, Michat Bortkiewicz, Michal Zajac, Razvan Pascanu, Lukasz Kucinski, and Piotr Miloš. Fine-tuning reinforeement learning models is secretly forgetting mitigation problem. arXiv preprint arXiv:2402.02866 2024.
Chengyue Wu, Yukang Gan, Yixiao Ge, Zeyu Lu, Jiahao Wang, Ye Feng, Ying Shan, and Ping Luo. Llama pro: Progressive llama with block expansion. arXiv preprint arXiv:2401.088 022415, 2024.
Wayne Wu, Honglin He, Chaoyuan Zhang, Jack He, Seth Z. Zhao, Ran Gong, Quanyi Li, and Bolei Zhou. Towards autonomous micromobility through scalable urban simulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 8, 18, 22, 23, 24
Ziyang Xie, Zhizheng Liu, Zhenghao Peng, Wayne Wu, and Bolei Zhou. Vid2sim: Realistic and interactive simulation from video for urban navigation. CVPR, 2025. 2,
Jie Xu, Viktor Makoviychuk, Yashraj Narang, Fabio Ramos, Wojeiech Matusik, Animesh Garg, and Miles Macklin. Accelcrated policy learning with parallel differentiable simulation. arXiv preprint arXiv:2204.07133 07.337 2022. 26
Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng. and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 10371-10381, 2024. 2
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023. 6
Yuke Zhu, Roozbeh Mottaghi, Eric Kolve, Joseph J Lim, Abhinav Gupta, Li Fei-Fei, and Ali Farhadi. drrget-driven visual navigation in indoor scenes using deep reinforcement learning. In 2017 1017 IEEE international conference on robotics and automation (ICRA), pp. 3357-3364. IEEE, 2017. 3,4
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A Statement on Large Language Model Usage
16
Demonstration Video
16
Real-World Experimental Results
17
C.I Scenarios with Static Obstacles
17
C.2 Scenarios with Moving Pedestrians
17
D Additional Experimental Results
17
D.1 Comparison of S2E variants
18
D.2 Qualitative Results on NavBench- GS Benchmark
18
D.3 Effectiveness of Reinforcement Learning
20
D.4 Cross-Embodiment Generality
20
D.5 Ablation Anchor-based Distribution Matching
21
D.6 Ablation on simulation environments:
22
Experimental Details
22
E.1 Details of Dataset and Environments
23
E.2 Details of Model Architecture
24
E.3 Details of Pretraining
24
E,4
Details of Finetuning
24
E.5 Details of Reward function designs.
25
E.6 Robots in Simulator
26
E,7 Robots in Real World
26
A STATEMENT ON LARGE LANGUAGE MODEL USAGE
We used GPT-5&40 large language model (LLM) as a writing assistant to polish sentences, refine word choices, and check grammar consistency. The LLM not involved in any reference collection, research ideation, experiment design, data analysis, or result interpretation. All technical contributions and conclusions in this work are solely the responsibility of the authors.
B DEMONSTRATION VIDEO
We highly recommend watching supplementary video for detailed demonstrations. It presents a variety of experiments in both the simulator and the real world that thoroughly evaluate our S2E model across various settings and embodiments. All real-world experiments conducted on different robols and scenes used the same S2E model. The video consists of four sections:
1) S2E Capabilities Demonstration: highlights the ability of S2E in zero-shot deployment, including obstacle avoidance, interaction with pedestrians.
2) Long- mogghorizon navigation: demonstrates the robustness of S2E in long-horizon urban environments.
3) Comparison with SOTA Methods: provides evaluations against representative baselines in real- world, demonstrating the performance of our method.
16

---

Published as a conference paper at ICLR 2026
4) Data and Benchmark: introduces training dataset for model pretraining, the interactive simulator used for policy finetuning, scenario overview of the NavBench-Gr benchmark.
C REAL-WORLD EXPERIMENTAL RESULTS
In this section, we present experimental results real-world scenarios, showcasing the zero-shot deployment performance of S2E. We first describe the setup and analysis of static obstacle evaluation C.1, followed by experimental results involving dynamic human interactions with robots in C.2.
Sonario 2 Soenzti
5s
25s
Figure 10: Evaluation on scenarios with static obstacles.
C.I SCENARIOS WITH STATIC OBSTACLES
We placed objects along the robot's path, such as rubbish bin on the first row and cone on the second row Figure 10, to evaluate obstacle avoidance. Figure 10 demonstrates the performance of our S2E model in static obstacle scenarios; Scenario 1 illustrates GO2's navigation in indoor environment, while Scenario shows COCO's performance in an outdoor park setting with benches. The results confirm that our method reliably avoids obstacles.
C.2 SCENARIOS WITH MOVING PEDESTRIANS
Cemrato Coaando
t 5s
t 20s
Figure Evaluation on scenarios with moving pedestrians.
We designed these experiments to evaluate robotic navigation capabilities in dynamic environments. In addition static obstacles (e.g., the cone shown Figure II), we introduced dynamic scenario where a pedestrian walks toward the GO2 robot, forcing it to replan its path. As demonstrated in Figure 11, when the pedestrian obstructs the robot's intended path, the robot dynamically adjusts its trajectory while still reaching the target destination.
D ADDITIONAL EXPERIMENTAL RESULTS
In this section, present additional experiments to further analyze the framework design and effectiveness of S2E. We begin by evaluating different variants of S2E to highlight the effectiveness of our design D.I. followed by validating the effectiveness of reinforcement learning in improving
17

---

Published as a conference paper at ICLR 2026
Empty
Obstacle
Pedestrian
Obstacle Pedestrian
Method
SR
RC
CT
SR 1
RC
CT 4
SR f
RC
CT
SR
RC1
CT
S2F-DDsccret
0.44
0.80
0.07
0.37
0.74
0.92
0.40
0.74
1.36
0.35
0.53
2.01
S2E- Difffusios
0.65
0.86
0.03
0.44
0.75
0.91
0.60
0.68
1.43
0.37
0.66
2.07
S2E-BC
0.65
0.88
0.08
0.42
0.71
0.87
0.63
0.74
1.61
0.40
0.70
2.22
S2E-PPO
0.15
0.27
0.93
0.02
0.10
2.37
0.04
0.12
1.94
0.01
0.08
4.94
S2E-Full
0.82
0.92
0.00
0.57
0.78
0.69
0.74
0.78
1.50
0.51
0.73
1.58
Table 4: NavBench-GSS Benchmark.
other offte-of-the--ee methods (Shah al., 2023b) D.3. We then measure the generalizability of the model across diverse embodiments in D.4, conduct ablations anchor points used for prediction in D.5. Furthermore, we provide qualitative results on the NayBench-GG benchmark in D.2.
D.1 COMPARISON OF S2E VARIANTS
As shown in Table 4, we first compare and DIS--IFFUSSON (replace the Transformer decoder by DiT' architecture as in DiffusionDrive (Liao al., 2025)) against the baseline S2E-BC. The results demonstrate that the discrete formulation struggles to capture multimodality, leading to poor performance. Moreover, diffusion does not yield additional gains, the GMM-based formulation is already sufficiently expressive fit the data, while also being more amenable to reinforcement learning fine-tuning, The performance gains are significant when comparing S2E-Full S2E-BC, with success rate improvements of 15% in obstacle settings, 1 11% in pedestrian scenarios, and 11 % in obstacle- obstacle-pedestrial pedestrian scenarios. These substantial improvements across increasingly complex environments demonstrate RL's critical role enhancing policy's 's interactive capabilities.
(trained with classification anchor points)
D.2 QUALITATIVE RESULTS ON NAVBENCH GS BENCHMARK
To enable fair comparison with existing navigation models, we developed the NavBench-GS bench- mark, which supports closed-loop evaluation for all navigation policies. Figure 12 presents repre- sentative test scenarios from our benchmark. Since these scenes typically lack roadside elements or interactive objects, we construct compositional scenes by adding static obstacles and dynamic pedestrians to increase the complexity and realism of the benchmark scenarios.
Figure 12: Scenes in NayBench-GS Benchmark.
Figure shows compositionally rendered results on the NavBench- benchmark. We construct NavBench on top of URBAN-SIM (Wu et al., 2025), and employ instance-mask- compositional rendering to seamlessly integrate backgrounds generated from 3D Gaussian Splatting (3DGS) Kerbl et al. (2023) with obstacles and pedestrians from the simulator. This design enables realistic scene synthesis and supports the evaluation physical interactions in complex navigation environments.
Figure 14 shows comparisons among different variants of S2E in scenes NayBench-GS benchmark. In each scenario, we give the observation and the trajectory of the agent, along with the starting and the goal point. Colored trajectories correspond to different policies- SPE---22EPPPPPP (red), S2E-BC under two different control schemes (pink and yellow), and the complete S2E-Full model (cyan). both scenarios S2F-PPO and S2E-BC either deviate from the intended path fail to reach the destination,
18

---

Published as a conference paper at ICLR 2026
Figure 13: Physical interaction in NavBench-GG GS Benchmark.
becoming trapped by roadside obstacles or crashing into open space. By contrast, the S2E-Full successfully navigates around obstacles and converges to the goal in all cases, demonstrating its robustness.
As illustrated in Figure 15, column shows our compositional scenes, where obstacles and pedestrians are overlaid onto base GS environments to simulate complex real-world navigation challenges. Column 2 and 3 demonstrate that NoMaD (Sridhar al., 2024) becomes trapped by roadside obstacles (e,g,, traffic lights) or deviates from the intended path, while Citywalker (Liu et al., 2024b) successfully reaches the destination in Scenario 2. Notably, column 4 demonstrates that our method outperforms both approaches, achieving robust navigation all scenarios containing both static and dynamic obstacles.
Obanvation
Obseraaioon
318-:: 2B0
SIS- 2NO
Startirn
Goal
Starting
Figure 14: Navigation Trajectories in NayBench-GS Benchmark.
Compositional scenes
NoMaD
CityWalker
Ours- Ours-S2E
Scenario
Scenario 2
Figure 15: Comparison with SOTAs on NayBench-Gr GS Benchmark.
19

---

Published as a conference paper at ICLR 2026
D.3 EFFECTIVENESS OF REINFORCEMENT LEARNING
To validate the effectiveness of RL in improving performance. we conduct more studies between the pretrained model and its finetuned weights in the simulator. Specifically, further conducted experiments on RL-based finetuning on top of ViNT* (Shah et al., 2023b) introduced in manuscript, where only the policy head is updated while the backbone remains fixed. The setting of simulation environments, reward design, curriculum, etc. are consistent with the one used in S2E. As shown in results on Table 5, the finetuned policy outperforms the pretrained ViNT across all evaluation metrics. Notably, it achieves significantly higher success rates and route completion, while also reducing collision cost. These results highlight the benefit of leveraging RL to continuously scale navigation foundation models that are trained solely on offline data.
Method
SR t
RC f
CT
ViNT*
0.27
0.31
2,01
VRL
0.39
0.55
1.41
Table 5: Performance comparison between pretrained and RL-finetuned VINT* (Shah al., 2023b). Finetuning the policy using significantly improves navigation performance across metries, which demon- strates the effectiveness reinforcement learning in scaling navigation performance.
150
Training Reward
100
50
gradient explosion
DecFT-RL
0
Ours
0.00
0.25
0.50 Step (le5)
0.75
1.00
Figure 16: Training curves of RL-finetuing on action decoder layers V.S. Ours- RAM.
Additionally, we provide training curves comparing DecFT-RL (reinforcement learning fine-tuning applied directly to the action decoder layers) with our method. As shown in Figure 16, DecFT-RL quickly suffers from gradient explosion, whereas our approach enables stable training. Furthermore, during training, DecFT-RI. requires nearly 40GB GPU memory, while our method only consumes 37GB single GPU with 64 parallel RL environments.
D.4 BROSS-EMBODIMENT GENERALITY
URBAN-SIM-Empty
NawBench-GSS-mmpee
Robot Type
SR 1
RC
SPL
SR
RC
SPL 1
Wheeled Rohot
0.99 + 0.06
0.99 + 0.09
0.81 + 0.26
0.92 0.03
0.96 0.02
0.90 0.03
Quadruped Robot
0.93 + 0.18
0.96 + 0.10
0.91 + 0.17
0.89
0.08
0.89
0.13
0.87 +: 0.08
Humanoid Robot
0.40 + 0.16
0.75 + 0.19
0.37 + 0.19
0.21 4 0.04
0.92 0.09
0.18 + 0.03
Table 6: Cross-embodiment generality. As general navigation model, S2E can be directly deployed on various robotic platforms without any modifications.
Generalization across different embodiments is important for deploying transferring policies in real-world applications. It would significantly reduce the retraining cost and improve the scalability of the model. To valid this capability, evaluate the policy across three types of embodiments,
20

---

Published as a conference paper at ICLR 2026
Le., wheeled, quadruped and humanoid robots. We test two scenarios: simulation scenes generated by URBAN-SIM and gaussian splatting scenes from NavBench-Gis As shown in Table 6, our approach maintains its performance across these different embodiments, showcasing the embodiment- agnostic property of the model. The wheeled and quadruped robots achieve high success rates, route completion, and SPL scores in all scenarios, demonstrating effective generalization across action controllers. While the humanoid robot shows lower success rate, its performance remains reasonable considering the increased joints. Notably, all evaluations on both simulated and real-world environments across all robots use one S2E model. We highly recommend referring to Section B and our demonstration video for more qualitative results and real environments.
D.5 ABLATION ON ANCHOR-BASER DISTRIBUTION MATCHING
Here, we provide the visualization results of the learned anchor points, shown in Figure 17.
Cluster centers
90
2
(-c (m)
0
4
44
49
X (+: forward) (m)
Figure 17: Visualization results anchor points from K-means. Red squares indicate the clustered end- points in the robot frame, while green circles denote the corresponding start-points in the robot frame.
We further investigate the influence the number of anchors used in the S2E model. To systematically study its impact, we conduct an ablation study on the RECON (Shah et al., 2021) testset by varying the number of anchor points used during both training and inference. All experiments share the same visual encoder, training configuration, and evaluation protocol to ensure fair comparison.Anche Anchor points serve as intermediate spatial targets that guide the policy toward the goal. As shown in Table 7, increasing the number of anchors from I to 64 improves performance, reducing minADE and improving mAP.
# Anchor
minADE
mAP 1
0.21
0.57
4
0.17
0.58
8
0.13
0.62
16
0.13
0.59
64
0.09
0.69
Table 7: Ablation on anchor point number. We vary the number of anchor points used in the S2E model and evaluate on the RECON (Shah et al., 2021) testset. Increasing the number of anchors improves performance up a point, with 64 anchors yielding the best results.
21

---

Published as a conference paper at ICLR 2026
# Anchor
SR f
CT
S2E-BC-Single
0.33
1.51
S2E-BC
0.42
0.87
Table 8: Comparison on NawBen-GSS-Ossacce
Figure 18: Examples of simulation environments in URBAN-SIM for RI. finetuning.
As shown in Table 8, we compare variants of S2E-BC with different anchors on the NavBench-GS- Obstacle benchmark. When using only a single mode for regression (S2E-BC-SINGLE) the model often fails to capture multimodal behaviors and thus exhibits significantly lower success rate and higher collision time. In contrast, employing multiple anchor modes (S2E-BC) provides a richer representation space, leading improved success rate and reduced collision time.
D.6 ABLATION SIMULATION ENVIRONMENTS
In the default setting URBAN-SIM (Wu et al., 2025), RL environ- ments randomly sampled across the region, which deviates from realistic urban layouts. To address this, we redesign the environment layout with as open source), enabling the creation of more structured and real- istic simulation scenarios, as illustrated in Figure 18 and Figure 19. As shown in Table 9, our redesigned environment distribution leads notable improvements both success rate and collision time, demonstrating the effectiveness of spatially coherent RL environments.
Methods
SR
CT 1
S2E-UrbanSim Ours
set of procedural generation rules (to be released
Table 9; Effectiveness the spa- tial distribution of RL envs.
E EXPERIMENTAL DETAILS
In this section, we provide more details regarding the model architecture, dataset and simulation environments, training strategies, and robots used in our experiments. We begin with description of the datasets and simulation environments in E. followed by the S2E model structure in E.2. E.3 and E.4, E.5 describe the pretraining and finetuning strategies, respectively. Finally, Sections E.6 and E.7 introduce the simulated and real-world robotic platforms.
22

---

Published as a conference paper at ICLR 2026
Figure 19: Examples of simulation environments in our experiments for RL finctuning.
X
9CA
URBAN-SIM
Figure 20: Examples of dataset for pretraining.
E. DETAILS OF DATASET AND ENVIRONMENTS
The S2E training dataset contains over 100 hours of navigation trajectories, sourced entirely from existing real-world records (Shah et al., 2021; Karnan et al., 2022b) or simulation platforms (Liu et al., 2024a; 2025; Wu et al. 2025). The dataset consists of a combination of navigation behaviors collected across distinct robotic platforms, as shown in Table 10 and Figure 20. For real-world datasets, trajectories labeled using odometry or fused GPS/IMU estimations. In simulation environments, ground-truth truth poses are directly extracted from the simulator. All data converted into standardized duservation-accion pairs by converting applying egocentric transformation.
As shown in Figure 19, TRBAN-SIM (Wu et al., 2025) and modified the procedural generation pipeline for finetuning. These environments designed for urban navigation, including static obstacles and varying scene layouts. The diversity and randomness in object placement, texture, and lighting encourage robust policy finetuning during reinforcement learning.
a diverse set of procedurally generated simulation environments from
23

---

Published as a conference paper at ICLR 2026
Dataset
Platform
Hrs. Used
Environment
RECON (Shah et al., 2021)
Jackal
25h
off-rood
SCAND (Karnan et al., 2022b)
Spot, Jackal
5h
sidewalks
FrodoBots-2K (Team, 2024; Hirose et al., 2025)
FrodoBot
65h
sidewalks
4
X-Mobility (Liu ct al., 2024a; 2025)
Nova Carter
2.5h
warchouse
5
URBAN-SIM (Wu et al., 2025)
COCO, GO2, G1
2.5h
sidewalks
Table 10: Overview of Training Datasets. The table summarizes the robot platforms, environments and useful details covered each dataset used S2E pretraining stage.
E.2 DETAILS OF MODEL ARCHITECTURE
In this section, we provide details of the model architecture used in S2E. We adopt DINOv3 -aaeedd visual encoder (Siméoni et al., 2025). Specifically, we first encode the past 10 frames sampled at 5Hz into frame-level tokens, and encode the current frame into path-level tokens via a grid pooling strategy, resulting in total of 10 + 64 tokens. The goal point is embedded through lightweight linear MLP, while the goal image is encoded using DINOv3, and the token length of goal image is 16 after grid pool. These tokens Transformer ( (Vaswani et al., 201 2017)-based architecture consisting of a 6-layer encoder and a 6-layer decoder, each with attention heads and a feedforward dimensionality of 3072 (i.e., 4666)) The encoder operates over the observation and goal tokens, while the decoder takes observations and goal keys K and values V, and anchors queries Q. generate action-relevant representations. Finally, the anchor features are decoded by three separate MLP heads to predict normalized trajectories with velocity and score. To mitigate shortcut learning, we apply stochastic masking over the goal signals: with probability of 0.35 neither the goal image nor the goal point is provided, with probability of 0.20 only the goal image is available, with probability of 0.40 only the goal point is available, and with probability of 0.05 both signals are provided.
then combined with anchor representations and processed by
E.3 DETAILS OF PRETRAINING
As shown in Table 11, we provide detailed list of hyperparameter used in pretraining stage of S2E,
S2E Model
S2E Pretraining
RGB Resolution
256 x 256
Epochs Nep
100
Encoder
Dinov3
Batch Size
256
Token Dimension
768
Learning Rate
2 x 10
Attention Hidden Dimension
3072
Optimizer
AdamW
# Attention Layers 7
6
LR Schedule
Cosine
Attention Heads ng
Scheduler Period
40
Temporal Context
10
Compute Resources
8x NVIDIA L40S
Prediction Horizon T
10
Table 11: Architectural and Pretraining Hyperparameters for S2E. Left: model architecture. Right: training
configuration.
E.4 DETAILS OF FINETUNING
In this section, we first introduce the finetuning strategy for the GMM-based policys.
Standard deviation reinitialization. We observed empirically that the action distribution learned from imitation learning tends to have very small standard deviation (i.e., - 0+), which hampers sufficient exploration in reinforcement learning. However, retraining the action decoder is undesirable, the anchor action mapping has already been well captured by imitation learning. To address this, we introduce an additional head to generate the log standard deviation, initialized to zero (corresponding to 1), thereby ensuring adequate exploration during RL fine-tuning.
We use the standard sampling strategy for GMM in RL finetuning. Given the mixture weights {g: m= means (Jx) JM - and standard deviations {a, 1 action a is sampled by first selecting
24

---

Published as a conference paper at ICLR 2026
component index Categorical(g), (g), then sampling from the corresponding Gaussian:
(J om))
(12)
Since the exact entropy of a GMM does not have a closed-form solution, we use simplified estimation of its lower bound. First, approximate based on the statement of (Huber et al., 2008; Wang et al., 2024):
M
M - Σ
H 2
Σ qm
2 log
((2Te)
qm log J.
(13)
m=1
m=1
pm oi o th m2
Em
(14)
pTh Tx
oy
we further set pm 0 during finetuning, so we have the estimation:
M
4m (2g (m
o
Σ
9m log 9m.
(15)
We train agent with 64 parallel environments, and the detailed list of hyperparameter used in finetuning stage of S2E shown Table 12.
Hyperparameter
Value
Epochs Tep
Minibatch Size
512
Learning Rate
1 10
Optimizer
AdamW
LR Schedule
Adaptive
KL. Threshold
0.01
Clip Range
0.2
GAE
0.95
Discount Factor y
0.99
Entropy Coefficient
0.001
Rollout Horizon
32
Gradient Clipping
1.5
Value Loss Coefficient
2.0
Compute Resources
1x NVIDIA L40S
Finetuning Time
8 hrs
Table 12: S2E PPO Finetuning Hyperparameters. PPO-related training configurations used for finetuning the policy in our experiments.
E.5 DETAILS OF REWARD FUNCTION DESIGNS.
We design the reward function R as:
R Rg + RiR + Rii:
(16)
Rg = Ry.d + Rg. + Re,d + Resr
(17)
RR Rwalkabler
(18)
R = Rerasectory Rheadingg
(19)
Dense goal-reaching reward Rg Defined as Rg.d = tanh(d, where o controls the distance scale and d, denotes the distance the goal point. We use standard
25

---

Published as a conference paper at ICLR 2026
deviations (Ocoarse 10, fine 2) for coarse and fine levels of shaping. Additionally, a small velocity-aligmmen term is included, defined as Rvel = cos(y coe(V-Pral' which P encourages the agent's velocity v to align with the relative pose Pret toward the goal.
Sparse goal-reaching reward Rg.s: A large terminal reward of +2000 is given upon successfully reaching the goal, and the episode would be terminated the same time.
Dense collision-avoidance reward Re.d Penalizes proximity to the nearest obstacle center Re,d -0.001 x 2i 20), where d, denotes the distance to the nearest obstacle center.
Sparse collision-avoidanee reward Re.s: A penalty of 200 is applied when a collision occurs, and the episode would be terminated at the same time.
Road-keeping reward Roookabble: the agent leaves the walkable area, the episode termi- nates with penalty of -10.
Trajectory-similariyy reward Rerajectory: Penalizes deviation from the reference global trajectory, scaled as -0.00005 x Aunj.
Smoothness reward Rheeding: Penalizes abrupt heading changes, computed as the sum of incremental heading differences Σ, | scaled as Rheading -0.005 x Σ, |-
E.6 ROBOTS IN SIMULATOR
All simulated robotic platforms are built upon NVIDIA IsaacSim (Xu et al., 2022; Dorbala et al., 2023), a high-fidelity and GPU-accelerrted simulation environment that supports physics-based interactions and photorealistic rendering. Each robot is modeled using official URDF descriptions and equipped with RGB cameras.
For locomotion, employ modular control stack that consists of a low-level joint controller and high-level policy trained with reinforcement learning. The policy trained using Proximal Policy Optimization (PPO)
aerain-randdmized environments with curriculum learning. The reward function encourages stability, velocity tracking. and energy efficiency, while penalizing collisions and falls. Domain randomization in texture, friction, and mass distribution is applied to improve sim-to-real transferability. We adopt a shared framework across robot types, enabling scalable embodiment-aware training in simulation.
Wheeled robot. A differential-drive robot, controlled via a kinematic model (Polack et al., 2017), 17), where linear and angular velocities (u,w) (calculated from waypoint via ideal PD controller (Sridhar et al., 2024)) are used as control commands. The simulator integrates these commands through rigid-body physics engine, with frictional contacts determining actual wheel-ground interaction.
Unitree-GO2 quadruped robot. A quadruped platform capable of versatile locomotion in complex terrains, the action is 3-dimensional vector (Ug, w) (calculated from waypoint via ideal PD controller (Sridhar et al., 2024)). The locomotion model is lightweight MLP trained based on the standard training environments provided by IsaacSim (Xu et al., 2022; Dorbala et al., 2023).
Unitree-G1 humanoid robot. A humanoid agent with articulated head, torso, and leg joints excluding hand actuation, the action is 3-dimensional vector (Va, My w) (calculated from waypoint via a ideal PD controller (Sridhar et al., 2024)). The locomotion model is lightweight MLP trained based on the standard training environments provided by IsaacSim (Xu et al., 2022; Dorbala et al., 2023).
E.7 ROBOTS IN REAL WORLD
Wheeled robot. differential-diee platform equipped with RGB cameras and differential drive odometry, deployed for sidewalk navigation. The robot is controlled via the same kinematic model as in simulation, where linear and angular velocities (v, w) are computed from waypoints using an ideal PD controller. The odometry is used for real-time position estimation and providing target position during navigation continuously.
Unitree-GO2 quadruped robot. The real-world GO2 robot is equipped with RGB sensors and low-level locomotion controller provided by Unitree. Instead of executing joint-level commands from trained policy, interface with the GO2 through its built-in velocity control API, sending
26

---

Published as a conference paper at ICLR 2026
high-level commands (Uge, Vy, W) to leverage its native gait generation and stability modules. There is a didar-based odometry used for real-time position estimation and providing target position during navigation continuously.
27


Reviews:
[{"review_id": "XRdQX07pGz", "reviewer": "Reviewer_SfZD", "summary": "This paper proposes S2E (Seeing-to-Experiencing), a framework for training navigation foundation models that combines offline pretraining on real-world videos with reinforcement learning (RL) fine-tuning in simulation. The key technical contributions include: (1) Anchor-Guided Distribution Matching (AGDM) for modeling multimodal navigation behaviors during pretraining, (2) a Residual-Attention Module (RAM) that enables RL fine-tuning while preserving pretrained knowledge, and (3) NavBench-GS, a benchmark built on 3D Gaussian Splatting scenes. The authors claim that RL alleviates diminishing returns from scaling offline data alone and enables zero-shot transfer to real-world scenarios.", "questions": "Data Efficiency: How many RL environment steps are needed? What's the sample complexity compared to collecting more offline data?\n\nReward Engineering: How sensitive is performance to reward function design? Have you tried learning rewards from human preferences (Christiano et al., 2017)?\n\nFailure Modes: What percentage of real-world failures are due to:\n\nPerception errors (misdetecting obstacles);\nPlanning failures (local minima);\nControl errors (locomotion instability);\nSim-to-real gap\n\n\nComparison Fairness: Can you provide results where all methods are:\n\nTrained on identical 100h dataset;\nEvaluated on an established benchmark (not self-proposed);\nUsing the same evaluation protocol and metrics\n\n\nGeneralization: How does performance degrade with:\n\nDifferent camera intrinsics/extrinsics;\nDifferent weather/lighting conditions;\nNovel obstacle types not seen in training\n\n\nRL vs. SFT: The paper claims RL is better than SFT (Figure 6b), but:\n\nHow was the SFT data collected? From the pretrained policy or optimal demonstrations?\nDid you try other offline RL algorithms (CQL, IQL, etc.) that might bridge the gap?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Comprehensive System: The paper presents an end-to-end system from data collection through real-world deployment, which requires significant engineering effort.\n\nCross-Embodiment Evaluation: Testing on wheeled, quadruped, and humanoid robots (Table 6) demonstrates some generality, though the humanoid results are quite poor.\n\nHonest Limitations Discussion: Section 5 acknowledges the limitation of vision-only approaches and collision failures.\n\nDetailed Implementation: The appendix provides extensive implementation details, which aids reproducibility.", "weaknesses": "1. Lack of Theoretical Insight: The paper doesn't explain why RL helps beyond showing empirical improvements. What specific failure modes of offline learning does RL address? What inductive biases make certain behaviors learnable only through interaction?\n\n2. Modest Improvements: Many reported improvements are marginal (e.g., Table 2: 0.51 vs. 0.32 success rate for wheeled robot). Given the additional computational cost of RL (8 hours on L40S GPU), the cost-benefit tradeoff is unclear.\nCherry-Picked Comparisons:\n\n3. CityWalker* (retrained on same 100h data) actually achieves 0.67 SR vs. S2E's 0.82 SR in empty scenes (Table 1), suggesting the RL contribution is partially due to other factors;\nFigure 6(a) compares against \"prior methods\" shown as dotted lines, but these prior methods use different evaluation protocols\n\n\n4. Limited Failure Analysis: The paper shows successful cases but doesn't systematically analyze failure modes. When does RL fine-tuning hurt performance? What environments or scenarios remain challenging?\n\n5. Questionable Design Choices:\n\nWhy freeze the visual encoder during RL? This prevents learning better visual features for interaction\nWhy use a simplified entropy approximation (Eq. 11) instead of more accurate estimators?\nThe stochastic goal masking strategy (Section E.2) seems arbitrary - no ablation justifies the specific probabilities chosen\n\n\n6. Reproducibility Concerns: Despite promising code release, the method depends on:\nProprietary Unitree robot APIs\nMultiple datasets with different licensing terms\nNVIDIA IsaacSim which requires expensive GPU resources\nHand-tuned reward functions that may not transfer to other environments\n\n7. Missing Related Work\n\nThe paper should discuss and compare with:\n\nOffline RL for Robotics: Kumar et al. (2020, NeurIPS), Nair et al. (2020, CoRL), Mandlekar et al. (2021, CoRL)\nVision-Language-Action Models: Driess et al. (2023, ICML - PaLM-E), Brohan et al. (2023, CoRL - RT-2)\nNavigation Benchmarks: Savva et al. (2019, ICCV - Habitat), Xia et al. (2018, CVPR - Gibson)\nSim-to-Real Transfer: Peng et al. (2018, ICRA), Tan et al. (2018, CoRL)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UeDJ4wsTdl", "reviewer": "Reviewer_vgu6", "summary": "This paper presents results from RL-finetuning of a navigation policy that is pretrained by SFT over static datasets. It empirically shows the improvements afforded by RL, introduing a way to provide data diversity and an architecture for the policy.", "questions": "Can you explain how the anchors are defined and chosen?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "An unusual combination of SFT and RL for learning-based navigation - while existing approaches focus on one or the other, this paper shows the value of doing both.", "weaknesses": "The anchor description is not very clear. It seems to be constant-curvature arcs - a clearer explanation is needed.\nUnclear how the constant curvature arcs approximate the full diversity of demonstration paths - is the matching performed instantaneously (i.e., for an instantaneous vx, w command mapped to the corresponding curvature?)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y6IOctZJAV", "reviewer": "Reviewer_FY2s", "summary": "This paper proposes the S2E framework, combining offline video pretraining (via AGDM strategy) and simulation RL finetuning (with RAM module). It builds NavBench-GS, verifies RL boosts navigation model performance, breaks diminishing returns of data scaling, and enables zero-shot transfer across robots.", "questions": "1. Given that the current model lacks 3D perception, what specific 3D information integration methods (such as depth prediction or occupancy prediction) do you plan to adopt in future work, and how will you balance the computational cost of 3D perception with the real-time performance of navigation?\n\n2. The NavBench-GS benchmark currently covers 26 scenarios, but real urban environments involve more dynamic elements (e.g., sudden appearance of vehicles, temporary road closures). Will you expand the benchmark to include such scenarios, and what criteria will be used to select new scenarios to ensure the benchmark’s representativeness?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. This paper proposes Anchor-Guided Distribution Matching (AGDM), which uses an anchor-guided Gaussian Mixture Model to model multimodal navigation trajectories, capturing diverse valid actions under the same observation while ensuring training stability.\n\n2. This paper designs the Residual-Attention Module (RAM), which freezes pretrained components and adds trainable residual branches to cross-attention layers, enabling the model to gain interactive skills via RL without losing pretrained generalizable knowledge.\n\n3. This paper establishes the NavBench-GS benchmark, built on photorealistic 3D Gaussian Splatting scenes with physical interactions, realizing closed-loop policy evaluation and solving the reproducibility issue of real-world navigation testing.", "weaknesses": "1. The model relies solely on visual input and lacks 3D perception capabilities, leading to occasional failure in obstacle avoidance in some scenarios, which is a persistent limitation for vision-only navigation approaches.\n\n2. The real-world evaluation scenarios are relatively limited (only 25 scenarios), and the generalization performance of the S2E framework in more complex and diverse urban environments (e.g., extreme weather, complex traffic conditions) has not been verified.\n\n3. The humanoid robot in cross-embodiment evaluations shows notably lower success rates compared to wheeled and quadruped robots, yet the paper does not deeply analyze the root causes (e.g., joint complexity impacts) or propose targeted optimization strategies for humanoid platform adaptation.\n\n4. The RL finetuning relies on modified URBAN-SIM environments, but the paper only briefly mentions procedural generation rule adjustments without detailing how these rules ensure the simulated environments fully align with real urban spatial layouts, potentially limiting the sim-to-real transfer reliability", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "R3TNrBpYvt", "reviewer": "Reviewer_UWpN", "summary": "The paper proposes Seeing-to-Experiencing (S2E), a training recipe for goal-conditioned navigation that keeps visual priors from offline videos and adds interaction skills with reinforcement learning in simulation. It introduces a new action representation, using an anchor-conditioned Gaussian mixture to represent multiple short-horizon motions under the same observation. It also adds a Residual-Attention Module that fine-tunes only residual cross-attention while freezing visual encoders and self-attention to limit sim-to-real drift.\nExperiments in the NavBench-GS benchmark show RL fine-tuning outperforms supervised fine-tuning under the same budget, scales better than more offline data alone, and transfers zero-shot to wheeled and legged robots, with ablations confirming both modules are important.", "questions": "1. In Fig. 6(b) left, why does SFT still sufer from overfitting even in the in-distribution Urban-sim evaluation?\n2. In reinforcement learning and simulative benchmark, there is no specific embodiment. Then how is collision checking done? Are they achieved by a navmesh or using a uniform collision volumn?\n3. In Tab. 1, what does ZeroPolicy mean? Is it a pure RL model? I'm also curious on the authors insights on why (or why not) pure RL training might not perform well in urban navigation while they work well in indoor environments like the HM3D dataset?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper shows reinforcement learning is extremely helpful for closed-loop navigation tasks.\n2. The paper proposes using several fixed anchor points and GMM to represent the action space. This is essential specifically to the navigation task. The authors also provide in-depth analysis and comparison to other representations.\n3. The paper provides insightful discovery on the data saturation margin in the embodied navigation task and shows RL to be a more robust and efficient learning technique.\n4. The paper shows zero-shot depolyment of the trained model on both wheeled and quadroped robots. Experiment result shows superior performance compared to other baselines.", "weaknesses": "1. The \"Residual attention module\" paragraph in section 3.2 (Line 258-268) is a bit confusing. The statement is not a well established conclusion in literature. The authors should rewrite the paragraph by providing more concrete derivation of the conclusion.\n2. In ablation study (Line 462-466 and Appendix Sec. D.5), there is no comparison with difussion policy models. The authors should add comparison to that to better support the discussion in Sec. 3.1.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)