GRL-SNAM: Geometric Reinforcement Learning with Differential Hamiltonians for Navigation and Mapping in Unknown Environments
Authors: Aditya Sai Ellendula
Yi Wang
Minh Phuong Nguyen
Chandrajit L. Bajaj
Venue: ICLR 2026 Poster (2026)
Keywords: Reinforcement Learning
Generalized Hamiltonian Manifolds
Differential Policy Optimization
Primary area: reinforcement learning
TL;DR: We propose a geometric RL method that navigates and maps using only local sensing, leveraging Hamiltonian dynamics and differential policy optimization to adapt quickly under dynamic, deformable conditions

Abstract: We present GRL-SNAM, a geometric reinforcement learning framework for Simultaneous Navigation and Mapping in unknown environments. GRL-SNAM differs from traditional SLAM and other reinforcement learning methods by relying exclusively on local sensory observations without constructing a global map. Our approach formulates navigation and mapping as coupled dynamics on generalized Hamiltonian manifolds: sensory inputs are translated into local energy landscapes that encode reachability, obstacle barriers, and deformation constraints, while policies for sensing, planning, and reconfiguration evolve stagewise under Differential Policy Optimization. A reduced Hamiltonian serves as an adaptive score function, updating kinetic/potential terms, embedding barrier constraints, and continuously refining trajectories as new local information arrives. We evaluate GRL-SNAM on 2D deformable navigation tasks, where a hyperelastic robot learns to squeeze through narrow gaps, detour around obstacles, and generalize to unseen environments. We compare our method against local reactive baselines (PF, CBF, staged DWA), global A* references (rigid, clearance-aware), and deep RL baselines (PPO, TRPO, SAC) under identical stagewise sensing constraints. GRL-SNAM achieves strong path quality while using the minimal map coverage, preserves clearance, and generalizes to unseen layouts. This demonstrates that our Hamiltonian-structured RL framework enables high-quality navigation through minimal exploration via local energy refinement rather than global mapping.

Published as a conference paper at ICLR 2026
GRL-SNAM: GEOMETRIC REINFORCEMENT LEARN- ING WITH DIFFERENTIAL HAMILTONIANS FOR NAVI- GATION AND MAPPING IN UNKNOWN ENVIRONMENTS
Aditya Sai Ellendula' Yi Wang Minh Nguyen Chandrajit Bajaj².2
*Department of Computer Science, University of Texas Austin
2 Oden Institute, University of Texas at Austin
á Department of Mathematics, University of Texas at Austin fadityase, panzer. wy, minhpnguyen, ja (atad))uuxxasee..
ABSTRACT
We present GRL-SNAM, a geometric reinforcement learning framework for Si- multaneous Navigation and Mapping in unknown environments. GRL-SNAM dif- fers from traditional SLAM and other reinforcement learning methods by relying exclusively on local sensory observations without constructing a global map. Our approach formulates navigation and mapping as coupled dynamics general- ized Hamiltonian manifolds: sensory inputs are translated into local energy land- scapes that encode reachability, obstacle barriers, and deformation constraints, while policies for sensing, planning, and reconfiguration evolve stagewise under Differential Policy Optimization. A reduced Hamiltonian serves score function, updating kinetic/potential terms, embedding barrier constraints, and continuously refining trajectories uate GRL-SNAM on 2D deformable navigation tasks, where a hyperelastic robot learns to squeeze through narrow gaps, detour around obstacles, and generalize to unseen environments. We compare our method against local reactive baselines (PF, CBF, staged DWA), global A references (rigid, clearance-aware), and deep RL baselines (PPO, TRPO, SAC) under identical stagewise sensing constraints. GRL-SNAM achieves strong path quality while using the minimal map coverage. preserves clearance, and generalizes to unseen layouts. This demonstrates that our Hamittonian-structured RL framework enables high-quality navigation through minimal exploration via local energy refinement rather than global mapping.
adaptive
local information arrives. We eval-
INTRODUCTION
Reinforcement learning has achieved strong performance in high-dimensional control, but contin- uous navigation in partially observed and changing environments remains challenging. Long hori- zons, multi-scall decisions, and online adaptation often make standard methods difficult to apply, es- pecially simultaneous navigation and mapping (SNAM). where an agent must move toward goal while incrementally building and exploiting an evolving representation of the environment. In these settings, purely model-free methods can require many interactions, while hierarchical approaches often depend on manually designed decompositions that do not transfer well across environments.
central difficulty is that most standard reinforcement learning (RL) policies do not explicitly encode the geometric and physical structure of navigation. As result, they may rely heavily on statistical regularities in training distribution rather than on invariant task structure, which can reduce robustness under distribution shift and degrade performance over long rollouts.
1.1 BEYOND STANDARD BELLMAN BOOTSTRAPPING VIA HAMILTONIAN UPDATES
Our framework departs from the Bellman-recursinn training loop that underlies most mainstream RL methods. Instead of learning value function through recursive bootstrapping and then improving policy from that estimate, our method parameterizes a Hamiltonian energy landscape whose gradi- ents induce local control actions. The Hamiltonian parameters learned directly from trajectory

---

Published as a conference paper at ICLR 2026
feedback and stagewise adaptation signals, so training remains fully interactive and data-driven. At execution time, actions are computed in a feedforward manner from the current state, local observa- tions, and the current Hamiltonian, avoiding rollout-based value propagation during control. While value-based methods and our dual formulations agree at optimality, our method explicitly learns in the non-optimal settings by modeling the dual Hamiltonian dynamics directly.
1.2
KEY INSIGHT: HAMILTONIAN STRUCTURE AS NAVIGATION INDUCTIVE BIAS
We propose to encode navigation through learned Hamiltonian:
H(q. p) (p) + P(g)
(1)
where kinetic and potential energies encode control objectives, constraints, and adaptation strategies. This formulation introduces three structural advantages:
(1) Energy conservation stabilizes long-horizon rollouts by preventing accumulation of numerical errors. (2) Symplectic geometry naturally separates fast reactive dynamics from slow strategic planning, addressing multi-scale temporal coordination. (3) Barrier encoding integrates safety and collision avoidance directly into potential functions, eliminating fragile reward shaping.
Hamiltonian structure transforms policy optimization into Differential Policy Optimization (Nguyen & Bajaj (2025)). where policies emerge as gradient flows of learned energies that respect geome- try. conserve invariants, and generalize across environments. Because the Hamiltonian is local encoding a long-horizon objective, local gradient steps remain globally goal-directed through the shared energy landscape.
1.3
ONL HAMILTONIAN SYNERGY
We distinguish between complementary learning regimes that exploit this geometric structure:
Offline learning fits reference Hamiltonians Ae" from interaction trajectories, capturing reusable geometric structure for sensing, path extraction, and reconfiguration local frames. These learned reference energies serve stable priors for downstream adaptation.
Online adaptation performs constrained, environment--ependeee corrections to the reference Hamiltonian,
h addatted 1e A context
where A context is inferred from the current obstacle configuration and task context. This yields conservative adaptation: the system preserves the learned reference geometry while adjusting only the energy components that are relevant to the new environment.
1.4 CONTRIBUTIONS
Our contributions include:
1. A dual Hamiltonian RL formulation for SNAM: a structured Hamiltonian energy landscape whose gradients induce local control actions, yielding a distinct dual alternative to standard value-function-basse RL.
We formulate navigation learning
2. Multi-scale geometric coordination: Differential policies for sensing, planning, and adap- tation unified through shared energy formulations, achieving temporal scale separation without manual hicrarchy design.
3. Physics-rouunded adaptation: Principled offline-online decomposition where stable ref- erence dynamics adapt through geometric alignment rather than catastrophic relearning.
4. Theoretical properties: We analyze stability under temporal scale separation, structure preservation for the conservative Hamiltonian substeps, and additive sample-complexity scaling under independent module training.
5. Empirical validation: We validate the framework on deformable navigation tasks and
compare against both generic deep RI. baselines and task-specific navigation baselines.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
We focus on structure- preserring.......g.... deployable navigation with deformable bodies. Our work lies the intersection of geometric control, continuous reinforcement learning, safety-critical plan- ning, and deformable robot navigation.
Geometric Structure in RL and Control. Most navigation-oriented RL methods operate in Eu- clidean spaces using standard PPO (Schulman al. (2017)), SAC (Haarnoja al. (2018)), TD3
(Fujimoto et al. (2018) 8)) objectives, without explicitly preserving geometric structure in the induced dynamics. Prior geometric approaches include Je(3vvrquvari policies (Hoang et al. (2025)
and Riemannian safe navigation (Klein et al. ) which incorporate symmetry or manifold- aware projections. Hamiltonian and port- Hamiltonian neural models (Desai et al. (202T) show that structure-per prrettur-preserving parameterizations can improve stability and generalization. However, these works focus on simpler control settings rather than modular, partially observed navigation.
peiti-proooggggaasseeeee RI, recent work has ex- plored continuous- Cattinoous-time formulations that avoid or reinterpret Bellman recursion. Jia and Zhou (Jia) & Zhou (2023) derive continuous-time g-learning through a martingale characterization. Settai et al. (Settai et al. (2025) study model-free lemporal-difference learning for stochastic continuous dynamics through an HJB-oriented lens, and Nguyen and Bajaj (Nguyen & Bajaj (2025) develop dual Hamiltonian control perspective. Our work is most closely aligned with this emerging line, but differs in its focus on simultaneous navigation and mapping, modular Hamiltonian subpolicies, and stagewise online adaptation in geometry-rich environments.
Continuous-Tinne RL. Beyond standard
Carety-Critical Navigation. Control Barrier Function (CBF) integration with RL achieves formal safety guarantees (Li et al. (2023)), but treats constraint satisfaction as orthogonal to navigation optimality, often resulting in conservative behaviors. Our Hamiltonian formulation integrates safety constraints directly within the energy structure.
Deformable Robot Navigation. Recent work demonstrates ring-like navigation through pre- programmed strategies: acrial gap navigation via fixed Liquid Crystal Elastomer responses (Qi al. (2024) and HAVEN (Mulvey Nanayakkara (2024)) using predetermined shape-changing sequences. These approaches rely on offline parameter optimization followed by deterministic exe- oution-ther cannot adapt deformation strategies online as environmental conditions change.
Neural Scene Representations. NeRe-based SLAM methods like NICE-SLAM (Zhu et al. (2022) provide rich environmental representations that complement our energy-based navigation formula- tion by supplying obstacle and free-space information for barrier and goal potential computation.
Simultaneous Navigation and Mapping. Most SNAM approaches prioritize building detailed maps before navigation. SGoLAM (Kim et al. (202I)) couples goal localization with occupancy mapping, CMP (Gupta et al. (2019) integrates a differentiable planner into learned mapping, and CL-SLAM (Vödisch et al. (2023)) maintains maps for long-term adaptability. In contrast, our GRL- SNAM framework aims to reach goals via high-quality, well-weighted paths while mapping as lit- tle of the environment as possible. To our knowledge, prior work explicitly targets minimal exploration; our method introduces progressive path refinement, continually improving least-cost trajectories as new observations arrive.
Positioning, GRL-SNAM extends Hamiltonian structure from low-dimensional control to simulta- neous navigation and mapping with sensing, path extraction, and deformable-body reconfiguration. Unlike approaches that append safety filters to learned controller or rely on pre-programmed de- formation strategies, our method learns an energy landscape through interaction and executes via local Hamiltonian gradients. The modular architecture preserves specialization across subpolicies, while the shared Hamiltonian coupling provides a globally goal-directed objective and a principled mechanism for stagewise online adaptation.
3 METHODOLOGY
We present GRL-SNAM (Geometric Reinforcement Learning for Simultaneous Navigation and Mapping), a Hamiltonian-strrctuure navigation framework that combines offline-eaamed subpoli- cies with online stagewise adaptation. The central object is surrogate Hamiltonian defined on
3

---

Published as a conference paper at ICLR 2026
phase space; its gradients generate local control actions for sensing, path extraction, and shape configuration, while a navigator adjusts environment-dependentt weights to reflect current obstacle configurations.
We consider a deformable robot with state 9: = (c,, 0,, Yr 4/)) E Q. where (c,, 8.) denotes the robot pose in the world frame and (ye, t44 captures sensing and internal configuration variables. The robot moves from Xg to x in an unknown environment with binary occupancy map / IR² - {0. 1}. At each stage, the current local obstacle realization is summarized by environment descriptor E. GRL-SNAM operates in two coupled layers: offline, it learns three module-specific Hamiltonian response models; online, assembles a surrogate Hamiltonian from these learned responses and performs stagewise corrections as & changes.
3.1 ONLINE STAGEWISE ADAPTATION
We now describe how the Hamiltonian structure ($3.2), the modular policy architecture ($3.3). and the meta-learning framework ($C.4) are combined at test time into unified navigation system.
Overview. At each navigation stage, the navigator sequentially queries the three -raiind policies Ty (sensor), Ty (frame/FPE), and To (object/reconfig) to obtain state-dependent control proposals. The meta-policy 9E maps the current environment and policy responses to environment- dependent energy weights and friction, from which assemble surrogate Hamiltonian and inte- grate the corresponding port-Hamiltonianl dynamics with dissipation and port correction. Observable quantities such clearance, goal progress, and speed are then used to perform short Jacobian- based update the active energy weights and friction coefficients, yielding a stagewise adaptation loop. The full pseudo-code, including initialization, query protocols, energy assembly, integration details, and adaptation rules, is deferred to Algorithm i Appendix EE] Although the control law is evaluated locally at each step, the resulting action field is globally conditioned through the Hamilto- nian weights, which depend on the sensed obstacle configuration and goal context.
3.2 NAVIGATION AS HAMILTONIAN OPTIMIZATION
From optimal control to Hamiltonian via Legendre-Fenchel conjugacy (fixed E). Fix an environment & and consider the control-afiine dynamics (gg, u; E) -R(g;& E) + puu), where R encodes goal//effectionnbbrriee terms and & penalizes effort. Pontryagin's 's principle introduces a costate p and the control Hamiltonian H(g, p, E) p (f(a) + A(g)u) - L(g, u;E), 8). Eliminating u amounts to taking the Legendre-Fenchel conjugate of yp:
f(g) + A(g) with stage cost
() sup (pT A(q)u - s(u)} +p f()) + R(g;
u =A{((())T +p f(4) + R)
provided i0 is proper, closed, and strictly convex. The optimal feedback is (g. = (A((A(q) p). In the common quadratic case (pu) u TTT a (with $ > 0 as a kinectic "term"), have 90 (w) 1 w --1w, hence
((() H(g. _ 1 - A(1) + ST((2) + R(g:E).
Identifying the inverse mass as -1 == A AAg)) and (optionally) absorbing p f()) into gauge term (or set f()) = 0) yields the mechanical form H(q. 2p M(g) R(g: E). The canonical equations, : v and أ = -V are therefore the Hamiltonian outcome of the fixed-E optimal control problem. Soft constraints (barriers) simply contribute additively R; nonconservative effects can be modeled as port inputs without altering the conjugate construction (esp. friction). Thus, for each scenario &, the inner motion law Hamiltonian with kinetic energy induced by the control penalty via conjugacy and potential shaped by the environment. Note that motion planning offline policy may or may not follow the surrogate Hamiltonian one wish to align but we caliberate the surrogate by interaction with response.
Search space for the Hamiltonian. The goal of navigator is to learn search in the energy space of H. The Hamiltonian defined in is iss a function on the cotangent bundle T Q:
H E H (() =1p² M -Tp + R( M Q Q- R E 8)
4

---

Published as a conference paper at ICLR 2026
We regard L as Hilbert space of admissible potentials on Q (and w.r.. environmental configu- ration) (e.g. [2 &).. For planar navigation we restrict the search to the enviromment-indext linear generated by task energies, where each policy governs distinct energy term and a joint dynamical barrier term. We model the meta navigator by parametrized map & - Vig ()((E) producing nonnegative dual weights that shape the primal potential. In general, each energy term may itself be parametrized:
Hiq.p:W.5: = MM] + R(g:w, Te(
R(g;w; nE(E)) Faensor (g: (g:E,w)) + B(E)) Figoal (1: wg) + X(E) Fobj(g
Σ
a;(E, ((d(;))(((((
iEC((E.))
RR ( Here w (wy: War w Wey wb) are intra- term parameters (e.g. metric, goal shape. deformation model, barrier template), while Te learns dual weights. The cardinality m(E,t) = 2+ (C, 9) is pewiirommettaaciveeeeee dependent, so 77 is implemented with permutation-invarian set encoder that outputs per-constraint scores a; t) 0, together with scalars ((E), X(E) 0. The active set C,(E, q) == {i d, E) s d} discovered online by sensing.
with (E( (BbE)))ob((E) (a, (a, ()))
the inter-term tradeoffs by mapping the environment
3.3 NAVIGATOR'S S MUBMODULAR ARCHITECTURE
Rather than learning single navigation policy, decompose the problem into three independent score functions, each dedicated to specific navigation aspect (see Figure[]for details):
Definition 3.1 (Independent Score Functions). Let K. = {u. f denote the set of policy indices corresponding to sensor, frame, and object domains respectively. For each k E K. define:
E Zx: the phase space state for policy k, where Zμ 2x X Pk with configuration space 2 and momentun space Pk
02 E Ob: the learnable parameters for policy k, where parameter sets satisfy disjointness:
θ: (nne 1 = for i (
h Zz x Rzo + R: learned energy functional parameterized by Ok
Each policy Tu is defined as an independent score function: 5k (zk: E,t) = Sax (V 1i (zk, E,t,),
The parameter disjointness ensures independence:
0a =0 for all + k, allowing parallel -Ø,
training while maintaining coordination through shared constraints Ct.
Policy A bstraction. Each policy is an offline-learned module exposed through a standardized query- response interface:
Sensor Policy (TT)): Adapts perception parameters energy gradients for information gathering
Frame Policy (TT)): Plans collision- paths energy gradients for goal attraction
Shape Policy (m,): Controls robot deformation -+ energy gradients for obstacle navigation
The key insight is that Navigator is agnostic policy implementation contribution is the Hamiltonian structure binding them together through dynamic constraint sets C,,
Algorithm details the online adaptation procedure, where the navigator issues sequential queries to the sensor, frame, and reconfig policies, integrates their energy gradients into a Hamiltonian update, and applies meta-corrections for contextual alignment generate stable trajectories for new environments.
Hamiltonian of modular sub-systems. Let K. = (v. f. of index three Hamiltonian submodules with local states (g+.Pk) E T Qr and local Hamiltonians
Hy(gh, E.E = PT. Mx(gx) -'px + Ra (q)n
module potential

---

Published as a conference paper at ICLR 2026
G
S,
Free Puth Extracter ny
Shupe Reconfigg To (S (..)
Loy E. 19 M (ey.
6,0%
of lzy 2.0 of (8, 2,00 Qurfflopens
Quatfooenee
Qurfllapes
Pas Qod, aa, Piatem Ty
R. (-20-1 Q44,)2
Navigator (Meta Hamilitomlnnnn Learner) Mil E) Migrose)"
EmSronment Rg).
Trajectory PP
Outpa: (2) (A(E), ME:; (au(E, Gls Retess
437
Logond: O
Qalic Qua, AC. Uol,- ). gel, (miin, de6d
Figure Independent score function architecture and query-response interface. The Navigator O issues queries containing local Hamiltonians Hg, initial states k4,u, and time horizons T. to each policy Ty E fo)). Each policy computes score functions S learned Hamiltonians hit backed by spatial indices Sş for efficient neighbor queries. Policies return standardized responses Rs containing state trajectories, aggregates these to update the surrogate Hamiltonian and generate meta-corrections ny (E))), and u for adaptive navigation.
Ss via energy gradients from
sequences, and Qols. The Navigator
The navigator represents the whole stack by surrogate Hamiltonian on T T* where 17(E) (E) R. (E..) is the meta policy(explorer) output for potential functional. For navigation, in particular the reference potential for each submodule is:
Rg Esensor (ggi E, wyy +
Σ (EC,(E.))
(E ((d
(5)
R,(gg;E,E) = B(EE) Egout(9 E, Eg +
b(dc))) E):
(6)
sEC,(E.g)
Ro(goiE,E) = X(E) Eobj(1o:
Σ
ai(E, ((:
VEC,(E.)
Remark. A black-box setup assume cannot observe the potential components {R&} yet can only observe kinectic terms by the integrated dynamics via policy Te- A gray-box setup can allow navigator reshape potential (and possibly kinectics) energy (equation
do not need to caliberate surrogate 3) with local Hamiltonian aggregation H EvEK H
Hamiltonian Dynamics of modular sub-systems. Each submodule E X. integrates its local dynamics for short horizon and returns standardized response Rg- Let the effective module Hamiltonian be he (gk, Pk: t) with initial condition h (qu. Pk. 0) H& (gu. Px; E). The local (port- Hamiltonian flow with dissipation and remaining nonconservative input is
ÿk V On (gk, Pk: t),
(8)
Ωk = V h (qk. Pe. t) 1 (gx::) V hh (gx, + Ge(gui (gx: E) uf Pk. t, E), (k.
(9)
with I' 2 Rayleigh/viscous damping and GLuf the nonconservative (non-potential) external input. We define the score of dynamics recorded by module & as the deterministic drift,
V
h
S4 (2L, t) ==
Pa
= (gk, Pk).
B
- h G a
non- non-Hamiltonian contribttonn (friction/ports)
Navigator meta-learning details. navigator as a meta-Hamin Hoeta-Hmiltonian learner including the construction of R(q: (E)). the training objective in equation 48 and the Qol-based online adaptation scheme to Appendix C.4
For clarity of exposition, defer the full formulation of the

---

Published as a conference paper at ICLR 2026
3.4 MULTI-SCALE TEMPORAL COORDINATION
The policies operate natural temporal hierarchies, creating stable multi- scale coordination:
Stage
Stage
Sensor
Czeu
C1
Path
Shape
Time
Jahape
Seath puth
Sarsar
sens 2
Tim
Figure 2: Temporal hierarchy. Sensor policy operates at low frequency (once per stage), estab- lishing environmental constraints C.. Path policy operates medium frequency within each stage, computing waypoints W. Shape policy operates at high frequency, continuously adapting at each in- tegration step. This creates natural hierarchy where slow sensor updates provide stable constraints for faster path and shape adaptations.
This temporal separation enables nested quasi-static approximation: the fastest dynamics (re- configuration) equilibrate within each frame update, and frame dynamics settle before the slower sensor policy evolves. This hierarchy prevents destabilizing interactions across timescales while preserving the necessary coupling for coherent, coordinated behavior.
3.5 OFFLINE PHYSICS LEARNING VS ONLINE ADAPTIVE CORRECTION
Our approach resolves the fundamental tension between learning complex dynamics and real-time adaptation through principled decomposition:
Standard RL
Our GRL-SNAM
Offiine: Leam Policy a(als) from dataset
Offiine: Learn Hamiltonian A ("(,, E) from trajectorics
Online: policy new environment
Online:
Contextual alignment A coneat to sensed C2
Challenge: Policy transfer domains
Advantago: Physies ensures stable adaptation
Figure Comparison between standard RL offline/online adaptation and our physics- grouiddd approach. Standard methods learn arbitrary policies and struggle with transfer, while our approach learns physically meaningful Hamiltonians that naturally adapt environmental variations.
3.6 THEORETICAL PROPERTIES
Our framework provides three key theoretical guarantees:
Theorem 3.2 (Multi-Policy Stability). Under temporal scale separation Trems >> Tpath >> Tim and bounded parameter updates, the coupled system maintains stability with ervor bound Exonal S
Theorem 3.3 (Symplectic Preservation). Each score function generates symplectic dynamics pre- serving the canonical structure wk (zk,+11) (=k.))).

---

Published as a conference paper at ICLR 2026
Theorem 3.4 (Linear Sample Complexity). Independent training achieves total sample complexity
Nuvtal = Eiely. O(e (2),, linear in the sum of policy dimensions rather than exponential in joint dimensionality.
We defer the proof of theorems in appendix. The system thinks in physics during offline training but adapts through energy corrections during online execution, combining principled dynamics stability with real-world deployment flexibility.
EXPERIMENTS
We evaluate GRL-SNAM across multiple dimensions that highlight the unique capabilities of our geometric approach compared to standard reinforcement learning and classical navigation methods. Our evaluation encompasses task performance, safety guarantees, and learning efficiency under min- imal sensing constraints. For more detailed results and analysis, refer to Appendix H
Experimental Setup. We evaluate GRL- in procedurally generated 2D deformable navi- gation tasks, where hyperelastic ring must traverse cluttered environments with narrow gaps and varying obstacle densities. The robot perceives only local window of size 2d x 2d, from which construct a Hamiltonian energy functional with goal-directed potential Fg. barrier potentials Fbs. and adaptive coefficients (B, Y, a:) modulated by context encoders.
Baselines. We compare GRL-SNAM against three classes of baselines under matched information constraints. In the global planning category. we consider rigid A with obstacle inflation and de- formable A with clcarance-aware penalty. In the local reactive category, we include Potential Fields (PF), Control Barrier Functions (CBF), and staged Dynamic Window Approach (DWA), all using the same local sensing window and stage management GRL-SNAM. We also include dearning-based deep RL baselines, PPO, TRPO, and SAC, trained with the same observation space method. This setup enables fair comparison against both task-
and short-rollout setting specific navigation pipelines and standard RL methods.
Metrics. Success Rate, Success-weighted Path Length (SPL), Detour Ratio, Minimum Clearance, Path Smoothness, Collisions, and Mapping Ratio (fraction of enviromment obseerrred
Codebase. Our implementation is available at: https///ghhbbbbcomCCCCcCaaaadddh
4.1
MAIN RESULTS
Q1. How efficiently does GRL-SNAM trade mapping for navigation quality? Table demon- strates that GRL-SNAM achieves CBF-level navigation quality (SPL = 0.95, Detour 1.09) while using essentially the same minimal map coverage PF (10.7% vs. CBF's 11.2%). This validates that our stagewise Hamiltonian refinement extracts maximum value per sensed environment unit.
For each deep RL algorithm (PPO, TRPO, SAC) we train three control parameterizations under the same short-rollout distribution and local observation space as GRL-SNAM: (i) a kinematic con- troller (policy outputs velocities), (ii) dynamic controller (policy outputs forces integrated by damped point-mass model), and (iii) a coefficient controller (policy outputs the Hamiltonian force- field coefficients (a, (, used by GRL-SNAM). The aggregated PPOSTRPO/SAC rows in Table 5 summarize the best-performing configuration for each family. The best TRPO/SAC results reach at most SPL 0.57 with almost grazing clearances (MinClear R 0) and require larger mapping ratios (<< 14-15%), while PPO collapses to SPL = 0.07 with negative effective clearance. In contrast, GRL-SNAM attains high SPL and positive clearance under strictly smaller sensing budget.
Q2. Does GRL-SNAM outperform classical, reactive, and RL planners in complex environ- ments? Yes. Figure 9 shows GRL-SNAM achieves near-perfect success rates ( 100%) across both in-distribution and outtof-distribttiin test cases, while all baselines degrade significantly. GRL- SNAM consistently maintains high SPL (r= 1.0) with low variance and produces the smoothest tra- jectories with lowest turning angles. The Pareto frontier analysis confirms GRL-SNAM uniquely dominates the perforneerfor trade-off.
8

---

Published as a conference paper at ICLR 2026
Table 1: Navigation quality comparison (success-only runs). GRL-SNAM achieves near-CBF ef- ficiency with minimal mapping budget, while deep RL baselines trained on the same short-rollout distribution and local observations yield lower SPL,, larger detours, and smaller clearances.
Method
SPL 1
Detour
Min. Clearance (m) f
Mapping Ratio (%)
PF
0.77
1.42
0.18
10.3
CBF
0.96
1.04
0.32
11.2
GRL-SNAM
0.95
1.09
0.26
10.7
PPO
0.07
1.65
-0.09
14.7
TRPO
0.57
1.44
0.004
14.3
SAC
0.57
1.53
0.004
14.6
(20015
15
1.0
15
15
1.5
-2
Figure 4: Hamiltonian force field composition. Left: goal force Ey: Middle: barrier forces Posi
Right: adaptive combination yielding safe, goal-directed trajectories.
Q3. How does the Hamiltonian formulation enable coherent navigation? Figure 4 illustrates how GRL-SNAM unifies goal attraction Fg and barrier repulsion Fbs into a coherent navigation field through adaptive coefficients. Unlike reactive methods that treat forces independently, our dif- ferential composition F = Fg + Fbs creates contextually balanced dynamics that simultaneously pursue goals and avoid obstacles, which aligns with the high SPL and positive clearances observed 5
Q4. What distinguishes GRI-SNAM's online adaptation from standard RL approaches? Unlike standard deep RL policies that learn fixed mapping from observations to actions (or co- efficients), GRL-SNAM modifies the entire local energy landscape as new obstacles are sensed. Figure 10 demonstrates that the coefficients (3, a) evolve dynamically to redefine the reduced Hamiltonian itself, ensuring energy-consistent posterior updates rather than heuristic reactive ad- justments. This online reshaping of the Hamiltonian explains why GRL-SNAM maintains high success, SPL, and clearance under the same sensing budget where PPO/TRPO/SAC (in all three control variants) either collide, stall, or take inefficient detours.
Deep RL baselines. To contextualize the algorithmic contribution of GRL-SNAM. addition- ally evaluate strong deep RL baselines (PPO, TRPO, and SAC) implemented under the same sensing pipeline, observation structure, action space, and Transformer encoder our method. For faimesss, all baselines operate on the identical short-rollout stagewise dataset derived from the dungeon envi- ronment of Liang et al. 2202)), with horizon H [2, 6], stage-exit goals, and locally reconstructed obstacles. Each agent outputs continuous 2D velocity actions and is trained with identical shaped rewards (goal progress, smoothness, and terminal imitation).
Table summarizes performance under these matched conditions: despite millions of interaction steps, PPO/TRPO/SAC achieve only 18-26% success, whereas GRL-SNAM attains 87.5% with an order-of-magniudde fewer updates. This highlights that, even under identical data, sensing, and architectures, GRL-SNAM's Hamiltonian force-learnigg offers substantially more reliable and sample-efficient mechanism for local navigation.

---

Published as a conference paper at ICLR 2026
Table 2: Short-rollout navigation performance under identical sensing, rollouts, and architecture.
Method
Success (%)
Mcan State Error (m) +
Mcan Goal Dist. (m) 4
PPO
26.1
1.8
1.2
TRPO
21.7
2.1
1.5
SAC
18.4
2.4
GRL-SNAM
87.5
0.3
0.1
4.2 ABLATION AND ROBUSTNESS STUDY
We conducted comprehensive ablation studies loss components (Litiction Lmmtt) confirming that friction matching is critical for stability while multi-start robustness prevents over-conservatism. Robustness evaluations under sensor noise and dynamics perturbations show significant degradation (87% success under severe noise vs 99% nominal) due to adaptive Hamiltonian framework. Sample analysis demonstrates faster convergence than RL baselines due to physics-informed informed structure. Refer to Appendix H for more details.
4.3 NAVIGATION IN COMPLEX MAP LAYOUTS
To test our method more complex map layouts, we also evaluate GRL-SNAM on a point-agent navigation task in dungeon-style environments from Liang al. (2023). The agent is contin- uous 2D point with action at = [Vz. Uy] E [-3,3] and receives only local observations. At each stage, Stagewisse seeenso provides the current macro-cell exit the local goal, while an ObstacleEx: GRL-SNAM uses a quadratic goal energy and radial obstacle barriers, and is compared against PPO, TRPO, and SAC under the same stagewise observation and action interface.
extracts visible obstacle primitives inside the sensing window W(g)).
Table 3: Short-rollout baselines vs. GRL-SNAM on the point-agent navigation task.
Algorithm
Success (%)
Mean state error (m) |
Mean goal dist. (m) |
Env. training steps
PPO
1.8
3.2M env. steps
TRPO
21.7
2.1
1.5
3. 8M env. steps
SAC
18.4
2.4
1.9
4.1M env. steps 500k gradient steps
GRL-SNAM
87.5
0.3
0.1
The RL baselines remain less reliable and less accurate than GRL-SNAM. Under the same local sensing and action interface, the Hamiltonian-stuucln surrogate achieves markedly higher success with lower state and goal error. This suggests that the gain comes primarily from the learning formulation rather than from the observation interface action parameterization. Additional details are provided in Appendix
5 CONCLUSION
We introduced GRL-SNAM, a reinforcement learning framework that leverages Hamiltonian struc- ture couple sensing, planning, and deformation into unified energy-based policy. Our formu- lation enables stable, feedforward navigation updates and achieves near-optimal path quality with minimal mapping effort challenging deformable-robo tasks. The results highlight that incorpo- rating geometric priors into RL can yield both efficiency and robustness, even under noisy sensing and -uitt--diittibutiin layouts. Future work will extend the approach richer sensing modalities and more complex environments, with the goal of validating its scalability to real robotic systems.
ACKNOWLEDGMENTS
This research was supported in part from the Peter O'Donnell Foundation, the Jim Holland- Backcountry Foundation and in part from grant from the Army Research Office accomplished under Cooperative Agreement Number W911NF-19-2-0333
10

---

Published as a conference paper at ICLR 2026
REFERENCES
J. I. Alora, Moses C. Beard, Thomas Libby, Philipp Rothemund, et al. Discovering dominant dy- namics for nonlinear continuum robot control. upj Robotics, 3(1):, 2025.
Karl Johan Aström and Björ Wittenmark. Adaptive Control. Courier Corporation, 2010.
Chandrajit Bajaj and Minh Nguyen. Physics-informed neural networks via stochastic hamilto- nian dynamics learning. In Kohei Arai (ed.), Intelligent Systems and Applications, pp- 182-197. Springer Nature Switzerland, 2024.
Atilim Günes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation machine learning: survey. Journal of Machine Learning Research, 18(153):1-43, 2018.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Joseph Dabis, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Tomas Jackson, Sally Jesmonth, Nikhil J Joshi, Ryan Julian, Dmitry Kalash- nikov, Yuheng Kuang, Isabel Leal, Kuang-Huei Lee, Sergey Levine, Yao Lu, Utsav Malla, Deek- sha Manjunath, Igor Mordatch, Ofir Nachum, Carolina Parada, Jodilyn Peralta, Emily Perez, Karl Pertsch, Jornell Quiambao, Kanishka Rao, Michael Ryoo, Grecia Salazar, Pannag Sanketi, Kevin Sayed, Jaspiar Singh, Sumedh Sontakke, Austin Stone, Clayton Tan, Huong Tran, Vincent Vanhoucke, Steve Vega, Quan Vuong, Fei Xia, Ted Xiao, Peng Xu, Sichun Xu, Tianhe Yu, and Brianna Zitkovich. Rt-1: Robotics transformer for real-world control at scale. arXiv preprint D1Xiv:2222.06817, 2023.
Brandon Caasenbrood, Alexander Pogromsky, and Henk Nijmeijer. Control-oriented models for hyperelastic soft robots through differential geometry of curves. Soft Robotics, 9(2):346-36, 2022.
Devendra Singh Chaplot, Ruslan Salakhutdinov, Abhinav Gupta, and Saurabh Gupta. Neural topo- logical slam for visual navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12875 12884, 2020.
Tian Qi Chen, Yulia Rubanova, Jesse Bettencourt, and David Duvenaud. Neural ordinary differential equations. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Mohammad Dehghnn Tezerjani et al. A survey reinforcement learning applications in slam. arXiv preprint arXiv:2408.4455 2024.
Shaan A Desai, Marios Mattheakis, David A Roberts, and Pavlos Protopapas. Port-hamiltonian neural networks for learning explicit time-dependent dynamical systems. Physical Review E, 104 (3):034312, 2021.
Aditya S Ellendula and Chandrajit Bajaj. Self- belanninnnn memory efficient, dynamic metric space data maintenance, for rapid multi-kernel estimation. arXiv preprint arXiv:2504.1800 2025.
Zhan Feng et al. Safer gap: A bap-b local planner for safe navigation with nonholonomic mobile robots. arXiv preprint arXiv:2303.0822 2023.
Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error in actor-
critic methods. In International conference machine learning, pp. 1587-1596, 2018.
Saurabh Gupta, Varun Tolani, James Davidson, Sergey Levine, Rahul Sukthankar, and Jitendra Malik. Cognitive mapping and planning for visual navigation. arXiv preprint arXiv:1702.03920. 2019.
Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Jennifer Dy and An- dreas Krause (eds.). Proceedings the 35th International Conference Machine Learning, vol- ume 80 of Proceedings of Machine Learning Research, pp. 1861-1870, Vienna, Austria, 10-15 Jul 2018. PMLR.
11

---

Published as a conference paper at ICLR 2026
Tai Hoang, Huy Le, Philipp Becker, Vien Anh Ngo, and Gerhard Neumann. Geometry-aware rl for manipulation of varying shapes and deformable objects. arXiv preprint arXiv:2502.07005, 2025.
Junwoo Jang and Maani Ghaffari. Social zone as a barrier function for socially-compliant robot navigation. arXiv preprint arXiv:2405.15101, 2024.
Yanwei Jia and Xun Yu Zhou. q-learning in continuous Journal of Machine Learning Research, 24(161):1-61, 2023.
Eshagh Kargar and Ville Kyrki. Macrpo: Multi-agent cooperative recurrent policy optimization. arXiv preprini arXiv:2109.00883 2021.
Nikhil Keetha, Jay Karhade, Krishna Murthy Jatavallabhula, Gengshan Yang, Sebastian Scherer, et al. Splatam: Splat, track & map 3d gaussians for dense rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Patrick Kidger, James Morrill, James Foster, and Terry Lyons. Efficient and accurate gradients for neural sdes. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Junho Kim, Eun Sun Lee, Mingi Lee, Donsu Zhang, and Young Min Kim. Sgolam: Simultancous goal localization and mapping for multi-object goal navigation. arXiv preprint arXiv:2 10.07177,, 2021.
Donald E Kirk. Optimal Control Theory: An Introduction. Dover Publications, 2004.
Holger Klein, Noémie Jaquier, Andre Meixner, and Tamim Asfour. On the design of region- avoiding metrics for collision-safe motion generation on riemannian manifolds. arXiv preprint 104v:23377.1440, 2023.
Hengyuan Lai et al. Roboballet: Planning for multirobot reaching with graph neural networks and reinforcement learning. Science Robotics, 2025.
Kyowoon Lee, Scongun Kim, and Jaesik Choi. Adaptive and explainable deployment of navigation skills via hierarchical deep reinforcement learning. In International Conference on Robotics and Automation, 2023.
Chengshu Li et al. Hrl4in: Hierarchical reinforcement learning for interactive navigation with mo- bile manipulators. In Conference Robot Learning, 2020.
Junjie Li et al. Learn with imagination: Safe set guided state-wise constrained policy optimization. arXiv preprint arXiv:2308. 13140, 2023.
Jingsong Liang, Zhichen Wang, Yuhong Cao, Jimmy Chiun, Mengqi Zhang, and Guillaume Adrien Sartoretti. Context-aware deep reinforcement learning for autonomous robotic navigation in un- known area. In Conference on Robot Learning, pp. 1425-1425 1436. PMLR, 2023.
Elisabetta Liu and Cosimo Della Santina. Physics-informed neural networks to model and control robots: theoretical and experimental investigation. Advanced Intelligent Systems, 2024.
David Martínez-Rube and Sebastian Pokutta. Accelerated riemannian optimization: Handling con- straints with prox to bound geometric penalties. In The Thirty Sixth Annual Conference Learning Theory, pp. 359-393. PMIRR, 2023.
Nicholas Mohammad and Nicola Bezzo. Soft actor-critie-based control barrier adaptation for robust autonomous navigation in unknown environments. arXiv preprint arXiv:2503.08477 2025.
Barry W Mulvey and Thrishantha Nanayakkara. Haven: haptic and visual environment navigation by shape-changing mobile robot with multimodal perception. Scientific Reports, 14(1):27018, 2024.
Minh Nguyen and Chandrajit Bajaj. A differential and pointwise control approach to reinforcement learning. arXiv preprint arXiv:2404. /5617, 2025.
Tariq Patanam, Eli Shayer, and Younes Bensouda Mourri. Deep dagger imitation learning for indoor scene navigation.
12

---

Published as a conference paper at ICLR 2026
L.S. Pontryagin, V.G. Boltyanskii, R.V. Gamkrelidze, and E.F. Mishchenko. The Mathematical Theory of Optimal Processes. Interscience, 1962.
Doina Precup. Temporal abstraction in reinforcement learning. University of Massachusetts
Amherst, 2000.
Qi, C. Zhou, H. Qing, H. Sun, and J. Yin. Aerial track-guided autonomous soft ring robot. Advanced Science, 2024.
Mohammad Roshanfar, Javad Dargahi, and Amir Hooshiar. Hyperclastic modeling and validation of hybrid-actuated soft robot with pressure-stiffening. Micromachines, 14(5):1001, 2023.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford. and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint MxXiv:1777.06447 2017.
Haruki Settai, Naoya Takeishi, and Takehisa Yairi. A temporal difference method for stochastic continuous dynamics, arXiv preprint arXiv:2505. /5544, 2025.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- fional Conference on Learning Representations (ICLR), 2021.
Edgar Sucar, Shikun Liu, Joseph Ortiz, and Andrew Davison. iMAP: Implicit mapping and po- sitioning in real-time. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 6229-6238, 2021.
Richard Sutton, Doina Precup, and Satinder Singh. Between mdps and semi-mdps: Learning, planning, and representing knowledge at multiple temporal scales. Technical report, Technical Report 98-74, University of Massachusetts, Amherst, 1998.
Hamid Taheri et al. Deep reinforcement learning with enhanced ppo for safe mobile robot naviga- tion. arXiv preprint arXiv:2405. /6266, 2024.
Lei Tai, Jingwei Zhang, Ming Liu, and Wolfram Burgard. Socially compliant navigation through raw depth inputs with generative adversarial imitation learning. arXiv preprint arXiv:1710.02544 2018.
Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, and Koray Kavukcuoglu. Feudal networks for hierarchical reinforcement learning. In International conference on machine learning, pp. 3540-3549, PMLR, 2017.
Niclas Vödisch, Daniele Cattaneo, Wolfram Burgard, and Abhinav Valada. Continual SLAM: Be- yond Lifelong Simultaneous Localization and Mapping Through Continual Learning, pp. 19-35. Springer Nature Switzerland, 2023.
Tianhao Wang et al. Pinn-ray: A physics informed ond neural network to model soft robotic fin ray fingers. arXiv preprint urXiv:2407.08222. 2024.
Weizheng Wang et al. Multi-agent Ilm actor-critic framework for social robot navigation. arXiv preprint wrXiv:2503.09755 2025.
Chi Yan, Delin Qu, Dan Wang, Dan Xu, Zhigang Wang, et al. Gs-slam: Dense visual slam with 3d gaussian splatting. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2024.
Songyuan Zhang, Zhangjie Cao, Dorsa Sadigh, and Yanan Sui. Confidence-aware imitation learning from demonstrations with varying optimality. arXiv preprint arXiv:2110.14754, 2022.
Chao Zheng et al. Semantic slam system for mobile robots based on large visual model in complex environments. Scientific Reports, 15(1): 1-15,, 2025.
Jun Zhu, Zihao Du Haotian Xu. Fengbo Lan, Zilong Zheng, Bo Ma, Shengjie Wang, and Tao Zhang. Navi2gaze: Leveraging foundation models for navigation and target gazing. arXiv preprint arXiv:2407.09053. 2024.
13

---

Published as a conference paper at ICLR 2026
Zihan Zhu, Songyou Peng, Viktor Lachner, Weiyang Xu, Michael Niemeyer, et al. NIAE-SLAM: Neural implicit scalable encoding for slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11786-12796 2022.
14

---

Published as a conference paper at ICLR 2026
A
EXTENDED INTRODUCTION AND MOTIVATION
This section provides expanded context for the challenges addressed by GRL-SNAM and detailed justification for our geometric approach.
A.1 COMPREHENSIVE ANALYSIS OF RL LIMITATIONS IN NAVIGATION
Contemporary reinforcement learning methods face several critical limitations that become particu- larly pronounced in continuous navigation tasks:
Sample Efficiency Bottlenecks. Standard RL algorithms like SAC (Haarnoja et al. (2018) and PPO (Schulman al. (2017) require millions of environment interactions to learn effective navigation policies. This inefficiency stems from the curse of dimensionality in continuous control settings where the action space infiniteddimensional and policies must simultaneously master fine-graines motor control and high-level strategic reasoning. In real-world deployment scenarios where data collection is expensive, this sample complexity becomes prohibitive.
The problem is exacerbated by the need for exploration in high-dimensional spaces. Unlike discrete control problems where systematic exploration strategies like e-greedy or UCB can provide theo- retical guarantees, continuous control requires sophisticated exploration mechanisms that often rely on injected noise or entropy bonuses. These mechanisms frequently lead to unsafe or inefficient exploration behaviors that unsuitable for real-world navigation tasks.
Generalization Failures. Policies trained in specific environments exhibit catastrophic performance degradation when deployed in novel settings, even when new enviromments share similar structure. This brittleness stems from the lack of inductive bias in standard neural network architectures. With- explicit encoding of physical principles geometric structure, learned policies tend memorize environment--peccficc features rather than discovering generalizable navigation principles.
The generalization problem is particularly acute in navigation because environmental variations can affect multiple aspects of the task simultaneously: obstacle configurations change collision con- straints, surface properties affect dynamics, and lighting conditions influence perception. Standard RL approaches learn monolithic mappings that cannot decompose these variations into their con- stituent factors, leading to behaviors that fail when any component deviates from training conditions.
Temporal Decomposition Challenges. Navigation inherently requires coordination across multiple timescales: immediate obstacle avoidance operates on millisecond timescales, local path planning unfolds over seconds, and strategic goal-directed behavior spans minutes or hours. Standard RL al- gorithms struggle to learn policies that reason effectively across these scales, often getting trapped in locally optimal behaviors that satisfy short-term objectives while failing to make long-term progress.
Existing approaches to multi-scale reasoning such as hierarchical RL (Sutton et al. (1998)) options frameworks (Preeup (2000)), feudal networks (Vezhnevets al. (2017), typically require manual decomposition of the task space and careful engineering reward functions for different levels. These methods introduce additional complexity without fundamentally addressing the structural issues that make multi-scale learning difficult.
A.2 THE SNAM CHALLENGE: WHY STRUCTURE MATTERS
Simultaneous Navigation and Mapping (SNAM) represents particularly challenging instance of the navigation problem where agents must build environmental representations online while traversing unknown spaces. This challenge amplifies limitations of RL approaches in several ways:
Memory and Representation Learning. SNAM requires policies to maintain and update spatial representations based on sensory observations. This places enormous demands on the policy's mem- ory architecture, requiring it to simultaneously master memory management, spatial reasoning, and motor control. Standard recurrent architectures like LSTMs GRUs struggle with this multifaceted learning problem, often failing to maintain coherent spatial representations over long episodes.
Exploration-Explee Exploitation Tradeoffs. In SNAM, exploration serves dual purposes: gathering in- formation about the environment for mapping and discovering navigation strategies. This creates complex exploration-exploitaaiio tradeoffs that standard RI. exploration mechanisms cannot handle
15

---

Published as a conference paper at ICLR 2026
effectively. Random exploration may discover regions but fails to systematically map environ- mental structure, while directed exploration based on current maps may miss critical features.
Dynamic Environmental Coupling. Unlike traditional navigation where environments SNAM requires reasoning about how the agent's actions affect both its position and its knowledge of the environment. This creates coupled learning problem where navigation decisions influence future mapping accuracy, and mapping quality affects navigation performance. Standard RL frame- works treat these separate problems, missing the critical coupling that enables efficient SNAM.
static,
Recent approaches in simultaneous navigation and mapping (SNAM) have coupled local mapping with policy learning to improve navigation performance. For example, SGoLAMM (Kim et al. [202] interleaves goal localization with occupancy mapping to enable point-goal navigation, while Cog- nitive Mapping and Planning (CMP) (Gupta et (2019)) integrates differentiable planner into learned mapping framework. Continual SLAM (CL-SLAM) (Vödisch et al. (202)) further empha- sizes long-term adaptability by maintaining and updating maps during navigation. However, these methods rely progressively constructing detailed maps of the environment before exploiting them for navigation. contrast, our objective to reach the goal along high-quality, well-weighted paths while mapping as little of the unknown environment possible. To the best of our knowledge, no prior work explicitly formulates navigation with minimal exploration as the central goal. Our pro- posed GRL-SNAM framework achieves this by progressively refining paths: from observed envi- ronmental variations, the policy differentially learns to identify the least-cost trajectory, such that the path improves continuously as new local information is revealed.
A.3 GEOMETRIC STRUCTURE: THE INEVITABLE SOLUTION
The limitations outlined above are not merely implementation details but fundamental consequences of treating navigation as unstructured optimization. Several lines of evidence suggest that geometric structure is not just helpful but inevitable for solving complex navigation problems:
Physical Realizability. Real robotic systems operate under physical constraints imposed by servation laws, kinematic limitations, and actuator dynamics. Policies that violate these constraints cannot be implemented on physical systems, yet standard RL approaches have mechanism to enforce such constraints during learning. Geometric formulations naturally incorporate physical constraints through the mathematical structure of the problem.
Stability Requirements. Long-horizon navigation requires numerical stability over extended roll- outs. Standard neural network policies accumulate errors over time, leading to unstable behaviors in long episodes. Hamiltonian formulations with symplectic structure preserve important invariants (energy, momentum) that ensure stability over arbitrarily long rollouts.
Compositionaliiy Needs. Complex navigation tasks require composing simpler behaviors: obsta- cle avoidance, path following, goal seeking, and environmental adaptation. Standard RL approaches learn monolithic policies that cannot decompose into interpretable components. Geometric formu- lations enable natural decomposition through energy terms that can be composed, weighted, and adapted independently.
A.4 DIFFERENTIAL POLICY OPTIMIZATION: BEYOND FIXED POLICIES
Traditional RL optimizes fixed policy parameters to maximize expected returns over discrete timesteps. Differential Policy Optimization (Nguyen & Bajaj (2025) fundamentally reconceptu- alizes this by learning dynamics operators through a continuous time differential dual formulation.
Mathematical Foundation. Rather than directly learning policies, Differential Policy Optimization reformulates RL through continuous tittiu optimal control. By approximating discrete reward sums with time integrals:
H Σ
~ max E [/" r(s,, dtt
max IE
r(sk, ax)
(10)
Applying Pontryagin's Maximum Principle introduces adjoint variables p and defines the Hamilto- nian function:
(p == (s, r(s, a)
(11)
16

---

Published as a conference paper at ICLR 2026
The key insight is that optimal actions can be implicitly represented through the stationarity condi- tion da = 0, yielding the reduced Hamiltonian:
f(s, p) =
(12)
Score Function Learning. Differential Policy Optimization learss a score function g(z) where (s. p) combines state and adjoint variables. The dynamics operator is constructed as:
(()))
G(x) ASVg(x)
(13)
where -1 0
is the canonical symplectic matrix and the discretization step.
Stagewise Learning Advantages. Unlike methods requiring backward-in-time adjoint calcula- tions (as in Pontryagin's 's Maximum Principle), Differential Policy Optimization enables feedforward learning where each stage defines local Hamiltonian H, integrated forward in time:
0++1 0g VeHe
(14)
This avoids the computational complexity and numerical instability of adjoint methods while main- taining theoretical guarantees through the geometric structure of the Hamiltonian formulation.
Goal Attraction
Sensor Cost
Deformation
Collision Barriers
gool
1obox
ScnOoe region
[c xyl
Euby (g)
Σ b(d, à)
Figure 5: Policy-aligned energy decomposition. Each policy governs distinct energy component: the Sensor Policy minimizes sensor cost, the FPE balances goal attraction and collision barriers, and the Reconfig Policy adapts size through deformation energy. Together these terms define the Hamiltonian reward R.
A.5 MULTI-POLICY ARCHITECTURE DETAILS
Our multi-policy decomposition addresses temporal scale separation through three specialized com- ponents operating at different timescales:
Sensor Policy (Ty): Operates slow timescales to adapt perception strategies based stagewise environmental feedback. This policy learns to focus attention on relevant environmental features, adjust sensor parameters for optimal information gain, and filter sensory noise. The sensor policy outputs constraints C, that inform slower planning processes.
Frame Policy (7 (πy): Operates medium timescales to plan collision-free trajectories in local co- ordinate frames. This policy takes constraints from the sensor policy and generates waypoints W, for shape control. The frame policy handles local obstacle avoidance and path optimization within a limited spatial horizon.
Shape Policy (R.): Operates at fast timescales to control robot morphological adaptation. For deformable robots, this includes shape changes, stiffness modulation, and configuration updates. For conventional robots, this might include gait transitions, tool selection, or behavioral mode switches.
The key insight is that these policies not manually designed hierarchies but emerge naturally from the temporal structure of the Hamiltonian dynamics. Fast variables (sensor adaptation) reach quasi-equilbrium before slower variables (shape changes) evolve significantly, creating natural scale separation without manual decomposition.
This extended analysis demonstrates that geometric structure not merely useful inductive bias but a necessary foundation for solving complex navigation problems that require multi-scale reasoning, online adaptation, and long-horizon stability.
17

---

Published as a conference paper at ICLR 2026
B EXTENDED RELATED WORK SURVEY
B.1
GEOMETRY AND MECHANICS PRIMER
Navigation learning methods can be categorized by their underlying mathematical spaces, with sig- nificant implications for performance and theoretical guarantees:
Euclidean Space Methods (RR"): Standard RL treats navigation as optimization in flat spaces using Euclidean distance metrics. Enhanced PPO TTaeri et al. 22244) demonstrate improved collision avoidance but ignore inherent geometric structure of robotic systems. Sample efficiency remains poor, typically requiring millions of environment interactions (Dehghami Tezerjani et al. (2024)).
Recognition of orientation constraints has led SE(2) and SE(3) formula- tions using equivariant neural architectures. These preserve rotational and translational symmetries but remain primarily limited to manipulation rather than navigation tasks.
Lie Group Methods:
Riemannian Manifold Approaches: Advanced geometric formulations employ differential ge- ometry for constraint handling through tangent space projections. MMartinez-Rubio & Pokutta (023) demonstrates constraint satisfaction through geometric structure rather than penalty meth- ods, achieving superior theoretical properties but limited practical deployment.
Hamiltonian and Symplectic Methods: Port-Hamiltoiian neural networks show significant per- formance improvements through symplectic integrators, proving that respecting geometric structure fundamentally improves learning dynamies. However, applications remain confined to simple trol problems.
B.2 SAFETY-CRITICA NAVIGATION TAXONOMY
External Safety Projection: Control Barrier Functions create safe action spaces through constraint projection. Neural Network Zeroing Barrier Functions (Feng et al. (2023) )) enable collision free
navigation, while adaptive safety constraints (Mohammad & Bezzo (2025) handle dynamic envi-
ronments. Social navigation approaches (Jang & Ghaffari (2024)) extend CBFs to human- robot interaction. These methods achieve formal safety guarantees but often exhibit conservative behav- iors due to the separation between safety and optimality.
Safety: Our approach incorporates safety directly within the Hamiltonian en- ergy structure via barrier potentials. This enables aggressive navigation while maintaining formal guarantees through symplectic structure preservation, avoiding the conservatism of external projec- tion methods.
B.3 DEFORMABLE AND SOFT ROBOT NAVIGATION
Hyperelastic Material Models: Recent advances include pressure-stiffen -tr.uuei--tiffeingg control with 6.40%
maximum error validation [Roshanfar (2023) and passivity-basse control using differential ge-
ometry of curves (Caasenbrood et al. (2022) ). Spectral Submanifold Reduction (Alora et al. (2025)
achieves computational speedup for time hyperelastic control with stability guarantees.
Ring and Circular Robots:
Liquid Crystal Elastomer responses enable acrial gap navigation (Qi
al. (2024) through predetermined actuation patterns. HAVEN (Mulvey & Nanayakkara (2024))
navigates constrained spaces via fixed shape-changing sequences based on multimodal perception. These approaches use offline parameter optimization with deterministic execution, lacking online adaptation capabilities.
Physics-Informed Learning: PINN-Ras -Ray (Wang et al. (2024)) achieves state-of saitee-of-hheaar ahe- hyperelastic displacement prediction, while extensions to non-conservative effects (Liu & Della Santina (2024)) provide experimental validation. However, these remain primarily modeling tools rather than adap- tive control frameworks.
B.4 NEURAL SCENE REPRESENTATIONS FOR NAVIGATION
HeRF-Based SLAM: Real-time dense reconstruction through VICE-SLAM (Zhu et al. (202)) and keyframe-free tracking via iMAP (Sucar al. (202)) provide rich environmental representations.
18

---

Published as a conference paper at ICLR 2026
Neural Topological SLAM (Chaplot et al. (2020)) combines learning with classical planning, while semantic approaches Zheng et al. (2025) integrate large vision models.
3D Gaussian Splatting: GS-SLAM (an et al. 2024)) and SplaTAM (Keetha al. (2024) demon-
strate state-o--theaar art reconstruction quality with real-time performance, offering dense 3D represen- tations suitable for navigation applications.
Integration with Energy Terms: Scene representations feed our energy formulation through:
Barrier Energy:
Ueurrier
Σ
b(SDF(x))
(15)
obstacles
Free-Space Energy:
Ufree
Σ w(x)
(16)
free regions
Goal Energy: Ugual
|x
Xgnal
||
(17)
B.5 MULTI-SCALE AND HIERARCHICAL METHODS
Hierarchical RL:
Task decomposition approaches like HRLAIN (Li et al. (2020) handle heteroge- neous navigation phases, while (Lee et al. (202)) learns specialized policy families with high-level coordination. These require manual decomposition and struggle with principled coordination, often leading to ad-hoc design choices without theoretical guarantees.
Multi-Agent Coordination: RoboBallet (Lai et al. (2025) achieves coordination for robots across 40 tasks using graph neural networks. MACRPO (Kargar & Kyrki (202I)) enhances information sharing beyond parameter sharing. However, these approaches lack the geometric structure preser- vation critical for deformable robot coordination.
B.6 IMITATION LEARNING NAVIGATION
Behavioral Cloning: RT-1 (Brohan et al. (2023) demonstrates impressive generalization across 700+ tasks using 130k demonstration episodes with transformer architectures achieving significant zero-shot performance improvements.
Inverse Reinforcement Learning: GAIL for Safe Navigation (Tai et (0188) combines gen- erative adversarial imitation with safety constraints. DAgger for Continuous Navigation (Patanam al.) iteratively improves policies through expert querying.
Sub-Optimal Demonstrations: Confident Imitation Learning (Zhang et al. (2022) handles demon- stration uncertainty through confidence-aware training, addressing distribution shift novel envi- ronments.
These approaches excel with high-quality demonstrations but assume expert availability and struggle with the full behavioral range needed for adaptive defirration strategies.
B.7 FOUNDATION MODEL INTEGRATION
I Large-scale models for navigation reasoning (Zhu et al. (2024); Wang et al. (2025) focus on high- level semantic understanding and multi-agent coordination at the symbolic level. Foundation models excel at reasoning and semantic understanding, while our GRL-SNAM provides principled low-level geometric control.
Integration Pathway: Foundation models could generate high-level objectives encoded as potential energy terms our energy functional R(q.). The geometric structure preservation ensures high- level semantic goals translate into physically consistent behaviors, addressing the critical gap where foundation model outputs often lack grounding in physical dynamics.
B.8 PARADIGM COMPARISON
This comprehensive survey positions GRL-SNAM uniquely addressing the intersection of geo- metric structure preservation, multi-scale coordination, and deformable robot control- cappailiiis that existing approaches handle separately or incompletely.
19

---

Published as a conference paper at ICLR 2026
Table Extended paradigm-level comparison of learning frameworks. Scoring: v comprehensive support, A= limited support, x= not supported.
Capability
Leamnigg Sampeissuerriied
Hecrurcicc RL
Enery Ouservation
Geomentc Seucture
Cossirint Isizention
Adaptation
Sample Eflieeny
Zen-Shee Canzralicriin
hoo-Wirk Deplosment
A
Defocauble Roboi Suggor.
Scoring Criteria:
Energy Conservation: Explicit conservation laws in dynamics
Geometrie Structure: Preservation of manifold properties
Constraint Integration: Safety/task constraints within optimization
Online Adaptation: Real-time policy modification during deployment
Multi-Scale Coordination: Principled coordination across temporal scales
Sample Efficiency: Learning with minimal environment interaction
Zero-Shot Generalization: Performance unseen environments
Deployment: Practical implementation feasibility
Deformable Robot Support: Explicit modeling of shape change
B.9 KEY INSIGHTS
Our framework builds upon of Hamiltonian and reinforcement learning principles, unifying offline reference dynamics with online adaptive updates. Below, we summarize the six key insights that form the backbone of GRL-SNAM.
1. Hamiltonian energy task reward. We define the Hamiltonian
H(g.p) = K(p) + P(g),
(18)
with kinetic energy K and task-specific potential P. In setup, P encodes navigation objec- tives (goal attraction, barrier avoidance, deformation penalties). Following Pontryagin et al. (1962); Bajaj Nguyen (2024), the Hamiltonian coincides with the surrogate objective policy gradient methods, I.c.
(oJ(
VolEz,(-H(q. p)],
(19)
linking task reward to the Hamiltonian gradient flow. This equivalence grounds the DPO surrogate in physical structure.
2. Offline Hamiltonian Online task reward. In offline training, the agent minimizes trajec- tories under fixed H constructed from synthetic local patches. Online, the environment is sensed, and task rewards Ren are parsed into Hamiltonian subtasks. By interpreting
Hantine Hollime + A
(20)
we align local sensory updates with the reference offline Hamiltonian. This mirrors the adaptive control interpretation in Áström & Wittenmark (2010).
3. Offline policy as reference Hamiltonian. Hamiltonian Hrat where the score function "
Every offline policy Tha is equivalent to a reference Ville defines canonical dynamics:
8Href
0Het
p =
(21)
Op
dq
Online adaptation then minimizes the divergence
D(Tonline
)eer)
x
E [ Voonine
V
(22)
structure exploited in score-based models
(Song al. 2021
20

---

Published as a conference paper at ICLR 2026
4. Advantages of stagewise updates. Rather than solving adjoint equations as in Pontryagin's Maximum Principle, we adopt a stagewise decomposition. Each stage defines a local Hi and is integrated feedforward:
0++ = 0, V
(23)
This avoids backward-in-time adjoint calculations and recovers the efficiency noted in adjoint-free feedforward networks (Chen et al. 2018 Kidger et al. 2021).
5. Universality of the pipeline. Our pipeline
Environment
Encoder
Context
Setup
Hadapted
(24)
universal. As long as H. is differentiable, adaptation reduces to evaluating its gradients, regardless of whether the system is white-box (explicit potentials) or black-box (sensor-leve inputs). This follows from the variational formulation of differentiable programming (Baydin al. 2018).
Navigator as meta-controller. The navigator policy Tmav interacts with three black boxes; the offline Hamiltonian Hief the online sensed reward Rem,, and the adaptive fusion Hadapt Its role is to formulate and solve
(25) where a is dynamically updated by the context encoder (e.g.. LSTM). This positions the navigator meta-controller that continually reforms the Hamiltonian problem, principle consistent with adaptive RL formulations Kirk (2004).
Hadapt a&Hee (1 Rev
B.10 SECANT GAUSS-NEWTON CONTROLLER WITH IMPLICIT OBSERVABLE TRANSITION
In the online setting, policy parameters (0 [8, 7, {a;))) do not act directly target observables Yigt. Instead, there exists an implicit transition chain:
policy induced energy
dynamics rollout
f(()
map
Yugt
(26)
where f() denotes observables (e.g., clearance, progress, admissible speed) produced after sym- plectic updates under the reshaped Hamiltonian Eg.
Implicit Jacobian. The true Jacobian of observables with respect to policy parameters is
of 0f dz
(27)
2 Oz 0e
which is expensive to evaluate through full rollouts. Instead, we maintain rank- secant estimate
J, f(0') - f(e--))
(28)
θ O
smoothed via exponential moving average to reduce noise. This captures the *implicit effect* of parameter changes on observables.
Energy Reshaping Step.
With the secant Jacobian, the Gauss update becomes
AO = G- W (f(0) Brgg), G ?-
(29)
followed by projection and per-head learning rates:
r++1 Projezo (Ø' + diag(nb, My na) A0).
(30)
Here, W is a weighting matrix over observables, and s/ stabilizes inversion.
Meta-Learning View.
The Navigator maintains meta-objective Luav ( Ø) ((f(O; ();2;,), Vige),
(31)
with context-dependent parameters & (e.g.. encoder weights for a). The implicit gradient is
VeCna JT ( Yegt),
(32)
which coincides with the secant Gauss Newton step above. Thus, meta-learning is implemented not by direct regression on θ but by observable alignment through the implicit policy - observables targets chain.
21

---

Published as a conference paper at ICLR 2026
Interpretation. This formulation clarifies that the controller does not operate in parameter space alone. Instead, continuously reshapes Hamiltonian parameters so that the induced observables approach task targets, effectively coupling multiple energy components (safety, progress, speed) without additional rollouts.
Sequential Qneey-RRepoone (as ports). At each the Navigator issues queries Ø and receives responses Ri which determine Migt (R²) and any weights in W; the update equation ?? (with the secant J) is then applied to each block o,o):
h = 0 + 1
20k G JJ Wa((((0)) l/gt (R))).
State Evolution.
With updated parameters, each policy advances its state before,
= 2 + Tk Jj 5k lltt) (et C. Condduted t).
Remark. If strict port-Hamiltonian view is desired, the J W - f()) term enters ii as an external port (input) rather than being baked into the potential; the resulting discrete update is identical the secant Bauss-Newton step above.
C HYPERELASTIC RING ROBOT MODEL
We model the deformable robot as closed hyperelastic ring with reduced-order dynamics to enable efficient navigation while capturing essential deformation behaviors.
ext
(tt)
o(l)
obstacles
Figure 6: Hyperelastic ring robot model showing generalized coordinates (s, o, 0),), spline sample
points Xj. and external force fields.
C.1 GEOMETRIC REPRESENTATION
The robot boundary is defined by a periodic cubic B-spline curve with nent control points:
nhai
(uu) Σ (u) E [0, 1]
(33)
where Nu,(u) are degree-3 B-spline basis functions with C continuity. The base shape is unit circle:
MosS2ti//ea (ea)) (in(2mi/nen))
P = Tbase
(34)
World coordinates are computed via similarity transformation:
P((t) o(t) ((t))
(35)
where s(t) is uniform scale, (t) E R center position, ((t) is orientation.
For physics computation, we sample K points the curve using B-spline evaluation matrix B E
R Xnai:
X, Σ
BiPr f= 1=,,..,, K
(36)
22

---

Published as a conference paper at ICLR 2026
Algorithm Hyperelastic Ring Deformation Policy
1: Input: State (s, š, f, w)), obstacles ((CK, target Xtarget
2: Output: Updated state (s', o', ò', ",0,, ,,')
3: Update geometry: X, sample curve at current state
4: Compute distances: dyk + ||x,
C&
Tke clearance: dmin <- ming dyt
5:
Conservative Forces-
6: IPC barriers: gj - Σx (bre(d,1)
8x;
7: Adaptive bulk: Fs +
Athu with Ararge (ongee(ddng (Amin)
8s
8:
Non-Conservative Forces
9: Stage forces: F sitage. t goal + radial tangential components
Friction: F Friition..j &- p tangent
tangent.velocity velocity
11:
Generalized Force Mapping
12: Map to coordinates:
Fs, Foy <- virtual work from {g) + singe.j + Priction..)
13:
Integration
14: Update velocities: + á + At Fa/Ms, etc.
15: Update positions: s' f- clamp(s 8 At 8'), o' +o At. etc.
16: return updated state
C.2 ENERGY FORMULATION
The total Hamiltonian combines kinetic and potential components:
H M2s² M
Mollo12
1 w + Ubbarier + Ubulk
(37)
2
IPC Barrier Energy: Collision avoidance using Incremental Potential Contact barriers:
Unartier
(dj()
(38)
|| 1/K, where wy
dyy is distance from sample j to obstacle k: (d- (dd-d)(logdd log d) if < d < d d 2 d if d <
bppC(d) (d))
0
(39)
penalty
Adaptive Bulk Energy: Area conservation with clearance-depennent target:
kbalk
Uhaik
(A(s) Ararget )2
(40)
where A(s) = 22 Aye and:
Augeet = [a + (1 a) tanh(B max(dmin- 0))] Aref
(41)
with a = 0.25, 8 = 2.5 encouraging compression in tight spaces.
C.3 GENERALIZED FORCE MAPPING
Forces spline samples map to generalized coordinates via virtual work:
0u
K
0x,
F, = 0s
F Fex (X,)
8
(42)
Øs
j=1
K
0u
F.
Σ
F Foxi(XX (X,) - 700
(43)
do
j=1
0u
K
5
wy ext (X,) (J(X, o)) 7ow
(44)
00
-1]
where J =
generates rotation and ox,
R(0)
ds
23

---

Published as a conference paper at ICLR 2026
C.4 NAVIGATOR AS META--AMILTTNIAA LEARNER
The GRL-SNAM operates as meta-learning system that coordinates multi-scale policies by learn- ing how to update their Hamiltonian energy functions rather than directly manipulating phase space states.
Navigation specialization. Throughout the paper we further simplify our navigator's Hamiltonian surrogate by assuming fixed throughout all environment when we primarily focus on learning to optimize the environment-inon -ndexeedd linear cone generated by task energies. The potential energy is denoted as follows and is shown in Figure 5
Riq ((E))
lall's
B||c X
Sensor Cost (Esenser)
Goal Attraction (Egual)
(Eobr)a(t)
a,b(di,
(45)
iECx(E.g)
Deformation Energy (Eorg)
Collision Barriers (FPE)
The Sensor Policy contributes the sensor cost Esensor. regularizing information acquisition. The FPE (Free Path Extractor) governs goal attraction and path-planning in current environment & via goal attraction force Egoa! and barrier potentials, balancing reachability with safety. The Reconfig Policy governs deformation, enabling radius modulation for narrow passages. This decomposition highlights that the total Hamiltonian is not a monolithic reward but a structured sum of physically interpretable energies, each attached specialized policy. In addition, assume friction and additional forces (e.g. derived from safety constraints) are happened FPE submodule only, and we pick a particular parametrization as:
r = 0, G 0,
y 0, 15. = us (E)L, G uf f0
(46)
= O, G 0, u = 0.
Thus the meta policy Ig directly produces the cone coordinates of potentials and non-conservative forces correction in FPE module friction and port correction term:
(E) [Te (E) (E(),
which defines the potential-and therefore the generalized force -for the stagewise motion- planning Hamiltonian rollouts. Learning 9ę gives meta-policy that maps environments to energy weights, i.e. a stagewise bilevel scheme where the inner layer optimizes motion under Hş and the outer layer trains & so that scenario-level Qols/constraints are satisfied across environments.
Learning (E), (E(E) throughout MMModlee-ssurrgaa reduction". Each submodule exposes response map that provides integrated dynamics rollout and additional quantity of interests (Qol) as the feedback:
R (Hx; O2, E,E) -
( (R) (k) QoT 17
k kEK.
where T& refers time scale for each submodule, To be more specific:
QoI, = AE, Qol, = ( = M ) QoI, = min d:(a : (E.4)
(47)
Namely, feedback Qols are: environmental update, velocity observation, and min distance clearance (and thus collision violation). Additional Qols, that can be deduced from (k)) St here explicitly. To train policy that output (7)) and (E) with different environments, we propose minimize
(k) not stated
- L(TEE, me) = Eg (w 111 gres gre5l12 112 + wo wullv vreslls + w. dulluu- preell2 + wa Lmatt] (48)
24

---

Published as a conference paper at ICLR 2026
Algorithm 2 Meta-policy training of (T()) and Me(E)
Require: dataset of different environments {E, avef Pret, Aress E D, step range e {2, 4, 5, 5, short rollout trials M, time step At, weights w.
1: Initialize {
2: for epoch 1. do
3: for batch B D do
4: Sample (go: po) per scene in
5:
Build tokens from active constraints C, qu). goal ag and current (go, Po) ({a,). +- n7 (tokens)) μ + pg
6:
7:
for m 1, ..... M do
8:
Sample near obstacle point fo. initialize po towards nearest obstacle at do- Integrate equation ald equation 9 for 7 steps from (īo. Po). D e.g. Symplectic Euler
9:
10:
11:
Compute clr mini,e di di(āi:E)
12:
Compute Lmuki Lmutti + xa softplus Emin -cr
13:
Tmin refers minimal radius of robot object.
14:
end for
15:
Integrate equation and equation 9or Tg steps from (qo.Po)
Evaluate loss [(TE (g)) in equation 48 Update Tl using V ne and V
17:
18:
E.g. Adam Optimizer
19:
end for
end for
of (k) seeds near obstacles discourage brittle nę (E) and (E(E) (details are addressed in Algorithm and w. 2 are user-input hyperparameters. The training via equation 448 can conducted offline, component- wise, even fine-tuned online, but state that it is important fully utilize the instantaneous response from a real navigation scenario which provides the scheme per-scneario online cor- rection when (E) and (E) properly trained under large-scale simulated dataset with reference potentials.
where Cmutt is a short multi-start robustness penalty that re-rolls from perturbed
Online Adaptation of (E) via Qols We state how response map for each environment can yield correction term under online navigation scenario. Given response R₁ at time / we construct an observable measurement vector and its reference goal:
clrs dist, speed,
Msafe
yt =
E R.,
y
dist, Epoog
max (speed,, 1 [elr) Umin)
where cir is the minimum clearance to inflated obstacles, dist is goal distance (to the global goal point), and speed = |/ul]l We update only the active barrier weights by selecting index set I, that represents nearby obstacles. Define the parameter vector
8,
ñ =
A
R
S. = Pe].
a
We denote an estimator of Jabocian Jr = Oy as ac,
J. = (v: - 91 1)(G 1T
= +
1[:. 1-1113 +E
with smoothing E [0. 1) and & 0. Then, given the desired observable change AVy = Vi yt.
one can upate meta-policy parameter via Tekuliov-regularized least square steps:
AC = arg min AC
|| 1 + A||ACII. + AG = (J, J. AT) -1) JJ Ayd.e.
25

---

Published as a conference paper at ICLR 2026
This yields updated Bi+1, YYt1, and a4+1,1 for i E I (inactive weights keep their previous values).
G++1 IR, ((1 K) Se + x AG), K2 (0.1).
Since the update is result of least square step, there exists residual:
Ay! des Je Acc
The online port correction term can amend the energy change by solving another least square prob- lem given port-observable sensitivity P. se 0y/Ouy:
|r r. r2 A. A..1 (
f.t arg min well
followed by componentwise clipping to feasible box U. A simple choice is to use P. d0ag(0, 0, (iy) so that the port primarily regulates speed while the energy weights steer clearance and goal progress; richer P. can be learned online by the same secant recipe as J..
D DOMAIN-SPECIFIC POLICY IMPLEMENTATIONS
D.1 SENSOR POLICY (Ty) DETAILS
The sensor policy maintains spatial index Ty of observations (x,, type, attr,) and derives three en- engy components from single neighbor queries:
Barrier Potential: Repulsion from obstacles
X; Cy 112
by Cc) =
Σ
1y exp
(49)
20
iENi
Free-Space Potential: Attraction to open regions
||x - cyll2
Vitee (Z C=)= =-
Σ
Wj exp
(50)
20²
JE.Neee
Density Potential: Information-theoree density measure
P(zy: Cc)
Σ RENa
wi log
11
nh [Wail
(51)
The complete sensor score function is:
b (Zy, Ca, t) = Ve,
Pyyl 1lll M + aobbx + ay Viiee + a&p
(52)
D.2 FRAME POLICY (TT) DETAILS
The frame policy uses Ty storing path samples with safety/contact distances and goal influence: Safety Field: Distance-bacc based safety measure
S(zy,
Σ
w: max (0,
threshold date
diate )2
(53)
Contact Field: Proximity to obstacles
N:
(dbonnaci) 20
C(zy, =
Σ
Wi exp
(54)
1=1
26

---

Published as a conference paper at ICLR 2026
Goal Field: Directional bias toward target
N2 Cizy,Ci) - Xy 19++
W cos
(55)
The frame score function integrates these fields:
1 o (=s, Vas 2 b a, + a.C +
(56)
D.3 SHAPE POLICY (T.) DETAILS
The shape policy controls deformation through reduced coordinates 2, (s, (,,,,,,, Smoothness Energy: Curvature regularization
Emooth L [Ix(w)) x
K Σ
|
(57)
j=1
Stretching Energy: Arc length preservation
Estrethh = ff (IIS (1))
Σ wylle let)
(58)
Target Energy: Configuration constraints
Eunget
|P
P Pargell ||2 (8,00,0) (Starget, Ouarget, Ourger )12
(59)
The complete shape score function is:
1 M2
(2o, C1, V=
2 M. + Tw2 O'sm Esmooth EStretttecc + Otg
(60)
ALGORITHM IMPLEMENTATION DETAILS
This section provides comprehensive implementation details for Algorithm 3 presented in $ We explain each algorithmic component, including parameter initialization, policy query protocols, Hamiltonian composition, integration schemes, observable extraction procedures, and online adap- tation mechanisms. These details are essential for reproducible implementation but are deferred to the appendix to maintain focus on the core methodological contributions in the main text.
E. INTTIALIZATION
Robot State. Initialize phase-spacs coordinates 2g (go- Pu) where qu = (Co, Pgr Sp, v/)) contains the starting position Co. orientation On. scale 80 1.0, and shape parameters v/n (e.g., B-spline control points initialized as a circle). The initial momentum is po
The integration time step r is typically set to 0.01-0.05 seconds. Query horizons (Ty, Ty To) determine how frequently each policy queried: sensor policy every T, steps, frame policy every Ty steps, and shape policy every step (or every T, steps). Typical values: T, 10, T, 5, =
Time Step and Horizons.
Maximum Time Horizon. Tmax is the maximum number of integration steps allowed before timeout, typically 5000-10000 steps corresponding 50-500 seconds of simulated time.
27

---

Published as a conference paper at ICLR 2026
Algorithm Online GRL-SNAM:
Navigator-Driven Hamiltonian Composition
1: Input: Goal Xg. initial žg = (go, 0), step T, horizons (T,, Ty,
2: Init: i + 0, Co + 0, meta-policy ge, Jacobians Jo, Po
3: while -REACHEDGOAL(C;, Xg) and < Tmax do
4: Query policies & collect responses:
5: if (mod Ty) then
6:
R Tyl (gy og. E,E) returns (-0.r, SU:T," AE}
(V)
EE.5
7:
1-EUAE:C C. t (a di(q::)) S dd
8:
end
9:
if = (mod T3) then
10:
(f) (1) I Rj + ( returns (obey Sury, {ot se=u/
E..6
11:
end if
(o) R t Fo(Ho: E, E,g) returns & son, {min, di d4 (0) : {
12:
SE.7
13:
Meta-polic proposal:
E.8
Build tokens T, from (Ct, 9t (x)
15:
Ing. US. E < 9 (T(:E) with vi (3t, X3, (af)ec.))
Compose surrogate Hamiltonian: Righ n 8) = Esensor + 3r Egoal + X1 Ebb + Eiec, (b((d)) H (gt, Priw &. E) Lp2 (Qq)) -pp: R(g n E)
E.
17:
Integrate dynamics: vpHli I- = ((:)) Pti VgHli ( = Vg + agEgoal + X X'V Eobi Eiec, a (dd)) Petr Pt v H1 TuE v H + Tu E.t
D F.10
18:
19:
20:
l++ = qi + ( PL+T
21:
Extract observables:
b E.11
22:
clr, min(min, d, (9( E), ming, di (0), E)}; dist, PP+Tl
|cttr
xyll: speed,
23:
-lc,,, diste, -speed,] T; 87 Eprog: max [ 2 mmar ) Umin)]
Bt
Maafe: diste
Parameter adaptation:
E.12
25:
Select I C Cis G = A {a. [a!
26:
= (y) - V1-7)( - G G--)
/(1G (G--)T/(IGC
--- te:: pJr- + +1 (T-
27:
Ayple Vi - yi AG (J Jr + >xT) -1 AeRes
28:
GAT = MR, ((1 - K) 3 Ge + K c (Ge AGe)
29:
Unpack to [get,, XHT {a hier, k [+7
l+
30:
Port correction:
> $E.13
Ti Aydes Jr Nri u g,tt clip P - 2.
31:
P. = (ye yt- (u - uuy - / /(1lu - - E): = pr (1 1-pr)
32:
33:
E+ttr
end while
35:
Return: Trajectory (ze)7 parameters (ng ub, us, yT
S.14
28

---

Published as a conference paper at ICLR 2026
E.2 PARAMETER STRUCTURE AND INITIALIZATION
Meta-Policy Network. The meta-policy Is : & + Me M4, u9] can be either:
learned neural network trained offline via Algorithm 2
Fixed constant values: 3 2.0, X 1.0, af = 1.5, vt = 0.1
If using learned network, are the network parameters. using fixed values, 9ę simply returns constants.
Jacobian Initialization. Initialize Jo [3x as small random matrix identity scaled matrix. Initialize port Jacobian Po E (3xdim((y) similarly. These will be refined online via secant updates.
Environment and Context. Initial environment & contains known static obstacles. Active con- straint set Co = 0 starts empty and will be populated by the sensor policy's 's first query.
E.3 TERMINATION CONDITIONS
The algorithm terminates when any of the following conditions is met:
Success: Goal Reached.
||c, 1 Xyl Egoal where Egoal 0.05 m tolerance).
Failure: Timeout.
t Timax without reaching the goal.
Failure: Collision. min, E) < O, indicating penetration into an obstacle.
Failure: Stuck. ||. 1c--6e Ct-50rl1 < Estuck where ésnac 0.01 m, indicating the robot has not moved more than 1 cm in the last 50 steps (indicating entrapment in a local minimum).
E.4 HIERARCIICCL QUERY FLOW
Policies are queried in specific sequence with information flowing forward:
Sensor Policy Query (Line 5). Executed every Tg steps. Takes input the previous context CT-T,, current state 2/ = (C,, i:), and goal Xg. Performs ray-casting from the robot's position to detect obstacles within sensing range.
Frame Policy Query (Line 8). Executed every Ty steps. Takes as input the updated context C; from the sensor policy, current full state 21 (qt, and goal Xg. Plans short-horizon path considering known obstacles.
Shape Policy Query (Line 10). Executed every step. Takes as input the context C, from sensor and the current shape state 2 = (st, th. St, vir). Computes deformation forces to navigate through constrained spaces.
C22y Frame implicit
all responses
Information Flow.
Sensor
Shape
Navigator
E.5 SENSOR RESPONSE
Response Structure.
R = (g)
P.T.T SO.T, Δε} where:
(v)
(v) 20T, nian H2
((gh (3) To
.Pr To is the integrated trajectory of the sensor policy's local Hamilto-
(v) BDT,
(5(0)
(9) Je=o the score function (dynamics drift) at each step
29

---

Published as a conference paper at ICLR 2026
A contains newly detected obstacles: AE (ypee)) with positions, radii, and types
Environment Update (Line 6). Merge new detections: < U AE. Update active constraint set: Ct < {i d:(q;; E) S d} where is the sensing/activation radius (typically 0.5-1.0 m).
Policy Hamiltonian (not directly observed). The sensor policy operates on Hy (gy Pyi E, = p Myy())) + (9gggiEE where Ry = Escusso (q; & 8,y) + Eiec, ac)(d,) as defined in the methodology.
E.6 FRAME RESPONSE
Response Structure.
R₂J = (200 (f) 1 ( o 1
Je where:
(n) =(9, 1 1 is the short-horizon rollout under H
oI,
(f) Bo,, the score functions
(og ( M Dezo are velocity observations used for extracting speed in observable
computation
Policy Hamiltonian. Hy(EE) 3 + RylgsiE,E) where R, Egoal + Eiec, a;b(d.).
Dissipation and Port in Frame Policy. The frame policy integrates with dissipation rỤ ME(E)I and port input G&uf as per the nott-Hamiltonian equations (5-6) in the methodology. This is re- flected in the Navigator's integration step (Line 16).
E.7
SHAPE RESPONSE
(0) (0) RS { Su:1. {min; ( where:
Response Structure.
O.T = (( Pi
(0) (0) 10) To is the deformation trajectory
(o)
SO, score functions
{min; d, (g( (o); E) tracks minimum clearance at each step of the shape rollout
Policy Hamiltonian. Holgo,PoiE,.8 1P. Mo(o)))))ppp
(o where R,
Eob) + Σiec, a,b(d,).
Usage in Clearance.
The clearance values from the shape rollout are used in Line 18 to compute clr,, accounting for the robot's actual deformed geometry when assessing collision risk.
E.8
META-POLICY PROPOSAL
Token Construction (Line 12). Build input tokens T, from current environment state:
Obstacle features: positions, radii, types for each E C₁
Robot state: position C₁ velocity M M- Pt. current scale $;
Goal information: relative position X, c.. distance ||x, - cell
These are typically encoded as ioopuuttaiioninvvaiin set representation (e.g., using attention mech- anisms over obstacle features).
30

---

Published as a conference paper at ICLR 2026
Meta-Policy Output (Line 13). 96 (T(:E) bin udg. u produces:
Energy weights: vg = (3", N fathee,) R 2+1cl
Friction coefficient: 1 R
Initial port suggestion: u/y" E (edim(u)) (will be refined online)
Fixed Learned. If Og is not trained, simply return fixed values: 5 2' 1.0, a
1.5 Vi, μ² 0.1, uet 0. The online adaptation (Lines 22-28) will refine these values regardless of initialiizatinn.
E.9 HAMILTONIAN ASSEMBLY
Energy Component Definitions.
The potential energy is decomposed as:
sensor (qi wyy
y ll's (sensor configuration cost)
(61)
Egoal (q; E,wg)
c; - x,1 (goal attraction) deformation energy of shape 1/2
(62)
Eobj (g: wa)
(63)
d²
M(d:gg:));ur barrier function, e.g., df
or IPC log- barrierrier
(64)
Total Potential (Line 15). R(ge ne. Esensor + Ba Eggal + X Eonj Eiec, a)b(d.)
This is the surrogate potential that aggregates contributions from all three policy domains, weighted by the meta-policy outputs vi = 1, faf)).
Surrogate Hamiltonian (Line 15). H(q, p. (, 1Pp M(qawM)-1pp + (q:n
The kinetic energy uses (possibly state-dependent) mass matrix M(:)). For planar navigation, this often constant: M = diag(m, m, J2z, m.....) for translation, rotation, scale, and shape degrees of freedom.
Fixed Parameters w. As stated in the methodology section, the intra-term parameters w (Wyy WM,, Wly wa, wa) assumed fixed throughout all environments. Only the dual weights ne are adapted online.
E.10
SYMPLECTIC INTEGRATION WITH PORT- HAMILTAAMLTNNIAN DYNAMICS
Gradient Computation (Line 16). Compute the Hamiltonian gradients:
V p H = M(g.) ''p
(65)
V H V V2R( m E)
(66)
V, Exnnsor + Egoaal + X'V Eabj +
Σ afV
(67)
iEC.
Each gradient ((b(d)) is computed using the chain rule: v,b(d,) Dd,
Oh
Vydi(g:
Symplectic Euler with Dissipation and Port (Lines 17-18). The integration follows the port- Hamiltonian structure from methodology equations (5-6):
Pt+- = 7 TV,
H|
V
H
+
(68)
G¹
r
Ut+r = lt + TM (ge) Petr
The dissipation term - - = - (4)) acts as propoiooaaooo damping. The
port input v² provides non-conservative forcing correct for residual errors.
31

---

Published as a conference paper at ICLR 2026
Frame Policy Only. As clarified, the dissipation uág and port uf apply only to the frame pol- icy dynamics. The sensor and shape policies have r 0,1% 0, and no port inputs in their local dynamics. However, since the Navigator integrates the full robot state using the composed Hamiltonian, the dissipation and port appear in the full-state integration.
E.11 OBSERVABLE EXTRACTION
Frame-Based but Multi-Module (Line 18-19). The observable vector 1/2 constructed from quantities that *primarily derived from the frame trajectory** but require information from all three modules:
Clearance (uses sensor shape):
min min ( clr, min min (2 E=0....T.
:88
(70)
This takes the minimum both the integrated Navigator state u++r and all states from the shape policy rollout (g Computing d, (q: E) requires:
Environment from sensor policy
Robot geometry (radius, shape) from shape policy
Distance (frame position):
dist, ||cett Xall
(71)
Directly from the frame component of (++r.
Speed (frame velocity):
speed, ||M (9(+)) 'ptrl
(72)
Computed from the updated momentum Pr+r-
Observable Vector and Target (Line 19).
clr² dist, speed,
Masae dist, - E pror
y1 =
, y
(73)
max(speed, 1 { 2 meste Umin).
Target parameters: Mgafe 0.15 (desired minimum clearance), Eprog (desired progress per step), Unin 0.3 m/s (minimum speed when safe).
Physical Interpretation.
[0] -clr;: negative clearance (larger value closer obstacles)
Target yf [0] -0.15 means want maintain at least 15 clearance
ye(1 [1] dist,; current distance to goal
Target vł [1] dist, Epro means want make progress (reduce distance)
ye [2] sppedd,; negative speed
Target encourages minimum speed Umin when clearance is safe
E.12 PARAMETER ADAPTATION VIA OBSERVABLE FEEDBACK
Active Parameter Selection (Line 21). Select subset I, C, of nearby obstacles (e.g., within 0.5 m) to limit the parameter vector size. Assemble:
G.
| X1 (af)iez, AE] E E
2+/Z++11
(74)
For obstacles not in I, their a; values remain unchanged from the previous step.
32

---

Published as a conference paper at ICLR 2026
Secant Jacobian Update (Line 22).
Approximate the Jacobian Jr 0y/0c using rank- 1 secant
update:
(Ur y/-r (S: - -(-r)
E (2x(2++1)++1)
(75)
Cr 1--113+
Apply exponential moving average for smoothing:
Ji pJl-r + (1 p),,, Pe (,
(76)
Typical value: p 0.9.
Tikhhnov-Regglarrized Least Squares (Line 23). Compute the desired observable change:
Avt des
y y
(77)
Solve the regularized least-squares problem:
Aç: arg min |J 25
Avt Acl1AC1E
(78)
Closed-form solution:
AC = (J J++ +,
(79)
Regularization parameter: X; 10 ensures numerical stability.
Parameter Update with Step Sizes (Line 24). Apply the update with componentwise step sizes [13,, RA, (Bag) } Ka]] where each Ki E [0, 1):
Se-r II ((1 - x) o Se + N (Gr + AC:))
(80)
The projection IRR enforces non-negativity: [Se+r]s = max (s+rli)
Typical step sizes: Ki 0.05, KΛ 0.05, Maa 0.1, Ki 0.02.
Unpacking (Line 25).
Extract the updated parameters:
[8t+,, X (aa+" heI. [ + Ce+
(81)
For obstacles j & I,, retain previous values: at = ajj
E.13 PORT CORRECTION FROM RESIDUAL
Residual Computation (Line 27). After the Jacobian -based parameter update, there may remain residual error:
= Ay des J,AG e R
(82)
This residual captures observable errors that cannot be corrected by reshaping the energy weights alone (c.g.. due model mismatch or unmodeled dynamics).
Port Correction via Least Squares (Line 27). Solve second least-squares problem to find port input that addresses the residual:
uf
arg
min ue aendim(uy)
|[uu -rell3 A, 11u1
(83)
Closed-form solution:
"
g.t+ 1.t+= ( P. XT.)) 'pT
(84)
Clip to feasible box: u g.t+7
clip(ug.ttt, ,21) where 21 [- Umuxx Umax dim(uy))
33

---

Published as a conference paper at ICLR 2026
Port-Observable Jacobian (Line 28). Estimate Pt 2 dy/Ou, using secant method:
P. (yt- yt-7 (us / 7T
(85)
|luy Uj
+
Apply smoothing:
P. = P- (1 (1- pr) pr E 10,
(86)
Simplified Port Jacobian.
If full P. estimation is noisy, simple diagonal form:
P. diag(0, 0, k.,)
(87)
where No 0 means the port primarily affects the speed observable (third component of yı), while clearance and distance are controlled via energy weights.
Role of Port Correction. The port correction u provides non-conservative forcing that can- not be represented by a potential function. It acts as a "corrective impulse" applied at each step to handle:
Residual errors after parameter adaptation
Transient disturbances
Model mismatch between surrogate and actual dynamics
This is computed online and is not part of the trained meta- meta-policy 9ç-
E.14 OUTPUT FORMAT
Trajectory. Return the sequence of states (z (gt,Pe)) where T = Linal is the final time step when termination occurred.
Meta-Policy History.
Return the time series of adapted parameters:
{Nt u uf Ye = {(s X (a.))) 15
(88)
This enables post-hoc analysis of:
How energy weights evolved during navigation
Which obstacles required stronger repulsion (large o:)
When goal attraction was increased/decreesee (8)
Friction and port correction patterns
Optional Diagnostics. Additional outputs may include:
Clearance history: {clre)
Observable error: (Aydes)7 17 Je=0
Hamiltonian values: (()
Residuals: {re)T=
F THEORETICAL ANALYSIS AND PROOFS
F.1 MULTI-POLICY STABILITY ANALYSIS
Theorem F1 (Multi-Policy Stability - Complete Statement). Consider the coupled multi-policy sys- By 8, tem with score functions {sy .s .. operating temporal scales {Trens: T path Time satisfying:
Tums
T Tpak
2 01 >
2 02 >
(89)
T path
T int
34

---

Published as a conference paper at ICLR 2026
Let each policy have Lipschitz constant L3 with respect to state and parameter variations:
1 C, t (z2, C, 1)l v Tx - -22ll
(90)
0 11 (z, 5x (z, c)1 - Lx 031
(91)
the parameter updates during training satisfy:
max
|10%+1 0i1l
(92)
(E(M.fo))
Lmax
11,
where Lmax maxk Lg,, then:
1. Stability: The coupled system state remains bounded: || S C(1 + || for some constant C.
Error Bound:
The total navigation error satisfies: Evtit S with probability 6.
3. Convergence: The system converges to neighborhood of the optimal trajectory:
lim; +o dist(zz, P*) E.
Proof. The proof proceeds in three steps:
The scale separation assumption ensures that fast dynamics (sensor) reach approximate equilibrium before slower dynamics change significantly. For the sensor policy operating on timescale sesss the quasi-static approximation gives:
By 7/ V Nyy (Zy. efixed t)
(93)
where Cfixel represents slowly varying constraints from path and shape policies.
Under the Lipschitz conditions, each policy detines contraction mapping its domain. The composed system inherits this property with contraction factor:
T4
p max
(94)
&
1 7KT
provided step sizes ты chosen appropriately.
Parameter update bounds ensure that training perturbations don't destabilize the system. The error propagates according to:
Linax
|| & p|| +
(95)
min((1)00))
which converges to the stated bound under the given conditions.
n
F.2 SYMPLECTIC STRUCTURE PRESERVATION
Theorem F.2 (Symplectic Preservation Complete Statement). Let (ZRyWa) be phase spuce coor- dinates with canonical symplectic form Wy E, dat A dpi- The score function update:
Zk,t+1 Eh,t + ThJusk Ok (zu,2, Ct,t
(96)
where Jr
Id and sk
V Bk preserves the symplectic structure:
Idz
wx(zH,411) = ( +
(97)
Proof. Since oh
VhL the update is discretized Hamiltonian flow. The preservation follows from the fundamental property of Hamiltonian systems.
For the continuous flow in = Js Vh dt (zk, 1), we have:
d
dt Wk = CXH, Wk =
(98)
where L is the Lie derivative and X = J Vh is the Hamiltonian vector field.
The discretization introduces O(T²) error due to the symplectic Euler scheme, but the leading-orde order symplectic structure is preserved.
35

---

Published as a conference paper at ICLR 2026
F.3 SAMPLE COMPLEXITY ANALYSIS
Theorem F.3 (Sample Complexity Complete Statement). For error tolerance > O and failure probability E (0, 1), consider training three independent score functions {& heluudoo) with phase space dimensions {dy, d do}-
Under standard smoothness and concentration assumptions, the total sample complexity is:
Nusal RE(U-io)
Σ
(99)
where each policy requires:
)))
3
No
log
(100)
&
with Lipschitz constants Lk and error allocation eg satisfying Σx S.
This achieves linear scaling Nova 0 dx).
OiE k dx) compared joint training requiring Njjiat
Proof. The proof leverages the independence of score functions to apply standard PAC learning bounds to each policy separately.
For each policy empirical risk minimization:
No
02
arg min 0%
Σ
1
(Zu.s)
(101)
N
1=1
achieves generalization error ex with probability 1 6/3 when NO 2 universal constant C.
C LLi
log(3/8) for some
of
The union bound over three policies gives total failure probability 6, and the error allocation ensures total error Σσ c S c.
The linear scaling follows from independence: total samples Σ NN. compared to joint training on the (dy + dd + do dd)-dimensional joint space requiring exponentially more samples.
G OTHER IMPLEMENTATION DETAILS
G. SPATIAL DATA STRUCTURES
Each policy maintains spatial index Te. can be implemented as dynamic octree Ellendula & Bajaj (2025), which supports the following operations and has been proved the optimal structure for apatio-tempore maintenance:
Insert: Ologg n) insertion of new spatial data.
Query: O(log k)) for k-nearest neighbor queries.
Update: Ologg n)) modification of existing entries.
Rebalance: Onnlogn) periodic rebalancing for efficiency.
The spatial indices enable multi-kernel evaluation: each score function query reuses the same Ollog n + k) neighbor search across multiple energy kernels, reducing computational complexity from ( dense evaluation to On log n) sparse computation.
G.2 DETAILED BASELINE IMPLEMENTATIONS
We provide here the full technical details all baseline planners evaluated.
Rigid * A standard A search is performed are inflated by the nominal rest radius Trea of the deformable ring, such that the resulting path is collision-free for rigid disc of radius Trest This serves conservative reference planner.
grid discretization of the workspace. Obstacles
36

---

Published as a conference paper at ICLR 2026
Deformable A Clearance at each grid cell æ is defined c(z), the distance to the nearest obsta- cle boundary. Feasibility requires c(x) 2 Tmin- The edge cost between cells u, v is augmented by a deformation penalty:
cost(u, v) E(u, u) + 2 (o(c(u) + <(c(o))) E(u, v), d(c) A - -
where f(u, v) is the Euclidean distance, and B, À control penalty strength. This formulation allows the planner to compress through tight gaps when unavoidable, while encoding energetic cost.
Potential Field (Stagewise). Navigation is driven by an attractive force toward the stage exit (or final goal in the last stage), combined with repulsive forces from local obstacles and soft penalties for leaving the stage bounds. Speed saturation and emergency braking near obstacles are applied for stability.
CBF (Stagewise). At each step, a nominal control toward the stage exit is filtered through Con- trol Barrier Function (CBF) quadratic program:
ut* arg min ||u - Unoml || s.t. Vh(z) 7/(x) 2 0,
where h(x) encodes the clearance from visible obstacles. This ensures forward invariance of the safe set within each stage.
DWA (Stagewise). We implement Dynamic Window Approach (DWA) adapted to the stagewise setting. Candidate (v, w) velocity pairs are sampled within dynamics limits, trajectories are rolled out over prediction horizon, and scored based on heading alignment, distance to target, velocity, and clearance with respect local obstacles only. Stage boundary penalties are also included. This contrasts with the conventional global DWA, which assumes full obstacle visibility; here show the stagewise variant for fairness, though known to underperform due to rigid-body kinematic assumptions.
Categories.
Global planning: Rigid A#, Deformable A#,
Local reactive: Potential Field (staged), CBF (staged), DWA (staged).
Ours: GRL-SNAM (local staged).
This categorization makes explicit which baselines share identical information constraints with GRL-SNAM, ensuring valid comparison.
491
Figure Online navigation of the hyperelastic ring through cluttered environments. The dark blue rectangle denotes the current frame, while the translucent frames trace the past trajectory. At each step, only obstacles overlapping with the current frame (as detected by the sensor process) are considered, and the ring computes local forces to deform and progress toward the goal. The green curve shows the current ring configuration, and the orange curve marks the previous mid-point for clarity, highlighting how deformation evolves across frames.
37

---

Published as a conference paper at ICLR 2026
H DETAILED EXPERIMENTAL EVALUATION
We evaluate GRL-SNAM across multiple dimensions that highlight the unique capabilities of our geometric approach compared standard reinforcement learning and classical navigation methods. Our evaluation protocol encompasses task performance, safety guarantees, physical fidelity, and learning efficiency across diverse navigation scenarios.
H.1 BASELINE PLANNERS
We compare GRL-SNAM against two categories of baselines: global planning methods based on A", and local reactive methods with the same stagewise information constraints GRL-SNAM. This ensures fair evaluation across fundamentally different planning paradigms.
Global Planning Methods
The deformable ring replaced with rigid disc of radius frest- Obstacles inflated by Trea, and a standard 8-connected A is run on the occupancy grid. This produces feasible shortest paths for rigid robot.
Rigid *
Deformable * A clearance-aware variant of A augments the step cost with deformation penalties that increase as clearance approaches the minimum admissible radius Tmin- This allows paths that squeeze through narrow gaps but penalizes excessive compression.
Local Reactive Methods To ensure fairness, all reactive methods use the same stage manager as GRL-SNAM: identical stage size, overlap, obstacle visibilityy and advancement logic. Each method navigates stage exit to stage exit until the goal reached:
Potential Field (Staged): Attractive force toward stage exit plus repulsive forces from local obstacles and stage boundaries.
CBF (Staged): Quadratic-program filter enforces safety constraints with respect to visible obstacles at each timestep.
DWA (Staged): Velocity samples (v, w) are rolled out over a short horizon using only local obstacles and stage bounds. Unlike the global DWA, which assumes full obstacle visibility, this stagewise variant ensures equal information constraints, though performs poorly due rigid-body assumptions.
Categorization Rigid and Deformable A form the global planning references, providing Leef for SPL and detour calculations. The stagewise Potential Field, CBF, and DWA baselines constitute local reactive category under identical information constraints. GRL-SNAM belongs to the same local category, enabling a fair
comparison.
H.2 DETAILED EXPERIMENTAL SETUP
We evaluate GRL-SNAM in procedurally generated 2D deformable navigation tasks, where hyper- elastic ring must traverse cluttered environments with narrow gaps and varying obstacle densities. Each environment randomized in obstacle positions, radii, and densities to span a spectrum of navigation difficulty. The robot perceives only local window of size 2d 2d, from which construct Hamiltonian energy functional.
Hamiltonian Decomposition
The energy functional decomposes into:
1. Goal-directed quadratic potential F,
2. Barrier potentials Fbs from signed distance fields
3. Frietion/regularizztion terms with adaptive coefficients (8, 7, modulated by context en- coders (LSTM)
Offline, GRL-SNAM integrates reduced Hamiltonian gradients to generate local trajectories. Online,
fuses newly sensed rewards Rew with the offline surrogate, adaptively refining navigation.
38

---

Published as a conference paper at ICLR 2026
Table 5: Comparison of navigation quality across methods (success-only runs). GRL-SNAM
achieves near-CBF path efficiency while consuming the same minimal mapping budget as PE. SPL Success weighted by Path Length; Detour executed path length / shortest path length.
Method
SPL
Detour +
Min. Clearance (m) f
Mapping Ratio (%) 4
PF
0.77
1.42
0.18
10.3
CBF
0.96
1.04
0.32
1 11.2
GRL-SNAM
0.95
1.09
0.26
10.7
Evaluation Metries
We evaluate all methods using:
Success Rate: Fraction of episodes reaching the goal
SPL: Success weighted path efficiency relative to A#
Detour Ratio: Executed path length relative to A*
Minimum and Mean Clearance: Distance nearest obstacle along the trajectory
Smoothness: Average turning cost (mean absolute change in heading)
Collisions: Number of obstacle intersections
Sample Efficiency: Normalized area under curve (AUC) for success and SPL, and steps required to reach 80% success or SPL > 0.7
Results are presented in question-answer format, emphasizing experimental questions and the corresponding insights.
H.3 RESULTS: NAVIGATION QUALITY UNDER MINIMAL SENSING
We first evaluate GRL-SNAM against two representative baselines: (i) Potential Fields (PF), a purely reactive controller that maps obstacle proximity into repulsive forces, and (ii) Control Barrier Functions (CBF), model-based method that enforces hard safety constraints via online quadratic programs. Both baselines use the same sensing budget as GRL-SNAM.
Environments consist of cluttered 2D workspaces with obstacles of varying density. Each trial starts from a random initial pose with a fixed goal. Performance averaged across 50 runs per environ- ment. Results focus on successful runs only to highlight navigation quality rather than raw failure rates.
Q1: How efficiently do we trade mapping for navigation quality? Table 5 shows that GRL- SNAM matches the SPL. and detour ratios of CBF despite using the same minimal map coverage as PF. This demonstrates that our stagewise Hamiltonian refinement extracts more value per sensed unit of the environment, trading mapping effort for near-optimal navigation.
Q2: What is the minimal mapping needed to reliably solve tasks? With 10- 11% map cov- erage. GRL-SNAM already achieves SPL > 0.95 and detour within 9% of the A shortest path. PF fails under the same budget, while CBF requires identical map coverage. Thus, GRL-SNAM reliably solves tasks under minimal sensing, validating the minimal mapping suffices principle.
Q3: Is the mapped information aligned with the subtask? Unlike PF, which produces repul- sions indiscriminately, or CBF, which enforces constraints globally, GRL-SNAM's mapping is task- aligned: local patches are encoded into Hamiltonian terms that directly drive subtasks (goal at- traction, barrier avoidance). The result is that every bit of mapped information yields functional guidance, as evidenced by SPI. and detour staying close to CBF even under tight sensing budgets.
Key Insight GRL-SNAM shows that Hamiltonian -ttruiuurrrrr policies can achieve CBF-level nav- igation quality while retaining the lightweight sensing footprint of PF. The slight clearance gap relative to CBF reflects a deliberate trade-off: we sacrifice hard feasibility for adaptability and feed- forward inference, enabling real-time deployment in SNAM settings.
39

---

Published as a conference paper at ICLR 2026
CALPRA P
Micg Succuis SPc 190
Suocetc 2.30
Succtii Si L.30
8000
dogo Doc
00 O
COLLO GOOROOO
PD
13
3.0
100
150
SAC
OOoO PO O COD
Ogg 1oooooo oo
Ogg 60
ooo
Sucenn SP-3 300
Seceue $222.800 200
Figure 8: Qualitative path comparison on representative Test-OOD environment. Fach panel shows a single rollout with success and SPL. annotated in the title. GRL-SNAM produces a smooth, short, and clearance- preeerrrin- path that threads the narrow passage. Kinematic and dynamic RL policies (PPO/TRPO/SAC- COATTRRPO//ACKK//DDIIIAI Kin/Dyn) typically stall, collide, or wander far from the goal, while the best coefficient- based variants TOPO/TRPO/SAC-COee reach the goal but hug obstacles and exhibit lower path efficiency than GRL-SNAM. Global planners (RigidA*, DeformA*) succeed but follow jagged or overly conservative routes, and reactive baselines (Potential Field, CBF, DWA) either oscillate, graze obstacles, or take longer, less structured paths.
H.4 RESULTS: COMPREHENSIVE NAVIGATION COMPARISON
Q4: Does GRL-SNAM outperform classical and reactive planners in both in-distribution (Test-ID) and diittiiiisiiittoonn (Test-OOD) settings? Yes. g summarizes the compari- son between our method and five baselines: Rigid A*, Deformable A Potential Field, Control Bar- rier Functions (CBF), and Dynamic Window Approach (DWA). GRL-SNAM achieves perrect success rates (== 100%) across both Test-ID and Test- cases, while all basclines degrade signif- icantly in cluttered or novel environments. Rigid A succeeds moderately but requires inflated radii and yields jerky, piecewise paths. Deformable A is less stable and highly sensitive to parameteri- zation. Reactive baselines (Potential Field, CBF, DWA) frequently fail reach the goal, producing oscillatory unsafe behaviors. Qualitative rollouts (Figure g further illustrate the superiority of GRL-SNAM in complex cluttered environments.
Does GRI-SNAM yield more efficient and smoother trajectories? Yes. The Success- weighted Path Length (SPL) distributions (Figure 9 top-middle) show that GRL-SNAM consis- tently stays near optimal efficiency (SPL R 1.0) with low variance. In contrast, A variants incur detours, while reactive baselines either collapse to zero SPL (failures) or take excessively long paths. Furthermore, GRL-SNAM generates the smoothest trajectories, with the lowest average turning gles (Figure 9 bottom middle),, ensuring physically realizable motions compatible with hyperelastic ring constraints.
Q5:
40

---

Published as a conference paper at ICLR 2026
pipe
Figure 9: Main comparison on Test-ID and Test-OOD. GRL-SNAM achieves near-perfect suc- cess, high SPL, smooth and safe trajectories. Classical (Rigid/Deform A*) and reactive (Potential Field, CBF, DWA) baselines are either unsafe, inefficient, or fail completely.
Q6: Does GRL-SNAM preserve safety margins? Yes. The minimum clearance analysis (Fig- bottom-left) shows that GRL-SNAM maintains consistently positive obstacle clearance, whereas A occasionally cuts too close and reactive baselines often enter collision regimes. The Pareto frontier plot (Figure 9 bottom-right) highlights that GRL-SNAM uniquely dominates the trade-off, achieving both high SPL and high clearance, while all baselines Pareto-dominated.
ure 9
H.5 RESULTS: HAMILTONIAN FIELD ANALYSIS
Q7: Does the Hamiltonian formulation unify attractive and repulsive forces into coherent navigation field? Yes. Figure 4 shows the isolated goal force Fg (left), the barrier force Fis (middle), and their differential composition F 3F, 7/% (right). While F, alone pulls the agent directly to the target, it ignores obstacles. Conversely, Fos encodes obstacle constraints but lacks task directionality. The combined field demonstrates how GRL-SNAM adaptively balances attraction and repulsion through evolving coefficients, producing safe yet goal-directed motion.
Q8: How does GRL-SNAM differ from ordinary online adaptation? Unlike standard RL poli- cies that merely adjust actions online, GRL-SNAM modifies the entire local energy landscape as new obstacles magnitudes (middle panel) not only rebalance between goal attraction [Fy) and barrier repulsion [F but also induce redefinition of the reduced Hamiltonian. This is reflected in the evolving coefficients (8,7,a) a:) (bottom panel). which do not act as heuristic gains but as dual variables gov- erning stagewise refinement. Thus, the adaptation is not reactive in the usual sense: GRL-SNAM performs posterior updates of the Hamiltonian itself, ensuring that each new frame redefines both the dynamics and the reward landscape in a principled, energy-conse consssset manner. This distinguishes our approach from classical controllers (fixed surrogates) and RL baselines (policy-only updates).
sensed. Figure 10 shows that when clearance decreases (top panel), the force
Q9: Does this lead to improved navigation performance compared to baselines? Yes. Across procedurally generated test cases, GRL-SNAM consistently achieves higher success and SPL while maintaining larger clearances than rigid A (fixed radius assumption), deformable A (static squeez- ing penalty), ) and reactive controllers (DWA, CBF).
Key Insights These experiments establish GRL-SNAM the first method to successfully unify global navigation objectives with local safety and deformation constraints in hyperelastic navigation. Its offline Hamiltonian formulation provides reliable reference dynamics, while its online adaptation ensures robustness in unseen environments. By contrast, classical and reactive baselines either fail outright, succeed only the cost of safety and efficiency.
41

---

Published as a conference paper at ICLR 2026
1.
genire derong
Figure 10: Quantitative validation of GRL-SNAM. Top: clearance stays above collision threshold, ensuring safety. Middle: force magnitudes adapt to environment complexity. Bottom: coefficients (3,7, 7)) evolve dynamically, confirming online adaptation and stagewise refinement of the Hamil- tonian.
Table 6: Ablation of loss terms. Qualitative summary from consistent runs on Test-ID//On Ar- rows denote desired direction. Numeric means-istd can replace the icons once collected.
Variant
Collisions
MinCir
Barrier Viol.
Progress/SPL 1
Smoothness
Observed behavior
High (x)
High
Poor
Poor
Penetrates obstacles
Mine 0. Weet 0.5
Low (
High
Low
Low
OK
Very slow, conservative
0.1.
None (7)
High
Low
High High
Best
Smooth, stable, last
Wine 0.1.
Wamm 0.5
None (V)
Slightly lower
Low
Good
Stable; tighter margins
H.6 ABLATION STUDY: LOSS COMPONENTS
Training Objective Our navigation surrogate is trained with weighted multi-term loss:
C = Cunj + Wivel Ceel + Wftictionn Ctriction + Wmulti Cmalti
(102)
where Limj and Lvel supervise trajectory and velocity matching, Ltriction 1| % encourages the learned damping to match the stagewise reference, and Lmulti penalizes failures under short rollouts from perturbed near-obstacle starts.
Ablated Settings
We toggle Lrietion and Lmulii to analyze their contribution:
No friction, no multi (Wric 0, Wmulti damped, unstable dynamics.
0): Agent penetrates obstacles due to under-
Multi only (Wie O, Wmmlti 0.5): Agent avoids collisions but moves very slowly,
sacrificing progress.
Friction only (Wiic 0.1, Wmuht 0): Produces smoother, stable paths, eliminating
penetrations and maintaining progress.
Friction + Multi (Wnic 0.1, Wmulu 0.5): Combines both benefits, but clearance slightly reduced as the agent cuts closer to obstacles.
Analysis Lriiction is critical for stability and smoothness, while Ceuuti improves robustness in clutter but can damp progress over-teig The best overall performance arises from combining both with moderate weights.
Litiction aligns dissipation and suppresses oscillations, yielding smoother, well-damped trajectories and preventing barrier "ringing" that causes penetrations when Wise 0. Lmain trains for near- contact robustness by sampling perturbed starts; if over weighte it down-scales the goal term, hence slow motion. Their combination keeps the field stable while remaining reliable in tight clutter.
42

---

Published as a conference paper at ICLR 2026
H.7 ROBUSTNESS ANALYSIS
Q10: Does GRL-SNAM remain reliable under sensor noise and dynamics shift? Yes. To evaluate robustness, systematically varied sensing fidelity (position jitter, radius estimation er- ror, missed obstacles, and false positives) and dynamics fidelity (velocity perturbation, damping coefficient ?). Each start-goal trial was rolled out across a grid of perturbation levels, producing total of N = Teay X Tuials X Ilperuubationss runs. For example, with environments, 5 trials each, and perturbation settings, this yields 135 rollouts.
Table 7: Robustness of GRL-SNAM to sensing noise and dynamics perturbations. Columns report success rate, success-weighted path length (SPL), minimum clearance, and average collisions per episode, Arrows indicate direction of improvement.
Perturbation Level
Success (%)
SPL
Min. Clearance (m) f Collisions
Nominal (0.0, 1.0)
98.7
0.82
0.36
0.3
Mild Noise (0.05, 0.9)
91.3
0.79
0.33
0.7
Severe Noise (0.10, 0.7)
87.1
0.72
0.29
1.1
Key Insights Despite significant perturbations, GRL-SNAM maintains high success rates and graceful degradation in SPL and clearance. Unlike fixed surrogate approaches that can fail catas- trophically under noise, our differential Hamiltonian adaptation continuously re-weights local forces, enabling stability highlights the feedforward, stagewise advantage of GRL-SNAM: it can adjust online without requir- ing adjoint or MPC-style corrections, ensuring reliable navigation in real-world uncertain conditions.
when sensing is imperfect dynamics deviate from training. This
H.8 DETAILED EXPERIMENTS ON DUNGEON POINT-AGENT NAVIGATION
We provides additional details for the dungeon point-agent navigation experiment used to test GRL- SNAM's performance in more complex map topologies. As stated earlier, we use dungeon layouts from Liang et al. (2023). The agent is point mass in continuous 2D with state d (position, and optionally velocity the dynamic parameterization) and action a, = [[a, E 1-,,3|² The task is to reach designated goal while navigating through narrow corridors, turns, and cluttered local geometry. Collisions with walls are penalized (or terminating, depending on the baseline), and reaching the goal yields reward.
To match the stagewise interface used by GRL-SNAM, a Stagewise identifies the cur- rently active macro-cell and exposes its exit as the local goal 9s- An ObstacleExt ractor fits circular obstacle primitives to wall segments visible inside the local sensing window W(:).. return- ing centers, radii, and visibility mask. The observation concatenates the stage-relative position, the local goal gr. and the extracted local obstacle set. GRL-SNAM instantiates a simple quadratic goal energy together with radial barrier energies from these obstacle primitives.
All methods share the same point-agent dynamics, local sensing budget, and action space. We compare two training protocols: (i) full-episode RL, where PPO/TRPO/SAC are trained end-to-end on the long-horizon partially observed MDP; and (ii) stagewise short-rollout RL, where the same algorithms are trained under the same structured observation/action interface used by GRL-SNAM. interface stagewise comparison in Table [3] Here we provide the more difficult full-episode baseline as additional context in Table & Despite substantial training budgets, all three RL methods remain unreliable: success is low, collisions are frequent, and the final distance to the goal remains large.
The main paper reports the matched-interface
Visualization. Figure II shows representative stagewise local-sensing rollouts on the same dungeon layout. All methods operate under the same local sensing window and matched action interface. GRL-SNAM reaches the goal with direct, efficient trajectory, while stagewise deep RL baselines are less reliable and often incur larger detours or fail to complete the task.
Across both protocols, the main difference is not the optimizer itself but the learning signal. Under matched dynamics, sensing, and action space, replacing sparse reward optimization with Hamiltonian-strucuuu supervision leads to substantially higher success and lower terminal emror.
43

---

Published as a conference paper at ICLR 2026
Table 8: Full-episode, end-to-end navigation baselines (PPO/TRPO/SAC) the dungeon point-
agent navigation task.
Algorithm
Success (%)
Mean goal dist. (m)
Collision (%)
Env. training steps
PPO
7.2
14.1
23.5
SM
TRPO
3.8
23.7
26.1
6.5M
SAC
1.5
20.2
28.3
8M
1RC-0.000
18
18
28
100
100
18
185
14
5-0.3% 0008
5-6 2R. 0008
5-3 28 000
5.
9.90
100
10
18
140
4
2.0
Figure 11: Representative dungeon rollouts under stagewise local sensing. Each panel shows the same layout with the goal (red x), start (yellow square), executed trajectory (colored by time), and the union visited sensing windows W(g:) (cyan boxes). Panel titles report episode success S and SPL (with Grid A* as the reference path length Lvet). GRL-SNAM reaches the goal with near-planner efficiency, whereas stagewise deep-RL baselines and the greedy potential field baseline fail this instance; classical planners and safety-filb filtted local methods succeed but follow more conservative routes.
This supports the view that GRL-SNAM's advantage comes primarily from learning structured energy model whose gradients encode goal attraction and obstacle avoidance, rather than from ad- ditional map access or more favorable control interface.
USE OF LARGE LANGUAGE MODELS (LLMs)
All content of the paper written by the authors. LLMs used for the aid of code implemen- tation, formatting LaTeX tables/figures, and apelling/gramman checking.
44


Reviews:
[{"review_id": "SA1XxmtZsu", "reviewer": "Reviewer_8Pzq", "summary": "The paper presents GRL-SNAM, a framework claiming to move beyond reinforcement learning (RL) by formulating navigation and mapping as Hamiltonian energy minimization on manifolds. It introduces “Differential Policy Optimization” (DPO) and claims that navigation arises as a gradient flow over learned Hamiltonians, evaluated on 2-D deformable-robot navigation tasks.", "questions": "Several notational and structural issues hinder readability:\n\nKey symbols in the reward/energy equations (Eq. 2) are undefined, including functions epsilon() and b()", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. Physically grounded formulation:\nConnects reinforcement learning with Hamiltonian optimal control, offering a structured and interpretable energy-based framework.\n\n\n2. Geometric and multi-scale design:\nPreserves energy and manifold structure through symplectic updates and coordinates multiple policies across different temporal scales.\n\n\n3. Applicability to deformable robots:\nDemonstrates potential in continuous and deformable robotic control, extending beyond traditional rigid-body navigation methods.", "weaknesses": "1. Theoretical Contributions Are Mischaracterized\n\nAlthough the paper repeatedly claims to go beyond RL or beyond Bellman optimization, the theoretical formulation remains within standard optimal control and RL principles:\nThe proposed Hamiltonian is written as:\n\n$\nH(q, p) = K(p) + P(q)\n$\n\nwhich directly corresponds to the Pontryagin Hamiltonian in optimal control with a linear dynamic system without drift.  \nIts gradients are:\n\n$$\n\\nabla_p H = f(x, u), \\quad \\nabla_q H = \\nabla_x V.\n$$\n\nis equivalent to the value-function gradient in the Hamilton–Jacobi–Bellman (HJB) framework.\n\nThus, the “Hamiltonian gradient” proposed here produces the same optimal policy direction as the value gradient in standard optimal control and RL.\n\nClaims that this method is “beyond Bellman” or “feedforward” are conceptually misleading — it effectively re-expresses the value-function gradient field under a different parameterization.\n\nHence, the theoretical novelty is overstated, and the method is a rebranding of existing value-based interpretations of RL/HJB rather than a fundamentally new control principle.\n\n2. Experiments Are Limited and Not Convincing\n\nThe experiments focus exclusively on simple 2-D environments with “a hyperelastic ring navigating among several circular obstacles”\nWhile the paper reports high success rates, these setups are not challenging:\n\nNavigation among a few separated obstacles is a standard and well-solved problem; methods like [1] (and even classical potential field methods) already achieve similar results.\n\nThe authors compare only against simple baselines (PF, CBF, A*, DWA) but not against modern RL or geometric planning baselines in complex environments.\n\nTo substantiate the claimed scalability, the authors should evaluate on realistic indoor navigation benchmarks such as Habitat, Gibson, or Matterport3D, where partial observability, long horizons, and mapping uncertainty actually challenge RL and control approaches.\n\nWithout such experiments, the empirical section does not demonstrate generalization or scalability.\n\n[1] Optimal obstacle avoidance based on the Hamilton–Jacobi–Bellman equation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ABXiyR6liz", "reviewer": "Reviewer_eoAa", "summary": "This paper introduces GRL-SNAM, a geometric reinforcement learning framework for simultaneous navigation and mapping in unknown environments. The key idea is to formulate navigation as a differential Hamiltonian system, embedding geometric priors directly into the policy dynamics without explicit value iteration or Bellman backups. By treating perception, mapping, and control under a unified Hamiltonian energy structure, the agent maintains spatial coherence and adaptively balances attraction toward goals with repulsion from obstacles.\nThe framework claims to bridge local and global navigation by allowing a continuous transition between locally reactive potential-based behaviors and globally consistent geometric flows, learned through reinforcement of energy-minimizing trajectories. Experimental validation includes multiple 2D simulated navigation tasks, showing improvements in mapping efficiency, trajectory smoothness, and robustness compared to traditional potential fields, CBF, and A*-based methods.", "questions": "How does GRL-SNAM maintain global goal consistency when only local energy gradients are available? Is there a mechanism analogous to global value propagation (e.g., a learned potential update or memory-based field fusion)?\n\nThe learning process seems to optimize parameters of the Hamiltonian rather than a standard RL policy. Could the authors clarify whether the reward signal still drives policy gradient updates, or if this is a purely energy-based optimization (akin to unsupervised RL or active inference)?\n\nHow does the system behave in environments with dynamically changing obstacles or shifting goals? Does the energy-based formulation adapt online, or does it require retraining?\n\nCould the authors compare with DRL-based navigation baselines (e.g., model-based/model-free, hierarchical RL, waypoint-/motion-based)? It would strengthen the claim that GRL-SNAM achieves similar or better coordination without an explicit hierarchy.\n\nGiven the Hamiltonian structure, could this framework naturally integrate with learned world models (e.g., latent-space dynamics) for true model-based RL on manifolds or navigation in 3D environments? It seems well-suited for that direction.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper reframes RL-based navigation as energy minimization on a manifold governed by Hamiltonian flows, avoiding discrete value propagation while retaining long-term optimality through implicit geometric consistency. This yields a clear bridge between local reactive control (as in potential fields) and global goal-directed planning (as in RL-based exploration).\n\nThe idea of using differential Hamiltonians introduces an interesting way to encode conservation and stability into DRL, improving interpretability and safety, which is sometimes the main concern in local navigation in unstructured environments.\n\nBy avoiding explicit value iteration, GRL-SNAM positions itself between model-free RL (which learns global policies but struggles locally) and geometric controllers (which react locally but lack long-term planning). This hybrid viewpoint is theoretically meaningful and could influence future works in hierarchical RL for navigation.\n\nExperiments show GRL-SNAM achieves nearly full success rates with lower mapping ratios, suggesting more efficient local exploration and better global coverage with limited sensing.", "weaknesses": "*Unclear role of learning vs. hand-crafted structure*\n\nAlthough stated as “reinforcement learning,” the optimization primarily tunes Hamiltonian parameters rather than learning a policy through interaction in the RL sense. The boundary between learning and analytic control design could be better clarified.\n\n*Insufficient analysis of global consistency*\n\nThe claim that the system enables global navigation without explicit mapping is not rigorously validated. It would be sufficient to show more whether the Hamiltonian field can produce loop-consistent paths or handle long-horizon navigation under partial observability.\n\n*Limited comparison to DRL baselines*\n\nWhile comparisons to potential fields and CBF controllers are provided, there’s no quantitative evaluation against or clarifications on any learning-/DRL-based navigation baselines (e.g., model-based/model-free, hierarchical RL, waypoint-/motion-based). Without these, it is unfair to assess whether the proposed framework is competitive in learning efficiency and generalization. Some examples include:\n- Tai, Lei, Giuseppe Paolo, and Ming Liu. \"Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation.\" 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, 2017.\n- Zhelo, Oleksii, et al. \"Curiosity-driven exploration for mapless navigation with deep reinforcement learning.\" arXiv preprint arXiv:1804.00456 (2018).\n- Liang, Jingsong, et al. \"Context-aware deep reinforcement learning for autonomous robotic navigation in unknown area.\" Conference on Robot Learning. PMLR, 2023.\n- Sathyamoorthy, Adarsh Jagan, et al. \"Terrapn: Unstructured terrain navigation using online self-supervised learning.\" 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2022.\n\n*Limited discussion on exploration strategies*\n\nThe “minimum mapping ratio” metric is novel, but it’s not clear how exploration is coordinated globally, whether the Hamiltonian field inherently guides global coverage, or if it only reacts locally to perceived gradients.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XIAtOzaQQ0", "reviewer": "Reviewer_aWJS", "summary": "This paper introduces GRL-SNAM, a geometric reinforcement learning framework for simultaneous navigation and mapping in unknown environments. The key contribution is formulating navigation as energy minimization on Hamiltonian manifolds rather than using traditional value-based RL. The system decomposes navigation into three independent policies, i.e., sensor, frame planning, and shape reconfiguration, which operate at different timescales and are unified through a shared Hamiltonian energy formulation. Instead of Bellman-based value iteration, the approach uses Differential Policy Optimization (DPO) for purely feedforward control. Empirically, the method is evaluated on 2D deformable robot navigation where a hyperelastic ring navigates through cluttered environments. It achieves near-optimal path quality with SPL=0.95 while using minimal map coverage at 10.7%.", "questions": "1. Regarding the reason of not comparing with modern RL methods, the authors claim that \"DPO has already been shown to outperform state-of-the-art policy learners\". However, this justification is insufficient since those comparisons were on different tasks. Could the authors provide stronger justification for this?\n2. Could the authors provide inspirations on extending the proposed work to more complex environment setup, like 3D or even real world?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The integration of Hamiltonian mechanics with RL presents a conceptually promising approach. The idea of avoiding value function optimization for feedforward control is innovative and the physical grounding provides strong inductive bias for navigation tasks.\n2.The paper develops a thorough and solid mathematical framework with clear definitions of multi-policy architecture, meta-Hamiltonian formulation, and provides theoretical guarantees for stability, symplectic preservation, and sample complexity.\n3. The focus on achieving high-quality navigation with minimal environment mapping addresses a practical concern in SNAM that has received limited attention in prior work.\n4. The paper provides detailed mathematical derivations, implementation specifics, and comprehensive baseline descriptions in the appendices.", "weaknesses": "1. Although the paper provides substantial theoretical contribution, the experimental validation is limited. The evaluation is restricted to 2D environments with a single robot type. This somewhat limits the generalizability and impact of the work.\n2. For presentation, frequent jumps between abstract concepts and implementation details disrupt flow, making it hard to follow. For example, Section 3.1 discusses abstract Hamiltonian optimization theory, including energy deposition by policy, then Section 3.2 abruptly introduces specific hyperelastic robot model, followed by Section 3.3 which jumps back to introduce policy abstraction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)