STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimization
Authors: Diqi He
Xuehao Gao
Hao Li
Junwei Han
Dingwen Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Vision-and-Language Navigation
Multi-Modal Reasoning
Primary area: deep_learning
TL;DR: STRIDER is a framework for zero-shot VLN-CE, featuring a SWG to constrain action space with spatial structure, and a TAR to adjust behavior based on task progress, helping agents align actions with spatial layout and task intent.

Abstract: The Zero-shot Vision-and-Language Navigation in Continuous Environments (VLN-CE) task requires agents to navigate previously unseen 3D environments using natural language instructions, without any scene-specific training. A critical challenge in this setting lies in ensuring agents’ actions align with both spatial structure and task intent over long-horizon execution. Existing methods often fail to achieve robust navigation due to a lack of structured decision-making and insufficient integration of feedback from previous actions. To address these challenges, we propose STRIDER (Instruction-Aligned Structural Decision Space Optimization), a novel framework that systematically optimizes the agent’s decision space by integrating spatial layout priors and dynamic task feedback. Our approach introduces two key innovations: 1) a Structured Waypoint Generator that constrains the action space through spatial structure, and 2) a Task-Alignment Regulator that adjusts behavior based on task progress, ensuring semantic alignment throughout navigation. Extensive experiments on the R2R-CE and RxR-CE benchmarks demonstrate that STRIDER significantly outperforms strong SOTA across key metrics; in particular, it improves Success Rate (SR) from 29\% to 35\%, a relative gain of 20.7\%. Such results highlight the importance of spatially constrained decision-making and feedback-guided execution in improving navigation fidelity for zero-shot VLN-CE.

STRIDER: Navigation via Instruction-Aligned Structural Decision Space Optimization
Diqi He Xuchao ; Hao , Junwei Han 13, Dingwen Zhang'
Northwestern Polytechnical University
Nanyyng Technological University
Chongging University of Posts and Telecommunications
https://github. com/ hww.:he66//////DDEE....
Abstract
The Zero-shot Usiionandd-Languae Navigation in Continuous Environments (VLN-CE) task requires agents to navigate previously unseen 3D environments using natural language instructions, without any scene-speeific training. A critical challenge in this setting lies in ensuring agents' actions align with both spatial structure and task intent over long-horizon execution. Existing methods often fail to achieve robust navigation due to a lack of structured decision-making and insuf- ficient integration of feedback from previous actions. To address these challenges, we propose STRIDER Instruction-Alignee Structural Decision Space Optimiza- tion), novel framework that systematically optimizes the agent's decision space by integrating spatial layout priors and dynamic task feedback. Our approach introduces two key innovations: a Structured Waypoint Generator that constrains the action space through spatial structure, and 2) Task-Alignment Regulator that adjusts behavior based on task progress, ensuring semantic alignment through- out navigation. Extensive experiments the R2R-CE and RxR-CE benchmarks demonstrate that STRIDER significantly outperforms strong SOTA across key metries; in particular, it improves Success Rate (SR) from 29% 35%, relative gain of 20.7%. Such results highlight the importance of spatially constrained decision-making making and becbbcck-guided execution in improving navigation fidelity for zero-shot VLN-CE.
1 Introduction
VLN-CE challenges agents to follow natural language instructions to navigate previously unseen 3D environments without any scene-specific training fine-tuning [42, 25, 60, 35). This task is a critical benchmark for embodied AI, requiring agents to generalize perception, reasoning, and action across diverse and dynamic scenes [3, 24, 40, 62, 47, 8). In comparison to discrete VLN tasks [3, 51, 26). VLN-CE more closely reflects real-world deployment conditions, where agents must process RGB-D inputs and make continuous movement decisions [18, 61, 21J. As result, zero-shot VLN-CE pushes the boundaries of language-grounder generalization in embodied navigation [38, 61, 35, 5].
central challenge VLN-CE lies not only in grounding instructions into perception, but also in ensuring that the agent's 's behavior remains aligned with the semantic intent of the instruction throughout the navigation process. In unfamiliar environments, agents may correctly understand the instruction yet still exhibit execution drift [47, 34, 42, 18, 35], such as stopping the target
"Equal contribution. t Corresponding author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Leavee the bedroom and take left in the hallwayy. Ge striight and go into the roum that the secund on the left. Stop in the doceway to the room.
Open-Navv
STRIDER
Step 1
Step 2
3
Step
Step3
Step
Top Down Map
Figure I: Navigation behavior comparison between STRIDER and Open-Nav [42]. Given the same instruction, Open-Nav demonstrates execution drift, such as prematurely turning away from a hallway, and accumulates deviations over time. contrast, STRIDER generates trajectories that more accurately follow the intended path and reach the goal region.
room without entering prematurely turning away from hallway, shown in Fig. 1. These failures highlight a significant gap between what the agent understands and how acts. Our key insight is that effective zero-shot VLN-CE agents must go beyond strong perception and reasoning- they must operate within an Instruction-Alignn Structural Decision Space, decision space that is explicitly structured by the environment and continuously regulated based task progress.
However, existing approaches typically rely on learned waypoint predictors or sequence-to -orennce---eeueeee policies that map instructions and visual inputs directly to actions [42, 24, 18, 8J. While effective at modeling local navigability, these models tend to ignore structured representations that capture the global layout or semantic task progression [15, 16, 41, 26, 34). Moreover, these methods often operate an open-loop fashion, inferring each action independently without feedback on prior decisions [2, 9, 56, 38, 61]. This limitation hinders their ability to assess whether actions have brought the agent closer to the goal, leading to deviations from the instruction's 's intent, especially in complex ambiguous scenes. While instruction grounding has significant progress, insufficient attention has been paid to optimizing the agent's decision space in a manner that aligns both with spatial structure and task instructions.
To address these challenges, we introduce STRIDER, a zero-shot VLN-CE framework built on principle of Instruction-Alin the observation that semantic misalignment often arises not from perceptual misunderstanding. but from the inability maintain alignment with both the spatial structure of the environment and the task's semantic progress over long-hore hong-horizon exccution [3, 233. STRIDER adopts a modeling-fing first approach: instead of directly predicting actions from visual inputs and instructions, it focuses optimizing the agent's s decision space by integrating spatial structure and task progress awareness [50]. By embedding spatial layout priors and continuous goal feedback into the agent's pocision-making process, STRIDER enables the agent to navigate within paths that are both spatially coherent and semantically aligned with the task. This dynamic adjustment of behavior reduces execution drift and improves instruction fidelity, enhancing both spatial generalization and instruction-leva adherence.
Structural Decision Space Optimizution. Our approach is grounded
STRIDER achieves this optimization through two tightly integrated modules. The Structured Waypoint Generator creates a layout-constrained action space by extracting skeletons from depth- based navigable regions [10, 36, 45]. This module ensures that the agent's movement decisions are limited to paths that are both spatially coherent and meaningful, grounded in the environment's structure. The Task- FaskmAllgnment Regulator continuously monitors task progress and adjusts the agent's 's behavior accordingly, ensuring actions remain aligned with the instruction-define goal. recalibrates behavior whenever deviations detected, while staying within the spatial constraints defined by the structured action space. Together, these modules optimize the agent's 's decision space by structuring it with spatial constraints and regulating according task progress, ultimately enabling more efficient and semantically faithful navigation.
Our contributions are summarized follows:

---

We propose STRIDER, zero-shot VLN-CE framework based on the principle of Instruction-Aligned Decision Space Optimization. STRIDER optimizes the agent's s de- cision space by integrating spatial structure and task feedback, enabling more coherent and cnaructioo-faithtii navigation behavior.
STRIDER consists of two tightly coupled modules that jointly optimize the agent's decision space: (I) A Structured Waypoint Generator that constructs a layout-constrained planning space from depth-based skeletons, embedding spatial priors into the action space; (2) A Task- Alignment Regulator that monitors semantic progress across steps and adjusts behavior accordingly, ensuring actions remain aligned with instruction goals.
We evaluate STRIDER on two standard zero-shot VLN-CE benchmarks, R2R-CE and navigation metrics,
RxR-CE, where it consistently outperforms strong baselines demonstrating the benefit of structuring and regulating the agent's 's decision space.
2 Related Work
2.1 Vision-and- Lang-uuag--- Navigation
Significant progress has been made in Vision-and Lan-Lang Navigation (VLN), especially in contin- uous settings [42, 24, 25, 18, 39, 3]. VLN approaches can be broadly categorized into supervised learning with environmeen--peecffe training and zero-shot generalization. Supervised methods, such as imitation learning and reinforcement learning [53], rely on human-annotated trajectories to train navigation policies [2, 39, 13, 58, 59, 19], often enhancing performance through visual-language alignment, attention modules, auxiliary tasks [39, 50, 62, I], and explicit or implicit 3D scene understanding and reconstruction [55, 31, 29, 30, 14, 32]. On the other hand, zero-shot methods aim to generalize to unseen scenes without task-specific fine-tuning. These approaches typically leverage instruction tuning, pretrained Vision- Llsoua-angguage Models (VLMs) or Large Language Models (LLMs) [42, 37, 46, 54, 48]. While zero-shot methods show promise, they primarily focus on local observations and immediate actions, lacking global spatial awareness and long-term planning capabil- ities. Few methods integrate feedback mechanisms to monitor task progress or correct deviations, limiting their adaptability in complex scenarios.
2.2 Decision Space Optimization
Decision Space Optimization originates from robotics and autonomous systems, where it involves structuring an agent's 's decision-male mannin within a "decision space" that integrates spatial constraints, task goals, and feedback from the environment [27, 52, 22, 33J. This structured decision allows agents to balance short-term actions with long-term objectives, improving task execution. In robotics, decision space optimization ensures that actions spatially coherent and consistent with global goals, aiding navigation and manipulation in dynamic environments [49, 43, 11]. In the field of VLN, early methods primarily map instructions to actions based on local observations but struggle with long-term planning and generalization to unseen environments [3, 13, 39]. More recent methods have incorporated memory, attention mechanisms, and reinforcement learning to improve decision-makinn action prediction and introduce curriculum learning for enhanced performance [62, 50]. However, challenges remain in maintaining global spatial awareness and ensuring long-term task alignment. STRIDER addresses these issues through Instruction-Alie Alliged Structural Decision Space Optimization. It optimizes docision-making by ensuring actions are consistent with both spatial constraints and task goals, enabling better navigation performance over long horizons.
(53, 17, 41J. These approaches also use visual-language alignment for
3 Method
In this section, we explore how VLN-CE can enhanced by structuring the decision space through spatial constraints and integrating task feedback. We begin with an overview of the VLN-CE task and introduce our proposed STRIDER framework (Sec. 3. We then present two core components of our approach: (1) the use of local perception to construct spatially organized representations that capture region connectivity (Sec.3.2), and (2) the integration of task feedback to monitor subgoal progress and regulate action selection during execution (Sec. 3.3).
3

---

Structured Waypoint Generator
Depth
Navigable Area
Skeleton Path
Structured Waypoint
Depth
Observation
Observation 6+1
Perception & Reasoning
RGB
Panoramic: Inscription: with setion roons and hilloay's
Action
RGB
Perception
nyponint doonery oyyoritt dost with lunging and open shelving trite..
Based the feedback, current tank to go in the bedroons.
Reasoning
Direction doows docnwey jeeding bedroom..... bicstmate belp cocppetts poung
Alsk-Alignment Regulator
Feedback
Feedback: agpeans that have approached the bodroom but have completaly gone it. should continue moving fonward into hedroom.
Feedback t+/
Views Comparison
Observation
Observation 64!
Figure 2: Overview of the STRIDER pipeline. The Structured Waypoint Generator constructs layout-constrained waypoint space by extracting skeleton paths from navigable depth observations. The agent performs perception and reasoning over visual descriptions and feedback to identify suitable actions in context. To maintain semantic alignment over time, the Task-Alignment Regulator compares current and previous observations and generates feedback that guides the next action.
3.1 Overview
Task Definition. the task of zero-shot VLN-CE, the agent begins at specified starting location and must reach target destination by interpreting both verbal guidance and visual observations. At each timestep. receives panoramic 3609 view composed of 12 RGB-D images captured at fixed intervals (0%, 30 ... 330%). From these observations, the agent predicts set of waypoints, candidate navigable locations, and selects one to move toward. The episode continues until the instruction is fulfilled or the agent reaches the goal.
STRIDER. Our proposed STRIDER framework follows a zero-shot VLN-CE pipeline, as illustrated in Fig. 2. At each timestep I, the agent receives RGB-D observation O, (I,, D.) and language instruction L. The Structured Waypoint Generator processes the depth D, to extract navigable regions and generates set of structured waypoints W2 organized by spatial connectivity. Subsequently, pretrained Vision Lisiuu-Language Model (VLM) describes the RGB input 4, attending to visual content in the directions of candidate waypoints W, to form a decision space A,. A Large Language Model (LLM) then reasons over the instruction L., the current decision space Ac, and the task feedback fe generated from the previous step select waypoint wi W, for movement.
After executing the action toward w the agent receives the next observation O,+1- The Task- Alignment Regulator compares O, and O.+1 using the VLM to detect progress toward the instruction goal and generates updated feedback fe+1. This feedback is leveraged in the next decision step, completing closed-loop control cycle grounded in structured perception and task alignment.
3.2 Structured Waypoint Generator
To optimize the agent's decision space in continuous environments, we generate mnyout-constrained set of candidate actions that explicitly reflect the spatial structure of the scene. Rather than relying on local navigability or unconstrained policy outputs, we propose Structured Waypoint Generator that transforms depth input into compact topological graph of navigable options. This process consists of three stages: (1) navigable region extraction, (2) topological skeleton abstraction, and (3) structured waypoint selection.

---

Navigable Region Extraction. At time step t, the agent receives panoramic RGB-D observation O, (I, where D, denotes multi-view depth input. Then we reconstruct local point cloud Pa C IRa where each point P. = (Ii, Vi, =4) represents 3D coordinate in the agent-centric reference frame. To isolate feasible movement areas, we filter for ground-level points and project them into
2D top-down plane:
24 11 (1p, E P pi(z) < on, ||p.(a,, w)1l < rr)),
where 8, is the height threshold, r is the local planning radius, and I1(-) denotes orthographic projection onto the horizontal plane. Ω defines the local traversable area around the agent.
Structured Selection
Regian Skeleton
Pixel V within skeleton
Topological Skeleton Extraction. To introduce structural priors into the local decision space, we abstract the raw traversable region Ω into a topological skeleton S; using morphological thinning [28]:
deg(v)=1
Endpoint
dog(v)=2 Midpoint
S Skeletonize(la).
(2)
where S. c Ω4 approximates the center axis of free space. Rather than interpreting navigable space as dense, unstructured area, the skele- ton captures its underlying topology by tracing the central lines of movement through open re- gions. This is analogous to how humans often form mental maps of environments based key corridors, intersections, and doorways, rather than memorizing complete spatial coverage. By reasoning over this abstract structure, the agent can better plan paths that respect the environment's layout and reduce unnecessary action noise.
dog(v)=2 Junction
Figure 3: Structured waypoint selection based on skeleton. We categorize skeleton nodes by their degree and select only degree-1 endpoints as candidate waypoints.
Structured Waypoint Selection. We model the skeleton S as an undirected graph G. = (Ve, & where nodes V; correspond skeleton pixels and edges & reflect 8-connected adjacency (including horizontal, vertical, and diagonal neighbors in the 2D grid). To reduce redundancy and focus on directionally meaningful locations, select sparse set of endpoints from the graph candidate waypoints. Specifically, we retain only nodes with degree 1:
(3) which correspond to the outermost reachable points on the local skeleton, as shown in Fig. 3. These endpoints naturally capture the agent's forward navigability and latent path divergence. Although we do not explicitly select junctions, their future branches are represented by the endpoints along each subpath.
We = foy E Vi deg(u,) = 1}
Each selected waypoint Wi E We is projected back into 3D. We input the RGB views I, into a VLM extract semantic information, focusing on visual content in the direction of each waypoint. For each wi, the VLM outputs a textual description D; of the corresponding direction. The final decision space defined as set of paired spatial- candidates:
A [w,, D,) wa W., D; VLM w.))
which couples structural layout with perceptual grounding. This layout-constrained action space forms part of the input for the LLM.
3.3 Task-Aligned Feedback Regulation
To ensure -astueetion-aligned behavior over long-horizon trajectories, we introduce feedback regulation mechanism that dynamically adjusts the agent's decision space based on recent observations and subtask progression. This module plays central role in principle of Instruction-Alie Decision Space Optimization, allowing the agent to refine its actions not only based on spatial layout but also on semantic alignment with the instruction over time.
Visual Feedback Generation. Through L.LM reasoning, the agent selects a waypoint wi from the decision space Ar, resulting in an action a, After executing ae it receives the next observation
5

---

O1+1 (1++1. Dz-1). To assess whether the agent has made progress toward the current subtask Te (derived from instruction L), we input the observation pair (O,, Op+1) and the subtask T, into the VLM to generate a feedback signal:
f(1 - VLM(Oo, O Ti)-
(5)
where J1+1 is textual reflection describing the change in scene with respect to T:-e.g., "partially entered the bedroom" or "moved away from the target." The feedback offers fine-grained progress estimation at each step, allowing the agent to detect incremental advances or errors.
Feedback-Guided Action Selection. To determine the next action, the agent reasons over the updated structured decision space Ar+1 (similarly defined in Eq. (4)), the full instruction I, and the generated feedback fi+1- The LLM outputs the next action via:
w = LLLM(AA+) Se+1, L),
a++1 Action
This loop forms closed feedsac cycle, in which action selection is continuously guided by semantic progress monitoring, As the agent moves through the environment, each observation is only encoded visually but interpreted in light of task intent, enabling corrective adjustments and reducing cumulative drift. In this way. the decision space is adaptively regulated by execution context, maintaining semantic coherence with the instruction over time. By iterating this closed-loop process for T steps, the agent produces an action sequence {01, Q2 02,...,ay' ar, that successfully navigates toward the cntructiionalignnn target.
4 Experiments
4.1 Experimental Setup
R2R-CE Dataset. We conduct experiments on the R2R-CE dataset, which extends the Room-to- Room (R2R) benchmark for visual language navigation (VLN) [3, 25J. This dataset consists of natural language instructions paired with navigation trajectories in realistic 3D indoor environments, derived from the Matterport3D dataset [4]. We follow the settings of OpenNav [42], conducting tests on 100 randomly selected episodes from the dataset. In these experiments, we leverage both VLM and LLM perform zero-shot navigation. Our goal is to fully leverage the generalization and reasoning capabilities of these pretrained models, enabling them to adapt to the current task without any additional training.
We also use the RxR-CE dataset, which extends the Room-Across-Row (RxR) benchmark with similar challenging conditions [26, 25]. RxR features longer and more diverse instructions across multiple languages and emphasizes global navigation capabilities. The CE variant (Continuously Evolving) simulates viewpoint changes and environmental variations, making suitable for evaluating the robustness and generalization of navigation agents under distribution shifts.
RxR-CE Dataset.
Evaluation metrics. We evaluate navigation performance using standard metrics. Navigation Error (NE) measures the shortest-pai path distance between the agent's 's final position and the goal. Success Rate (SR) is the percentage of episodes where the agent stops within 3 meters of the goal. Success weighted by Path Length (SPL) balances success and path efficiency [3]. Normalized Dynamic Time Warping (NDTW) reflects the similarity between the predicted and reference trajectories [20]. Oracle Success Rate (OSR) indicates the best possible success assuming the agent stops optimally along its path [25]. Trajectory Length (TL) records the average length of agent trajectories. Soft-DTW (SDTW) is a relaxed version of DTW that tolerates slight deviations in trajectory matching [12].
Implementation details. All experiments are conducted in simulated VLN-CE environments. At each step, the agent receives an RGB-D observation, where the RGB input is resized to 244 x 244 x 3 and the depth map to 256 256. Structured waypoints are generated by extracting skeletons from depth without relying on any pretrained waypoint predictor. For perception and feedback generation, as the Vision-Language Model (VLM). The action selection process is guided by GPT-4o, which reasons over the instruction, structured perception, and feedback to choose the next waypoint. Our VLM and L.M are accessed via API rather than deployed locally; for local deployment using open-source models, please refer to Open-Nav [42].
we use Owen-VL-Me

---

Table 1: Performance comparison on the R2R-CE dataset. The dash "-" indicates that the corresponding metric was not reported the original work. Bold indicates the best result. We report the relative percentage change of our method compared to the previous SOTA in parentheses. Red denotes improvement, while Green indicates degradation.
Method
NE
NDTW
OSR
SR
SPL1
Supervised Learning
CMA [25]
6.92
50.77
45
37
32.17
RecBERT [57]
5.8
54.81
57
48
43.22
BEVBert [1]
5.13
61.40
64
60
53,41
ETPNav [2]
5,15
61.15
58
52
52.18
HNR [55]
4.42
67
61
51
Zero-Shot
Random [42]
8.63
34.08
12
2
1.50
LXMERT [25]
10.48
18.73
22
2
1.87
DiscussNav-GPT4 [38]
7.77
42.87
15
11
10.51
Open-Nav-Llama3.1 [42]
7.25
44.99
23
16
12.90
Open-Nav-GPP4 [42]
6.70
45.79
23
19
16.10
SmartWay [46]
7.01
51
29
22.46
Ours
6.91(3.1%)
51.8(13.2%%
(9(2.55%)
35(20.6%)
30.30(34.9%)
Table 2: Performance comparison on the RXR-CE dataset. The dash "." indicates that the corresponding metric was not reported in the original work. Bold indicates the best result. We report the relative percentage change of our method compared to the previous SOTA in parentheses. Red denotes improvement, while Green indicates degradation.
Method
NEG
NDTW
SDTW1
SR1
SPL
Supervised Learning
LAW [44]
11.04
37.0
8.0
10.0
9.0
ILNCBERT [19]
8.98
46.7
27.1
23.7
GridMM [56]
8.42
48.2
33.7
36.3
30.1
ETPNav [2]
5.64
61.9
45.3
54.8
44.9
WS-MGMap [6]
9.83
15.0
12.1
HNR [55]
5.51
63.56
47.24
56.39
46.73
Zero-Shot
Na [7]
16.8
6.3
CA-Nav [5]
10.37
13.5
5.0
19.0
6.0
Ours
11.19(7.9%)
30.1(122.9%)
8.9(7%))%)
21.2(11.5%)
9.6(52.3%)
4.2 Main Results on Zero-Shot VLN-CE
We evaluate STRIDER on two standard zero-shot VLN-CE benchmarks: R2R-CE (Tab. 1) and RxR-CE (Tab. 2). Across both datasets, STRIDER consistently outperforms prior zero-shot methods on key metrics such as SPL, SR, and NDTW, indicating more reliable goal completion and higher trajectory fidelity. While supervised methods benefit from task-specific training, STRIDER remains competitive despite operating without fine-tuning or environment-specffic adaptation.
On R2R-CE, STRIDER achieves substantial improvements in SPL and NDTW over all prior zero-shot models, driven by two key design factors. The Structured Waypoint Generator constrains the agent's behavior to fayout-consisterr paths, reducing detours and spatial drift, while the Task-Alignment Regulator provides real-time feedback to maintain semantic alignment and correct deviations. This combination enables STRIDER to balance spatial feasibility and instruction adherence, leading to gains across both path-quality and goal-completion metrics. On the more diverse RxR-CE benchmark, STRIDER continues to outperform other zero-shot methods, though with narrower margins margins-likelyy likely due to RxR's higher linguistic and geographic variability. Even such settings, STRIDER'S structured decision space offers strong prior that compensates for ambiguity, while the feedback
7

---

Table 3: Effect of Structured Waypoint Generator (SWG) under different node degree con- figurations. Bold indicates the best result. We report the relative percentage change compared to the baseline in parentheses. Red denotes improvement, while Green indicates degradation. The gray-shaded row denotes the primary experimental configuration used our main results.
SWG
Node Degree
NE
NDTW1
OSRI
SR1
SPL1
7.19
48.78
29
24
21.07
6.91(3.9%)
51.87
(34.5%))
35(45.8%
30.30(43.8%)
>
7.34(2.1%)
19.88(2.3%)
33(13.8%)
28(16 28(16.7%)
25.02(18.7%)
#
2
(.83()
51.12(4.8%)
38(31.0%
33(37.5%)
29.21
Original Waypolnt Predictor
Structured Waypolnt
Original Waypolnt Predictor
Structured Waypolnt
O
O
O
O
Curdisbae Wasgainl
History Wingoint
Taret Ptilion
Agent Pteilio
Trijecoryy
Region Skeleton
Novigable
Figure 4: Comparison between original waypoint predictor and Structured Waypoint Gener- ator. Our Structured Waypoint Generator extracts layout-consistent waypoints that align with the environment's topology, resulting in trajectories that
goal-directed and spatially coherent.
loop supports adaptive planning. Together, these elements demonstrate the robustness and generality of decision space optimization under zero-shot conditions.
is worth mentioning that STRIDER does not achieve the lowest Navigation Error (NE), which is expected given its emphasis instruction alignment and structural feasibility over exact endpoint proximity, The agent may stop at semantically appropriate locations slightly offset from the goal, reflecting preference for coherent, interpretable paths shortcut-baa based precision acceptable tradeoff essruction-guided navigation tasks.
4.3 Ablation Study
We conduct ablation experiments analyze the impact of STRIDER's two core modules: the Structured Waypoint Generator and the Task-Alignment Regulator. For each module, we present both quantitative comparisons and qualitative visualizations to highlight their effects on navigation performance.
4.3.1 Effect of Structured Waypoint Generator
We evaluate different configurations of the Structured Waypoint Generator to understand how layout- aware guidance affects agent behavior. As shown in Tab. 3, we consider two key design choices: (1) whether to use the Structured Waypoint Generator, and (2) how to guide waypoint selection based on skeleton connectivity-ce. (deg(v:)>2) or both. We use the original waypoint predictor trained on the R2R dataset the baseline for STRIDER without the Structured Waypoint Generator. We present the visualization comparison between the original waypoint predictor and the Structured Waypoint Generator in Fig. 4.
using only endpoint nodes (deg(v;)=I)) higher-degree junctions
Using degree-1 nodes (endpoints) yields the best overall performance across all metrics, as these points typically lie at corridor tips or face the goal direction, providing clear guidance with minimal ambiguity. In contrast, using only high-degree nodes (>22), such as junctions, leads to degraded performance with higher NE (7.34) and lower SR/SPL, likely due increased behavioral uncertainty.

---

Table 4: Effect of the Task-Alignment Regulator (TAR) on navigation performance. Bold indicates the best result. We report the relative percentage change compared the baseline in parentheses. Red denotes improvement, while Green indicates degradation. The gray-shaded row denotes the primary experimental configuration used in our main results.
TAR
NEJ
NDTWT
OSR
SR
SPL1
X
6.77
51.06
42
29
26. 11
6.91
(1.67)1.6%)
(9(1.)%)
35(20.7%
30.30 16.0%)
Without Feedback
Thoucht: have completed the provious instrmction Keep walking along the kitchen area towards the doorway; right and ar area." To execute this, I need to select a direction that
The next instruction
kitchen
effectively me right.
2
Action Prediction:
2
Candidate
With Feedback
Fsedbackk have entered the kitchen which sugpests that the instrac tion 'Keep walking alung the kitchen ares towards the doorway" is not fullyy completod. need cuntinue moving forward follow yoth along kitchen. Thooeht: Based on the foetback. I need to contimme executing the currett instraction ensue that the task of walking along kitchen area towards the doorway fully comsleted.
Start Position
Target Posttion
Agent Position
Trujectory
Refereace Fath
ction Prediction:
Instruction: W Walk towards the kitchen area. Keep walking along the kitchen area towards the doorway. Turn right and exit the kitchen area walking across the small living room area and into the bedroom to you left.
Figure 5: Comparison of agent behavior under no-feedback and feedback-driven execution strategies. Without feedback, the agent prematurely infers task completion, resulting in incorrect action (Action 2). With feedback, the agent leverages the intermediate state to refine its understanding. yielding more semantically consistent action (Action 1) aligned with the instruction.
Combining non-2-degree nodes (i.c., #2) provides a good compromise, incorporating both endpoints and informative junctions to achieve strong performance (NE: 6.83, SPL: 29.21) at the cost of a slightly longer trajectory. Based on these results, we adopt the degree-1 configuration the default in our main experiments.
4.3.2 Effect of Aligmenig Regulator
As shown in Tab. 4, enabling the Task-Alignment Regulator (TAR) leads to consistent improvements across SPI,, SR, and NDTW, indicating that feedback-driven behavior adjustment helps the agent maintain semantic alignment and recover from drift during long-horizon navigation. While NE slightly increases (6. (6.77...91), this reflects a more conservative execution pattern, where the agent prioritizes instruction fidelity and avoids premature termination, ultimately leading to higher task success and trajectory consistency. We illustrate the impact of the Task-Alignment Regulator on navigation behavior in Fig. 5. Overall, TAR enhances instruction fidelity without compromising spatial plausibility and adopted in our main model configuration.
4.3.3 Effect of Model-Agnostic Design
STRIDER is not tied to any single model and can operate effectively across components of similar capabilities. To verify this, we conduct experiments using various VLMs of different sizes and providers. As shown in Tab. 5, GPT-4o achieves the best performance in terms of NE (6.75) and SR (36). However, STRIDER also shows competitive results using similar models, such as Qwen-VI. VL- Max and Qwen2.5- an overall solid performance. Additionally, STRIDER continues to deliver good results with smaller models like Qwen2.5-VL-32B and Swen2.5-VL-7BB. The consistent performance across different
which perform strongly in NDTW, OSR and SR, while still maintaining

---

Table 5: Ablation different VLMs. We test our method using various VLMs of different sizes and capabilities. Bold indicates the best result. The gray-shaded row denotes the primary experimental configuration used in our main results.
VLM
TL
NEL
NDTWT
OSR
SRT
SPL
Qwen-VL-Max VL-Max
8.13
6.91
51.87
39
35
30.30
Qwen2.5-VL-72B
8.30
6.78
51.99
39
34
29.07
Qwen2.5-VL-22B
8.56
7.12
48.02
33
24.20
Qwen2.5-VL-7B
8.92
7.46
46.35
29
24
21.12
GPT-40
8.01
6.75
50.12
39
36
31.37
Gemini-2.5-Pro
8.34
6.92
51.35
37
29.85
Gemini-2.5-Flash
7.68
7.08
49.87
34
29
25.30
Claude-3.5
7.81
6.86
52.10
36
33
29.40
Claude-4
8.22
7.14
45.25
31
29
26.10
Table 6: Applying SWG to BEVBert. We compare the vanilla BEVBert with the one applying our SWG. Bold indicates the best result. We report the relative percentage change compared the baseline in parentheses. Red denotes improvement.
Method
OSR
SR
SPL1
BEVBert
4.57
67
59
50
BEVBert w/ SWG 4.37(4.3%
70(4.4%)
61.3.3%
53(6.0%)
models suggests that STRIDER's design amplifies the strengths of the underlying model without relying on a specific model.
For primary experiments, selected Qwen-VL-Man VL-Max. This choice highlights STRIDER's model- agnostic design, which enables our method to work with any similar model and ensures that its results driven by the strength of our approach, rather than reliance on specific foundation model.
4.3.4 Applying SWG to Fine-Tuned Models
further assess the effectiveness of Structured Waypoint Guidance (SWG) fine-tuned models. Specifically, we incorporate our SWG module into the BEVBert model by replacing its original waypoint prediction mechanism. This modification leads to improvements across all key evaluation metrics, shown in Table 6.
Even in fine-tuned settings, SWG's ability integrate environmental structure as a strong prior helps compensate for uncertainties in the navigation task. This further reinforces the versatility and effectiveness of SWG in different contexts, demonstrating that structured priors can be seamlessly integrated into existing models, thereby improving their robustness and reliability across range of navigation tasks.
5 Conclusion
We presented STRIDER, zero-shot VLN-CE framework that optimizes agent behavior through Instruction-Alinaee Structural Decision Space Optimization. By combining a Structured Waypoint Generator with Task-Alignment Regulator, STRIDER enables agents navigate in complex, un- seen environments in a manner that is both structurally coherent and semantically faithful to natural language instructions. Extensive experiments on VLN-CE benchmarks demonstrate that our approach outperforms strong zero-shot baselines across multiple metrics, with ablations confirming the comple- mentary contributions of structural planning and feedback-drive regulation. This work highlights the importance of structuring and modulating the decision space for long-horizon instruction following and opens up future directions integrating richer spatial priors, adaptive subgoal modeling, and instruction-awe aware exploration strategies.
10

---

6 Acknowledgement
This work was supported by the National Natural Science Foundation of China under Grant 62293543, Grant 62322605.
References
[1] Dong An, Yuankai Qi, Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, and Jing Shao. Bevbert: Multimodal map pre-training for language-guided navigation. arXiv preprint 2tti::2212.0438333 2022.
[2] Dong An. Hanqing Wang, Wenguan Wang. Zun Wang, Yan Huang, Keji He, and Liang Wang. Eip- nav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[3] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-an and -anguaga navigation: Interpreting navigation instructions in real environments. In Proceedings of the IEEE conference Computer Vision and Puttern Recognition, pages 3674-3683, 2018.
[4] Angel Chang, Angela Dai, Thomas Funkhouser, Maciej Halber, Matthias Niessner, Manolis Savva, Shuran Song, Andy Zeng, and Yinda Zhang. Matterport3d: Learning from rgb-d data in indoor environments. arXiv preprint arXiv: 170.00667990588 2017.
[5] Kehan Chen, Dong An, Yan Huang, Rongtao Xu, Yifci Su, Yonggen Ling, Ian Reid, and Liang Wang. Constraint-aware zero-shot vision-language navigation in continuous environments. arXiv preprint arXiv:2412.10137, 2024.
[6] Peihao Chen, Dongyu Ji, Kunyang Lin, Runhao Zeng, Thomas Li, Mingkui Tan, and Chuang Gan. suaely-supervised multi-granularii map learning for vision-and-languaae navigation. Advances in Neural Information Processing Systems, 35:38149 1433819--38611 2022.
[7] Peihao Chen, Xinyu Sun, Hongyan Zhi, Runhao Zeng. Thomas H Li, Gaowen Liu, Mingkui Tan, and Chuang Gan. A2 nav: Action-aware zero-shot rohot navigation by exploiting lioon-and-language ability of foundation models. arXiv preprint arXiv:2308. 07997, 2023.
[8] Shizhe Chen, Picrre-oouss Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for vision-and-language navigation. In Proceedings of the Advances in Neural Information Processing Systems, volume 34, pages 5834-5847, 2021.
[9] Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act local: Dual-scale graph transformer for anddo--and-o aad-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16537-16547, 2022.
[10] Xinyi Chen, Boyu Zhou, Jiarong Lin, Yichen Zhang, Fu Zhang, and Shaojie Shen. Fast 3d sparse topological skeleton graph generation for mobile robot global planning. In 2022 IEEE/RS/ International Conference on Intelligent Roboty and Systems (IROS), pages 10283-10289. IEEE, 2022.
[11] Zihao Chen, Kunhong Li, Haoran Li, Zhiheng Fu, Hammo Zhang, and Yulan Guo. Metric localization for lunar rovers via cross-view image matching. Visual Intelligence, 2(1):12, 2024.
[12] Marco Cuturi and Mathieu Blondel. Soft-dtw: a differentiable loss function for time-series. In International conference on machine learning, pages 894 903. PMLR, 2017.
[13] Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Kerg-Kiikp,,rick, Kate Saenko, Dan Klein, and Trevor Darrell. Speaker-follower models for vision-and-lann and-language navigation. Advances in neural information processing systems, 31, 2018.
[14] Yuanyuan Gao, Yalun Dai, Hao Li, Weicai Ye, Junyi Chen, Danpeng Chen, Dingwen Zhang, Tong He, Guofeng Zhang, and Junwei Han. Cosurfgs: Collaborative 3d surface gaussian splatting with distributed learning for large scene reconstruction. arXiv preprint arXiv:2412.1 1761.1 2024.
[15] Pierre-Loui Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. Airbert: Tn- domain pretraining for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1634-1643, 2021.
[16] Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning a generic agent for vision-an and language navigation via pre-training, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13137-13146, 2020.
11

---

[17] Yicong Hong, Cristian Rodriguez-Opazo. Qi Wu, and Stephen Gould. Sub-instruction aware vision-and- language navigation. arXiv preprint arXiv:2004. 02707, 2020.
[18] Yicong Hong, Zun Wang, Qi Wu, and Stephen Gould. Bridging the gap between learning in discrete and continuous enviromments for iision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15439-15449, 2022.
[19] Yicong Hong. Qi Wu, Yuankai Qi, Cristian Rodriguez- Opazo, and Stephen Gould. A recurrent vision-and language bert for navigation. arXiv preprint arXiv:201 /.3392, 2020.
[20] Gabriel Ilaarco, Vihan Jain, Alexander Ku, Bugene Ic, and Jason Baldridge. General evaluation for instruction conditioned navigation using dynamic time warping. arXiv preprint urXiv:1907.05446, 2019.
[21] Seongjun Jeong, Gi-Cheon Kang, Joochan Kim, and Byoung-Tak Zhang. Zero-shot vision-and- languagg navigation with collision mitigation in continuous environment. arXiv preprint arXiv:2410.17267 2024.
[22] Leslie Pack Kaelbling and Tomás Lozano-Pérez. Integrated task and motion planning in belief space. The International Journal of Robotics Research, 32(9-100 10): 1 194 1227, 2013.
[23] Liyiming Kc, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, and Siddhartha Srinivasa. Tactical rewind: Self-correction via backtracking in vision-and-land lan--laa navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6741-6749,
2019.
[24] Jacob Krantz, Aaron Gokaslan, Dhruv Batra, Stefan Lee, and Oleksandr Maksymets. Waypoint models for cerructiin-guidded navigation continuous environments. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15162-15171, 2021.
[25] Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav- nav-graph: Vision-and languuge navigation in continuous environments. In Proceedings of the European Conference on Computer Vision, pages (04-120,220 2020.
[26] Alexander Ku, Peter Anderson, Roma Patel, Eugene Ie, and Jason Baldridge. aoom-across-rooe Multilingual aasoon--nd-aanuuaae navigation with dense spatiotemporal grounding, arXiv preprint arXiv:2010.07954. 2020.
[27] Steven M Valle. Planning algorithms. Cambridge university press, 2006.
[28] Ta-Chih Lee, Rangasami Kashyap, and Chong-Nam Chu. Building skeleton models via 3-d medial surface axis thinning algorithms. CVGIP: graphical models and image processing.5666)442222 56(6))462-478, 1994.
[29] Hao Li, Yuanyuan Gao, Haosong Peng, Chenming Wu, Weicai Ye, Yufeng Zhan, Chen Zhao, Dingwen Zhang, Jingdong Wang, and Junwei Han. Dgtr: Distributed gaussian turbo-rcconstruction for sparse-viee view vast scenes. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 203-213. IEEE, 2025.
[30] Hao Li, Yuanyuan Gao, Chemming Wu, Dingwen Zhang, Yalun Dai, Chen Zhao, Haocheng Feng, Errui Ding, Jingdong Wang, and Junwei Han. Ggrt: Towards pose-free generalizable 3d gaussian splatting in real-time. In European Conference on Computer Vision, pages 325-341. Springer, 2024.
[31] Hao Li, Roy Qin, Zhengyu Zou, Diqi He, Bohan Li, Bingquan Dai, Dingewn Zhang, and Junwei Han. Lang- surf: Language-embedder surface gaussians for 3d scene understanding. arXiv preprint 27X2::4412.17635. 2024.
[32] Hao Li. Dingwen Zhang, Yalun Dai, Nian Liu, Lechao Cheng, Jingteng Li, Jingdong Wang, and Junwei Han. Gp-nerf: Generalized perception nerf for context-award 3d scene understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 21708 21718, 2024.
[33] Wenji Li, Zhaojun Wang, Ruitao Mai, Pengxiang Ren, Qinchang Zhang, Yutao Zhou, Ning Xu, JiaFan Zhuang, Bin Xin, Liang Gao, et al. Modular design automation of the morphologics, controllers, and vision systems for intelligent robots: a survey. Visual Intelligence, 1(1):2, 2023.
[34] Xiangyang Li, Zihan Wang, Jiahao Yang, Yaowei Wang, and Shuqiang Jiang. Kerm: Knowledge enhanced reasoning for lado--uuee navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2583-2592 2023.
[35] Xiwen Liang, Liang Ma, Shanshan Guo, Jianhua Han, Hang Xu, Shikui Ma, and Xiaodan Liang. Cornav: Autonomous agent with self corrected planning for sero-shot vision-and and-aand-nnnn language navigation. arXiv preprint arXiv:2306. 10322, 2023.
12

---

[36] Jyh-Ming Lien, John Keyser, and Nancy M Amato. Simultaneous shape decomposition and skeletonization. In Proceedings of the 2006 ACM symposium Solid and physical modeling, pages 219-228, 2006.
[37] Yuxing Long, Wenzhe Cai, Hongeheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: sero-shot system for generic instruction navigation in unexplored environment. arXiv preprint orXiv:2406. 04882, 2024.
[38] Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. Discuss before moving: Visual language navigation via multi-expert discussions. In Proceedings of the International Conference on Robotics and Automation, pages 1 17380-17387, 2024.
[39] Chih- Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, and Caiming Xiong. Self- Seff-monitoring navigation agent via auxiliary progress estimation. arXiv preprint arXiv:1901.088 03035, 2019.
[40] Chi Ma, Zuxuan Wu, Ghassan AlRegib, Caiming Xiong, and Zsolt Kira. The regretful agent: Hcuristic-aided navigation through progress estimation. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 6732-6777770 6740, 2019.
[41] Arjua Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, and Dhruv Batra. Improving ssion-and-language navigation with image-texi pairs from the web. In Proceedings of the European Conference on Computer Vision, pages 259-274, 2020.
[42] Yanyuan Qiao, Wenqi Lyu, Hui Wang, Zixu Wang, Zerui Li, Yuan Zhang, Mingkui Tan, and Qi Wu. Open- nav: Exploring zero-shot vision-and and--nnd-dnaa language navigation in continuous environment with open-source Ilms. arXiv preprint (XXx::2409187999 2024.
[43] Nathan D Ratliff, David Silver, and Andrew Bagnell, Learning to search: Functional gradient techniques for imitation learning. Autonomous Robots, 27:25-53, 2009.
[44] Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, and Angel X Chang. Language-aligen waypoint (law) supervision for vision-and- and-languaee navigation in continuous environments. arXiv preprint arXiv:2109. 15207, 2021.
[45] Hossein Memarzadch Sharifipour, Bardia Yousefi, and Xavier PV Maldague. Skeletonization and recon- struction based on graph morphological transformations. arXiv preprint arXiv:2009.09977 2020.
[46] Xiangyu Shi, Zerui Li, Wenqi Lyu, Jiatong Xia, Feras Dayoub, Yanyuan Qiao, and Qi Wu. Smartway: Enhanced waypoint prediction and backtracking for zero-shot laignaan navigation. arXiv preprint arXiv:2503, 10069, 2025.
[47] Chan Hee Song. Jiaman Wu, Clayton Washington, Brian Sadler, Wei-Lun Chao, and Yu Su. LIm-planner: Few-shot grounded planning for embodied agents with large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2998-3009 2023.
[48] Yuliang Sun, Xudong Zhang. and Yongwei Miao. review of point cloud segmentation for understanding 3d indoor scenes. Visual Intelligence, 2(1):14, 2024.
[49] Aviv Tamar, Yi Wu, Garrett Thomas, Sergey Levine, and Picter Abbeel. Value iteration networks. Advances in neural information processing systems, 29, 2016.
[50] Hao Tan, Licheng Yu, and Mohit Bansal. Learning navigate unseen environments: Back translation with environmental dropout. arXiv preprint arXiv: 04 195, 2019.
[51] Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog navigation. In Conference Robot Learning, pages 394 406. PMLR, 2020.
[52] Mare Toussaint. Logic-geometric programming: An optimization--aseed approach to combined task and motion planning. In IJCAI, pages 1930-1936, 2015.
[53] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Yuan-Fang Wang, William Yang Wang, and Lci Zhang. Reinforced cross-modal matching and self-supervised imitation learning for languaee navigation. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pages 6629-6638, 2019.
[54] Yizhou Wang, Longguang Wang, Qingyong Hu, Yan Liu, Ye Zhang, and Yulan Guo, Panoptic segmentation of 3d point clouds with gaussian mixture model in outdoor scenes. Visual Intelligence, 2(1):10, 2024.
[55] Zihan Wang, Xiangyang Li, Jiahao Yang. Yeqi Liu, Junjie Hu, Ming Jiang, and Shuqiang Jiang. Lookahead exploration with neural radiance representation for contiquous vision vision-langue language navigation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13753 13762, 2024.
13

---

[56] Zihan Wang, Xiangyang Li. Jiahao Yang. Yeqi Liu, and Shuqiang Jiang. Gridmm: Grid memory map for vision-and-language navigation. In Proceedings of the IEEE/CVF International Conference Computer Visian, pages 15625-15636, 2023.
[57] Richard Wu. Recbert: Semantic recommendation engine with large language model enhanced query segmentation for k-nearest neighbors ranking retrieval. Intelligent and Converged Networks, 5(1):42-52. 2024.
[58] Jiazhao Zhang, Kunyu Wang. Rongtao Xu, Gengze Zhou, Yicong Hong. Xiaomeng Fang. Qi Wu, Zhizheng Zhang, and He Wang. Navid: Video-basee vlm plans the next step for vision aed--nn--aaa language navigation. arXiv preprint arXiv:2402. 2024.
[59] Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang. Jing Zhang, Zhongyuan Wang, Shanghang Zhang, and Renjing Xu. Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and- -sion-and-language navigation. arXiv preprint 2314v 2025.
[60] Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang- Towards learning generalist model for embodied navigation. Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 13624-13634, 2024.
[61] Gengze Zhou, Yicong Hong, Qi Wu. Navgpt: Explicit reasoning vision and language navigation with large language models. Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 7641-7649, 2024.
[62] Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. Vision- laaiga- navigation with self- supervieed auxiliary reasoning tasks. In Proceedings of the IEEEJCVE conference on computer vision and pattern recognition, pages 10012-10022, 2020.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The key assertions presented in the abstract and introduction effectively encapsulate the research goals, methodologies, and contributions of this work, all of which are further developed and rigorously supported throughout the whole paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We provide the limitation discussion the supplementary materials.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
15

---

Answer: [NA]
Justification: Our paper is based on experimental results and does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proof' in the paper should be numbered and cross referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: We provide detailed description of the method and experimental setup in the paper, to ensure the reproducibility of the methods.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
16

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes)
Justification:
We provide the code in the supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https:
wttps.cc/ppblccc//cccccccpcppppp guwic//gueee...e CoodeSummii www.nnoolicciooeooo for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones omitted from the script and why.
At submission time, preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
The experimental setting and details are provided in the Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: Our paper does not include error bars or other statistical experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
17

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The information is provided in the Sec. 4.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: htt////eerr../.eee cc/public/Ethi ctwieetteteiieeeeeeeeeeeeeeeeeee dinneesieelleee
Answer: [Yes]
Justification:
Our research complies with the NeurIPS Code of Ethics in all aspects.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer:
[Yes]
Justification: provide this the supplementary material.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: Our paper poses no such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The existing assets covered in the paper are subject to their license and terms of use, and are explicitly cited and described.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide the code in the supplementary material.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/cooe/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification: Please Section 4.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy (https 220 for what should or should not be described.
21


Reviews:
[{"review_id": "fTlMB6z2b7", "reviewer": "Reviewer_NbAg", "summary": "This paper proposes a structural framework STRIDER for navigation tasks in a zero-shot manner. Specifically, STRIDER introduces a structural module to generate waypoint candidates and employs a regulator to adjust agent actions throughout action execution, ensuring alignment with the instruction's intent. Experimentally, STRIDER achieves significant performance improvements on both the R2R-CE and RxR-CE benchmarks.", "questions": "- Why are different models (Qwen and GPT4o) used for feedback generation and action selection?\n- Could the authors provide a comparison of inference time to better understand the computational cost of STRIDER?\n- Given that the feedback plays an important role in STRIDER’s performance, it would be valuable to include an ablation study using feedback of varying quality, like from open-source models of different sizes, to assess the impact on performance.\n- What stopping strategy does STRIDER employ? Is the decision to stop explicitly predicted by the model?", "limitations": "Yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "### Strengths\n- The paper is well-written and easy to follow.\n- STRIDER addresses an important real-world challenge: long-horizon navigation aligned with human intent. It demonstrates promising improvements on two benchmarks.\n### Weaknesses\n- While STRIDER improves SR and SPL across both benchmarks, its performance on NE is notably worse than the baselines. The authors discuss this betwen lines 259–263, but the explanation does not address my questions. In particular, it remains unclear why STRIDER struggles more than other methods with stopping at semantically appropriate locations—an issue that commonly affects zero-shot models.\n- Additionally, STRIDER shows a 12% drop in OSR on R2R-CE compared to baselines. This degradation needs further discussion.", "was_revised": true, "final_justification": "Generally, this work introduce multiple VLM calls to perform long-horizon navigation aligned with human intent. I mainly concern about the diverse performance gains and the extra costs. The current stop strategy could also be improved. Therefore, I maintain my rating of bordline reject, but acknowledge the potential of this work.", "rebuttal": "**Weakness 1:**\n\nWhile NE (Navigation Error) is a meaningful metric, it is not the most decisive indicator of navigation quality, as shown in recent works such as Smartway, which accepts slightly higher NE (7.01), in exchange for improved SR, SPL, and NDTW. This is because NE measures the Euclidean distance from the agent's final position to the goal, but it does not account for whether the agent successfully followed instructions or generated a coherent, goal-directed path. In contrast, SR (Success Rate) reflects whether the agent stops next to the goal, SPL (Success weighted by Path Length) evaluates path efficiency while accounting for success, and NDTW (Normalized Dynamic Time Warping) captures trajectory fidelity concerning the reference path. These metrics are more indicative of instruction-following accuracy, semantic alignment, and task-level success, especially in instruction-following navigation. To this end, recent works mainly use SR, SPL, and NDTW to evaluate the overall performance of the methods.\n\n**Weakness 2:**\n\nWhile STRIDER’s OSR (39) on R2R-CE is lower than SmartWay's (51), we believe this difference is largely attributable to SmartWay’s use of a backtracking mechanism and significantly longer trajectories (TL = 13.09) as shown in Table 1. OSR considers a path successful if any visited point comes within a threshold distance of the goal. Therefore, methods with longer and more exploratory trajectories inherently have higher OSR, even if they do not stop correctly or follow instructions precisely. In contrast, STRIDER produces concise, forward-directed trajectories (TL = 8.47) without backtracking, yet still achieves a strong OSR. This suggests that our agent’s paths are well-aligned with the goal, not by chance, but through accurate perception and structured reasoning. STRIDER also ranks highest in SR and SPL, further indicating its strength in task completion and trajectory quality, rather than relying on incidental proximity. \n\nWe see this as a design trade-off: STRIDER avoids unnecessarily long or exploratory paths, and instead focuses on semantically faithful, efficient navigation.\n\n**Table 1: Comparison of TL and OSR**\n\n| Method              | TL    | OSR↑ | SR↑ |\n|---------------------|-------|------|-----|\n| DiscussNav          | 6.27  | 15   | 11  |\n| Open-Nav-Llama3.1   | 8.07  | 23   | 16  |\n| Open-Nav-GPT4       | 7.68  | 23   | 19  |\n| Smartway            | 13.09 | 51   | 29  |\n| STRIDER         | 8.47  | 39   | 35  |\n\n**Question 1:**\n\nWe would like to emphasize that our architecture adopts a modular design, where VLMs are used for perception and LLMs for reasoning. This design is not tied to any specific models and is model-agnostic across components of similar capabilities.\n\nTo verify this, we conduct an ablation study across multiple VLMs of varying sizes and providers (Table 2). The results demonstrate that while stronger models do yield better performance, STRIDER maintains competitive results even with smaller or different models, confirming that the design generalizes well across models of similar capacities and does not rely on any single pretrained model. \n\n**Table 2: Ablation on different VLMs**\n\n| VLM                 | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|---------------------|------|------|-------|------|-----|--------|\n| Qwen-VL-Max         | 8.13 | 6.91 | 51.87 | 39   | 35  | 30.30  |\n| Qwen2.5-VL-72B      | 8.30 | 6.78 | 51.99 | 39   | 34  | 29.07  |\n| Qwen2.5-VL-32B      | 8.56 | 7.12 | 48.02 | 33   | 28  | 24.20  |\n| Qwen2.5-VL-7B       | 8.92 | 7.46 | 46.35 | 29   | 24  | 21.12  |\n| GPT‑4o          | 8.01 | 6.75 | 50.12 | 39   | 36  | 31.37  |\n| Gemini-2.5-Pro      | 8.34 | 6.92 | 51.35 | 37   | 34  | 29.85  |\n| Gemini-2.5-Flash    | 7.68 | 7.08 | 49.87 | 34   | 29  | 25.30  |\n| Claude-3.5          | 7.81 | 6.86 | 52.10 | 36   | 33  | 29.40  |\n| Claude-4            | 8.22 | 7.14 | 45.25 | 31   | 29  | 26.10  |\n\n**Question 2:**\n\nWe measure the average inference time per episode for both STRIDER and the Open-Nav baseline. STRIDER takes 3.9074 minutes per episode compared to 3.6606 minutes for Open-Nav. Despite incorporating additional reasoning steps via the Task-Alignment Regulator, STRIDER introduces only a minor runtime overhead. We believe this slight increase is acceptable given the clear performance improvements, and it confirms that STRIDER remains computationally practical for real-world applications.\n\n**Question 3:**\n\nThank you for the insightful suggestion. We evaluate the effect of feedback quality by conducting an ablation study using open-source VLMs of varying sizes and capabilities, including Qwen2.5-VL-72B, 32B, and 7B (Table 2). The results confirm that higher-quality feedback (e.g., from larger or more capable models) improves performance. However, STRIDER continues to offer improvements over baselines even when paired with smaller models, suggesting that our structural guidance provides added value regardless of model scale.\n\n**Question 4:**\n\n STRIDER does not rely on the model generating an explicit “stop” signal. \n\nSpecifically, the instruction is first broken down into a series of subtasks by the LLM, and the maximum number of execution steps is set to be equal to the number of subtasks, with a hard minimum of 6 steps to ensure completion. Once the max step is reached, the system stops.\n\nThis approach avoids reliance on potentially unreliable stop predictions from the model and ensures stable behavior across tasks."}, {"review_id": "TpSdeDax7I", "reviewer": "Reviewer_mq4X", "summary": "The paper proposes STRIDER, a zero-shot Vision-and-Language Navigation (VLN-CE) framework that optimizes an agent’s decision space using spatial layout priors and dynamic task feedback. Key innovations include a Structured Waypoint Generator (constraining actions via environmental skeletons) and a Task-Alignment Regulator (adjusting behavior based on task progress). Experiments on R2R-CE and RxR-CE show STRIDER improves Success Rate from 29% to 35% and outperforms SOTA on metrics like SPL and NDTW. The approach addresses long-horizon execution drift by integrating spatial structure and feedback loops.", "questions": "1. Are there limitations in skeleton extraction for complex topologies or non-standard layouts (e.g., open spaces)?\n\n2. Can the task-alignment regulator handle fine-grained semantic differences in instructions?\n\n3. Why do some candidate waypoints in Figure 2 have a degree greater than 1？\n\n4. In the ablation study of Table 3, what is the performance without using TAR?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. Novel Framework Design: STRIDER’s dual-module approach (structured waypoints + feedback regulation) effectively combines spatial constraint and semantic alignment, addressing core VLN-CE challenges.\n\n2. Strong Empirical Validation: Comprehensive experiments on two benchmarks (R2R-CE, RxR-CE) demonstrate significant performance gains over zero-shot baselines, with SR improvements and higher SPL.\n\nWeaknesses:\n\n1. Computational Overhead: The pipeline involves multiple model calls (VLM for descriptions, LLM for reasoning), which could impact real-time execution. \n\n2. The authors should indicate in the paper which LLMs or VLMs are used by other zero-shot methods. Additionally, it is important to clarify how much of the performance gain remains when the same LLM or VLM is used across methods.", "was_revised": true, "final_justification": "Due to the concerns on the computational overhead and the lack of deep analysis of limitations in skeleton extraction for complex topologies or non-standard layouts in simulator environments, I recommend the borderline reject rating.", "rebuttal": "**Weakness 1:**\n\nWe understand the reviewer’s concern. While STRIDER’s pipeline involves multiple calls to a VLM and an LLM, our measurements show that the overall inference time per episode remains comparable to the Open-Nav baseline. Specifically, STRIDER requires 3.9074 minutes per episode, compared to 3.6606 minutes for Open-Nav. We believe this minor overhead is acceptable given the substantial gains in navigation performance, and it demonstrates that STRIDER is practical for offline applications.\n\n**Weakness 2:**\n\nThank you for the helpful suggestion. To assess how much of STRIDER’s performance gain comes from model design rather than model choice, we conduct controlled experiments where STRIDER and prior methods use the same LLM and VLM. As shown in Table 1, STRIDER outperforms the other two baselines under identical LLM and VLM settings, confirming that the performance gain stems from our framework design.\n\n**Table 1: Comparison Under Identical VLM and LLM**\n\n| Method     | VLM                | LLM     | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|------------|--------------------|---------|------|------|--------|------|-----|--------|\n| DiscussNav | InstructBLIP + RAM | GPT-4   | 6.27 | 7.77 | 42.87  | 15   | 11  | 10.51  |\n| STRIDER    | InstructBLIP + RAM | GPT-4   | 8.24 | 7.21 | 46.61  | 29   | 23  | 19.05  |\n| Open-Nav   | Spatial-Bot + RAM  | GPT-4   | 7.68 | 6.70 | 45.79  | 23   | 19  | 16.10  |\n| STRIDER    | Spatial-Bot + RAM  | GPT-4   | 8.31 | 6.82 | 49.30  | 31   | 26  | 22.37  |\n\n**Question 1:**\n\nOur method is designed with a focus on indoor navigation scenarios, where environments typically exhibit well-defined spatial constraints and structured topologies (e.g., hallways, rooms, doorways). In these settings, skeleton extraction yields meaningful connectivity graphs that align with human spatial reasoning and support robust, layout-consistent planning.\n\nWe agree that in open or non-standard layouts, such as large open spaces without clear boundaries, the extracted skeleton may become overly simplified or lack informative branching. However, for the indoor environments targeted in VLN-CE benchmarks like R2R-CE and RxR-CE, which are exactly our tasks, our abstraction is well-suited and highly effective.\n\n**Question 2:**\n\nYes, the Task-Alignment Regulator is explicitly designed to handle fine-grained instruction alignment by generating feedback based on scene changes. At each step, the regulator compares pre- and post-movement observations, and reasons over whether the intended semantic progression (e.g., fully entering a room vs. partially entering) has been achieved.\n\nThis allows the agent to detect subtle differences and adjust accordingly in the next decision step. As shown in Fig. 5 in the full paper, the regulator distinguishes between visually similar but semantically distinct states.\n\n**Question 3:**\n\nWe appreciate the reviewer’s attention to detail. The candidate waypoints in Figure 2 are intended for illustrative purposes only, to convey the general idea of skeleton-based waypoint sampling. As shown, the figure includes various keypoints along the skeleton, including some with degree > 1, to better visualize the underlying graph structure. However, as described in Section 3.2 (and shown in Figure 3), our actual method only retains candidate waypoints with degree = 1, i.e., skeleton endpoints.\n\n**Question 4:**\n\nThe performance without using the Task-Alignment Regulator (TAR) is reported in Table 4 in the full paper, specifically in the first row labeled “×”. We intentionally separate this analysis from the Structured Waypoint Generator (SWG) ablation in Table 3 in the full paper to isolate the individual contribution of each component."}, {"review_id": "illUqBm2WX", "reviewer": "Reviewer_iwMP", "summary": "This paper introduces STRIDER, a framework for zero-shot Vision-and-Language Navigation in Continuous Environments (VLN-CE). STRIDER combines a *Structured Waypoint Generator* that encodes spatial priors into the action space, with a *Task-Alignment Regulator* that dynamically adjusts actions based on feedback about task progress. The authors report performance improvements across two VLN-CE benchmarks (R2R-CE and RxR-CE), claiming that STRIDER provides more structured, instruction-aligned navigation without fine-tuning.", "questions": "Typos:\n* Page 2, Line 77: \"**An** Task-Alignment Regulator\" -> \"**A** Task-Alignment Regulator\"\n* Page 9, Line 297-298: \"navigate complex, unseen environments\" -> \"navigate **in** complex, unseen environments\"\n\nSuggestions\n*  The LLM selects a waypoint from a candidate set, but it's unclear if this includes high-level reasoning (e.g., subgoal decomposition) or just ranking visual directions. The authors could detail the **reasoning scope** handled by GPT-4o.", "limitations": "See Weaknesses#4.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### ***Strengths***\n\n* The Structured Waypoint Generator is a clever use of skeletonization and topological abstraction to impose environmental constraints without explicit training.\n\n\n### ***Weakness***\n\n1. **Model Selection Justification**\n\n   - The paper uses Qwen-VL-Max as the VLM and GPT-4o as the LLM, but does not **justify these choices**. Why not use a more recent VLM such as Qwen2-VL-72B, especially since it has shown strong results in VLN tasks?\n\n   - It remains unclear whether STRIDER’s performance gains are **architecture-agnostic** or dependent on these specific models. This limits the generalizability of the method.\n\n2. **Lack of Baseline Comparison for Waypoint Generation**\n\n   - The \"original waypoint generator\" is frequently mentioned but **never clearly referenced** in the main text. What are its core mechanics, and how does it compare structurally to the proposed method?\n\n   - There is **no direct comparison under a controlled setup** where Task-Alignment Regulator is held constant, and only the waypoint generation method varies. Such an experiment is essential to validate the structured waypoint hypothesis.\n\n3. **Missing Prompting System Details**\n\n   - Given that the entire method operates in a zero-shot prompting-based setup, **the absence of details on the prompt engineering** used with GPT-4o is a critical omission.\n\n   - How do these prompts compare to those used in **Open-Nav** or **SmartWay**? SmartWay integrates history reflection and allows backtracking — capabilities not clearly discussed in STRIDER. A deeper comparison here would be informative.\n\n4. **Zero-Shot Performance Remains Far from Supervised Upper Bound**\n\n   - Although the improvements in zero-shot settings are appreciated, the **gap with supervised models remains significant** (e.g., SR of 35 vs. 61 in R2R-CE).\n\n   - As zero-shot VLN-CE is still relatively immature, it is debatable whether this research constitutes a **paradigm-shifting contribution** or a well-executed empirical exploration. The paper could better contextualize its significance within current research trends, where fine-tuning and hybrid systems are dominant.", "was_revised": true, "final_justification": "Thanks for the rebuttal. \nThe added experiment about different VLMs is appreciated. \nMy qustions regarding the details of prompts and the definition of original waypoint generator are resolved. \n\nMy major concern still remains that the framework replies heavily on superior VLMs. And it remains unclear how the SWG would influence the fine-tuning based methods. Therefore, I choose to keep my score.", "rebuttal": "**Weakness 1.1:**\n\nWe select to use Qwen-VL-Max (latest) because it is the most recent (updated on 25-04) and stable release. In our experiments, it produces clear, concise descriptions under our prompt design, showing comparable performance with other VLMs.\n\nFor the LLM, although most baselines adopt GPT-4, we choose GPT-4o due to its more consistent formatting, reduced verbosity, faster and more stable response under API usage.\n\n**Weakness 1.2:**\n\nWe would like to clarify that STRIDER is fundamentally architecture-agnostic. Its design centers around a structured planning-regulation pipeline that can operate with a variety of models.\n\nWe present ablation results in Table 1 using different VLMs. The results demonstrate that while stronger models do yield better performance, STRIDER maintains competitive results even with smaller or different models, confirming that the design generalizes well across models of similar capacities and does not rely on any single pretrained model.\n\n**Table 1: Ablation on different VLMs**\n\n| VLM                 | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|---------------------|------|------|-------|------|-----|--------|\n| Qwen-VL-Max         | 8.13 | 6.91 | 51.87 | 39   | 35  | 30.30  |\n| Qwen2.5-VL-72B      | 8.30 | 6.78 | 51.99 | 39   | 34  | 29.07  |\n| Qwen2.5-VL-32B      | 8.56 | 7.12 | 48.02 | 33   | 28  | 24.20  |\n| Qwen2.5-VL-7B       | 8.92 | 7.46 | 46.35 | 29   | 24  | 21.12  |\n| GPT‑4o          | 8.01 | 6.75 | 50.12 | 39   | 36  | 31.37  |\n| Gemini-2.5-Pro      | 8.34 | 6.92 | 51.35 | 37   | 34  | 29.85  |\n| Gemini-2.5-Flash    | 7.68 | 7.08 | 49.87 | 34   | 29  | 25.30  |\n| Claude-3.5          | 7.81 | 6.86 | 52.10 | 36   | 33  | 29.40  |\n| Claude-4            | 8.22 | 7.14 | 45.25 | 31   | 29  | 26.10  |\n\n**Weakness 2.1:**\n\nThank you for pointing this out. In the paper, our reference to the “original waypoint generator” corresponds to the learning-based waypoint predictor used in prior VLN works. This predictor is typically trained on the R2R dataset using RGB or depth inputs to regress a set of candidate waypoints (angles and distances). It operates as a perception network with no explicit structural grounding, relying entirely on visual cues and learned priors. We use the version from Open-Nav in all comparison experiments in the full paper.\n\nIn contrast, STRIDER replaces this component with a geometry-based, layout-consistent skeleton extraction process that requires no training and offers interpretable and spatially constrained waypoint candidates.\n\n**Weakness 2.2:**\n\nWe appreciate the reviewer’s suggestion, and we would like to clarify that Table 3 in the full paper already includes the requested controlled comparison.\n\nSpecifically, the first row (marked with \"×\" under SWG) uses the original learning-based waypoint predictor from Open-Nav, while keeping other components unchanged. In contrast, the remaining rows apply our structured waypoint generator (SWG) under different configurations. As shown, enabling SWG instead of the original waypoint predictor yields consistent improvements across all metrics, confirming the effect of structured waypoint design.\n\n**Weakness 3.1:**\n\nThank you for pointing this out. Our prompting setup for GPT-4o follows the prompt templates from Open-Nav. We intentionally adopt these prompts without major modifications to ensure a fair comparison with prior methods and to isolate the contribution of our structural planning architecture. Since our focus is not on prompt engineering for LLMs, we kept this component consistent across experiments.\n\n**Weakness 3.2:**\n\nSTRIDER adopts the same prompt as Open-Nav, which includes historical observations, previous decisions, and prior reasoning steps, along with the current view. While SmartWay supports explicit backtracking when the agent detects failures, STRIDER follows a forward-only design. \n\n**Weakness 4.1:**\n\nWe acknowledge the performance gap between STRIDER and fully supervised models. However, it is important to highlight that STRIDER operates entirely in a zero-shot manner without any task-specific training. This design is particularly valuable for practical deployments, where collecting annotated navigation data is often infeasible. \n\nDespite this constraint, STRIDER achieves strong results across two challenging datasets (R2R-CE and RxR-CE). As shown in Table 1 and Table 2 in the full paper, STRIDER improves SR and SPL significantly (e.g., +20.6% SR and +34.9% SPL over baseline on R2R-CE; +11.5% SR and +52.3% SPL on RxR-CE), demonstrating its robust generalization to diverse environments and languages.\n\nWe believe this performance offers a complementary perspective to zero-shot VLN.\n\n**Weakness 4.2:**\n\nWe appreciate the reviewer’s broader reflection on the maturity of zero-shot VLN-CE. While we agree that this research area is still developing, we believe this is what makes STRIDER a timely and meaningful contribution. \n\nSTRIDER offers a structural shift in zero-shot VLN by combining skeletonized spatial abstraction and action regulation. It complements existing approaches by offering an interpretable alternative that can generalize across environments without fine-tuning. While not aiming to replace fine-tuning or hybrid systems, STRIDER opens a new direction for zero-shot, interpretable navigation that we believe will grow increasingly relevant as foundation models continue to evolve.\n\n**Question 1:**\n\nThank you for pointing out the typos. We appreciate your attention to detail and have carefully proofread the manuscript to fix such issues.\n\n**Question 2:**\n\nIn STRIDER, the LLM’s role extends beyond ranking visual cues. GPT-4o is responsible for interpreting and decomposing the full instruction, reviewing the agent’s historical observations, past decisions, and prior reasoning. In this sense, the LLM performs structured, high-level reasoning grounded in both spatial layout and linguistic intent."}, {"review_id": "6XIj61uzCR", "reviewer": "Reviewer_RNwD", "summary": "This paper addresses the task of Zero-shot Vision-and-Language Navigation in Continuous Environments (VLN-CE), where an agent must follow natural language instructions to navigate unseen 3D spaces without any scene-specific training or fine-tuning. The authors propose STRIDER, a novel framework that structures the agent’s decision space using spatial layout priors—via a Structured Waypoint Generator—and dynamically regulates agent behavior through task progress feedback—via a Task-Alignment Regulator. STRIDER is evaluated on the R2R-CE and RxR-CE benchmarks, achieving substantial improvements over state-of-the-art zero-shot methods, particularly in Success Rate (SR) and Success weighted by Path Length (SPL), thereby demonstrating the effectiveness of spatially-constrained and feedback-driven navigation.", "questions": "Along with the queries mentioned in the \"Major weaknesses\" part, it would be great if the authors could address the following:\n\n1. What is the per-step latency introduced by skeletonization and LLM reasoning? Can STRIDER operate in real time? How does it compare to Open-Nav with an API call instead of a local deployment?\n2. How sensitive is STRIDER to the choice of VLM/LLM? Would a smaller or domain-specific language model still yield benefits?\n3. Have the authors observed cases where the Skeleton graph omits valid paths or where feedback misleads the agent due to issues in VLM? How are such issues mitigated?", "limitations": "No limitation is mentioned explicitly.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Major strengths:\n\n1. The paper identifies a key gap in zero-shot VLN-CE: existing agents often drift from the intended instruction due to unstructured decision spaces and lack of integrated feedback, particularly in long-horizon tasks.\n2. STRIDER achieves state-of-the-art zero-shot performance on both R2R-CE and RxR-CE benchmarks.\n3. The ablation studies—especially on the Task-Alignment Regulator—clearly demonstrate its consistent positive impact on navigation metrics, with an insightful discussion of metric trade-offs (e.g., slightly higher Navigation Error (NE) due to more conservative path execution).\n\nMajor weaknesses:\n\n1. The framework’s dependence on a VLM for semantic descriptions and an LLM for action selection may obscure the core contributions; it remains unclear how much of the observed gains stem from STRIDER itself versus from the use of powerful pretrained models. Further ablations with varying backbones would help clarify this.\n2. STRIDER’s success heavily relies on the robustness of the underlying VLM and LLM. If the VLM misinterprets visual scenes (e.g., under poor lighting or challenging viewpoints, as noted in the supplementary), or if the LLM provides inaccurate progress feedback, errors may compound. An analysis quantifying the impact of incorrect feedback on overall performance is needed.", "was_revised": true, "final_justification": "Thanks for the rebuttal. While I acknowledge the experiments, one major bottleneck I find is the model's over-reliance on the capabilities of the underlying VLM, which may hinder its more real-time application. Given this, I prefer to stay on my previous rating.", "rebuttal": "**Weakness 1:**\n\nThank you for the thoughtful comment. To disentangle the contribution of STRIDER’s framework from the underlying pretrained models, we conduct controlled comparisons using the same LLM and VLMs across STRIDER and baseline methods. As shown in Table 1, STRIDER outperforms the other two baselines under identical LLM and VLM settings, confirming that the performance gain stems from our framework design.\n\n**Table 1: Comparison Under Identical VLM and LLM**\n\n| Method     | VLM                | LLM     | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|------------|--------------------|---------|------|------|--------|------|-----|--------|\n| DiscussNav | InstructBLIP + RAM | GPT-4   | 6.27 | 7.77 | 42.87  | 15   | 11  | 10.51  |\n| STRIDER    | InstructBLIP + RAM | GPT-4   | 8.24 | 7.21 | 46.61  | 29   | 23  | 19.05  |\n| Open-Nav   | Spatial-Bot + RAM  | GPT-4   | 7.68 | 6.70 | 45.79  | 23   | 19  | 16.10  |\n| STRIDER    | Spatial-Bot + RAM  | GPT-4   | 8.31 | 6.82 | 49.30  | 31   | 26  | 22.37  |\n\n**Weakness 2:**\n\nThank you for raising this important point. We agree that the robustness of STRIDER depends in part on the reliability of its underlying VLM and LLM components. To evaluate the impact of incorrect feedback, we conduct a controlled experiment where we randomly corrupt the progress feedback provided to the LLM during reasoning. As shown in Table 2, this experiment offers a concrete quantification of STRIDER’s tolerance to feedback noise and supports its practical usage even in imperfect settings.\n\n**Table 2: Robustness to incorrect feedback**\n\n| Method  | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|---------|------|------|-------|------|-----|--------|\n| STRIDER | 8.21 | 7.24 | 45.62 | 27   | 24  | 20.54  |\n\n**Question 1:**\n\nSTRIDER introduces a per-step latency of approximately 34.2754 seconds, only slightly higher than Open-Nav’s 32.1105 seconds. In terms of overall runtime, STRIDER takes 3.9074 minutes per episode, while Open-Nav takes 3.6606 minutes. Notably, STRIDER’s waypoint generation is faster, requiring only 0.4001 seconds compared to 0.8892 seconds for Open-Nav. These results demonstrate that STRIDER remains efficient and practical for real-world use.\n\n**Question 2:**\n\nWe clarify that the capabilities of the underlying VLM/LLM are fundamental to the success of STRIDER. \n\nOur method is designed to leverage and amplify the strengths of powerful models like GPT-4o,  which help improve decision robustness. This makes the method less sensitive among models of similar capability, e.g., between GPT-4 and GPT-4o.\n\nWe present ablation results in Table 3 using different VLMs. The results demonstrate that while stronger models do yield better performance, STRIDER maintains competitive results even with smaller or different models, confirming that the design generalizes well across models of similar capacities and does not rely on any single pretrained model.\n\n**Table 3: Ablation on different VLMs**\n\n| VLM                 | TL   | NE↓  | NDTW↑ | OSR↑ | SR↑ | SPL↑  |\n|---------------------|------|------|-------|------|-----|--------|\n| Qwen-VL-Max         | 8.13 | 6.91 | 51.87 | 39   | 35  | 30.30  |\n| Qwen2.5-VL-72B      | 8.30 | 6.78 | 51.99 | 39   | 34  | 29.07  |\n| Qwen2.5-VL-32B      | 8.56 | 7.12 | 48.02 | 33   | 28  | 24.20  |\n| Qwen2.5-VL-7B       | 8.92 | 7.46 | 46.35 | 29   | 24  | 21.12  |\n| GPT‑4o         | 8.01 | 6.75 | 50.12 | 39   | 36  | 31.37  |\n| Gemini-2.5-Pro      | 8.34 | 6.92 | 51.35 | 37   | 34  | 29.85  |\n| Gemini-2.5-Flash    | 7.68 | 7.08 | 49.87 | 34   | 29  | 25.30  |\n| Claude-3.5          | 7.81 | 6.86 | 52.10 | 36   | 33  | 29.40  |\n| Claude-4            | 8.22 | 7.14 | 45.25 | 31   | 29  | 26.10  |\n\n**Question 3:**\n\nWe have indeed observed occasional cases where the skeleton graph misses viable paths or where feedback from the VLM introduces minor errors.\n\nTo mitigate these issues, we enforce a maximum step-size limit of 1.5 meters during navigation, which ensures fine-grained control and prevents large deviations. As a result, even if an error occurs at one step, the agent does not stray far from the intended path and can often correct itself in subsequent steps after gaining a broader view.\n\nOur robustness evaluation, shown in Table 2 in the supplemental material, further supports this claim: when subjected to mid-trajectory perturbations that displace the agent from its planned waypoint, STRIDER demonstrates better recovery."}]

Author rebuttal:

Meta-review: This paper presents a method to perform instruction-guided navigation in real world environments (e.g. VLN-CE). Towards this goal, the authors propose a framework that makes heavy use of VLMs and LLMs to reason about where the agent should navigate to next and propose a structured-waypoint generator to create navigable options.

After the rebuttal, the remaining concerns of the reviewers fall into two categories:
1. The heavy use of VLMs/LLMs makes the method slow and possibly not applicable to real-time applications.
2. The method is sensitive to the choice of VLM/LLM.
3. The method relies heavily on the strength of modern VLMs/LLMs.

The AC disagrees with the reviewers.

1. While the method is not applicable to real-time applications, its inference time is not meaningfully different than existing methods, such as DiscussNav[1] and Open-Nav[2] (Tab 1 in response to NbAg). It is also worth noting that, in the view of the AC, zero-shot VLN-CE has low enough success rates that performance, not inference time, is the current bottleneck for deployment.
2. The sensitivity of methods to the choice of VLM or LLM is a common issue and present in many other published works (both in VLN-CE and more broadly). As pointed out by the authors in the rebuttal, Open-Nav[2] and CA-Nav[3] show similar sensitivity to the choice of VLM and LLM. Further, the majority of this sensitivity appears to stem from the underlying capabilities of the model rather than over-reliance on a specific model, e.g. the proposed method achieves very similar results with Qwen-VL-Max, Qwen2.5-VL-72B, GPT-4o, and Gemini-2.5-Pro (Tab 2 in response to NbAg).
3. While the proposed method does heavily rely on the strength of modern VLMs/LLMs, this is only an issue if the improvements compared to existing work come solely from the choice of VLM/LLM. Experiments provided in the rebuttal show that while some of the improvements do come from these models, other significant improvements come from how the authors use these models (Table 1 in response to RNwD and mq4X).

Turning to the strengths of the paper, here the AC agrees with the reviewers. Specifically, reviewers noted the importance of the task (NbAG), the novelty and elegance of the proposed method (mq4X, iwMP, RNwD), the strength of its improvements over existing methods (mq4X, RNwD), and the value of the ablation studies presented (RNwD).

Overall, this paper presents a valuable contribution to the VLN-CE community, especially the more nascent zero-shot sub-community, and the criticisms levied against it have either been rebutted or do not warrant rejection. As such, the AC recommends accepting this paper.


[1] https://arxiv.org/abs/2309.11382, ICRA 2024

[2] https://arxiv.org/abs/2409.18794, ICRA 2025

[3] https://arxiv.org/abs/2412.10137
Decision: Accept (poster)