Active Test-time Vision-Language Navigation
Authors: Heeju Ko
Sungjune Kim
Gyeongrok Oh
Jeongyoon Yoon
Honglak Lee
Sujin Jang
Seungryong Kim
Sangpil Kim
Venue: NeurIPS 2025 poster ()
Keywords: Vision-Language Navigation
Active Learning
Test-time Adaptation
Primary area: applications

Abstract: Vision-Language Navigation (VLN) policies trained on offline datasets often exhibit degraded task performance when deployed in unfamiliar navigation environments at test time, where agents are typically evaluated without access to external interaction or feedback. Entropy minimization has emerged as a practical solution for reducing prediction uncertainty at test time; however, it can suffer from accumulated errors, as agents may become overconfident in incorrect actions without sufficient contextual grounding. To tackle these challenges, we introduce ATENA (Active TEst-time Navigation Agent), a test-time active learning framework that enables a practical human-robot interaction via episodic feedback on uncertain navigation outcomes. In particular, ATENA learns to increase certainty in successful episodes and decrease it in failed ones, improving uncertainty calibration. Here, we propose mixture entropy optimization, where entropy is obtained from a combination of the action and pseudo-expert distributions—a hypothetical action distribution assuming the agent's selected action to be optimal—controlling both prediction confidence and action preference. In addition, we propose a self-active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions. As a result, the agent stays actively engaged throughout all iterations, leading to well-grounded and adaptive decision-making. Extensive evaluations on challenging VLN benchmarks—REVERIE, R2R, and R2R-CE—demonstrate that ATENA successfully overcomes distributional shifts at test time, outperforming the compared baseline methods across various settings.

Active Test-time Vision-Language Navigation
Heeju Ko'
Sung June Kim' Gyeongrok Oh' Seungryong Kim*
Jeongyoon Yoon' Sangpil Kim 1x
Honglak Lee² Sujin Jang'
Korea University 2 University of Michigan Samsung Al Center, DS Division KAIST AI
Abstract
Vision-Language Navigation (VLN) policies trained on offline datasets often ex- hibit degraded task performance when deployed in unfamiliar navigation environ- ments at test time, where agents are typically evaluated without access to external interaction or feedback. Entropy minimization has emerged as a practical solution for reducing prediction uncertainty test time; however, it can suffer from mulated errors, as agents may become overconfident incorrect actions without sufficient contextual grounding. To tackle these challenges, introduce ATENA (Active TEst-time Navigation Agent), a test-time active learning framework that enables practical human-robot interaction via episodic feedback on uncertain navigation outcomes. In particular, ATENA learns to increase certainty in suc- cessful episodes and decrease it in failed ones, improving uncertainty calibration. Here, we propose mixture entropy optimization, where entropy is obtained from combination of the action and pseudo-expert distributions hypothetical ac- tion distribution assuming the agent's 's selected action to be optimal-controllinng controlling both prediction confidence and action preference. In addition, we propose a self- active learning strategy that enables an agent to evaluate its navigation outcomes based on confident predictions. As a result, the agent stays actively engaged throughout all iterations, leading to well-grounded and adaptive decision-making. Extensive evaluations on challenging VLN Becthbaarrs-REERRRR R2R-CE Ce-ddsoottrateee that XTENA successfully overcomes distributional shifts at test time, outperforming the compared baseline methods across various settings.
R2R, and
Introduction
Vision-Language Navigation (VLN) is fundamental multimodal task in embodied Al systems, which requires an agent to interpret natural language instructions and navigate through complex visual environments [T]. Despite recent advancements in VLN, distributional shifts between offline training and online testing environments remain a critical challenge for robust and reliable deployment [2] 3. To address this issue, many prior works focus on enhancing generalizability during offline training to better handle potential domain shifts 4 S o However, these approaches are limited in addressing real-world variability, as collecting diverse expert demonstrations across environments is often impractical. Therefore, test-time adaptation (TTA) the ability to directly adapt to test-time environments--is crucial for real-world robotic navigation.
Test-Time Adaptation (TTA) refines pre-trained model during inference using unsupervised sig- nals-such prediction entropy [7], consistency (8), pseudo-labels 9 offerring practical yet challenging approach to improving robustness at test time. Entropy minimization is one of the widely accepted TTA strategies, based the assumption that greater model certainty correlates with improved accuracy during inference [7 10 [ However, applying entropy minimization uniformly
"Corresponding author: spk7 korea.ac.kr
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

across all decision points in sequential tasks such as VLN may cause the policy to overfit to failure patterns, thereby increasing the likelihood of incorrect actions. Consequently, the policy accumulates emoos throughout iterations and loses resilience on failure cases. Thus, blindly increasing prediction certainty without considering the navigation status leads to suboptimal behaviors.
How, then, can we provide the contextual cues necessary for a VLN agent to properly leverage entropy as a meaningful signal test time? To answer this, we propose an active learning (AL) [12 [3]
strategy, enabling the agent to query a human oracle for necessary contextual labels. Here, we must consider practical constraints in the online test-time navigation setting for utilizing human feedback: (1) Latency- human involvement should not introduce delays during navigation rollout; and (2) Accessibility- human input must be intuitive, requiring minimal expertise and effort. Therefore, is unrealistic to expect human feedback at the same level of detail as stepwise expert demonstrations in VL.N at test time.
To address these practical concerns, we define the active label as an episodic, binary evaluation indicating navigation success or failure, rather than requiring detailed stepwise supervision. Inspired by the uncertainty sampling paradigm in AL [5] we design the agents to inquire feedback whenever the average action uncertainty throughout each navigation task exceeds predefined threshold. Given the sparsity of the feedback, we introduce a novel technique called mixture entropy optimization (MEO) to effectively leverage it. Specifically, based on the binary outcome, we guide entropy optimization by minimizing entropy for successful navigation and maximizing it for failed ones. Here, entropy is derived from mixture of two distributions: an action distribution, representing the likelihood assigned to each possible action, and a pseudo-expert distribution, a one-hot probability distribution centered on the agent's chosen action, assuming this action as optimal. By combining these two distributions, MEO not only controls the certainty of the decisions but also explicitly suppresses incorrect actions and encourages actions that led to successful navigation.
Additionally, we introduce novel paradigm of self-active learning (SAL), enabling the navigation agent remain actively engaged throughout all iterations for continuous feedback. Traditional AL methods typically request human feedback only when the model prediction is uncertain, potentially overlooking the subtle errors that are hidden beneath high confidence in certain predictions. In contrast, our method allows the agent to determine the navigation outcome by itself in relatively certain predictions. This is achieved through a self- preficcediition head, initialized at test time and trained during streaming test episodes using both human-providee As result, the agent receives continuous guidance for the direction of entropy optimization, which is crucial for precise adaptation. Ultimately, SAL reduces reliance on human intervention, thereby improving the agent's autonomy and robustness in online test-time environments.
labels and the agent's own predicted outcomes.
We name our overall framework as ATENA (Active TEst-time Navigation Agent), and validate its effectiveness through comprehensive evaluation on challenging VLN benchmarks: REVERIE [17]. R2R [1] and R2R-CE [18). ATENA achieves significant gains over the underlying target policies and outperforms strong TTA baselines. Our empirical results and in-depth analysis indicate that ATENA effectively addresses test-time distribution shifts and provides strong foundation for future research on active human-robot interaction vision-and- -nd-aagg navigation.
The contributions of this work are summarized as follows:
We introduce ATENA, the first active learning framework for online VLN that leverages human input to guide entropy- based optimization.
Mixture entropy optimization enhances confidence calibration by explicitly suppressing incorrect actions and encouraging desired actions.
The self-active learning phase provides a strategic solution to provide continuous active labels, with reduced burden of human labeling in online environment.
2
Related Work
2.1
Vision-Language Navigation
Vision-Language Navigation (VLN) is pivotal task of bridging human communications with embodied Al system [19 21 The sequential natures of the decision making process in VI.N led
carly research to adopt recurrent neural network-based architectures [ 22 23]. Following works

---

utilized the Transformer network 24 to capture complex multimodal dependencies and achieved substantial performance gains
28 29 However, these offline training methods merely anticipate domain shifts and suffer performance degradation when the online navigation environment deviates from the training distribution. To overcome the discrepancy, large-language models came into play as zero-shot navigation agents, but their reasoning capabilities without fine- tuning have yet to yield reliable performances (30) 31
25 27
32
33
34
35| 36]. Recently, another
line of research has focused online test-time adaptation of trained policies, using either unsupervised entropy minimization [37] or feedback-based reinforcement learning [38]. In this work, explore how the core principle of VLN human-robot interaction can be effectively leveraged to facilitate online test-time adaptation.
2.2 Entropy-based Test-time Adaptation
Entropy minimization is widely adopted learning objective in domain adaptation [3 and semi-supervised learning 42
Recently, entropy minimization has emerged as a foundational technique in test-time adaptation (TTA) due to its simplicity and effectiveness in the absence of labeled target data. 45 46.. The core idea is encourage the model to make confident predictions by minimizing the entropy of output distributions at test time, assuming that well-adapted models should be confident in in-distribution samples. Tent (7] introduced lightweight yet effective approach that minimizes prediction entropy by updating only batch normalization parameters during test time. Building upon this, numerous studies across various fields began integrating entropy minimization into their TTA strategies [47 488
43 44
10 I1. In VIN, ESTTA (37 extends entropy minimization by accounting for the sequential and episodic nature of the task. Although effective in many settings, blindly minimizing the entropy can lead to the propagation of overconfident mistakes, making particularly problematic in sequential tasks like VLN where early errors can cascade work addresses the problem by enabling agents to actively query oracles for feedback on uncertain navigation outcomes, providing crucial guidance for antropy-basce test-time adaptation.
49 [0]. This
2.3 Active Learning
Active Learning (AL) is a machine learning strategy designed efficiently reduce labeling costs by selectively querying labels for the most uncertain or informative data points [2] 51 52 13].
Traditional AL methods primarily utilize uncertainty sampling, prioritizing data points when the model's predictive confidence is low; typical metrics for quantifying uncertainty include entropy, margin sampling, and least-confidence measures [5 [6]. Initially focused on relatively simple classification tasks, AL techniques have progressively evolved to tackle increasingly complex, real- [6]. Recent advancements further extend AL concepts into TTA, enabling models to dynamically adapt during inference by utilizing uncertainty estimates, thus reducing the reliance on extensive retraining or large amounts of labeled data (57] Inspired by these developments, our research pioneers the integration of Active
world scenarios [ 54
55
Adaptation into VLN,
effectively overcoming practical constraints in real-world navigation.
3 Method
3.1 Task Description
Vision- Language Navigation (VLN) tasks agent with interpreting natural language instructions I to navigate through a visual environment. Starting from an initial visual observation oo, at each timestep I, the agent perceives visual observation or, selects an action a, according to its policy ma. and transitions into the next state. Repeating this process until the agent selects a stopping action produces a trajectory - = {(o., ax) ) where T is the total number of steps taken. In this work, specifically consider online VLN scenario where the navigation policy encounters stream of test instructions and environments during deployment.
3.2 Overview
Our proposed framework, ATENA (Active TEst-time Navigation Agent), enables active learning in online VI.N by integrating human guidance into entropy- bbased optimization. ATENA consists of two core components:
3

---

State Memory
Entropy Memory
Episode Rollout @ step
End-of-Episode Adaptation
Navigation Policy
Action Distribution
State Memony
a0 Pal
DD D
SP Head
Salf-Active
Active Learning
Success / Fai
Success / Fai
Action Head
Panoramic Image
"Go to the kitchen on the second floor and
Entropy Memory
2V8 pook
Language Instruction
Untertainty
Epeode
Entropy-based Thresholding
Got it!
I' m here!
step 0
step t
step 1-1
Figure 1: Overview of the ATENA adaptation framework. At each navigation step, the agent stores state and entropy information in its memory. Once the episode ends, the stored entropy is used to determine the feedback source: human oracle for uncertain episodes, and self oracle for certain episodes. Self oracle utilizes self-prediction head, trained during online test-time, enabling the agent to autonomously predict navigation success or failure by itself.
Mixture Entropy Optimization (MEO): A method that refines the agent's policy using outcome- conditioned entropy signals, leveraging sseudo-expert-guiie action distribution to more effec- tively amplify correct behavior and penalize failure.
Self-Active Learning (SAL): A strategy that enables the agent to autonomously request or replace feedback based on internal uncertainty and self-assessment, allowing robust adaptation even when explicit feedback is sparse unavailable.
Together, these solutions allow the agent to adapt online by jointly leveraging episodic outcomes and self-predicter performance, without relying on ground-truth trajectories or dense human supervision.
3.3 Mixture Entropy Optimization (MEO)
A traditional entropy minimization method in VLN 37 aim to reduce uncertainty by decreasing the entropy of the predicted action distribution. However, indiscriminately minimizing entropy can in incorrect actions, leading to compounding errors during navigation. To address this, we optimize the entropy based on success or failure the navigation episode. Specifically, minimize it for successful episodes to reinforce the selected action, and maximize for failed episodes to penalize incorrect decisions. Furthermore, this entropy-based test-time adaptation is facilitated by novel mixture entropy optimization.
reinforce confidence
3.3.1 Mixture Action Distribution
First, we define the Mixture Action Distribution as a convex combination of the predicted action distribution and pseudo-expert distribution Opscudo- The pseudo-expert distribution is a one-hot probability distribution that assigns full probability (i.e. 1.0) to the selected action at which refers to the action with the highest predicted probability under the current policy, i.c., at = arg max, ie (a or, 1). In other words, the pseudo-expert distribution treats as if agel is the optimal expert action. The mixture action distribution formalized as:
Umix (a oz I) = (apecndo aببي + (1 A)TT(a or I), 0 O5A<1.
(I)
This mixture formulation sharpens the distribution around the selected action, with the combination weight controlling how strongly the pseudo-expert guides the distribution. Accordingly, the entropy of the mixture action distribution at timestep / is defined as:
H(gmia (-1 [))
Σ aeAA.
Fmix (a oz: I) log quix (a ot, I),
(

---

Fail! Mottre Entropy
Pseudo Eeeudo-Expert Distribution
Mature Entrapy Minimitztion
Mixture Action Distribution
Action
or
Distribution
Oracle
(a) Mixture Action Distribution
(b) Mixture Entropy Optimization
Figure 2: Ilustration of Mixture Entropy Optimization (MEO). (a) The Mixture Action Distribu- tion is constructed by combining the action distribution (yellow) with pseudo-expert distribution (red). (b) Mixture entropy is minimized for successful episodes to encourage the correct actions, and maximized for failures to penalize incorrect ones.
where A is the set of all possible actions step t. We average the entropy over all steps and obtain H'' (2mix) ) as the optimization signal of the episode. Then, the mixture entropy loss function can be formulated as:
Liix Tuseess H'' (Amix) - (1 Ilsuceess) H' H(mmixx,
where Iuccess is binary indicator that is 1 if the navigation was successful, 0 otherwise.
3.3.2 Effect on Policy Adaptation
Since the mixture action distribution inherently sharpens the original predicted action distribution, this strategy amplifies the feedback further boosting correct actions when successful, and more strongly suppressing incorrect ones when failed (see Figure 2b. This quantitatively evident from the selected action's probability:
Amix (a))) Oz, 1) = (1 (am
oo: I),
which is strictly greater than (a o,, when 0. As result, entropy-based optimization applied to Zmix exerts stronger influence the selected action compared to directly using my. Specifically, when minimizing this entropy successful episodes, the gradient increases Zmix ) more sharply than optimizing alone would. Conversely, maximizing entropy in failed episodes suppresses Qmix (att)) more aggressively, allowing MEO to drive stronger and more directional updates to the policy, improving sample efficiency and reducing the reliance on active learning during test time as demonstrated in Table
3.4 Self-Active Learning (SAL)
Mixture Entropy Optimization enables policy refinement based on navigation outcomes, requiring episodic feedback during test-time adaptation. In practice, however, acquiring feedback-especially from human annotators- be costly or delayed. Moreover, uncertainty alone may fail to capture subtle but critical navigation errors in seemingly confident predictions. To address these challenges, we propose Self-Active Learning (SAL), where the agent selectively queries human feedback or uses its own predictions on navigation outcomes to self-supervise, allowing for more robust and autonomous adaptation.
3.4.1 Uncertainty-Guite Query Strategy
At each timestep, the agent computes the entropy of its action distribution, H( or: 7)), and stores it in the entropy memory. At the end of an episode, the agent determines the source of supervision eieee Human (human-provided feedback) or Agent (self-generated feedback) on the average entropy. Technically, we consider as a function of 7 to predict lluccessi
Human, i DI H(mo( oc, I)) >
(5)
Agent, otherwise,
where 5 a pre-defined uncertainty threshold. In other words, the agent requests supervision from human in uncertain navigation and self-supervise in relatively certain navigation.
5

---

Algorithm Self-Active Learning for Online Adaptation
Require:
Pre-trained policy To, entropy threshold learning rate 17, loss weight Y 1: Initialize parameters 6, <
2: for each episode do
3: Follow instruction I and collect trajectory {(or: 1r T-1
4: Compute average entropy H + Lt= H(TT(( Or: I)) (Eq.
5:
if H then
6:
Receive human feedback: Lsuccess
O Human (T)
7:
else
8:
Receive agent feedback: Lsucesss
O Agem (7) (Eq. 6
9:
Compute mixture entropy loss Liix (Eq. 3)
10:
Compute self-prediction loss Liell (Eq.
11:
L= mmix + L (Eq. 8
Update: (0, + (0,$) Vo.L
return Adapted policy parameters (0", .6))
3.4.2 Self-Prediction Head
Predicting Navigation Outcome. To enable autonomous self-supervision, we incorporate self- prediction head fo into the pre-trained policy Ty. trained online to predict the episodic outcome (i.e., success or failure) from its internal states. Specifically, at step i, the D-dimensional hidden state vector E RRD is stored in the state memory and averaged over the episode as Sang. This is then fed into the self-prediction head to determine navigation outcome success
1,
if o (fo(wa))) > 0.5 otherwise,
success
(6)
0,
where a is the sigmoid activation function.
Training Self-Prediction Head. To train the self-prediction head, we use binary cross-entropy between (Savg) and the binary episodic outcome Lsuccess E {0,1): 1}:
Eyell [lucesss (f + [uccess) log(1 (of(ll-ffffsse (Swg))))
This loss is used during test-time adaptation regardless of the label source. If the feedback oracle human, assume that Isucess mostly accurate. Alternatively, if the feedback oracle is the agent itself, this can be interpreted as a self-training paradigm with pseudo label derived from the agent's own assessment of task completion, enabling continual self-improvement without external supervision as shown in Table Algorithm summarizes the full adaptation process of SAL.
Total Adaptation Objective of ATENA. We combine the mixture entropy loss from Eq. and the self-prediction loss into a unified test-time adaptation objective:
L
Lmix -
where y balances the influence of self-assessment. This joint objective reinforces correct decisions, penalizes errors, and improves the agent's ability to assess its own performance during deployment.
Experiments
4.1 Datasets & Metrics
We conduct experiments on three challenging VLN benchmarks REEESSEEEEEREEEEEE 17 R2R II and R2R- CE [8]. REVERIE evaluates agents' ability to follow high-level, gral-oriented instructions to locate remote objects in indoor environments; navigation episode is considered successful if the agent stops within meters of the target. For REVERIE, the performance is measured with Success Rate (SR). Oracle Success Rate (OSR), Success penalize by Path Length (SPL. and Remote Grounding SPL. (RGSPL.). R2R, in contrast, emphasizes fine-grained instruction following, providing detailed bysp-b guidance and using the same 3-meter success criterion. R2R-CE extends R2R by

---

Table 1: Experimental results the REVERIE dataset. f implies that the results are obtained from
our re- e-implementation (same for Table 2 and Table 3).
Methods
Val Seen
Unseen
Unseen
OSR
SR1
SPL 1
RGSPL
OSR1 1
SR
SPL 1
RGSPL
OSR 1
SR
SPL
RGSPL
HAMT [S]
47.65
43.29
40.19
25.18
36.84
32.95
17.28
33.41
30.40
13.08
w/ TTETT 7
46.03
43.43
40.78
25.81
32.60
30.56
28.23
14.48
25.06
23.73
21.78
10.82
w/ ESTTAT
37
48.21
42.87
39.56
24.58
36.78
30.51
17.20
33.39
30.39
13.61
w/ ATENA (Ours)
52.92
57.34
48.08
29:60
38.85
34.00
30.96
17.51
38.19
32.55
28,38
14.32
DUET E
73.86
71.75
63.94
51.14
51.07
46.98
33.73
23.03
56.91
52.51
36.06
22.06
w/ TENT
73.72
71.89
64.06
50.41
S1 /43
47. 55
33.99
23.32
57.12
52.61
36.17
22, 16
w/ ESTTA
75,59
15.48
65.84
52.23
56.26
54.15
36.41
23,56
58.44
53.40
36,43
22.40
ATENA (Ours)
85.52
84.33
74.31
59.99
71.88
45.82
31.26
57.74
54,28
25.01
GOAT 60
82,36
80.74
73:44
58.82
57.97
53.82
37.52
27.00
61.44
57.72
40.53
26.70
w/ TENT!
82.43
80.74
73.47
58.75
$7.68
53.51
37.49
26.99
62.00
57.28
39.82
26.97
w/ ESTTA!
82.36
80.74
73.42
58.82
57.94
53.79
37.50
26.95
62.35
57.52
39.49
26.88
w/ ATENA (Ours)
85:03
83.35
76.45
61.60
70.29
66
53.15
39.80
64.26
62.03
46.82
31.54
Table 2: Experimental results on the R2R dataset. Table 3: Experimental results on the R2R-CE
dataset.
Methods
Vall
Val Enseca
T1.1
NE
SR
SPL
TL.
NE
SR
SPL.
Methods
Val Seen
Val Unseen
DUET
12.35
2.28
79
73
13.94
2.31
72
66
TT.L
NE
OSR
SR
SPC.
TL.
NE
OSR
SR
SPL.
ATENA (urs)
333
11:27
3.08
14.66
12.31
2.90
75
62
66
11.08
3.95
72
66
59
11.99
4.71
65
57
49
RRViee [9
13.56
3.87
84
14.55
2.31
15
64
33
11.35
3.93
59
11.57
49
PSTTA
12.38
3.31
13.96
1.50
74
67
(Ouns)
10.81 3,56
12.89
o/ ATANAN (Ours)
10.79
2.26
12:23
2.78
76
68
BEVBen
13,98
3.77
60
13.27
w/ PSTTTA
GOAT 12
11.87
11.67
1.70
1.68
$4.52
$4.93
79.60
$0.08
13,45
13.26
1.12
2.33
77.91
00
62 AM
67.34
w/ ESTTA
14:00 4,11
74
69
00
13.11
4.79
66
601
51
W/ AAAATAAA (Ours)
11:56
1.64
55.01
10.15
12.92
79.0
69 90
w/ ATENA (Ouns)
11.31
13.48
replacing the discrete action space with continuous one, increasing the difficulty of low-level control and decision decision-making For R2R variants, we use Trajectory Length (TL), Navigation Error (NE), SR and SPL evaluation metrics.
4.2 Baselines
For experiments, we apply our ATENA on pre-trained HAMT [5]. DUET 4 BEVBert [59], ETP- Nav [6] and GOAT 60). HAMT is end-to-end transformer-basee VLN network trained via reinforcement learning. DUET exploits both global topology and local visual information for decision- making. BEVBert enhances spatial understanding by encoding the environment into Bird's- Eye-View representation. EPTNav emphasizes long-range planning for agents operating in continuous environments. Lastly, GOAT is a unified structural causal model for VLN. We compare our method against Tent 77 and FSTTA (37 Tent is a TTA method that minimizes entropy to adjust normaliza- tion statistics. FSTTA further applies the concept of entropy minimization the sequential VLN task. However, due to reported issue in the official codebase + we re-implement the method to ensure accurate evaluation. Throughout our experiments, a indicates results obtained from our version.
4.3 Main Navigation Results
REVERIE. Table L reports the comparisons of the navigation results on the REVERIE dataset, where the TTA methods including ATENA is applied to HAMT (5), DUET 4 and GOAT [1). Unlike previous methods that utilize entropy minimization a substantial performance increase from ATENA. Specifically, TENT and FSTTA brings minimal performance gains, or rather hinders the navigation performances in several metrics when applied to HAMT and GOAT. However, ATENA improves the SR metric in the validation unseen split up to 3.19%, 44.98 44.9% and 25.72% in HAMT, DUET and GOAT, respectively. Furthermore, ATENA A also excels in the test unseen split, improving GOAT by 4.59% 7.47%, 15.52% and 18. 13.3% in OSR, SR, SPL and RGSPL respectively.
test-time adaptation signal, we notice
R2R& & R2R-CE. In Table 2 we present the experimental results on the R2R dataset. Consistent e
with the findings from the REVERIE dataset, ATENA demonstrates superior effectiveness compared to FSTTA. While FSTTA improves the SPI. metric of GOAT by 0.21% on the validation unseen split, ATENA achieves 2.91% % improvement. Moreover, for the SR metric the validation unseen split of DUET, although the success rate is the same as FSTTA, ATENA achieves this with an 11.69%
attps://github http:////wee.eeaaaaaa//////00000
7

---

Table 4: Comparison of Test-Time Adaptation (TTA) methods with Active Learning (AL). Aster- isks (*) indicate methods integrated with Active Learning (AL), meaning they receive episodic feedback (success or failure) at uncertain navigation to guide entropy minimization or maximization. Active (%) denotes the ratio of navigation steps where feedback requested.
Val Seen
Val Unseen
Methods
SRt
SPL1
RGSPL
Active (%)
SRt
SPL1
RGSPLt
Active (%)
DUET TENT
75.69
67.20
54.67
65.92
55.69
38.76
26.16
90.34
DUET + FSTTA"
71.47
64.19
51.28
89.39
46.95
33.75
23.03
85.25
DUET MEO (Ours)
80.53
72.84
58.78
25.44
63.70
42.49
27.83
55.52
reduction in trajectory length, highlighting its high navigation efficiency. We observe similar results in the R2R-CE dataset, which is reported in Table 3 Specifically, ATENA increases 6.7% of SPL for BEVBert in the validation seen split. Lastly, we observe that given that the R2R variants rely on dense, step-wise guidance during training, the episodic feedback is relatively sparse to drive significant performance enhancements compared to that of the REVERIE'S.
4.4 Comparison of TTA Methods with Active Learning
Since the compared baseline methods do not employ active learning (AL) in their framework, integrate AL into the baselines to highlight the impact of MEO. Specifically, we apply TENT and FSTTA to the pre-trained DUET policy, and allow the agent update the parameters based on the human evaluation of navigation success or failure at uncertain episodes. Similar to ATENA, these baselines also minimize entropy for successful navigation, and maximize for failed ones. Furthermore, we evaluate a variant of ATENA without Self-Active Learning to assess the individual contribution of MEO with AL. For this experiment, the entropy threshold is equally set = 0.1 and we evaluate on the REVERIE dataset. The results are reported in Table 4 from which we draw the following observations. First, compared to the result in Table T TENT shows substantial performance increase when guided by human interactions. However, AL provides minimal benefit to FSTTA, which we attribute to its internal mechanism for modifying gradient directions- guidance on entropy optimization. MEO demonstrates strong synergy with AL,
conflicting with
the numan-provide leading to superior navigation performance across SR, SPL, and RGSPL metrics. Moreover, MEO achieves these improvements with significantly fewer human interventions, suggesting that the model progressively gains confidence as navigation proceeds.
4.5 Effect of Self-Active Learning
Table demonstrates the effectiveness of our pro- pose Self-Active Learning (SAL). We compare the performance of ATENA with and without SAL, ap- plied to the DUET navigation policy. The evaluation is done using the REVERIE dataset. Even without SAL, as previously observed in Table 4 ATENA shows solid performance increase from the base policy. However, enabling the agents to train online from episodic labels and autonomously evaluate its navigation outcome brings notable improvements. Specifically, we observe in the validation unseen split 6.92%, 7.84% and 12.32% enhancements in SR. SPL, and RGSPL, respectively. This clearly demonstrates the effectiveness of remaining continuously active throughout the adaptation process.
Table 5: Comparison of performance on the REVERIE dataset demonstrating the effec- tiveness of Self- Learning (SAL).
Val Seen
Val Unseen
Methods
SRT
SPL1
RGSPL
SRT
SPL
RGSPL
SAT.
80.53
72,84
58.78
63.70
42.49
27.83
w/ SAL
84.33
74.31
59.99
68.11
45.82
31.26
4.6 Combination Weight of Mixture Entropy Optimization
We vary the combination weight À in Eq. within À E (0.0,0.1.0.2...... 1.0) and evaluate its effect on adaptation performance. À = 0.0 implies that the agent relies solely on its original action distribution, serving as the baseline in Figure We omit the policy relies exclusively on the entropy of the pseudo-expert distribution, which is identically zero and thus provides no informative signal. As the weight of the pseudo-expert distribution increases, observe consistent improvements across SR. SPL, and RGSPL, demonstrating the
1.0 from our results because, this case,
8

---

70
50
45
30
40
RGSPL
8
28
50
35
1:00 (vanilaa entropy)
N-Olvanila entropyi
28
1-0 (vanila entrapy)
40
25
24
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.6
0.8
0.0
0.2
0.4
0.8
0.8
Figure 3: Effect of the combination weight X. Performance comparison with different combination
weight À in Mixture Entropy Optimization. À = 0 corresponds to vanilla entropy without distribution mix. The results are averaged across three experiments with different seeds.
benefit of distribution sharpening. The performance peaks at À 0.4, where the balance between the predicted distribution and the pseudo-expert distribution appears to be optimal. As À increases further, we observe decrease in SR and SPL, yet the benefits compared vanilla entropy remains solid. These results collectively suggest that while the optimal performance depends on careful balance between the predicted and pseudo-expert distributions, incorporating the mixture itself is consistently beneficial for adaptation.
4.7 Sampling Strategies for Feedback Episodes
We compare the uncertainty-base active learning strategy with two different sam- pling baselines: (1) Random Episodes, where episodes that receive feedback are
Random Episoces
Consecutive Epiodes
Uncertain Episodes
Full Edisodes
66
selected randomly; and (2) Consecutive Episodes, where feedback is provided on a contiguous block episodes starting from the beginning of the dataset. The number of samples the baselines are equally set to match that of our m 60% in this
28
42
26
SPL
RGSPL
Figure 4: Different Episode Sampling Strategies. Our uncertainty -based sampling outperforms baselines and remains competitive against full-feedback settings.
experiment- as the portion of uncertainty- based selection cannot be approximated heuristically. Figure 4 our uncertainty- bbased sampling outperforms the baselines, indicating that optimization guided by informative uncertainty signals is more effective than relying simple rule-based selection. Furthermore, when compared to the setting where feedback is provided for all episodes, our method achieves higher performance in SR and RGSPL, while maintaining petitive results in SPL. These results suggest that our amcertainty-basee strategy achieves superior performance with reduced supervision by selectively focusing the most informative episodes, outperforming both heuristic baselines and full feedback in terms of efficiency and effectiveness.
5 Conclusion
We introduce ATENA, novel TTA framework that leverages active human-robot interaction to enhance online vision-language navigation. Specifically, we propose Mixture Entropy Optimization, explicitly reinforeing correct actions and penalizing incorrect ones based episodic outcomes. Additionally, through Self-Active Learning, we enable the agent to autonomously predict navigation outcomes during episodes where it has relatively high confidence. Extensive experiments demonstrate that ATENA substantially outperforms baseline approaches, effectively addressing distribution shifts between training and testing environments. By integrating human-guided and self-guided active learning mechanisms, ATENA allows the agent to handle uncertainty through continuous adaptation and self-refinement. Ultimately, our approach opens promising avenues for future research by integrating human- interaction with automated asf-smssessmnt to support robust and efficient online adaptation across diverse interactive embodied AI tasks.
Limitations and Future Work. A limitation of ATENA is that the policy is updated only after the episode ends, rather than during navigation. This design minimizes external interaction latency but prevents the agent from receiving fine-grained step-level feedback during execution. As result, the agent cannot immediately adapt its behavior or identify which specific steps contributed to performance degradation. Future work could explore lightweight mechanisms for incorporating intermediate, segment- or step-level signals to enable more timely and precise adaptation while maintaining low-latency operation.

---

6
Acknowledgment
This work was supported by Culture, Sports and Tourism R&D Program through the Korea Creative Content Agency grant funded by the Ministry of Culture, Sports and Tourism (International Collabo- rative Research and Global Talent Development for the Development of Copyright Management and Protection Technologies for Generative AI, 203-2024-00345502 technology for copyright protection of generative AI 3D content, National Research Foundation of Korea(NRF) grant funded by the Korea governnen(MSSITTRS 2025-00521 602. 48%), Institute of Information & communications Technology Planning & Evalua- tion (ITTP) & ITR CInforrmatioo Technology Research Center) grant funded by the Korea govern- ment(MSIT) Noo.R--2011 2019-11190079, Artificial Intelligence Graduate School Program(Kove (Koraa Uni- versity), 1%; LITP-2025 RS-2024-00436833 1%), and Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT(MSIT, Korea)&Gwangju Metropolitan City.
25%; Research on neural watermark
25%), the
References
[1] Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and- -nd---n navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3674-3683, 2018.
[2] Amin Parvanch, Ehsan Abbasnejad, Damien Teney, Javen Qinfeng Shi, and Anton Van den Hengel. Counterfactual Assion-and-languee navigation: Unravelling the unseen. Advances in neural information processing systems, 33:5296-530 53076,5377222 2020.
[3] Jialu Li, Hao Tan, and Mohit Bansal. Envedit: Environment editing for vision-and-language navigation. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 15407-15417, 2022.
[4] Shizhe Chen, Pierre-Louis Guhur, Makarand Tapaswi, Cordelia Schmid, and Ivan Laptev. Think global, act local: Dual-scale graph transformer for vision-and-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16537-16547, 2022.
[5] Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for vision-and- languuuee navigation. Advances in neural information processing systems, 34:5834-5847, 2021.
[6] Dong An, Hanqing Wang, Wenguan Wang. Zun Wang, Yan Huang, Keji He, and Liang Wang. Etpnav: Evolving topological planning for vision-language navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
[7] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 10726, 2020.
[8] Qin Wang, Olga Fink, Lue Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7201-7211, June 2022.
[9] Sachin Goyal, Mingjie Sun, Aditi Raghunathan, and J Zico Kolter. Test time adaptation via conjugate pscudo-labels. Advances in Neural Information Processing Systems, 35:6204-6218, 2022.
[10] Hao Yang, Min Wang, Jinshen Jiang, and Yun Zhou. Towards test time adaptation via calibrated entropy minimization. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 7746-3746, 2024.
[11] Xingzhi Zhou, Zhiliang Tian, Boyang Zhang, Yibo Zhang, Ka Chun Cheung, Simon See, Hao Yang, Yun Zhou, and Nevin L. Zhang. Test-time adaptation on noisy data via model-pruning- based filtering and flatness-aware entropy minimization. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10852-10866 2025.
10

---

[12] Burr Settles. Active learning literature survey. 2009.
[13] Dongyuan Li, Zhen Wang, Yankai Chen, Renhe Jiang, Weiping Ding, and Manabu Okumura. survey on deep active learning: Recent advances and new frontiers. IEEE Transactions on Neural Networks and Learning Systems, 2024.
[14] Jiaxi Wu, Jiaxin Chen, and Di Huang. Entropy-based active learning for object detection with progressive diversity constraint. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9397-9406, 2022.
[15] Alex Tamkin, Dat Nguyen, Salil Deshpande, Jesse Mu, and Noah Goodman. Active learning helps pretrained models learn the intended task. Advances in Neural Information Processing Systems, 35:28140-2222 40-221153 2022.
[16] Alexandru Tifrea, Jacob Clarysse, and Fanny Yang. Margin- based sampling in high dimensions: When being active is less efficient than staying passive. In International Conference on Machine Learning, pages 34222-34262. PMLR, 2023.
[17] Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9982-9991, 2020.
[18] Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav- graph: ladia-annd navigation in continuous environments. In Computer Vision-ECCC ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16, pages 104-120. Springer, 2020.
[19] Peng Gao, Peng Wang, Feng Gao, Fei Wang, and Ruyue Yuan. Vision-language navigation with embodied intelligence: A survey. arXiv preprint arXiv:2402. 14304, 2024.
[20] Wansen Wu, Tao Chang, Xinmeng Li, Quanjun Yin, and Yue Hu. Vision-language navigation: a survey and taxonomy. Neural Computing and Applications, 36(7):3291-3316 2024.
[21] Jing Gu, Eliana Stefani, Qi Wu, Jesse Thomason, and Xin Eric Wang. Vision iisoon-and-lamge languagee navigation: A survey of tasks, methods, and future directions. arXiv preprint arXiv:2203. 12667, 2022.
[22] Dong An, Yuankai Qi, Yan Huang, Qi Wu, Liang Wang, and Tieniu Tan. Neighbor- en- hanced model for vision and language navigation. In Proceedings of the 29th ACM International Conference on Multimedia, pages 5101-5109, 2021.
[23] Xin Wang, Qiuyuan Huang, Asli Celikyilmaz, Jianfeng Gao, Dinghan Shen, Cuan-Fang Wang, William Yang Wang, and Lei Zhang, Vision- navigation policy learning and adaptation. IEEE transactions on pattern analysis and machine intelligence, 43(12):4205-44 4216, 2020.
[24] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, tukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
[25] Rui Liu, Wenguan Wang, and Yi Yang. Volumetric environment representation for vision- language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16317-16328. 2024.
[26] Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning a generic agent for vision-and-languag navigation via pre-training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13137-13146, 2020.
[27] Liuyi Wang, Zongtao He, Jiagui Tang, Ronghao Dang, Najjia Wang, Chengju Liu, and Qijun Chen. A dual semantic-aware recurrent global-adaptive network for vision-and -ad-auuu---- navigation. arXiv preprint arXiv:2305.03600 2023.
[28] Federico I Landi, Lorenzo Baraldi, Marcella Cornia, Massimiliano Corsini, and Rita Cucchiara. Multimodal attention networks for lowel vision-and- language navigation. Computer vision and image understanding, 210: 2021.

---

[29] Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, and Yejin Choi. Robust navigation with language pretraining and stochastic sampling. arXiv preprinl arxiv: 0tti:::999.002000 /909.02244, 2019.
[30] Kehan Chen, Dong An, Yan Huang, Rongtao Xu, Yifei Su, Yonggen Ling, lan Reid, and Liang Wang. Constraint-aware zero-shot vision- lisoonlanguage navigation in continuous environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[31] Yanyuan Qiao, Wenqi Lyu, Hui Wang, Zixu Wang, Zerui Li, Yuan Zhang, Mingkui Tan, and Qi Wu. Open-nav: Exploring zero-shot vision-and-langune navigation in continuous environment with open-source llms. 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 6710-66 6717. IEEE, 2025.
[32] Jiaqi Chen, Bingqian Lin, Ran Xu, Zhenhua Chai, Xiaodan Liang, and Kwan- Yee K Wong. Mapgpt: sup-guided prompting with adaptive path planning for and-language navigation. arXiv preprint arXiv:2401.07314 2024.
[33] Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment. arXiv preprint orXiv:2406.04885 2024.
[34] Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: navigation with large language models. Proceedings of the AAAI Conference on Artificial Intelligence, pages 7641-7649, 2024.
Explicit reasoning in vision-and-languae
[35] Yuxing Long, Xiaoqi Li, Wenzhe Cai, and Hao Dong. Discuss before moving: Visual language navigation via multi-expert discussions. In 2024 IEEE International Conference Robotics and Automation (ICRA), pages 17380-17387. IEEE, 2024.
[36] Gengze Zhou, Yicong Hong. Zun Wang, Xin Eric Wang, and Qi Wu. Navgpt-2: Unleashing navigational reasoning capability for large vision-language models. In European Conference on Computer Vision, pages 260-278. Springer. 2024.
[37] Junyu Gao, Xuan Yao, and Changsheng Xu. Fast-slow test- adaptation for online vision-and- language navigation. In International Conference on Machine Learning, pages 14902-14919. PMLR, 2024.
[38] Sungjune Kim, Gyeongrok Oh, Heeju Ko, Daehyun Ji, Dongwook Lee, Byung-Jun Lee, Sujin Jang, and Sangpil Kim. Test-time adaptation for online vision- languaee navigation with feedhack-based reinforcement learning. In Forty-second International Conference on Machine Learning.
[39] Ning Ma. Jiajun Bu, Lixian Lu, Jun Wen, Sheng Zhou, Zhen Zhang, Jingjun Gu, Haifeng Li, and Xifeng Yan. Context-guidee entropy minimization for semi-supervised domain adaptation. Neural Networks, 154:270-288 2022.
[40] Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez. Advent: Ad- versarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2517-25266 2019.
[41] Xiaofu Wu, Suofei Zhang, Quan Zhou, Zhen Yang, Chunming Zhao, and Longin Jan Latecki. Entropy minimization versus diversity maximization for domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 34(6) 2907, 2021.
[42] Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in neural information processing systems, 17, 2004.
[43] David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances neural information processing systems, 32, 2019.
12

---

[44] Jiawei Wu, Haoyi Fan, Xiaoqing Zhang, Shouying Lin, and Zuoyong Li. Semi-supervised semantic segmentation via entropy minimization. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1-6. IEEF, 2021.
[45] Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribution shifts. International Journal of Computer Vision, 133(1):31 2025.
[46] Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey. arXiv preprint arXiv:2411.03687, 2024.
[47] Zhengqing Gao, Xu-Yao Zhang, and Cheng-Lin Liu. Unified entropy optimization for open-set test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23975-23984, 2024.
[48] Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee. Note: Robust continual test-time adaptation against temporal correlation. Advances in Neural Information Processing Systems, 35:27253-27266 2022.
[49] Mingkui Tan, Guohao Chen, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Peilin Zhao, and Shuaicheng Niu. Uincertintt-ccalbbaate rest-timee model adaptation without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
[50] Jonghyun Lee, Dahuin Jung, Saehyung Lee, Junsung Park, Juhyeon Shin, Uiwon Hwang, and Sungroh Yoon. Entropy is not enough for test-time adaptation: From the perspective of disentangled factors. arXiv preprint MxXi:2400300366 2024.
[51] Pengzhen Ren, Yun Xiao, Xiaojun Chang, Po- Yao Huang, Zhihui Li, Brij B Gupta, Xiaojiang Chen, and Xin Wang. A survey of deep active learning. ACM computing surveys (CSUR). 54(9):1-40, 2021.
[52] Samuel Budd, Emma C Robinson, and Bernhard Kainz. A survey active learning and human- in-the-loop deep learning for medical image analysis. Medical image analysis, 71:102062, 2021.
[53] Martin Mundt, Yongwon Hong, Iuliia Pliushch, and Visvanathan Ramesh. A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning. Neural Networks, 160:306-336, 2023.
[54] Ali Ayub and Carter Fendley, Few-shot continual active learning by a robot. Advances in Neural Information Processing Systems, 35:30612-30624, 2022.
[55] Jachyun Park, Dongmin Park, and Jae-Gil Lee. Active learning for continual learning: Keeping the past alive in the present. arXiv preprint arXiv:2501. /4278, 2025.
[56] Bo Fu, Zhangjie Cao, Jianmin Wang, and Mingsheng Long. Transferable query selection for active domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 7272-7281, 2021.
[57] Shurui Gui, Xiner Li, and Shuiwang Ji. Active test-time adaptation: Theoretical analyses and an algorithm. arXiv preprint arXiv:2404.05500 2024.
[58] Guowei Wang and Changxing Ding. Effortless active labeling for long-term test-time adaptation. arXiv preprint arXiv:2503. 14564, 2025.
[59] Dong An, Yuankai Qi, Yangguang Li, Yan Huang, Liang Wang, Tieniu Tan, and Jing Shao. Bevbert: Multimodal map pre-training for language-guider navigation. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023.
[60] Liuyi Wang. Zongtao He, Ronghao Dang, Mengjiao Shen, Chengju Liu, and Qijun Chen. Vision- and-lan navigation via causal learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13139-131500 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims and contributions of our work are clearly presented in both the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We have included discussions of the limitations in the conclusion section.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
Our paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Detailed descriptions our experimental settings, including hardware specifi- cations and hyperparameters, provided in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer:
[No]
Justification:
The source code will be publicly released upon acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://niis.. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https guwdes/dde/eee Cooeeeoodeesssseeeeee 7/nips.com /ttc: swiniioiscyp for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification: Detailed descriptions of the experimental conditions can be found in both the main manuscript and the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[No]
Justification: Our paper does not report error bars or statistical significance information.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Detailed descriptions of the computer resources are provided the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
No potential ethical harms are expected in our research.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We have provided in the appendix a discussion of both potential positive and negative societal impacts our research.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [NA]
Justification: No such risks are expected in our paper.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We have provided the licenses and credits for the benchmark dataset and the baseline models.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the Datasets.papersstt
package should be provided. For popular datasets, paperswi thcode. cww/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: Our paper does not introduce any new assets.
Guidelines:
The answer NA means that the paper does release new assets. Researchers should communicate the details of the dataset/cooeemmodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our research does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According to the NeurIPS Code Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our research does not involve crowdsourcing experiments or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
19

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: Our work does not involve LLMs as important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standarr components.
Please refer to our LLM policy ttttps://neuriiee wwwparripppccoooeeeeeeeeeeee ///Cooffreences/22225 for what should or should be described.
20


Reviews:
[{"review_id": "DQvAOBUNqx", "reviewer": "Reviewer_1sz2", "summary": "This paper introduces ATENA (Active Test-time Navigation Agent), a novel framework for Vision-Language Navigation (VLN) that addresses the challenge of domain shifts between offline training and online test environments. ATENA incorporates active learning at test time, using Mixture Entropy Optimization (MEO) to adjust the agent's confidence based on navigation success or failure. Additionally, it proposes Self-Active Learning (SAL), enabling the agent to autonomously assess its navigation outcomes and reduce reliance on human feedback. Extensive evaluations on benchmark VLN tasks (REVERIE, R2R, and R2R-CE) demonstrate that ATENA outperforms existing test-time adaptation methods, improving performance while maintaining efficiency and minimizing human intervention.", "questions": "Please refer to the Weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. ATENA introduces a novel approach to test-time adaptation in VLN by combining active learning with entropy optimization. The use of Mixture Entropy Optimization (MEO) and Self-Active Learning (SAL) makes it a significant advancement over existing methods that rely solely on entropy minimization.\n2. The paper provides thorough experiments on REVERIE, R2R, and R2R-CE, showing that ATENA outperforms baselines like TENT and FSTTA in multiple metrics, such as Success Rate (SR) and Success weighted by Path Length (SPL).\n3. ATENA is lightweight and efficient, suitable for real-world deployment with minimal human feedback. Its ability to adapt autonomously through Self-Active Learning makes it highly scalable and practical for real-time VLN tasks.\n\nWeaknesses:\n1. The method is tested primarily on VLN tasks, and its generalization to other types of navigation tasks, like object-goal navigation, remains unexplored. Further evaluation in diverse settings is needed to assess its broader applicability.\n2. Although ATENA reduces human feedback reliance, it still depends on episodic feedback for uncertain episodes. Ambiguity or inconsistency in human feedback could introduce errors, affecting performance, especially in complex environments.\n3. The Self-Prediction Head contributes to reducing human feedback, but its accuracy is not flawless. False predictions could accumulate over time, potentially impacting the adaptation process and leading to suboptimal decisions.\n4. While ATENA is relatively efficient, it still introduces some additional adaptation latency compared to simpler methods like TENT. The minor increase in computational cost may be a limitation in highly time-sensitive environments.\n5. While Self-Active Learning (SAL) improves performance, its effectiveness in sparse feedback scenarios could be better evaluated. Further analysis on how SAL handles more challenging environments with limited feedback would strengthen the claims.", "was_revised": true, "final_justification": "The authors have sufficiently addressed my main concerns. In light of the rebuttal, I have adjusted my score to 4 (Borderline accept).", "rebuttal": "### **[W1] Generalization to other navigation tasks**\n\nTo demonstrate the general applicability of ATENA to other types of navigation tasks, we present additional experimental results on the Image-Goal Navigation task, where the agent is asked to navigate and locate a given visual target. We use TSGM [1] as the baseline policy and report the adaptation result of ATENA in the ‘hard’ difficulty environment (Please see 4.2 of [1] for details). Also, we use the mixture rate of $\\lambda=0.4$ and the learning rate of 1e-6. The clear increases of SR and SPL in the table below strongly demonstrate ATENA’s superior adaptability also in the Image-Goal Navigation task, underscoring its potential as a **general TTA method for robotic navigation tasks**. Furthermore, please refer to our response for [W2] of Reviewer zXCt, where we empirically demonstrated the capability of ATENA in the Iterative-VLN benchmark as well. We will incorporate these new analysis into the final version of the paper.\n\n| Methods         | Hard SR ↑  | Hard SPL ↑ |\n|-|-|-|\n| TSGM [1]           | 70.30          | 50.00            |\n| w/ ATENA         | **72.49**      | **53.23**    |\n\n[1] Kim et al., “Topological Semantic Graph Memory for Image-Goal Navigation”, CoRL 2022\n\n### **[W2] Reliability of human feedback**\n\nWe understand the reviewer’s concern on the reliability of human feedback. To address this, we conducted additional experiments by manually varying the proportion of verified human feedback from 50% to 100%, with incorrect feedback provided to the remaining proportion. This setting closely resembles the real-world scenario where even the human feedback may be noisy. The results on the 'val_unseen' split of REVERIE, averaged over three experiments conducted with different random seeds, are summarized below:\n\n| | DUET       | w/ TENT     | w/ ATENA (50\\%)     | w/ ATENA (60\\%)     | w/ ATENA (70\\%)     | w/ ATENA (80\\%)     | w/ ATENA (90\\%)     | w/ ATENA (100\\%)\n|-|-|-|-|-|-|-|-|-|\n|SR ↑|$46.98$|$47.55$|$46.09 \\pm 0.33$|$48.48 \\pm 0.49$|$56.89 \\pm 0.77$|$60.21 \\pm 0.52$|$63.28 \\pm 1.45$ |$67.34 \\pm 0.99$|\n\nAlthough a slight performance decrease is observed at the lowest feedback accuracy level (50%), **ATENA consistently improved the DUET baseline at feedback accuracy levels of 60% and above**. This clearly indicates that even when feedback includes up to 40% of incorrect signals, ATENA demonstrates robust and effective adaptation performance in complex environments. Moreover, we argue that the 40% noise level tested in this experiment represents a highly conservative estimate of real-world error. **Providing a binary success/fail judgment is a trivial and straightforward task for a human user**, suggesting that the actual feedback reliability in practice would be very high, placing performance closer to the upper end of our experimental results.\n\n### **[W3] Reliability of the Self-Prediction Head**\n\nWe agree that the Self-Prediction head is not flawless; however, we empirically demonstrate its high reliability in the appendix (Table 7), with an **overall accuracy of 82.19%**. This functions as a self-training mechanism, a well-established paradigm where a model beneficially learns from its own high-confidence pseudo-labels to improve performance [1-3]. Importantly, our Self-Active Learning (SAL) strategy only uses these self-predictions on episodes with low uncertainty, while querying a human for more challenging cases, which inherently minimizes the risk of learning from incorrect labels. Therefore, this **combination of high empirical accuracy and confidence-based filtering** ensures the benefits of autonomous adaptation significantly outweigh the minimal risk of error accumulation from imperfect pseudo-labels. We also address a related concern in our response to Reviewer PkQQ (Q 3.1), which further discusses and emphasizes the reliability of using the Self-Prediction Head.\n\n[1] Wang, et al., \"Deep insights into noisy pseudo labeling on graph data.\", NeurIPS 2023.\\\n[2] Zhang, et al., \"Pseudo label refinery for unsupervised domain adaptation on cross-dataset 3d object detection.\", CVPR 2024.\\\n[3] Chen, et al., \"SAT: Improving semi-supervised text classification with simple instance-adaptive self-training.\", ACL 2022\n\n### **[W4] Minor increase in computational costs**\n\nTo clearly highlight the efficiency of ATENA against TENT, we present an additional experimental analysis on the 'val_unseen' split of REVERIE, where we provide less feedback to the policy by varying feedback intervals (after every 1, 2, 4, 10, 20, and 100 iterations). As the result in the table below shows, the total inference time drops proportionally as the feedback interval increases. However, even if the success rate also reduces from increased interval, **the performance stays superior compared to TENT with much less inference time**. Furthermore, we would like to emphasize that, since the increased latency is mostly due to the parameter updates in between the episodes, **it does not hinder the navigation latency itself**. \n\n| | DUET       | w/ TENT     | w/ ATENA (interval=100)     | w/ ATENA (interval=20)     | w/ ATENA (interval=10)    | w/ ATENA (interval=4)    | w/ ATENA (interval=2)     | w/ ATENA (interval=1)|\n|-|-|-|-|-|-|-|-|-|\n|SR ↑| $46.98$ | $47.55$ | $47.97 \\pm 0.37$ | $50.09 \\pm 0.73$ | $52.75 \\pm 0.18$ | $57.33 \\pm 1.04$ | $63.44 \\pm 0.54$ | $67.43 \\pm 0.83$ |\n|Inf. Time (s)| $157.82$ | $489.90$ | $201.27$ | $213.42$ | $238.56$ | $280.87$ | $367.73$ | $516.04$ | \n\n### **[W5] Performances in sparse-feedback scenario**\n\nAs the reviewer mentioned, although the effectiveness of Self-Active Learning (SAL) is verified, assessing its robustness in sparse-feedback environments is crucial. Therefore, we analyzed this using the experimental setting and results from our response for [W4] above, and additionally reported the impact of SAL at each interval. As seen in the table below, although we provide feedback less frequently, ATENA still improves the baseline policy DUET and surpasses TENT, even in the extremely sparse feedback condition (interval = 100). Furthermore, as we remove SAL from ATENA, we observe performance drops from all intervals, indicating that **SAL poses strong benefits regardless of feedback scarcity**.\n\n| | DUET       | w/ TENT     | w/ ATENA (interval=100)     | w/ ATENA (interval=20)     | w/ ATENA (interval=10)    | w/ ATENA (interval=4)    | w/ ATENA (interval=2)     | w/ ATENA (interval=1)|\n|-|-|-|-|-|-|-|-|-|\n|SR ↑|$46.98$|$47.55$|$47.97 \\pm 0.37$|$50.09 \\pm 0.73$|$52.75 \\pm 0.18$ |$57.33 \\pm 1.04$ |$63.44 \\pm 0.54$  |$67.43 \\pm 0.83$ |\n|SR w/o SAL ↑ |N/A|N/A|$47.72 \\pm 0.24$|$49.63 \\pm 0.64$|$51.48 \\pm 0.41$|$54.88 \\pm 1.06$ |$60.47 \\pm 0.64$  |$63.98 \\pm 1.65$  |"}, {"review_id": "VIsT3d0nPr", "reviewer": "Reviewer_PkQQ", "summary": "The authors address the problem of domain adaptation in Vision-Language Navigation by proposing to further optimize the policy in the test environment through Mixture Entropy Optimization and Self-Active Learning. The experimental results demonstrate the effectiveness of the proposed method.", "questions": "1. In Section 3.3.2, could you provide the gradient update formulas before and after the modification to more formally illustrate that the method behaves as intended?\n\n2. Is the purpose of the trained Self-Prediction Head to determine whether the task is completed, or the additional training of the self-prediction head during testing also help improve the policy itself? What happens if the agent determines at deployment that the task is not completed? Also, what is the accuracy of the Self-Prediction Head? I did not see relevant data in the experiments.\n\n3. Could you comment on how the Self-Prediction Head would behave if it were trained only with human-verified feedback, instead of using its own pseudo-labels? I am concerned that the current self-training scheme may introduce drawbacks such as confirmation bias and error accumulation.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The paper is clearly written and easy to follow.\n\n2. It focuses on a timely and important topic, and naturally integrates entropy and active learning techniques.\n\nWeaknesses:\n\n1. The motivation for introducing a pseudo-expert distribution in the Mixture Action Distribution is not well explained, lacking an intuitive or theoretical justification. There are more general ways to control the sharpness of the distribution, such as introducing different temperatures in the softmax.", "was_revised": true, "final_justification": "After reviewing all the current materials, I maintain my original assessment.", "rebuttal": "### **[W1,Q1] Motivation & justification for the pseudo-expert distribution**\n\n**1. Theoretical analysis and formulation of the gradient updates** \n\nThe core motivation and intuition for introducing the pseudo-expert distribution is **to create a larger gradient gap between the selected action and all other non-selected actions**. We provide a theoretical analysis of how MEO with pseudo-expert distribution can achieve a larger gradient gap compared to the standard entropy optimization. For simplicity, we modify the binary indictor in Eq (3) to be $\\mathbb{I}=1$ for success, and $\\mathbb{I}=-1$ for failure, formulating the general entropy objective function as $\\mathcal{L}=\\mathbb{I}\\cdot\\mathcal{H}(P)$ for a certain probability distribution $P$.\n\nIn standard entropy optimization, the objective function at time $t$ is:\n\n$$\n\\\\mathcal{L}\\_\\\\text{standard}=\\\\mathbb{I}\\\\cdot\\\\mathcal{H}(\\\\pi\\_{\\\\theta}(\\\\cdot|o\\_t, I))=-\\\\mathbb{I} \\\\sum_{a\\\\in\\\\mathcal{A}\\_{t}} \\\\pi_{\\\\theta}(a|o\\_t, I) \\\\log \\\\pi\\_{\\\\theta}(a|o\\_t, I)\n$$\n\nNext, we compute the gradient of this loss w.r.t the policy parameter $\\theta$:\n$$\n\\\\nabla\\_\\\\theta \\\\mathcal{L}\\_\\\\text{standard}=-\\\\mathbb{I} \\\\cdot \\\\nabla\\_\\\\theta \\\\Big(\\\\sum_{a}\\\\pi\\_{\\\\theta}(a) \\\\log \\\\pi\\_{\\\\theta}(a) \\\\Big)\n$$\n\nApplying the product rule to the term inside the summation, we get full gradient for standard entropy optimization:\n$$\n\\\\nabla\\_\\\\theta \\\\mathcal{L}\\_\\\\text{standard}=-\\\\mathbb{I} \\\\sum_{a\\\\in\\\\mathcal{A}\\_{t}} \\\\big(1+\\\\log \\\\pi\\_{\\\\theta}(a|o_t, I) \\\\big)\\\\nabla\\_{\\\\theta}\\\\pi\\_{\\\\theta}(a|o_t, I)\n$$\n\nLeveraging the same structure from above, the gradient of the MEO loss with pseudo-expert distribution is given as:\n$$\n\\\\nabla\\_\\\\theta \\\\mathcal{L}\\_\\\\text{MEO}=-\\\\mathbb{I} \\\\sum\\_{a\\\\in\\\\mathcal{A}_{t}} \\\\big(1+\\\\log q\\_{\\\\text{mix}}(a|o_t, I) \\\\big)\\\\nabla\\_{\\\\theta} q\\_{\\\\text{mix}}(a|o_t, I)\n$$\n\nAccording to the definition of $q_{\\text{mix}}$ from Eq (1) in our paper, the parameters of $\\theta$ only influence $q_{\\text{mix}}$ through $\\pi_{\\theta}$. Therefore, the gradient of $q_{\\text{mix}}$ w.r.t $\\theta$ is $\\nabla_\\theta q_{\\text{mix}}=(1-\\lambda)\\nabla_\\theta \\pi_{\\theta}$. Substituting this into the equation above, we get:\n$$\n\\\\nabla\\_\\\\theta \\\\mathcal{L}\\_\\\\text{MEO}=-\\\\mathbb{I} \\\\sum\\_{a\\\\in\\\\mathcal{A}\\_{t}} \\\\big(1+\\\\log q\\_{\\\\text{mix}}(a|o_t, I) \\\\big)(1-\\\\lambda)\\\\nabla\\_{\\\\theta} \\\\pi\\_{\\\\theta}(a|o_t, I)\n$$\n\nThe gradient formulation of each objective function can be seen as a weighted average of the individual gradient contributions from each action $\\nabla_\\theta \\pi_{\\theta}(a)$, where the weighting term determines how much influence each action has on the final direction of the update. In other words, the larger gap between the weight on the selected action $a^{\\text{sel}}$ compared to an unselected action $a^{\\prime}$ indicates effectiveness. \n\nThe weight gap of the standard optimization is:\n$$\n\\\\Delta w\\_{\\\\text{standard}}=(1+\\\\log \\\\pi\\_{\\\\theta}(a\\^{\\\\text{sel}}))-(1+\\\\log \\\\pi\\_{\\\\theta}(a\\^{\\\\prime}))=\\\\log \\\\frac{\\\\pi\\_{\\\\theta}(a\\^{\\\\text{sel}})}{\\\\pi\\_{\\\\theta}(a\\^{\\\\prime})}\n$$\n\nFor MEO, the weight gap is:\n$$\n\\\\Delta w\\_{\\\\text{MEO}}=(1-\\\\lambda)(1+\\\\log q\\_{\\\\text{mix}}(a\\^{\\\\text{sel}}))-(1-\\\\lambda)(1+\\\\log q\\_{\\\\text{mix}}(a\\^{\\\\prime}))=\\\\log \\\\frac{q\\_{\\\\text{mix}}(a\\^{\\\\text{sel}})}{q\\_{\\\\text{mix}}(a\\^{\\\\prime})}\n$$\n\nThe ratio of the actions for the mixture distribution $q_{\\text{mix}}$ is:\n$$\n\\\\frac{q\\_{\\\\text{mix}}(a\\^{\\\\text{sel}})}{q\\_{\\\\text{mix}}(a\\^{\\\\prime})}=\\\\frac{\\\\lambda+(1-\\\\lambda)\\\\pi_{\\\\theta}(a\\^{\\\\text{sel}})}{(1-\\\\lambda)\\\\pi\\_{\\\\theta}(a\\^{\\\\prime})}=\\\\frac{\\\\lambda}{(1-\\\\lambda)\\\\pi\\_{\\\\theta}(a\\^{\\\\prime})}+\\\\frac{\\\\pi\\_{\\\\theta}(a\\^{\\\\text{sel}})}{\\\\pi\\_{\\\\theta}(a\\^{\\\\prime})}\n$$\nFor any $0<\\lambda<1$, $\\frac{q_{\\text{mix}}(a^{\\text{sel}})}{q_{\\text{mix}}(a^{\\prime})}$ is strictly larger than $\\frac{\\pi_{\\theta}(a^{\\text{sel}})}{\\pi_{\\theta}(a^{\\prime})}$, which also holds true in the log-space:\n$$\n\\\\Delta w\\_{\\\\text{MEO}}\\>\\\\Delta w\\_{\\\\text{standard}}\n$$\n\nTherefore, **this larger gap formally demonstrates that our MEO approach creates a more focused and directional gradient signal than standard entropy optimization**. This ensures the final update to the policy parameters is dominated by the gradient contribution from the selected action, either reinforcing it on success or suppressing it on failure with greater precision. \n\n**2. Comparison with temperature scaling of softmax** \n\nThe fundamental difference of our mixture distribution against the temperature scaling of softmax is in the **isolation of specific action**. While softmax divides all logits with a uniform temperature value to adjust the sharpness, ours directly injects a probability mass to one specific action. By doing so, the temperature globally adjusts the sharpness between all action pairs, while ours **amplify the gap between the selected action and the rest in a more targeted way**. \n\nFor example, consider a scenario where the policy outputs the following logits for three possible actions $(a_1,a_2,a_3)$ : $z=[2.0,1.0,0.5]$, where $a_1$ is the selected action. Using a standard softmax with $\\tau=1$, we get the probability distribution of $[0.63,0.23,0.14]$. We calculate the probability ratios of two pairs $a_1$ : $a_2$ and $a_2$ : $a_3$, which are approximately $2.72$ and $1.65$, respectively. Next, if we sharpen the distribution with $\\tau=0.5$ and make a scaled distribution $[0.84,0.12,0.04]$, we get two ratios of $7.40$ and $2.71$. The gap between the selected action $a_1$ and the unselected action $a_2$ grew significantly, but the gap between the unselected action also grew. On the other hand, if we apply our MEO with a mixture weight of $\\lambda=0.5$, we get a distribution $[0.81,0.12,0.07]$, with two ratios being $7.02$ and $1.65$. Notice that the gap remained exactly the same for the unselected actions, indicating that our approach preserves the policy's initial understanding of the relationships between the unselected alternatives, while directly focusing the signal to reinforce or penalize the selected action. \n\n### **[Q2,Q3] Questions regarding the Self-Prediction Head (SPH)**\n\n**Q 2.1: What is the purpose of the trained SPH?**\\\nThe purpose of the SPH is in building an **automatic feedback system that evolves throughout online iterations to gradually reduce the burden of human feedback**. As iteration proceeds, its accuracy in predicting navigation outcome grows by learning from both self-predicted and human-verified feedback. Therefore, once trained to a certain degree, it can be utilized not only for determining the task completion, but more importantly, providing itself with a reliable feedback for adaptation in unseen environments with limited human feedback.\n\n**Q 2.2: Does additional training of the head during testing also help improve the policy itself?**\\\nAdditional training of the head on the policy—without employing Self-Active Learning (SAL)—does have some effect on the parameters and the performances (SR:68.11→68.45 on the 'val_unseen' split of REVERIE). However, the improvement is marginal, and we argue that this training must be properly integrated into our proposed SAL framework to fully leverage its adaptive capabilities and realize the intended contribution of our approach.\n\n**Q 2.3: What happens if the agent determines at deployment that the task is not completed?**\\\nIn that case, the agent will use its self-generated signal to update and re-plan a new trajectory. If the agent's determination is correct, it will penalize its past decisions and explore better alternatives. Even if it’s a false signal, it still pushes the agent to find a more efficient path. This process of seeking a more definitive success state simultaneously trains the prediction head to become more robust for future attempts.\n\n**Q 2.4: What is the accuracy of SPH?**\\\nWe have provided the performance of the SPH in our appendix. The confusion matrix in Table 7 demonstrates high accuracy (82.19) and precision (80.78) of the SPH under the supervision of online pseudo-labels, making it a reliable source for self-training. \n\n**Q 3.1: How would SPH behave if it were trained only with human-verified feedback? (Concerns on error accumulation)**\\\nWe agree that self-training via pseudo-labeling inherently suffers from error accumulation. However, pseudo‑labeling maintains widespread use across diverse machine learning fields [1-3]. Importantly, these works do not assume pseudo-labels are flawlessly accurate; instead, they demonstrate that the benefits of using pseudo-labels (e.g., improved generalization, enhanced robustness under limited annotations, and reduced annotation costs) clearly outweigh their drawbacks.\n\nSimilarly, our SPH employs pseudo-labeling to continuously guide a robust adaptation during test-time navigation with limited human feedback. To address the reviewer's question more directly, we have conducted an experiment by training SPH only with human-verified feedback. Here, we observed that the accuracy of SPH in this setting shows 78.93, which is less than the accuracy under the default self-training setting (82.19, please see Q 2.4 above). This clearly indicates that a **certain amount of noise and self-correction through pseudo-labels brings robustness in SPH training**. Moreover, as also demonstrated by our empirical results in Sec 4.5 of our paper, the **practical advantages clearly surpass the risks associated with noisy labels**, justifying our approach within the active test-time VLN framework. \n\n[1] Wang, et al., \"Deep insights into noisy pseudo labeling on graph data.\", NeurIPS 2023.\\\n[2] Zhang, et al., \"Pseudo label refinery for unsupervised domain adaptation on cross-dataset 3d object detection.\", CVPR 2024.\\\n[3] Chen, et al., \"SAT: Improving semi-supervised text classification with simple instance-adaptive self-training.\", ACL 2022"}, {"review_id": "EJ86dwoKet", "reviewer": "Reviewer_9Upr", "summary": "This paper proposes ATENA, a test-time active learning framework that enables a practical human-robot interaction on uncertain navigation outcomes, to enhance the VLN performance in unseen test environments. Two core modules are constructed. Firstly, in the mixture entropy optimization module, the entropy is derived from a mixture of the action distribution and the pseudo-expert distribution. Then, this module minimizes entropy for successful navigation and maximizes it for failed ones. Secondly, a self-active learning strategy is designed, where the agent evaluates its navigation outcomes based on confident predictions, which enables the agent to actively engage through all iterations for adaptive decision-making. Experimental results on multiple VLN benchmarks, including REVERIE, R2R, and R2R-CE, demonstrate the superiority of the proposed approach over the compared baseline methods.", "questions": "As mentioned in the Weaknesses section, I would like to see: 1) detailed definition and pipeline of the proposed test setting, 2) the explanation about the comparison fairness, 3) the effectiveness evidence of the proposed approach in real-world experiments, 4) the visualization demonstration about the intermediate process of how the proposed approach impact the action decision.", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1 The proposed approach has good generalization ability, and the experimental results show that the proposed approach brings significant performance gain compared with the baselines on multiple VLN benchmarks.\n\n2 The motivation for designing mixture entropy optimization (MEO) and self-active learning (SAL) is described clearly. The paper is well-written and easy to follow. Figures and tables are nicely presented.\n\n3 Supplementary materials are provided with implementation details, computational cost comparison, and trajectory visualizations.\n\n&nbsp;\n\n### Weaknesses\n\n1 A major problem is that in Section 3.1, the authors did not define the test setting utilized in the \nproposed approach in detail. Since there exists obvious difference between traditional test\nsetting and the one utilized, it is important to describe the test setting utilized in their approach\n concretely (better with formulations for clarification), e.g., what is the agent’s input and output \nduring the test setting, and what is the form of information provided by the human/agent\noracles, how the proposed approach is integrated into the agent decision-making process, etc., to \nclearly demonstrate the difference between the two kinds of test settings.\n\n2 Since the proposed approach introduces the prior information of the test data, it seems that the direct comparison with the traditional baselines such as HAMT, DUET, BEVBERT, etc, is not fair. Moreover, relying on the human oracle looks like more unpractical than the traditional test setting in real-world applications. I would like to see more explanations and clarifications about these concerns. \n\n3 In Line 28-31, the authors claim that test-time adaptation is crucial for real-world robot navigation. However, there are no real-world experiments in the manuscript. Moreover, the results on the R2R-CE dataset, which is more similar to real-world setting, show that the proposed approach brings only ~1 point improvement over the baseline ETPNav and BEVBert on Val Unseen. Such results cause doubts about the effectiveness of the proposed method in the real-world setting.\n\n4 In the trajectory visualization in the supplementary material, the authors did not clearly present and explain the intermediate result (e.g., the information provided by the human/agent oracle and the discriminative effect of MEO and SAL) about how the proposed approach impacts the action decision. As a result, it makes the action decision process lack interpretability.\n\n5 In Section 2.1 in the Related Work section, some important references about the zero-shot discrete/continuous VLN [1-4] are missing, e.g., MapGPT, Open-Nav, etc.\n\n**References**\n\n[1] Chen K, An D, Huang Y, et al. Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments[J]. arXiv preprint arXiv:2412.10137, 2024.\n\n[2] Qiao Y, Lyu W, Wang H, et al. Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs[C]//ICRA, 2025.\n\n[3] Chen J, Lin B, Xu R, et al. Mapgpt: Map-guided prompting with adaptive path planning for vision-and-language navigation[C]//ACL, 2024.\n\n[4] Long Y, Cai W, Wang H, et al. Instructnav: Zero-shot system for generic instruction navigation in unexplored environment[C]//CoRL, 2024.", "was_revised": true, "final_justification": "The authors have addressed most of my concerns, including the task definition of test-time vision-language navigation, comparison fairness, explanation of the R2R-CE performance, etc. Based on the rebuttal and other reviewer's comments, I raise my score to 4 (Borderline accept).", "rebuttal": "### **[W1] Detailed test setting**\n\nWe apologize for our lack of detailed explanations of the task setting in Section 3.1. We will revise the section in our final draft as follows:\n\n>**3.1 Task Description**\n>\n> **Traditional Test-Time Vision-Language Navigation**\\\n> Vision-Language Navigation (VLN) requires an agent to follow natural language instructions, $I$, to navigate through a visual environment. In a standard offline evaluation, a pre-trained policy, $\\\\pi\\_{\\\\theta}$, is frozen and evaluated on a batch of $N$ distinct test episodes, $\\\\{e_n\\\\}\\_{n=1}\\^{N}$. For each episode $e_n$, the agent starts at a given position, receives the instruction $I\\^{(n)}$ and the initial panoramic visual observation $o\\_0\\^{(n)}$, and sequentially selects actions. At each timestep $t$, the policy outputs a probability distribution over the action space, and the agent executes the action with the highest probability, $a\\_t = \\\\arg \\\\max\\_a \\\\pi\\_{\\\\theta}(a|o\\_t\\^{(n)}, I\\^{(n)})$. This process is repeated until the agent selects a 'stop' action, generating a trajectory $\\\\tau\\^{(n)} = \\\\{(o\\_t\\^{(n)}, a\\_t\\^{(n)})\\\\}\\_{t=0}\\^{T-1}$. Performance is then calculated by averaging metrics across all $N$ episodes. The key aspect of this setting is that the policy does not change during evaluation.\n>\n> **Online Setting with Active Policy Adaptation**\\\n> Our work introduces an online, active adaptation setting that simulates a real-world deployment scenario. Unlike the traditional batch evaluation, test episodes are executed sequentially (i.e., with a batch size of one), allowing the policy to be updated between episodes. Specifically, after the agent completes a navigation episode, it receives a simple, binary success/fail feedback. This episodic outcome is then used to guide an immediate adaptation of the policy parameters. The updated policy is then carried over to the subsequent navigation episode, allowing the agent to learn from its experience on-the-fly. By doing so, we can study how an agent adapts on-the-fly to overcome distribution shifts using simple, interactive feedback, rather than requiring dense, ground-truth supervision.\n\n### **[W2] Comparison fairness and practicality**\n\n**1. Clarification on the comparison fairness**\n\nWe would like to respectfully clarify the nature of the feedback system and our experimental design. \n\nFirst, our approach does not use 'prior information' of the test data during adaptation. Rather, as mentioned from our response for [W1] above, the $N$ distinct test episodes are streamed only once without repetition. Furthermore, the feedback is post-hoc, meaning that it is only provided after an episode is finished. This is a highly challenging yet practical setting where the agent must learn to adapt its policy for future, unseen episodes based solely on the high-level outcome of its past performance, without ever revisiting the same task. \n\nNext, our comparison with baselines like DUET or HAMT is not to claim ATENA is a superior model architecture, but to demonstrate that any given offline-trained policy can significantly improve when augmented with our ATENA framework in this online setting. The adaptation performance itself is fairly assessed through comparison against other TTA methods, TENT and FSTTA, by applying them onto the same baseline policy. Our experiments clearly report the performance of the base policy and then the results after applying TTA methods, isolating the gains from the adaptation (e.g., 'DUET' and 'DUET w/ ATENA' in Table 1).\n\n**2. Practicality of the human oracle**\n\nWe agree that relying on a constant human oracle would be impractical. However, **our proposed framework was designed specifically to address this challenge** and maximize autonomy:\n\n- **Minimal expertise and effort:** As we have stated in L46, human feedback should require low effort for real-world practicality. In our framework, the feedback is a simple yes or no signal at the end of each episode, which is a highly practical form of human-robot interaction. \n\n- **Active Learning based adaptation:** The agent only requests human feedback for episodes where it is most uncertain (uncertainty-based thresholding in Sec 3.4.1). This active learning strategy reduces the need for human intervention by focusing only on the most informative scenarios.\n\n- **Self-Active Learning (SAL):** As one of our key contributions, the SAL mechanism enables the agent to autonomously predict the outcome of its own navigation for episodes where it is confident. As shown in Table 5, SAL is highly effective and provides a substantial performance boost. This drastically reduces the reliance on the human oracle and is a core contribution toward practical real-world application.\n\n### **[W3] Real-world Applicability**\n\n**1. Lack of real-world experiments**\n\nWe agree that sim-to-real transfer is a crucial step in evaluating test-time robotic navigation. As we do not possess a physical mobile robot, we are unable to conduct real-world experiments at this point. However, we are actively seeking opportunities to expand our work to the real-world, which may open many new interesting and challenging research directions to solve. \n\nNonetheless, we believe that our simulated experiments provide a strong and necessary foundation for this line of research for several key reasons. First, simulation offers **reproducibility and controlled evaluation**, which is critical when assessing the impact of an adaptation algorithm. Second, simulation **enables validation at a scale and diversity** (thousands of varied scenarios in the REVERIE, R2R, and R2R-CE benchmark) that would be infeasible in the real world. For these reasons, many recent studies emphasize these advantages of simulation, highlighting its reliability as a proxy for real-world evaluation [1,2].\n\n[1] Li, et al. \"Evaluating real-world robot manipulation policies in simulation.\", CoRL 2024.\\\n[2] Yang, et al, \"EMBODIEDBENCH: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents\", ICML 2025\n\n**2. Regarding the performance of R2R-CE**\n\nWe would like to emphasize that the comparatively less improvements on the R2R-CE dataset, relative to the substantial gains on REVERIE, does not reflect a limitation of ATENA's generalization to real-world scenarios. Instead, it **results from fundamental differences in task objectives and instruction granularity** between the two datasets. \n\nSpecifically, the REVERIE dataset is designed as a goal-oriented task, characterized by high-level and coarse instructions. In contrast, R2R and its extension R2R-CE consist of detailed and fine-grained navigation instructions, naturally requiring step-wise guidance for policy training. Therefore, the episodic feedback might be relatively sparse to bring notable performance enhancements for R2R variants compared to REVERIE.\n\nIn summary, the differential performance observed across datasets does not indicate a limitation in ATENA's ability to reflect the real-world but **rather highlights its specialized strength in goal-oriented navigation scenarios**.\n\n### **[W4] Visual Demonstrations of the Intermediate Results**\n\nWe apologize for the lack of interpretability of our visual demonstrations. However, it is a shame that, due to the rebuttal policy, we could not present an improved visual illustrations during this phase. We firmly promise to incorporate your feedback and replace the current version with more clear and detailed figures including the intermediate processes. \n\nIn the meantime, we would like to provide details of the current version and clarify once more our initial intention. To explicitly visualize how ATENA improves the trajectory, we sampled episodes that initially failed before adaptation, and repeated the same episode after processing the failure feedback through ATENA. Here, we observe that the policy selects a different action from the previous trial at major junctions (e.g., corridors connecting rooms or open spaces with multiple branches). This can be interpreted as our Mixture Entropy Optimization (MEO) using the 'failure' signal to penalize the overconfident but incorrect action by maximizing its entropy. This correction at the critical junction enables the agent to select the correct path on its next attempt, leading to a successful trajectory.\n\n### **[W5] Additional references in related work**\n\nThank you for the suggestion. We will revise the sentence in L92-L94 to incorporate the following discussions on the referred zero-shot navigation methods (bolded reference numbers are from the paper): \n\n> To overcome this discrepancy, recent research focus on enhancing zero-shot navigation by leveraging Large Language Models (LLMs) for sophisticated planning and reasoning [1-4, **30,31,32**]. Approaches like Open-Nav [2] and MapGPT [3]  aim to equip agents with stronger intrinsic capabilities to interpret complex instructions and navigate unseen environments from the outset, without task-specific fine-tuning. However, their reasoning capabilities without fine-tuning have yet to yield reliable performances compared to the trained ones. \n\n[1] Chen et al., \"Constraint-Aware Zero-Shot Vision-Language Navigation in Continuous Environments\", arXiv 2024.\\\n[2] Qiao et al., \"Open-Nav: Exploring Zero-Shot Vision-and-Language Navigation in Continuous Environment with Open-Source LLMs\", ICRA 2025.\\\n[3] Chen et al., \"Mapgpt: Map-guided prompting with adaptive path planning for vision-and-language navigation\", ACL 2024\\\n[4] Long et al., \"Instructnav: Zero-shot system for generic instruction navigation in unexplored environment\", CoRL 2024."}, {"review_id": "O8GnLHB817", "reviewer": "Reviewer_zXCt", "summary": "This paper presents a novel test-time adaptation (TTA) method that leverages human (or self-) feedback to update the navigation policy. The method is evaluated on the Visual Language Navigation (VLN) task using widely adopted benchmarks such as REVERIE, R2R, and R2R-CE. It is grounded in entropy minimization, and the authors further propose a mixture entropy optimization approach that decouples the entropy of the pseudo-expert distribution from that of the action distribution. A key contribution is a novel mechanism for incorporating human or self-feedback to dynamically maximize or minimize entropy based on the predicted outcome (success or failure). The method demonstrates strong performance, outperforming all prior TTA approaches, including those combined with active learning. This work represents a significant advancement in test-time adaptation and human-in-the-loop visual language navigation.", "questions": "1. If I understand the method correctly, the feedback is binary and based solely on whether the trial succeeds. This feedback is applied to the entire action trajectory. Have you considered cases where the navigation is technically successful but inefficient, for example, when the robot takes an unusually long or suboptimal path?\n\n2. Test-time adaptation is typically used for handling out-of-domain inputs, such as visually corrupted observations. However, in your setup, all observations are clean and in-domain. What motivates the use of entropy minimization in this context?\n\n3. Have you tried alternative approaches that may not have worked but are worth discussing? For instance, since human feedback is available for the entire navigation trajectory, could reinforcement learning (RL) be applied during test time?", "limitations": "Line 333: I don’t see a strong reason why your method cannot be applied to other visual navigation tasks. Most of these tasks share similar input and output formats; the main difference lies in the goal specification. The more critical limitation, as I noted earlier, is the reliance on trial-and-error and the assumption of a discrete action space, both of which may not always hold.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "$\\textbf{Strengths}$\n\n1. The innovative combination of test-time adaptation and human feedback has strong potential for real-world robot deployment with a human in the loop.\n\n2. The method is evaluated on multiple benchmarks against strong baselines and demonstrates promising results.\n\n\n$\\textbf{Weaknesses}$\n\n1. The method relies on trial-and-error; that is, the navigation episode must either succeed or fail before the model can adapt. In other words, the robot can attempt a task multiple times to complete a given episode. This assumption may be impractical in scenarios where the robot must adapt immediately, for example, to avoid obstacles, since failure could result in a crash.\n\n2. The method also assumes that the robot can reset itself to the original starting position. For instance, in the Supplementary Figures, after completing Trial 1 and receiving human feedback, the robot begins Trial 2 from the reset position rather than from where it previously stopped.\n\n3. Real-world navigation typically involves a continuous, rather than discrete, action space. As a result, this method may not be directly applicable to real robots such as [NOMAD]. However, I am not aware of any existing TTA work that proposes entropy minimization for continuous action spaces, so this limitation is acceptable given the current state of the field.\n\n$\\textbf{Reference}$\n\n[NOMAD] Nomad: Goal masked diffusion policies for navigation and exploration (ICRA'24)", "was_revised": true, "final_justification": "The authors have mitigated most concerns during rebuttal. I would like to maintain a positive score toward acceptance.", "rebuttal": "### **[W1] Reliance on episodic trial-and-error**\n\nWe fully agree with the reviewer's concern regarding the importance of immediate, step-by-step adaptation, particularly in scenarios requiring timely obstacle avoidance. However, the primary focus of our work was to emphasize **how a simple human-robot interaction through episodic feedback can be utilized to control the prediction entropy and improve test-time adaptation**. Therefore, our work addresses the distinct but equally crucial challenge of adaptation, where the goal is to learn from high-level task outcomes, a problem fundamental to language-guided navigation.\n\nFurthermore, as demonstrated in our experiments and discussions regarding FSTTA, performing intermediate adaptations in between navigation steps can sometimes degrade baseline performance, as frequent parameter update at test time has the potential to hinder general navigation knowledge acquired during training. Also, as we have shown in Table 8 in the appendix, such adaptation methods significantly increase the latency during navigation rollout, making them computationally impractical compared to episodic methods like ATENA. Nonetheless, we agree this is a valuable direction, and we believe a key challenge for future work is to merge the benefits of our efficient, sparse adaptation with more immediate, step-wise corrections, without performance degradation or high latency shown in our analysis.\n\n### **[W2] Resetting the starting position**\n\nWe would like to respectfully clarify that our method **does not assume nor necessitates the robot to reset itself to the original position**. The scenarios illustrated in the supplementary material were a deliberate setup, designed to clearly show the direct impact of ATENA's adaptation on a single, controlled problem (i.e., before vs. after feedback). For all results in our main tables, we strictly adhere to the standard benchmark protocols where each distinct test episode is attempted only once without repetition.\n\nHowever, we agree with the spirit of the reviewer's concern regarding real-world continuity. To directly address this, we conducted an additional experiment on the more realistic IVLN-CE [1] benchmark. In this setting, episodes are chained together—the end of one path is the start of the next—eliminating arbitrary resets and better simulating a single, continuous deployment. ATENA's episodic framework adapts naturally to this setting by treating each distinct instruction in the sequence as a self-contained episode for feedback and adaptation. Using the MAP-CMA [1] policy as our baseline, we evaluated  ATENA under the 'Inferred via RedNet' map source with the 'Iterative' mapping procedure (Please see L14 of Table 4 in [1]), and demonstrated clear improvements across diverse metrics for the 'val_unseen' split:\n\n| Method   | SR ↑ | SPL ↑ | nDTW ↑|\n|-|-|-|-|\n| MAP-CMA | 35 | 32 | 54 |\n| w/ ATENA | **36**| **34** | **56** |\n\nThese strong results show that ATENA's adaptation mechanism is **robust and effective not only in standard single-episode evaluations but also in more realistic, continuous navigation scenarios**.\n\n[1] Krantz, et al., \"Iterative vision-and-language navigation.\", CVPR 2023.\n\n### **[W3] Entropy-based TTA in continuous action spaces**\n\nAs the reviewer correctly noted, adapting to continuous action spaces using entropy-based TTA is a novel and under-explored area. In our work, we evaluated ATENA on the R2R-CE benchmark , which explicitly replaces the discrete action space of R2R with a continuous one to better simulate real-world robotic control. As shown in Table 3 in our paper, applying ATENA to baseline policies like ETPNav and BEVBert in this continuous environment leads to notable performance improvements. Furthermore, thanks to your constructive feedback, we have also demonstrated ATENA's performance in the IVLN-CE benchmark (Please see [W2] above), which simulates a more realistic continuous environments. We believe our positive results in both of these challenging benchmarks represents a valuable step forward for applying the method to real-world robotic systems.\n\n### **[Q1]  Success cases with suboptimal and inefficient trajectory**\n\nDuring training, the navigation policies explicitly learn to follow the optimal expert trajectories. However, even though the policy learns some general navigation knowledge through training, navigating in unseen environments presents significant challenges due to differences in environment structure and conditions. Therefore, we argue that a **certain degree of exploration is necessary and allowed to facilitate successful adaptation in an unseen environment**. Accordingly, we classify the task as successful even if the trajectory itself may appear suboptimal or inefficient compared to the optimal path.\n\nTo support our claim, we have conducted an additional experiment, where we uniformly divided the test episode sequence in the 'val_unseen' of REVERIE into four splits. Here, we aim to compare the average trajectory length during adaptation against the average ground truth trajectory length at each split. As seen in the table below, the initial split shows a long average trajectory length compared to the ground truth, indicating early stage explorations. However, as the episodes proceed, the gap reduces and the trajectories generated during adaptation become closer to the optimal path, with increasing success rates. This demonstrates that ATENA enables the agent to **progress from initial, inefficient exploration to increasingly optimal navigation paths as it adapts, rather than simply settling for any kind of successful route**. \n\n|  Scenario A. | Pred Length | GT Length | Pred:GT Ratio | SR ↑ |\n|-|-|-|-|-|\n| split 1/4   | 23.14      | 9.54     | 2.43 | 70.27      |\n| split 2/4  | 21.13      | 9.37     |  2.26  | 86.49     |\n| split 3/4   | 19.74      | 9.52     | 2.07   | 86.11     |\n| split 4/4  | 13.73      | 8.31     | 1.65   | 91.67    |\n\n| Scenario B.  | Pred Length | GT Length | Pred:GT Ratio | SR ↑|\n|-|-|-|-|-|\n| split 1/4 |21.71 | 9.45 | 2.30 |   54.05 |\n| split 2/4 |20.06 | 9.26  | 2.17 | 91.89  |\n| split 3/4 |16.01| 8.86 | 1.81 | 94.44 |\n| split 4/4 |14.44 | 9.18 | 1.57 | 94.44  |\n\n### **[Q2]  Test-time adaptation and the motivation for using entropy**\n\nThe datasets we used (REVERIE, R2R, and R2R-CE) explicitly include the ‘validation_unseen’ and ‘test_unseen’ splits, which evaluate navigation performance in entirely new indoor environments never encountered during training. Therefore, these validation splits are considered as the ‘out-of-domain scenarios’ in the literature [1,2], despite having visually clean observations. \n\nFurthermore, test-time adaptation (TTA) methods are not only limited to visually corrupted inputs but are broadly applicable to any domain shifts encountered during inference [3,4]. In specific, entropy minimization is a prevalent TTA technique for adapting models with stochastic outputs, since it explicitly handles output distribution in an unsupervised manner.\n\nVLN policies also output a probability distribution over action spaces, naturally leading to increased prediction uncertainty with high entropy in the out-of-domain scenarios. For example, the baseline policy, DUET, shows significantly higher average entropy on the out-of-domain validation set (val_unseen: 6.345) compared to the in-domain validation set (val_seen: 2.945). Therefore, entropy was an essential selection as a learning signal in our framework, where we creatively further guide its optimization direction through episodic feedback, which clearly improves navigation performance and adaptation.\n\n[1] Zheng, et al., \"Towards learning a generalist model for embodied navigation.\", CVPR 2024.\\\n[2] Liu, et al., \"Vision-language navigation with energy-based policy.\", NeurIPS 2024.\\\n[3] Zhang, et al., \"Domainadaptor: A novel approach to test-time adaptation.\", ICCV 2023.\\\n[4] Su, et al., \"Beware of model collapse! fast and stable test-time adaptation for robust question answering.\", EMNLP 2023.\n\n### **[Q3] Could RL be used as an alternative?**\n\nThank you for bringing up this insightful question. Motivated by this perspective, we conducted additional experiments on the 'val_unseen' of REVERIE, incorporating the Reinforcement Learning from Human Feedback (RLHF) paradigm into DUET. Specifically, for a fair comparison with our method, we provided episodic rewards of +1 for success, and -1 for failure. The policy is updated using the REINFORCE policy gradient algorithm. The results in the table below show DUET + RLHF achieving notable improvements (SR: 60.98, SPL: 36.04), clearly surpassing the original DUET (SR: 46.98, SPL: 33.73) as well as established TTA baselines such as TENT (SR: 47.55, SPL: 33.99) and FSTTA (SR: 54.15, SPL: 36.41). These results strongly validate the effectiveness of leveraging episodic human interaction to enhance navigation adaptation. However, updating the policy with a fixed integer of -1 or 1 regardless of the episodes shows limitations in handling the stochastic nature of the policy, naturally yielding lower performance compared to ATENA which **dynamically controls the confidence of the policy itself through the Mixture Entropy Optimization**.\n\n|Methods| DUET | w/ TENT |w/ FSTTA |w/ RLHF |w/ ATENA|\n|-|-|-|-|-|-|\n|SR ↑|46.98|47.55|54.15| 60.98| **68.11**|\n|SPL ↑|33.73|33.99|36.41|36.04| **45.82**|\n\n### **[+] Additional Comments**\n\nThank you for your thoughtful interpretation of our limitations. As you have pointed out, we have indeed demonstrated ATENA's generalizability to other visual navigation tasks, and we respectfully direct you to our response to Reviewer 1sz2 (W1) for detailed results. Regarding other limitations, we hope our responses have resolved your concerns about the trial-and-error system and action spaces. Thank you."}]

Author rebuttal:

Meta-review: This paper provides a novel TTA method for improving vision-language navigation policies from sparse (binary) human feedback. The framework ATENA is based on two simple techniques: mixture entropy optimization (MEO) and self-active learning (SAL). Experiments in the paper show that ATENA outperforms existing TTA baselines across several VLN benchmarks, and further experiments from the rebuttal show that ATENA also outperforms in an image-goal navigation task and in continuous environments (IVLN-CE). 

The rebuttal/discussion period significantly strengthened the paper, addressing all major concerns (regarding generalizability of the method, the role of SAL and how accurate the SAL predictor needs to be, the required accuracy of human feedback, providing more theoretical justification for MEO) raised by the reviewers. In the end, the reviews concluded with a positive impression of the work and its timeliness. I strongly encourage the authors to incorporate all proposed changes from the final remark into the camera-ready, which will significantly strengthen the quality of the manuscript.
Decision: Accept (poster)