M$^3$E: Continual Vision-and-Language Navigation via Mixture of Macro and Micro Experts
Authors: Yongliang Jiang
Huaidong Zhang
Xuandi Luo
Shengfeng He
Venue: ICLR 2026 Poster (2026)
Keywords: Vision-and-Language Navigation
Continual Learning
Embodied AI
Large Language Models
Primary area: transfer learning, meta learning, and lifelong learning

Abstract: Vision-and-Language Navigation (VLN) agents have shown strong capabilities in following natural language instructions. However, they often struggle to generalize across environments due to catastrophic forgetting, which limits their practical use in real-world settings where agents must continually adapt to new domains. We argue that overcoming forgetting across environments hinges on decoupling global scene reasoning from local perceptual alignment, allowing the agent to adapt to new domains while preserving specialized capabilities.
To this end, we propose M$^3$E, the Mixture of Macro and Micro Experts, an environment-aware hierarchical MoE framework for continual VLN. Our method introduces a dual-router architecture that separates navigation into two levels of reasoning. A macro-level, scene-aware router selects strategy experts based on global environmental features (e.g., office vs. residential), while a micro-level, instance-aware router activates perception experts based on local instruction-vision alignment for step-wise decision making. To preserve knowledge across domains, we adopt a dynamic momentum update strategy that identifies expert utility in new environments and selectively updates or freezes their parameters.
We evaluate M$^3$E in a domain-incremental setting on the R2R and REVERIE datasets, where agents learn across unseen scenes without revisiting prior data. Results show that our method consistently outperforms standard fine-tuning and existing continual learning baselines in both adaptability and knowledge retention, offering a parameter-efficient solution for building generalizable embodied agents.

Published as a conference paper at ICLR 2026
E CONTINUAL VISION-AND-LANGUAGI NAVIGA- TION VIA MIXTURE OF MACRO AND MICRO EXPERTS
Yongliang Jiang' Huaidong Zhang Xuandi Luo" Shengfeng He
South China University of Technology, SSingapore Management University
{ftyljiang ftxuandi. 1uo) @mail. scut.edu. huaidongz@scut. edu. cn, shengfenche& smu. edu.sq
ABSTRACT
Vision-and- Lnd-Laggu Navigation (VLN) agents have shown strong capabilities following natural language instructions. However, they often struggle to gener- alize across environments due to catastrophic forgetting, which limits their prac- tical use in real-world settings where agents must continually adapt to new do- mains. We argue that overcoming forgetting across environments hinges on de- coupling global scene reasoning from local perceptual alignment, allowing the agent to adapt to new domains while preserving specialized capabilities. To this end, we propose E the Mixture of Macro and Micro Experts, an environment- aware hierarchical MoE framework for continual VLN. Our method introduces dual-router architecture that separates navigation into two levels of reasoning. A lacro-level, scene-aware router selects strategy experts based on global en- vironmental features (e.g., office vs. residential), while a micro-level, instance- aware router activates perception experts based on local instruction-visite align- ment for step-wise decision making. To preserve knowledge across domains, we adopt a dynamic momentum update strategy that identifies expert utility in new environments and selectively updates or freezes their parameters. We evaluate M in domain-incremental setting the R2R and REVERIE datasets, where agents learn across unseen scenes without revisiting prior data. Results show that our method consistently outperforms standard fine-tuning and existing con- tinual learning baselines in both adaptability and knowledge retention, offering a parameter-effin eerfmeetr-efficientt solution for building generalizable embodied agents. Our project page is available at
INTRODUCTION
Vision-and-Language Navigation (VLN) tasks require embodied agents to follow free-form natu- ral language instructions to reach target locations in photorealistic indoor environments Anderson et al. (2018): Qi et al. 2020); Ku al. (2020); Thomason al. (2020); (Zuu et al. (2021). This setting demands the tight integration of visual perception, language grounding, and sequential deci- sion making under partial observability. Significant progress has been made in recent years. Early approaches focused on aligning panoramic visual inputs with language using multimodal encoders and imitation learning Hao al. (Majumdar al. (2020), often leveraging large-scale vision- language pretraining Chen et al. (2020); [Tan & Bansal 2019 and data augmentation to improve generalization Tan et al.J (2019): ; Ma et al. (2019a). Beyond cross-modal alignment, subsequent work introduced memory modules Hong et al. (2021). I). backtracking mechanisms Ma et al. (2019b), and topological map construction Chen al. (2021a) to support long-horizon reasoning. More re- cently, large language models (L.LMs) have been incorporated to enhance world knowledge access and compositional reasoning, either through prompting or fine-tuning Pan et al. (2033: Zheng et al. 2023 Zhang et al. (2024).
Despite these advances, most VLN systems are trained on static datasets and exhibit poor general- ization unseen environments. Adapting new domains often requires expensive retraining and can lead to catastrophic forgetting of previously learned skills. Continual learning VLN remains
Corresponding Author.

---

Published as a conference paper at ICLR 2026
underexplored. Existing methods primarily rely rehearsal buffers that replay past trajectories dur- ing training]eeome et al. (2024); [i al. (2024), which increases storage and computational overhead and may raise privacy concerns. Furthermore, VLN presents unique challenges not encountered in classification tasks (e.g., CL--MoE Huai et al. (2025), ) such as the need for sequential planning under partial observability and fine-grained instruction grounding.
We argue that overcoming forgetting in continual VLN requires separating high-level scene rea- soning from low-level perceptual alignment. Scene-level understanding captures structural patterns that generalize across domains, such as layouts typical of offices or residential spaces. In contrast, token-level grounding provides fine-graine -ra-irained cues for local decision making. Conflating these two levels of reasoning can result in brittle policies and poor transfer across tasks.
To address this, we propose E Mixture of Macro and Micro Experts for continual vision-and- language navigation. M is replay-free continual learning framework based on an end-to-end trainable LLM agent. introduces hierarchical mixture-of-exxe of-eeertss architecture with dual routing to separate global and local reasoning. The macro router converts the agent's cognitive map of visited and frontier nodes into a sparse graph, performs topology-aware propagation using graph neural network, and attends over nodes using instruction embeddings to produce scene-level expert weights. In parallel, the micro router derives token-wise expert activations from the hidden states of the LLM. These two routing signals fused to select experts in MoE-LoRA layers, enabling context-sensitive specialization while maintaining sparse and efficient computation.
To support continual learning without replay, E introduces dynamic momentum update strategy that consolidates knowledge across domains. For each layer, we track expert usage by accumulating the fused routing weights over all tokens in the current task, normalize obtain per-expert contri- butions, and select the top-perforning experts. These are updated with larger momentum, while less relevant experts are updated conservatively preserve generalizable knowledge. This mechanism enables rapid adaptation to new environments while mitigating forgetting of prior domains.
We evaluate E in agents learn in disjoint scene sets without revisiting prior data. We report standard VLN metrics such as success rate (SR) and success weighted by path length (SPL), along with continual learning met- rics including forward transfer (FWTT and backward transfer (BWT). M consistently outperforms strong baselines, improving both generalization and retention across domains.
setting on the R2R and REVERIE benchmarks, where
Our contributions are as follows:
We define the disionannd-lagguae navigation continual learning (VLNCL) setting, with task protocol and evaluation metrics, and present the first replay-free mixture-of-experts framework for this setting.
We introduce E dual-router hierarchical architecture that separates global scene reasoning from local teven-level grounding, enabling more precise and modular expert specialization.
We propose dynamic momentum update mechanism that consolidates task-critical experts using differentiated update rates, supporting fast adaptation without data replay.
We conduct extensive experiments R2R and REVERIE, showing that M improves both navigation success and continual learning performance, demonstrating stronger generalization and reduced forgetting across environments.
2
RELATED WORK
2.1
VISION-AND-LANGUAAA NAVIGATION
Cross-modal grounding. Early VLN methods focused on learning joint representations of panoramic visual inputs and language instructions Hao et al. (2020); Majumdar et al. (2020); Guhur et al. [202I); Chen et al. (2021b), typically using multimodal transformers. Subsequent improve- ments leveraged large-scale Vision-language pretraining Chen et al. (2000) Li al. (2020): Su et al.
(2019) Tan & Bansal 2019). , data augmentation Tan et al. (2019); ; Li et al. (2022) Dou & Peng
(0022: ; Li & Bansal 2023), and advanced training regimes such as imitation learning combined
with reinforcement learning Wang et al. (20T8): Ma et al. (019a): Zhu et al. (2020). These strate-
2

---

Published as a conference paper at ICLR 2026
gies enhanced the agent's 's ability to ground linguistic references in visual observations and improved generalization to unseen environments.
Planning and memory. Beyond alignment, effeecive navigation requires reasoning over long trajec-
torics. Several approaches have addressed this through memoory-assee encoders Hong et al. (2021)
Chen et al. (202Tc), self-correction and backtracking mechanisms Ke et al. (2019): Ma et al. (2019b)
and explicit topological or semantic mapping An et al. (2021); Chen et al.
(0211))] Liu et al. (2023)
Wang et al. (2023); ) Liu et al. (2024). External knowledge sources and semantic priors have also
been incorporated to guide planning in unfamiliar environments Lin et al. (2022) Li et al. (2023)
Language models for navigation. Recent work has explored using large language models (LLMs)
to inject world knowledge and enable more structured reasoning Pan et al. (2023); Zhou et al. (2024)
2025| Qiao al. (2025): Chen et al. (2024): Lin et al. (2024); ); Long et al. (2024); Zhang et al.
(2024); Zheng et al. (2023). Some approaches treat the LLM as frozen API, while others pursue
end-to-end training of multimodal LLM--bsed agents [Song et al. (2025); Mu al. (2023); Zhang
ee al.] (2024), aiming to unify language understanding, perception, and action within a single policy.
These models demonstrate promising generalization, but still struggle to adapt continuously across
diverse domains.
Despite these advances, most VI.N agents are trained on fixed datasets and perform poorly when deployed in novel environments. Adapting new domains often requires full retraining, which is computationally expensive and risks catastrophic forgetting of prior knowledge. These limitations motivate the development of continual learning (CL) capabilities for VLN agents.
2.2
CONTINUAL LEARNING FOR EMBODIED AGENTS
A critical capability for embodied agents is to learn incrementally across domains while preserving prior knowledge, a goal that CL explicitly targets. In the context of VLN, however, research in CL remains limited. Most existing approaches rely on vehearsal-basee al. (2024), where past trajectories are stored and periodically replayed to mitigate forgetting. For example, PerpR/ESR JJeong al. (2024) maintains buffer of navigation episodes and interleaves them during training on new environments. While effective, these methods suffer from scalability and privacy concerns. Maintaining large buffers imposes storage and compute overhead, and re- playing raw trajectory data raises concerns in applications where privacy or memory constraints are critical. To address these issues, recent work in related multimodal domains has explored replay- free solutions. For instance, CL-MoE Hual et al. (2025) proposes a Mixture-of-Expertt (MoE) framework for continual visual question answering. where task-specific experts enable selective knowledge retention. However, this line of work has not yet been extended to VLN, which poses additional challenges due to its sequential, partially observable nature and the need for consistent spatial reasoning over time.
methods Jeong et al. (2024); Li
Our work bridges this gap by introducing a replay-free MoE-based framework specifically tailored for continual VLN. In contrast to prior CL approaches in vision or language tasks, we explicitly account for the hierarchical structure of navigation: separating global scene reasoning from local token-level grounding. This separation allows for targeted expert specialization, improves transfer- ability, and supports continual adaptation without relying on trajectory replay.
PROBLEM FORMULATION
3.1 VISION-AND-LANGUAAE NAVIGATION CONTINUAL LEARNING (VLNCL)
A lasionaand-language navigation (VLN) task is defined on an indoor environment represented as a navigation graph 9 = (V, 8), where nodes V are viewpoints and edges & denote navigable con- nections. Given natural- lanural-language instruction I = (W1,... w2). agent starts at node vo and sequentially observes panoramic views oe and selects actions ag E A(vx) according to a policy To(a, sc, I). where s; denotes the agent's 's state at step t. Executing the policy produces trajectory T = (8o, Op, Cp,,... ".,....,8r) Sr). Training generally seeks to minimize the expected cumulative loss:
T Σ [(rr, at)
min Eyy, EraTe 0
(1)
t=0
3

---

Published as a conference paper at ICLR 2026
where L can be instantiated as imitation learning, reinforcement learning, hybrid objectives.
In continual VLN, the agent faces a stream of tasks {Di where each D, contains navigation graphs and language instructions from (possibly) new domain. At stage t, the agent starts from consolidated parameters O,-1, adapts them on D, to obtain task-specific parameters 2,, and then fuses them into consolidated parameters O₁:
+ arg mjn BGG,D-,, B
Σ ((sk
θ, C(0,-1, Φt).
(2)
where -(--)) a consolidation operator (instantiated by our momentum update in Sec. 4.2 Unless otherwise noted, past-task data are not revisited (no rehearsal), making catastrophic forgetting central challenge.
3.2 LLM-BASED NAVIGATION AGENT
Our agent follows the paradigm of end-to-end trainable LL.M-based navigation, in the spirit of generalist models such as NaviLLM Zheng et al.] (2024). EmbodiedGPT Mu et al. (2023). and Nadid Zhang et al. (2024). Unlike approaches that query frozen APIs, we directly fine-tune mul- timodal LLM backbone on VLN. The system has two components: scene encoder and a language model.
Scene Encoder. At each step the environment provides a discretized panoramic observation (e.g., 36 views). A pretrained Vision Transformer (ViT) first extracts per-view features, which are then fused by a lightweight multi-view encoder (Transformer or attentive pooling) to contextualize spatial relations across views and yield viewpoint-level scene embeddings. These embeddings update the cognitive map (visited and frontier nodes) used by downstream routing (Sec. 4.1
LLM Backbone and Action Generation. We use a decoder-only Transformer (e.g., a 7 LLM) as the policy core. The instruction, navigation history, and scene embeddings are serialized into unified input schema and fed to the LLM; the hidden state at the action query position is passed to small action head to score reachable candidate viewpoints, defining single-step autoregressive decision. The process repeats until STOP is emitted or the step budget is exhausted.
Modification for M Our key architectural innovation is the replacement of the standard Feed- Forward Network (FFN) layers within the LLM backbone with our proposed M layers, as detailed in Sec. This modification allows the agent dynamically route computations through specialized experts based on both high-level strategic needs and low-level semantic context, while supporting continual learning across domains.
METHOD
We propose E a framework that equips large language model-based VLN agents with continual learning capabilities. It consists of two main components. The Macro-Micro MoE (Sec. 4.1 uses a dual-router design: a Macro Router captures global, scene-level strategies, while a Micro Router handles local, token-level semantics. The Dynamic MoE Momentum Update (Sec. 4.2) reduces forgetting by identifying task-critical experts and updating them with differeentated momentum, allowing rapid adaptation while preserving transferable knowledge.
4.1 MIXTURE OF MACRO AND MICRO EXPERTS
4.1.1 MACRO ROUTER
The macro router Gma is designed capture global, structural regularities of the navigation envi- ronment and align them with the high-level task intent, Instead of naively aggregating visual infor- mation, we propose Topology-Aware, Task-Focused (TATF) Routing. This method uses GNN to understand the structure of the agent's known world (the "Where"), then applies cnoruction-uuiden attention to focus relevant areas (the "What").
Step 1: Global Map Representation. From the cognitive map available at the current step (includ- ing both nodes already visited and nodes discovered but not yet explored), we derive pairwise node
4

---

Published as a conference paper at ICLR 2026
Seen Task Domain
Continual Learning
Wh-l
Phol
Dynamic Mosccttu Update
Dynamic Momceluu Updott
Dynamic Momcotura Update
Train
Agett 6
Agant
Agent
Agent
Agent By.
Mixture of Macro Micro Experts (M-
Representation
Topology aware Propagation
Task-focused Aggregation
Expart
Vacro Router
vnlenan, (2) Im
Hgu (N,(AA,
edfraaz (M
Esport
Input
Tokanized
Plase the Uichee
Pheas
the
Mothen
Expert nd
image Input
Micro Rouber
Export
Figure 1: Overall architecture of E The framework decouples macro-level scene reasoning and micro-level token grounding via dual-router design. The Macro Router (blue) builds a task-aware scene representation using GNN-based propagation over cognitive map. The Miero Router (pur- ple) computes token-wise expert weights from hidden states. Both signals are fused to route experts the MoE-LoRA layers for planning.
distances and threshold them form sparse adjacency A, encoding connectivity among N nodes. Each node E V, is initialized with a feature vector that integrates panoramic visual embeddings, spatial location, temporal step embeddings, and navigation state indicators. These node features are collected into matrix:
[x]] Vule, E ARR.ERNN Nxd
Importantly, the cognitive map is constructed online from the agent's exploration history, including visited and frontier nodes, rather than using any pre-defined global map.
Step 2:
Topology-aware Propagation The node features X and adjacency matrix A, are fed into Graph Neural Network (GNN). The GNN performs message passing learn the spatial relationships between nodes, yielding topology-awan representations:
H GNN (A, X) Nxd.
Step 3: Task-focused A Aggregation. Let Emb E [Rdx be the embedding of the navigation instruc- tion. We use Embra query to compute attention scores on topology-awand node representations Hgm (where h, is the representation for node v):
Os = softmax,(h, Emb
S
Σ a h Rdh
(5)
WeVi
The resulting vector s; is aggregated scene representation that is both structurally informed and semantically focused on the current task.
Step 4: Scene-level routing. Finally, the aggregated scene vector s, is passed through routing head to produce the macro-level expert weights:
w ma
Gmm (st) = Softmax (MLP(s:)) E IR".
(6)
where n is the number of experts. This routing mechanism provides the model with a richer, more actionable understanding of the scene, enabling more effective macro-level planning.
5

---

Published as a conference paper at ICLR 2026
4.1.2 MICRO ROUTER
At the micro (token-level) granularity, we introduce a router G that operates on each token em- bedding to produce distribution over experts. Given the hidden state h of token in the navigation step, the generie form follows the MoE gating paradigm:
W mi (h) = Softmax (MLP(h)) E R",
(7)
where w ToT ..... W R denotes the token-wise routing weights experts, and with the probability of choosing expert Ej. This micro router captures fine-grained semantics within the instruction stream (e.g., in "go to the kitchen," the verb token "go" is inclined to action reason- ing experts, while the noun token "kitchen" prefers object/scene understanding experts), enabling context-sensitive expert specialization. We train G the task data D, so that it learns select local experts conditioned on token-level representan,.....nnnnss.
4.1.3 DUAL-ROUTER FUSION
Finally, the macro router Tm and the micro router G are fused to produce the final expert routing weights. The macro weights w provide global, scene-level prior that reflects the structural layout and high-level task intent, while the micro weights w offer token-level, fine- tine-graine judgments that adapt the current linguistic context. We combine them through a convex interpolation:
W B TT + (1 3) w R"
(8)
where is a hhper--prannnn parameter that balances the importance of global priors and local adaptivity. This dual-router fusion equips the model with both strategic awareness from the macro router and fine-grained adaptivity from the micro router, leading to more precise and context-sensitive expert selection.
4.2 DYNAMIC MOE MOMENTUM UPDATE
In continual VLN setting, idcal agent must assimilate knowledge from the current task t while preserving the general navigation capabilities acquired from previous tasks ......f-1.. To achieve this balance, introduce Dynamic MoE Momentum Update mechanism that differentiates perts by their contribution to task and updates their parameters with task-aware momentum.
Step 1: Expert workload and contribution. For each MoE layer, denote by w(x) R the fused routing weights of a token & E D, obtained from the dual-router fusion in Eq. equation We first accumulate per-expert workload over all tokens of Di.
Σ w(x) E R..
(9)
DED
and normalize obtain a contribution distribution,
ExED, wi(x) EE EvED, wj(x)
(E,))
A pl
(10)
E
ulil
Here, (E,) is the normalized importance of expert E, on task L.
Step Top-K expert selection. We select the K most active experts as important experts and treat the rest as non-important:
cimp = (p,
E -EE (EI...., En) !
(1I)
Step 3: Momentum-basse parameter consolidation. Let θ,-1 be the consolidated expert pa- rameters from previous tasks and o, the parameters obtained by fine-tuning on D, (initialized from 1) We assign a momentum coefficient X; to each expert E, based on group:
if E, E cimp
7,
d; =
? E (0,0.5),
(12)
1-7,
if E, E ETOn,

---

Published as a conference paper at ICLR 2026
so that important experts lean more toward 4 (aggressive adaptation), while non- -on-mportant experts preserve 6t-1 (conservative update). Let A = [At..... An] denote the per-expert momentum vector (broadcast to the parameter shapes). The consolidated parameters are then computed by an element- wise interpolation:
+ A 0 (1 A) @ t
(13)
where @ denotes element-wise multiplication and is the all-ones mask.
This task-aware momentum consolidation adapts important experts rapidly to the current task while preserving generalizable knowledge non-important experts, thereby mitigating catastrophic for- getting and improving transferability in sequential navigation.
5 EXPERIMENTS
Datasets & Protocol. We evaluate the proposed method on R2R Anderson et al. (2018) and REVERIE Qi et al.) (2000) (goal-oriented, object-anchored instructions) in Matterport3D photo- realistic indoor environments. In R2R, instructions are received step-by-step. while REVERIE al- ways receives goal-oriented and object-anchored instructions. For each dataset, we adopt scene- partitioned continual stream. Concretely, a base agent is first pretrained on the train split. Its performance is then evaluated on the val-seen split, which consists of previously encountered environments but novel instructions. This evaluation, denoted as BaseAgentO, serves as a reference for in-distribution capability, and is further used assess both performance on trained environments and generalization to new instructions within them.
We then partition the val-unseen scenes by scene id into D disjoint domains (Si...., Sp.....,ppk, Sp). Each domain defines one continual-leaminng task Ta, where all episodes belong to environments in Sa. To construct reliable per-task splits, we adopt an 8:2 ratio for training and validation within each val-unseen domain, while discarding scenes with fewer than or equal 10 validation episodes to ensure sufficient validation(see Appendix IA.I).
At stage d {I....,) D), the agent is initialized with consolidated parameters Oa-1 and fine-tuned only on the training data of Ta, without accessing prior-task data (replay-free continual learning). After finishing Ta. the updated policy is evaluated all tasks O:d to track both adaptation to new domains and retention of prior knowledge; after the final stage, we additionally report performance on all D tasks. To control training budgets across methods, the number of optimization steps for each task is fixed, with the same early-stopping and validation protocol. Episode sampling follows standard VLN practice, with fixed maximum step limit per episode and candidate viewpoint set. This protocol provides controlled, privacy-friendl) benchmark for continual learning in VLN, stressing cross-domain adaptation and catastrophic forgetting.
Evaluation Metrics. We report both standard VLN and continual-learnigg metrics.
VLN metrics: (i) Success Rate (SR) the fraction of episodes where the agent stops within 3m of the goal; (ii) Success weighted by Path Length (SPL) SR penalized by the ratio between the agent's 's trajectory length and the shortest path; (iii Navigation Error (NE) the average distance (meters) between the stopping point and the goal; (iv) Oracle Success Rate (OSR) - the fraction of episodes which any visited viewpoint is within 3m of the goal.
Continual learning metrics: After completing training on task f, the policy is evaluated on all tasks Out using the above VLN metrics. Let Re, denote the performance on task i after training up to task t. We report the continual-learning metrics:
1 BWT T- Σ RRr, R..)..FT T
Σ RR,1, RR.))
(14)
Here, Backward Transfer (BWT) measures how learning tasks influences performance pre- vious tasks, while Forward Transfer (FWT) evaluates zero-shot generalization by testing each task before it is trained. Note that, unlike [i et al. (2024). our definition of BWT explicitly includes the BaseAgent (i = 0), so that it also measures the extent of forgetting relative to the initial in- distribution capability.
Baselines. To demonstrate the effectiveness our method, we compare against several represen- tative continual learning strategies, built the same M.L.-b navigation backbone (Sec. 3.2
7

---

Published as a conference paper at ICLR 2026
Table I: Domain-incres incremental learning in R2R environment. Methods are categorized by strategy: Reg (regularization), Reh (rehearsal), and RF (replay-free).
Strategy
Method
AvgSR%
AvgSPL
AvgNE
BWT
FWT
Reg
Reh
Finetune
X
63.28
59.08
3.72
-5.42
-2.41
L.2
58.78
56.20
4.23
-5.00
.3.43
FWC
X
X
64.
60.21
3.60
-3.50
-2.80
ER
66.35
62.10
3.45
PerR
V
x
67.05
62.93
3.38
-1.35
0.62
X
68.12
63.88
3.25
-1.10
Dual-SR
70.25
65.40
3.05
-0.45
1.85
M (ours)
X
71.92
66.96
2,95
0.04
2.15
specifically NaviLLM Zheng et al. (2024). For regularization--asee methods, adopt L2 and EWC Kirkpatrick et al. (2017), which constrain parameter updates to preserve prior knowledge. For rehearsal- methods, we evaluate ER [Schaul al. (2015), PerR, ESR [eong et al. (2024), and Dual-SR [Li et al. (2024), which mitigate forgetting by storing and replaying past trajectories with different sampling strategies. As a replay-free baseline, we include naive Finetune, which sim- ply trains sequentially without any CL mechanism. Our proposed ' belongs to the replay-free category, introducing a dual-router MoE and momentum -aase consolidation to achieve continual adaptation without replay. Please see Table for the categorization of methods.
Implementation Details. All methods are built on the NaviLLM backbone Zheng et al. (2024) with MoF-LLRA integration. We use total of 6 experts, with Top- K = 2 routing and fusion weight 3 = 0.3. For momentum-basee
consolidation, we set = 0.4 for task-critical experts (Sec. 4.2) LoRA adapters rank 16 with scaling factor
32, injected into both feed-forward (FFN)
and attention projection layers. Training is performed with a batch size of 4 and a learning rate of 2 10-5 Each task is trained with five epochs and AdamW optimization. For rehearsal-based baselines, the memory bank size is fixed to 100 episodes per task. All experiments are conducted on NVIDIA A800 80GB GPUs.
5.1
MAIN RESULTS
Comparison with continual-leanning base- lines. Trepots averaged navigation perfor- mance across domains (AvgSR/AvgSPLLAvgEE) and continual learning metrics (BWT/FWT) on R2R. All methods share the same training modget.replay-bbsee methods report results with their buffer memory included in the accounting.
Table 2:
Domain-incremental learning in
REVERIE environment.
Method
SR% 1 SPLS
BWTT
FWT 1
Finetune
50.12
39.86
-16.91
-10.26
M' (ours)
51.23
48.30
-5.91
-8.09
From the results, we highlight the following observations. First, our method M' achieves the best overall performance among all strategies, outperforming both regularization-ban based and rehearsal- based baselines. For instance, it improves AvgSPL by +1.6%% over the strongest rehearsal method Dual-SR (66.96 vs. 65.40), while also reducing AvgNE (2.95 vs. 3.05). Second, unlike most base- lines that suffer from large negative BWT (e.g., -5.42 for Finetune, -3.50 for EWC), approach achieves near-zero BWT (0.04), showing that effectively mitigates catastrophic forgetting without replay. Third, E also demonstrates strong forward transfer (FWT 2.15), indicating improved zero-shot generalization to new domains before training. Fourth, while rehearsal-baie methods (ER, PerR, ESR, Dual-SR) alleviate forgetting some extent, they rely on storing past trajectories, raising scalability and privacy concerns. In contrast, our replay-free free method achieves better per- formance with no access to previous- task data. In addition to R2R, we also evaluate on the more challenging REVERIE dataset (Tab. 2). While other methods suffer from severe forgetting in this goal-oriented setting, M substantially alleviates degradation: compared Finetune, it improves SPL by +8.44% (48.30 vs. 39.86) and reduces forgetting (BWT 5.91 vs. - 16.91). This demon- strates that our replay-free design also transfers effectively to object-centric navigation tasks, where precise grounding and long-how horizon reasoning not only improves navigation success but also establishes a stronger balance between generalization and knowledge retention, setting new of-th-o-thhe-e art replay-free baseline.
required. Overall, these findings show that M'E
Forgetting and generalization under full val-unseen training. Table B evaluates dif- ferent setting from our continual-leaming protocol. Instead of task-by-task domain-incremental
8

---

Published as a conference paper at ICLR 2026
Table 3: Performance before and after training on the full val-un.een split. Before denotes the released checkpoint; After denotes the performance after post training; A denotes the gain from post training (After Beforre.
R2R
REVERIE
Method
State
val-seen
lest-unsecn
val-seen
lest-unsecn
SR% ?
SPI.%
SR%
SPL%
SR%
SPI.%
SR%
SP1.% 1
Before
70.13
66.30
62.30
57.57
43.29
40.19
33.41
26.67
HAMT
After
$7.59
52.67
58.01
52.06
42.38
37,90
28.67
24.23
A
-12.54
-13.63
4.29
-5.51
-0.91
-2.29
-4.74
-2.44
Before
73.36
68.93
67.60
59.84
50.11
47.24
39.53
32.51
NaviLLM
After
71.11
67.97
63.52
58.27
38.93
36.05
34,98
28.93
A
-2.25
-0.96
-4.08
-1.57
-11.18
-11.19
-455
-3.58
Before
50.11
39.53
M (ours)
After
75.51
71.94
67.28
61.33
46.24
37.79
31.64
+2.15
+3.01
-0.32
+1.49
-3.87
-4.31
-1.74
-0.87
training, we directly load the officially released checkpoints and continue training on the entire val-unseen split. This setup probes aspects: (i) how much knowledge about previously seen environments (val-seen) is forgotten, and (it) how the overall generalization to novel environ- ments (test-unseen) changes after further training.
As shown in the table, HAMT suffers from substantial forgetting on val val-seen (e.g., 100-12.. 12.54 SR on R2R) and clear drops on unseen scenes. NaviL.M exhibits milder degradation on R2R but severe forgetting on REVERIE, especially in val-seen performance (-11.18 SR). In contrast, our M achieves positive gains on R2R val-seen and maintains competitive performance on test-unseen, while exhibiting much smaller drops on REVERIE compared to NaviLLM. These results suggest that E not only resists catastrophic forgetting when adapting to new domains in bulk training but also preserves cross domain generalization better than both strong baselines.
5.2 ABLATION STUDY
Table 4 reports comprehensive ablation of the three core components R2R, covering all eight possible combinations. Starting from naive finetuning, the agent suffers from catastrophic forget- ting (BWT -5.42) and poor transfer. Applying the Momentum Update directly to the dense back- bone (cquivalent to EMA) improves retention (BWT 2.1 but restricts plasticity, lowering the success rate to 61.52%: this indicates that stabilization without specialized expert routing is insuf- ficient for adapting to new domains. Introducing the Micro Router or Macro Router individually improves navigation accuracy (SR 65.51%) via token-level specialization and generalization (FWT +1.80) through topology-aware reasoning, respectively, yet forgetting remains severe without con- solidation. Hybrid combinations reveal distinct trade-offs: Mltcro+MMomentun secures structural knowledge (FWT +1.92 but lacks Ane-grained grounding, while curacy but limits transfer. Notably, the dual-router design (Micro+Macra) yields the strongest plas- ticity (67.80 (773 among m-mmmnttum methods but fails to retain past knowledge (BWT -6.05). Only when all three components are combined does the model achieve the best overall trade-off, reaching 71.92% SR and nearly eliminating forgetting (BWT 2 0). This confirms that routing (Micro+Macro) and momentum-based consolidation are complementary: the former improves spe- cialization and generalization, and the latter ensures stability across domains.
improves
6 CONCLUSION
We presented M replay-free continual learning framework for Vision-and-Languagge Navigation that separates global scene reasoning from local instruction-visiion grounding via macro-micro Mixture-of- Built on a trainable LLM-based agent, M uses dual routing for expert activa- tion and consolidates task-critical experts through differentiated momentum, enabling fast adapta- tion without storing past data. Evaluated on R2R and REVERIE under scene parttttiati protocol, E improves navigation performance and shows stronger forward and backward transfer compared to rehearsal- and regularization-basse baselines.

---

Published as a conference paper at ICLR 2026
Table 4: Ablations R2R under domain-incremennal learning. We evaluate all combinations of the Micro Router, Macro Router, and Dynamic MoE Momentum Update. Note that applying Momen- tum without routers (Row 2) is equivalent to applying Exponential Moving Average (EMA) the dense model.
Components
R2R Metries
Micro
Macro
Momentum
AvgSR (%)
(vg)PI.(%)
AvgNE
BWT
FWT
x
X
63.28
59.08
3.72
-3.42
-2.41
x
61.52
57.90
3.95
-2.15
-2.80
X
V
X
64,20
60.15
3.68
-5.65
1.80
65.15
61.05
3.60
-0.12
1.92
65.51
3.55
66.72
62.34
3.48
-0.35
-1.48
67.83
64.12
3.34
-6.05
2.15
71.92
66.96
2.95
0.04
2.15
Limitations and future work. First, while our experiments are conducted discrete simulators adhere standard VLN protocols, the proposed macro-micro decomposition is inherently com- patible with continuous control. In such settings, 8 functions as a high-level waypoint planner for local controllers (such ROS MoveBase). Bridging the Sim-to-Real gap requires further ad- dressing challenges such as visual domain gaps (e.g., lighting, motion blur) and real-wo well-woldd SLAM estimation errors, though our robustness analysis (Appendix (A.3) suggests resilience to map noise. Second, the fixed expert configuration may limit flexibility under extreme open-ended scenarios. Future work includes dynamic expert allocation, privacy-aware hybrid replay, Sim-to-Real transfer and extension to physical robotic platforms.
ACKNOWLEDGMENTS
The work is supported by National Natural Science Foundation of China (No.623021 Guang- dong Basic and Applied Basic Research Foundation (No. 1..AA155000 1510110187)))) Guangdong Natural Science Funds for Distinguished Young Scholars (Grant 2023B 1,15020097), the Singapore Min- istry of Education Academic Research Fund Tier 2 (Award No.
the Sin- gapore Ministry of Education Academic Research Fund Tier 1 (Proposal ID: 24-S1S-SMU-0155) and the Lee Kong Chian Fellowships.
REFERENCES
Dong An, Yuankai Qi, Yan Huang, Qi Wu, Liang Wang, and Tieniu Tan. Neighbor-view enhanced model for vision and language navigation. In Proceedings of the 29th ACM International Confer- ence on Multimedia, pp. 5101-5109, 2021.
Peter Anderson, Qi Wu, Damien Teney Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton Van Den Hengel. Vision-and-languaae navigation: Interpreting visually-grounded navigation instructions in real environments. In Proceedings of the IEEE con- ference on computer vision and pattern recognition, pp. 3674-3683, 2018.
Jiaqi Chen, Bingqian Lin, Ran Xu, Zhenhua Chai, Xiaodan Liang, and Kwan- Yee K Wong- Mapgpt: Map-guided prompting for unified vision-and-language navigation. arXiv preprint arXiv:2401.03314, 2024.
Kevin Chen, Junshen K Chen, Jo Chuang, Marynel Vázquez, and Silvio Savarese. Topological planning with transformers for iision-add-lanuual navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11276-1 1286, 2021a
Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for iision-and-language navigation. Advances in Neural Information Processing Sys- tems, 34:5834 1148844-7777,, 2021b.
Shizhe Chen, Pierre-Louis Guhur, Cordelia Schmid, and Ivan Laptev. History aware multimodal transformer for dasiion-nd-Languaaeee navigation. Advances Neural Information Processing Sys- fems, 34:5334-5844 2021c.
10

---

Published as a conference paper at ICLR 2026
Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In European conference on computer vision, pp. 104-120. Springer, 2020.
Zi-Yi Dou and Nanyun Peng. Foam: A follower-awarr speaker model for rision-and-- -annuage navigation. arXiv preprint arXiv:2206.0422 2022.
Junyu Gao, Xuan Yao, and Changsheng Xu. Fast-slow test-time adaptation for online vision-and- language navigation. 2024.
Pierre-Louis Guhur, Makarand Tapaswi, Shizhe Chen, Ivan Laptev, and Cordelia Schmid. Airbert: In-domain pretraining for dision-and-language navigation. In Proceedings of the IEEE/CVE In- ternational Conference on Computer Vision, pp. 1634-1643. 2021.
Weituo Hao, Chunyuan Li, Xiujun Li, Lawrence Carin, and Jianfeng Gao. Towards learning a generic agent for vision-and-lagguage navigation via pre-training. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13137-13146, 2020.
Yicong Hong, Qi Wu, Yuankai Qi, Cristian Rodriguez-Opazz. and Stephen Gould. recurrent Assionand-language bert for navigation. In Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition (CVPR), pp. (643-1653, June 2021.
Tianyu Huai, Jie Zhou, Xingjiao Wu, Qin Chen, Qingchun Bai, Ze Zhou, and Liang He. Cl-moe: Enhancing multimodal large language model with dual momentum mixture-of-expert for contin- ual visual question answering. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19608-19617, 2025.
Seongjun Jeong, Gi-Cheon Kang, Seongho Choi, Joochan Kim, and Byoung- Tak Zhang. Continual vision-and-language navigation. arXiv preprint arXiv:2403. 15049, 2024.
Liyiming Ke, Xiujun Li, Yonatan Bisk, Ari Holtzman, Zhe Gan, Jingjing Liu, Jianfeng Gao, Yejin Choi, and Siddhartha Srinivasa. Tactical rewind: Self-correction via backtracking vision-and- language navigation. In Proceedings of IEEE/CVF conference on computer vision and pattern recognition, pp. 6741-6749, 2019.
Sungjune Kim, Gyeongrok Oh, Heeju Ko, Daehyun Ji, Dongwook Lee, Byung-Jun Lee, Sujin Jang, and Sangpil Kim. Test-time adaptation for online vision- laiionla navigation with feedhack-baas -ased reinforcement learning. In Forty-second International Conference on Machine Learning.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan. John Quan, Tiago Ramalho, Agnieszka Grrabsa--Brwwinte et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3511-3526 2017.
Alexander Ku, Peter Anderson. Roma Patel, Eugene Ic, and Jason Baldridge. Room-across-room Multilingual vision-and and-o-- navigation with dense spatiotemporal grounding. Proceed- ings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4392 4412, 2020.
Jialu Li and Mohit Bansal. Panogen: Text-conditioned panoramic environment generation for vision-and-language navigation. arXiv preprint arXiv:2305. 19/95, 2023.
Jialu Li, Hao Tan, and Mohit Bansal. Envedit: Environment editing for vision-an and--add-aaaaa -an-aag naviga- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15407-15417, 2022.
Xiangyang Li. Zihan Wang, Jiahao Yang, Yaowei Wang, and Shuqiang Jiang. Kerm: Knowledge enhanced reasoning for vision-and and-language navigation. Proceedings of the IEEE/CVF Con- ference Computer Vision and Pattern Recognition, pp. 2583-2592. 2023.
Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantee aligned pre-training for vision-language tasks. In European Conference on Computer Vision, pp. 121-137. Springer, 2020.
11

---

Published as a conference paper at ICLR 2026
Zhiyuan Li, Yanfeng Lv. Ziqin Tu, Di Shang. and Hong Qiao. Vision-language navigation with continual learning. arXiv preprint vrXiv:2409.02561, 2024.
Bingqian Lin, Yi Zhu, Zicong Chen, Xiwen Liang, Jianzhuang Liu, and Xiaodan Liang. Adapt: navigation with aliggnd action prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15396-15406, 2022.
Bingqian Lin, Yunshuang Nie, Ziming Wei, Jiaqi Chen, Shikui Ma, Jianhua Han, Hang Xu, Xiaojun Chang, and Xiaodan Liang. Navcot: Boosting Ilm-based vision-and-lame and-language navigation via learning disentangled reasoning. arXiv preprint arXiv:2403.07376, 2024.
Rui Liu, Xiaohan Wang, Wenguan Wang, and Yi Yang. Bird' sird''-eye-view scene graph for vision- language navigation. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 10968-10980, 2023.
Rui Liu, Wenguan Wang, and Yi Yang. Volumetric environment representation for vision-language navigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, pp. 16317-16328, 2024.
Yuxing Long, Wenzhe Cai, Hongcheng Wang, Guanqi Zhan, and Hao Dong. Instructnav: Zero- shot system for generic instruction navigation in unexplored environment. arXiv preprint arXiv:2406.04882 2024.
Chih-Yao Ma, Jiasen Lu, Zuxuan Wu, Ghassan AlRegib, Zsolt Kira, Richard Socher, and Caim- ing Xiong. Self-monitoring navigation agent via auxiliary progress estimation. arXiv preprint arXiv:1901.03035. 2019a.
Chih-Yan Yao Ma, Zuxuan Wu, Ghassan AlRegib, Caiming Xiong, and Zsolt Kira. The regretful agent: Heuristic-aided navigation through progress estimation. In Proceedings of the IEEE/CVF confer- ence on Computer Vision and Pattern Recognition, pp. 6732-6740, 2019b.
Arjun Majumdar, Ayush Shrivastava, Stefan Lee, Peter Anderson, Devi Parikh, and Dhruv Batra. Improving usion-anddlanguage navigation with image-text pairs from the web. In European Conference on Computer Vision, pp. 259-274. Springer, 2020.
Yao Mu, Qinglong Zhang, Mengkang Hu, Wenhai Wang, Mingyu Ding, Jun Jin, Bin Wang, Jifeng Dai, Yu Qiao, and Ping Luo. Embodiedgpt: Vision-language pre-training via embodied chain of thought. Advances in Neural Information Processing Systems, 36:25081-25094 2023.
Bowen Pan, Rameswar Panda, YouYoung Jin, Rogerio Feris, Aude Oliva, Phillip Isola, and Yoon Kim. Langnav: Language as perceptual representation for navigation. arXiv preprint arXiv:2310.07889 2023.
Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environ- ments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9982-9991, 2020.
Yanyuan Qiao, Qianyi Liu, Jiajun Liu, Jing Liu, and Qi Wu. Llm as copilot for coarse graiiedd vision- and-language navigation. In European Conference on Computer Vision, pp. 459-476. Springer, 2025.
Tom Schaul, John Quan, loannis Antonoglou, and David Silver. Prioritized experience replay. arXiv preprint arXiv:1511.0599 1.05552, 2015.
Xinshuai Song, Weixing Chen, Yang Liu, Weikai Chen, Guanbin Li, and Liang Lin. Towards long- horizon vision-language navigation: Platform, benchmark and method. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 1077--1288, 2025.
Weijie Su, Xizhou Zhu, Yue Cao, Bin Li, Lewei Lu, Furu Wei, and Jifeng Dai. Vl-bert: Pre-training of generic visual-linguistic representations. arXiv preprint arXiv: 1908.08530 2019.
12

---

Published as a conference paper at ICLR 2026
Hao Tan and Mohit Bansal. Lxmert: Learning cross- coos--modality encoder representations from trans- formers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP.
IJCNLP), pp. 5100-5111 11, 2019.
Hao Tan, Licheng Yu, and Mohit Bansal. Learning to navigate unseen environments: Back transla- tion with environmental dropout. In Proceedings of NAACL-HLT, pp. 2610-2621, 2019.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialos navi- gation. In Conference on Robot Learning, pp. 394-406, 2020.
Xin Wang. Wenhan Xiong, Hongmin Wang, and William Yang Wang. Look before you leap: Bridg- ing model-free and model-based reinforcement learning for planned-ahead vision-and-language navigation. In Proceedings of the European Conference Computer Vision (ECCV), pp. 37-53, 2018.
Zihan Wang, Xiangyang Li, Jiahao Yang. Yeqi Liu, and Shuqiang Jiang, Gridmm: Grid memory map for vision-and- lang-uad- navigation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 16625-15633 2023.
Jiazhao Zhang, Kunyu Wang, Rongtao Xu, Gengze Zhou, Yicong Hong, Xiaomeng Fang, Qi Wu, Zhizheng Zhang, and Wang He. Navid: Video-based vlm plans the next step for vision-and- language navigation. arXiv preprint arXiv:2402. 15852, 2024.
Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. Towards learning generalist model for embodied navigation. arXiv preprint arXiv:2312.02010, 2023.
Duo Zheng, Shijia Huang, Lin Zhao, Yiwu Zhong, and Liwei Wang. Towards learning generalist model for embodied navigation. Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13624-13633 13634, 2024.
Gengze Zhou, Yicong Hong, and Qi Wu. Navgpt: Explicit reasoning in vision-and- and-language naviga- tion with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 7641-7649 2024.
Gengze Zhou, Yicong Hong. Zun Wang, Xin Eric Wang, and Qi Wu. Navgpt-2: Unleashing naviga- tional reasoning capability for large vision-language models. In European Conference on Com- puter Vision, pp. 260-278. Springer, 2025.
Fengda Zhu, Yi Zhu, Xiaojun Chang, and Xiaodan Liang. Vision-language navigation with self- supervised auxiliary reasoning tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10012-10022, 2020.
Fengda Zhu, Xiwen Liang, Yi Zhu, Qizhi Yu, Xiaojun Chang, and Xiaodan Liang. Soon: Sce- nario oriented object navigation with graph-based exploration. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12689-12699, 2021.
13

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 INCREMENTAL DATASET CONSTRUCTION
We construct incremental datasets for both R2R and REVERIE following the same protocol. For clarity, we use R2R illustrative example below.
A.1.1 R2R DATASET STATISTICS.
Table D summarizes the official R2R splits. The training set contains 61 scans with 15,060 instruc- tions and 4,675 paths. The validation splits are divided into val-seen and val-unseen. Val-seen consists of scans that overlap with training environments but have disjoint paths and instructions (1,021 instructions, 340 paths), Val-unseen contains 11 novel scans that are completely disjoint from both train and val-seen. The test split further includes novel seans (1,391 paths, 4,173 instructions) that do not overlap with any other split.
Table 5: R2R dataset splits and statistics.
Split
Scans
Paths
Instructions
Notes
Train
61
4.675
15,060
standard training set
Val-seen
56
340
1,021
overlapping scans, new paths/instruction disjoint scans from all other splits disjoint scans from all other splits
Val-unseen
11
783
2,349
Test-unseen
18
1,391
4,173
Val-unseen distribution. Within val-unseen, the number of trajectories per scan is highly imbalanced. The top scans include 2az01b91c22. OLBMNvç9in,, TbHJrupSAjP, and 4 (each with 100 paths), whereas smaller scans such 8194nk5LbLE (15 paths) and p0ee4 (6 paths) contain very few samples.
Table 6: Statistics of R2R val fal-unseen scans and their corresponding incremental dataset entries. Scans with < validation trajectories are excluded from the incremental benchmark (shown in gray).
Scan ID
# Trajectories (R2R)
Train entries
Val entries
2azQ1b91cZZ
100
80
20
oLBMNvg9in8
100
80
20
TbHUrupAAjP
100
80
20
zsNo4HB9uLZ
100
80
20
X7HyMhZNoso
98
78
20
QUCT<6BB5sX
93
74
19
EU6Fwq7SyZv
64
51
13
Z6MFQCViBuw
60
48
12
x8F5xyUWy9e
47
37
10
8194nk5LbLH
15
12
3
PLe4wQe7grG
4
2
Total
883
624
159
Incremental split construction. To construct the incremental benchmark, we sort val-unseen scans by trajectory count and adopt an 8:2 split the scene level. Scans with less than or equal to 10 validation trajectories are removed to ensure reliable validation performance. The resulting incremental dataset provides: (i) training stream for continual learning evaluation, and (ii) held- out validation set that is sufficiently large and diverse to measure both VLN navigation accuracy and continual learning metrics (B (BWT/FWT).
14

---

Published as a conference paper at ICLR 2026
(x) Cansulative Average Perfermince (Stabilits)
Currcal Task Performsece (Plaslicitt)
72.
70.0
05.0
69.0
Talk
Tall 3
Taa
Tasi
Taa
Taak
Task
Taak
Tolk
Taak
Tak
Tel.4 Taak
Tisk
Tak
Takk
Takk
Figure 2: Performance Trend Analysis. (a) Cumulative Stability: The Finetune baseline shows a downward trend due to interference, whereas M maintains stable upward trend, indicating effective knowledge accumulation. (b) Plasticity: E consistently achieves higher performance on the current task being learned compared to Finetune, demonstrating sustained learning capability.
A.2 SCALABILITY AND PERFORMANCE TREND ANALYSIS
To further validate the effectiveness of E in continuous adaptation, we analyze the temporal dynamics of the learning process. Verifying performance trends across episodes is crucial for estab- lishing the scalability of continual learning agents.
Metric for Performance Trend. To quantify the performance trend across lifelong learning stages, we adopt the Cumulative Average SPL on Seen Tasks (A,) as the primary metric. Let RG denote the SPL on task i after the agent has finished training on task t. The metric stage is defined as:
A
R
(15)
This metric dynamically tracks how well the agent performs the entirety of domains encountered so far. non-decreasing A, curve indicates that the agent successfully learns new tasks (plasticity) without degrading previous ones (stability).
Empirical Evidence of Scalability. We plot the trend of A, in Figure and the detailed performance matrix in Figure
Performance Trend (Figure 2): In terms of Cumulative Stability (Left), the Finetune baseline (blue line) exhibits monotonic downward trend in At, dropping from ~70% ~59% due severe in- terference. In contrast. M (red line) stabilizes after the initial tasks and exhibits a gradual upward trend from Task Task This confirms that our method effectively accumulates knowledge and scales gracefully as it experiences more episodes. Regarding Plasticity (Right), M consistently maintains high performance on the current task (red line blue line), demonstrating that the model's ability to learn new, complex scenes does not degrade over time due to capacity saturation.
Lifelong Transfer Matrix (Figure B): The heatmaps visualize R₁,; across all sequential tasks. For Finetune (Right), the lower-triangulat region (representing past tasks) fades rapidly (color turns lighter) as training progresses, visually confirming catastrophic forgetting. For M (Left), the color intensity remains consistent and deep along the columns (c.g., Task I performance remains high even at Stage 8). Furthermore, the diagonal elements (learning new tasks) remain high throughout the cur- riculum. This empirically proves that the expert-isolation mechanism prevents capacity saturation, ensuring late-stage learning is as effective as early-stage learning.
This analysis confirms that M scales effectively. The decoupled expert architecture allows the agent to expand its domain coverage continuously, maintaining high performance on both early and late tasks.
15

---

Published as a conference paper at ICLR 2026
(i)
Ayw
Apw
Ape
Ape
Apm
Toa 13
Figure 3: Lifelong Transfer Matrix (Ra.:) visualization. Each cell (f, represents the SPL on Task i after training on Task t. Left EM*E): The color intensity remains consistent and deep along the columns, indicating robust retention of past tasks. Right (Finetune): The lower-triangular region fades rapidly, visually confirming catastrophic forgetting.
A.3 ROBUSTNESS ANALYSIS: RESILIENCE TO MAP IMPERFECTIONS
To address concerns regarding the dependency on accurate cognitive mapping, empirically ana- lyze the system's 's resilience to map imperfections. We demonstrate robustness through both archi- tectural design and stress-test experiments the R2R dataset.
Architectural Resilience. The M framework does not rely solely on the cognitive map. The final routing weight is fusion:
_w + (1 w". In implementation, meaning the routing decision heavily weighted (70%) towards the Micro Router, which operates immediate visual observations and instruction tokens. This design ensures graceful degradation: even the cognitive map is noisy or incomplete (impairing w=*), the Micro Router continues to guide the agent based on local perception, preventing catastrophic failure.
set / 0.3,
Empirical Evidence. We conducted a robustness analysis by introducing varying levels of noise to the cognitive map input of the Macro Router during inference:
Edge Drop:
Randomly removing 10% 30% of edges from the topology graph simulate connectivity errors.
No Frontiers: Removing all frontier nodes, simulating poor exploration awareness.
As shown in Table 7 even under severe noise conditions such as a 30% edge drop, the perfor- mance degradation minimal (~~1.1 absolute points SR). Furthermore, the robust M continues to significantly outperform the Micro-only baseline (70.83% vs. 66. (6.72%) confirming that even degraded cognitive map provides valuable structural context compared having no map all.
Table 7: Robustness of M under Cognitive Map Noise (R2R), Performance drops are minimal
compared to the complete removal of the Macro Router (Micro-only).
Setting
Noise Type
SR (%) f
SPL (%) f
A SR (Points)
Full E (Standard)
None (Original)
71.92
66.96
M (Robustness)
Edge Drop (10%)
71.65
66.58
-0.27
M (Robustness)
Edge Drop (30%)
70.83
65.90
-1.09
E (Robustness)
No Frontier Nodes
70.42
65.21
-1.50
Micro-only Baseline
Macro Router Disabled
66.72
62.34
-5.20
16

---

Published as a conference paper at ICLR 2026
A.4 ABLATION ON EXPERT CAPACITY
To verify whether the fixed expert configuration limits performance under domain shifts, ducted an ablation study on the R2R dataset with varying numbers of experts (N E {4, 6, 8)).
As shown in Table [] increasing the expert count from 4 to 6 yields significant gain (+1.77% SR), indicating that sufficient capacity is needed to capture diverse navigation primitives. However, further increasing from 6 to 8 results in diminishing returns (+0.13% SR). This saturation indicates that the capacity of experts is sufficient to cover the domain variations in standard benchmarks. Furthermore, while the total parameter count increases with more experts, our Top-K K routing (K 2) ensures that the inference computational cost remains constant. Thus, our chosen configuration of experts represents an optimal trade-off between performance and efficiency.
Table 8: Impact Expert Count on R2R Performance. Increasing experts beyond yields dimin- ishing returns.
# Experts
Trainable Params
Avg SR (%) f
Avg SPL (%) f
BWT (%)
FWT (%) f
4
226.0M
70.15
65.10
-1.12
1.85
6 (Ours) 8
310.6M
71.92
66.96
0.04
2.15
395. 1M
72.05
67.08
0.05
2.18
A.5 RELATION TO TEST-TIME ADAPTATION (TTA)
To explicitly position E against recent Test-Time Adaptation (TTA) approaches in VLN, such as FSTTA (ICMI. '4 Gao et al. (004)) and FeedTTA (ICMI 22m|Kim et al. we clarify the theoretical distinctions and provide quantitative comparison.
Theoretical Positioning. We distinguish M from TTA methods along three dimensions:
Task Setting (Lifelong vs. Transient): TTA methods focus inference-time adaptation to single incoming stream. M targets lifelong learning across distinct domains, where retaining knowledge of past domains (BWT) is critical.
Supervision Dependency: FeedTTA relies on external Oracle (success/failure feedback) dur- ing testing. M self-contained and requires no external signals during inference.
Mechanism: TTA methods often use gradient regularization smooth updates on shared pa- rameters. E uses architectural decoupling (MoE) to structurally isolate knowledge, providing robust long-term memory.
Quantitative Comparison. We adapted FSTTA and FeedTTA to our R2R benchmark under a Continual TTA (CTTA) protocol using LoRA. As shown in Table while TTA methods improve adaptation (AvgSPL) over naive finetuning, they suffer from severe catastrophic forgetting (negative BWT) because they destructively update shared parameters. In contrast, M achieves the highest performance while uniquely maintaining positive BWT.
Table 9: Comparison with TTA methods on R2R Continual Stream. TTA methods suffer from forgetting (negative BWT) in lifelong settings.
Method
Type
Test-time Input
AvgSPL (%) f
BWT +
Finetune (Baseline)
CL
Frozen
59.08
-5.42
ESTTA (ICML'24)
TTA
Unlabeled Stream
63.45
-6.04
FeedTTA (ICML'25)
TTA
Stream Oracle Feedback
63.84
-5.78
E (Ours)
CL
Frozen
66.96
0.04
17

---

Published as a conference paper at ICLR 2026
Rouler
Router
61
Tack
-60
Tado
Figure 4: Visualization of Expert Specialization across sequential domains. (a) Macro Router exhibits domain-specific activation shifts, indicating it captures changing scene contexts. (b) Micro Router shows consistent activation across tasks, indicating it captures shared linguistic and visual semantics.
A.6 VISUALIZATION OF MACRO-MICRO DISENTANGLEMENT
To empirically verify our dual-router design and demonstrate the effective disentanglement of global (scene) and local (instruction) reasoning, analyzed the Expert Contribution Distribution (as de- fined in Eq. T0 across 8 distinct domains in our domain-incremennal training stream.
Visualization of Expert Specialization. We visualized the activation heatmaps for both routers across the sequential tasks. As shown Figure the results reveal two distinct patterns that validate our architectural hypothesis:
Macro Router: Task-Specific Sparsity (Figure 4a). The Macro experts exhibit distinct shifts in activation as the agent moves between domains. For example, Expert dominates in Task but drops significantly in Task 2, where Expert 3 and Expert 4 take over. Similarly, Expert is highly specialized for Task 5. This sparsity and variability confirm that the Macro Router effectively captures Global Scene Context-suee as specifie topological layouts or room types unique to each environment and routes to experts specialized for those structural priors.
Cross- Tass Consistency (Figure 4b). In sharp contrast, the Micro experts dis- play consistent activation pattern across all 8 domains. For instance, Expert 2, Expert and Expert maintain high activity levels throughout the entire stream (visible as horizontal bands). This aligns with our design that the Micro Router focuses on Local Semantics (e.g., grounding verbs like "go" nouns like "door"), which are linguistic and visual primitives that generalize across different environments.
Micro Router:
Overall, this contrast-Marn experts shifting with the domain versus Micro experts remaining sta- ble -ee-provides strong empirical evidence that M successfully disentangles scene-level structural reasoning from token-level perceptual grounding.
Table 10: Inference Latency Breakdown per Step (ms).
Map Size (N)
Macro Router (GNN) Micro Router (MLP) LLM Backbone (7B)
Relative Overhead
10 nodes
0.80 ms
0.17 ms
152.10 ms
+0.5%
50 nodes
3.07
0.17
152.48
+2.0%
100 nodes
6.10 ms
0.17 ms
152.80 ms
+4.0%
A.7 INFERENCE LATENCY PROFILING
To quantify the exact overhead, we profiled the inference latency of different components on single NVIDIA A800 GPU. We measured the per-step processing time the cognitive map size
18

---

Published as a conference paper at ICLR 2026
(N) increases from 10 to 100. As shown in Table 10 the Macro Router introduces approximately
3 ms of overhead for typical map size (N = 50), which is orders of magnitude smaller than the LLM decoding time 152 ms). The relative overhead is only around 2%, which does not create computational bottleneck.
LLM USAGE STATEMENT
Large Language Models (LLMs) were employed to aid in the writing and polishing of this manuscript. Specifically, used LLM to refine the language, improve readability, and en- sure clarity in selected sections. The model's assistance included tasks such as sentence rephrasing, grammar checking, and enhancing the overall flow of the text.
is important to emphasize that the LLM was not involved in ideation, research methodology. experimental design, or analysis. All research concepts, ideas, and results presented in this paper were entirely developed and conducted by the authors. The LLM's role was limited to improving the linguistic presentation of the paper, without any contribution the scientific content data interpretation.
The authors take full responsibility for the entirety the manuscript, including any text generated or refined with the help of the LLM. We have ensured that the use of the LLM complies with ethical standards, does not involve plagiarism, and does not constitute scientific misconduct.
19


Reviews:
[{"review_id": "DEV3igvVEh", "reviewer": "Reviewer_P3Q9", "summary": "This paper presents a novel continual learning benchmark on vision - language navigation task and proposes a replay - free method - Mixture of Macro and Micro Experts (M3E) approach that effectively deals with the catastrophic forgetting problem in the continual learning setting. Two core components are responsible for the superior performance of the M3E approach: A macro - router conditioned on the scene context and a micro - router conditioned on egocentric observation. By designing a mixture - of - expert frameworks with both routers, the M3E is shown to work better than most of the continual learning methods.", "questions": "(1) Although the experimental results shown in Table 4 demonstrate that the M3E method can effectively solve (or greatly alleviate) the catastrophic forgetting problem, there is a lack of results to confirm whether the task performance of M3E continues to improve as it experiences more episodes under lifelong learning settings—consistent with the paper’s continual learning framework. Therefore, I wonder if there is a specific metric that quantifies the performance trend of M3E across different numbers of experienced episodes?\n(2) In Table 4, the performance metrics of M3E on the R2R and REVERIE benchmarks are inconsistent (e.g., M3E alleviates forgetting on R2R but still suffers from it on REVERIE). What are the key reasons for this inconsistency in performance metrics between the two benchmarks? Furthermore, what primary factors cause M3E to still struggle with catastrophic forgetting in the REVERIE task?\n(3) Currently, this work is built on the NaviLLM framework. Given this, can the M3E approach be generalized to other navigation foundation models? If not, what technical constraints limit its generalization across different foundation models?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The continual learning settings designed for the VLN task are not only theoretically interesting but also serve as a data-efficient strategy to enhance the generalization ability of VLN approaches.\n2. The proposed method (M3E) is well-justified in addressing the catastrophic forgetting problem, particularly in the aforementioned continual learning settings for VLN.", "weaknesses": "1. The proposed method (M3E) relies on meta-information (i.e., scene topology) to construct the macro-router and adheres to conventional discrete VLN settings—both factors widen the gap for its practical real-robot deployment. Discrete settings, for instance, often fail to align with the continuous perceptual and motion requirements of physical robots, while the dependence on pre-defined scene topology limits adaptability to unstructured environments.\n2. No evidence is provided in this paper to demonstrate that the proposed method can scale effectively under continual learning settings. Specifically, there is a lack of analysis on whether the model’s performance on episodes in later (or more complex) scenes remains comparable to, or outperforms, those in the first few scenes—an essential indicator of scalability in continual learning.\n3. As shown in Table 4, the M3E fails to address the catastrophic forgetting problem under the REVERIE benchmark. This limitation makes it challenging to verify the model’s generalization ability across different VLN tasks, especially within the continual learning framework emphasized earlier.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YPHY0GEMLm", "reviewer": "Reviewer_eMkh", "summary": "In this paper, the authors propose M3E (Mixture of Macro and Micro Experts) for continual end-to-end vision-language navigation. For replay-free continual learning, the authors use dynamic momentum update for the MoE layers by selecting the top-performing experts, which helps them prevent catastrophic forgetting. The navigation policy consists of a visual scene encoder and a transformer decoder as a policy, with the FFNs replaced with M3E. The architecture is dual router: There is a global router captures global structural regularities of the environment and uses a GNN to build spatial relationships between different nodes in the scene graph and eventually pass it to experts; the local router performs token-wise routing over N experts.\n\nThey experiment with R2R and REVERIE benchmark in MP3D datasets. They evaluate in both seen and unseen environments. In addition to traditional VLN metrics (SR, SPL, NE, OSR), they also compute continual learning metrics (backward transfer, forward transfer). The baselines used are fine-tuned policies, L2, EWC, ER, etc, consisting of regularization-based and rehearsal-based methods. M3E performs the best, achieving the highest average SPL, reducing average navigation error significantly, and having lesser back-ward transfer error showing that it prevents catastrophic forgetting. They also perform additional experiments to train on val-unseen and show that the performance on val-seen does not reduce, while maintaining competitive performance on test-unseen, against HAMT and NaviLLM. They also perform ablations on micro/macro routers and momentum based updates and show that all of them are important for best performance.", "questions": "1. How easy is it to apply this approach to other tasks such as mobile-manipulation or table-top manipulation?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The authors tackle a relatively under-explored problem of continual learning and catastrophic forgetting in vision and language navigation, and propose a methodical approach to solve it.\n2. The idea of decoupling scene level and token level information is pretty important and useful for handling the continual learning task.\n3. The approach actually helps in preventing catastrophic forgetting across different environments as shown by low BWT metric.", "weaknesses": "1. The $M^{3}E$ framework \"relies on accurate cognitive mapping\". This implies that the performance of the macro-router, and thus the entire system, could degrade if the underlying cognitive map representation is poor, especially since it relies on a topological graph.\n2. The method uses fixed expert configurations. This may limit flexibility under extreme domain shifts, as the agent cannot dynamically add new experts if a new environment is radically different from all previous ones.\n3. The authors do not discuss how the approach is applicable to settings beyond VLN - VLN is a hard problem, but there are existing approaches which perform very well on these tasks.\n4. While BWT is near-zero on R2R, the method is not a perfect for preventing forgetting. In the \"full val-unseen training\" experiment (Table 3), $M^{3}E$ still shows some performance degradation (e.g., -4.31 SPL, -3.87 SR) on the REVERIE val-seen split, though it is much more resilient than the NaviLLM baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vBJ0mLvn2z", "reviewer": "Reviewer_Jsee", "summary": "This paper presents M3E, a replay-free continual learning framework for Vision-and-Language Navigation (VLN).\nM3E is designed to disentangle global scene reasoning from local instruction–vision grounding via a macro–micro Mixture-of-Experts (MoE) structure.\nThe framework further introduces a dual routing mechanism for expert activation and a differentiated momentum update to consolidate task-critical experts without relying on replay data.\nExperiments conducted on R2R and REVERIE demonstrate that M3E achieves improved navigation performance and alleviates catastrophic forgetting compared to existing continual learning approaches.", "questions": "Please refer to the weakness section where I also listed questions and suggestions.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "* The paper provides a clear motivation for leveraging a macro–micro MoE strategy to enhance transferability and stability in lifelong VLN tasks.\n\n* The proposed topology-aware propagation using GNNs and the dual-routing fusion methods are conceptually simple yet appears to be an effective way to capture structural relationships in navigation environments.\n\n* The replay-free continual learning design aligns well with realistic online settings, making the proposed approach practically meaningful.", "weaknesses": "* The paper lacks a review and comparison with recent online test-time adaptation approaches for VLN, such as FeedTTA (ICML '25) and FSTTA (ICML '24). In particular, FeedTTA also investigates catastrophic forgetting through a gradient regularization mechanism, which is conceptually similar to the replay-free and momentum-based update proposed in this paper. Please consider positioning M3E more explicitly in relation to these methods to highlight its distinct contributions by providing quantitative comparisons if possible.\n\n* The topology-aware, task-focused dual routing mechanism is interesting, but the individual contributions of each component remain unclear. For example, Tab.3 does not isolate the effects of the macro-level router or the momentum update in ablation studies, making it difficult to assess their independent impact.\n\n* While the dual-router fusion mechanism is intuitively reasonable, it remains unclear how effectively the model disentangles and learns global versus local information in practice. The authors should provide analytical or visualization-based evidence (e.g., expert activation maps or attention distributions per task/environment changes) to support the claim that such separation emerges through the proposed design.\n\nAt this stage, I lean toward a borderline rejection. However, I'm willing to increase the score based upon the authors responses and further clarifications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "6nGiHRKrWu", "reviewer": "Reviewer_tcXs", "summary": "This paper introduces a new framework for continual learning in Vision-and-Language Navigation (VLN) tasks. The core problem addressed is catastrophic forgetting, where an agent's performance on previously learned environments degrades after being trained on new ones. The proposed solution is a replay-free method based on a Mixture-of-Experts (MoE) architecture integrated into an LLM-based navigation agent. \n\nThe key innovation is a dual-router system that separates reasoning into two streams: a Macro Router for global, scene-level strategic planning using a graph neural network over a cognitive map, and a Micro Router for local, token-level semantic grounding. To consolidate knowledge without replaying old data, the authors propose a Dynamic MoE Momentum Update mechanism that selectively updates or freezes expert parameters based on their utility in the current task. \n\nThe method is evaluated on a domain-incremental benchmark constructed from the R2R and REVERIE datasets, where it is shown to outperform standard fine-tuning and several continual learning baselines in both navigation success and knowledge retention.", "questions": "(1) Could the authors comment on the challenges they foresee in deploying your method on a physical robot?\n\n(2) Could the authors provide some analysis of the inference-time overhead introduced by the Macro Router, especially as the size of the cognitive map increases throughout an episode?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "To begin with, the paper tackles  a critical and well-recognized challenge that must be overcome to build autonomous systems capable of lifelong learning in dynamic environments. Decoupling high-level spatial reasoning from low-level instruction grounding is a very sensible approach to modularizing knowledge and seems well-suited to the hierarchical nature of navigation. And the  method's ability to mitigate forgetting without storing and replaying past trajectory data is a significant strength.", "weaknesses": "I have a few concerns that temper my enthusiasm.\n\nIf I understand well, the primary motivation of introducing continual learning to  VLN is explicitly tied to real-world agents that must continually adapt to new domains. However, the evaluation is performed exclusively in a simulated environment on an artificially constructed stream of tasks (partitioning the val-unseen set). I think this is a significant limitation, as the challenges of a simulated, sequential dataset may not fully represent the complexities of real-world continual learning. The paper would be much stronger (and from my point, it ought to contain it) if it discussed the potential challenges of Sim2Real transfer.\n\nAnother concern is that I wonder about the computational complexity of the Macro Router. It appears to construct a graph from the agent's cognitive map and perform GNN propagation at each decision step. As the agent explores a large environment, this map could grow significantly. More discussion on the computational overhead and scalability of this component seems necessary for a complete picture of the method's practicality.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)