Lifelong Embodied Navigation Learning
Authors: Xudong Wang
Jiahua Dong
Baichen Liu
Qi Lyu
Lianqing Liu
Zhi Han
Venue: ICLR 2026 Poster (2026)
Keywords: Embodied Navigation
Lifelong Learning
Robotics Learning
Primary area: applications to robotics, autonomy, planning
TL;DR: We propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA).

Abstract: Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowledge. To tackle this problem, we propose Uni-Walker, a lifelong embodied navigation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design a knowledge inheritance strategy and an experts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a navigation-specific chain-of-thought reasoning mechanism to capture specific knowledge and enhance instruction-style understanding. Extensive experiments demonstrate the superiority of Uni-Walker for building universal embodied navigation agents with lifelong learning. We also provide the code of this work in the Supplementary Materials.

Published as a conference paper at ICLR 2026
LIFELONG EMBODIED NAVIGATION LEARNING
Xudong Wang*.1.2, Jiahua Dong" Baichen Liu Qi Lyu',3 Lianqing Liu', Zhi Han
State Key Laboratory of Robotics and Intelligent Systems, Shenyang Institute of Automation, Chinese Academy of Sciences,
2 University of Chinese Academy of Sciences,
3 Mohamed bin Zayed University of Artificial Intelligence
Task 1
Task t-1
Task
Uni-Walker lifelong learning in multi-instructiin styles
Vision-Language Navigation
Object Localization Navigation
Dialogue Understanding Navigation
(Follow user step-by-step instructions)
(Explore object destination)
(Reasoning
dialogue)
Uber & Lun Jundes
walang
comp ste ravgation
(Ak e leat the abjech And how to MAAK
User B. YOU stoud
uei rengenes
massring
Uni-Walker
Uni-Walker V2
Uni-Walker V3
Uni-Walker lifelong learning in multi-navigation scenes
Scane
Scane
Scane 3
Scene 4
Scane
Scona 6
Scone
Scone
Figure 1: Illustration the proposed lifelong embodied navigation learning (LENL) task. The pro- posed Uni-Walker is able to evolve and continually learn multiple new navigation tasks based on learned navigation knowledge, for developing universal embodied navigation. In lifelong learning, the sequential navigation tasks include multi-scenes and multi-instruction styles (VLN, OLN, DUN).
ABSTRACT
Embodied navigation agents powered by large language models have shown strong performance on individual tasks but struggle to continually acquire new navigation skills, which suffer from catastrophic forgetting. We formalize this challenge as lifelong embodied navigation learning (LENL), where an agent is required to adapt to a sequence of navigation tasks spanning multiple scenes and diverse user instruction styles, while retaining previously learned knowl- edge. To tackle this problem, we propose Uni- Walker, a lifelong embodied nav- igation framework that decouples navigation knowledge into task-shared and task-specific components with Decoder Extension LoRA (DE-LoRA). To learn the shared knowledge, we design knowledge inheritance strategy and perts co-activation strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. To learn the specific knowledge, we propose an expert subspace orthogonality constraint together and a navigation-spectt chain-of offin-of-thouuht reasoning mechanism to capture specific knowledge and enhance instruction-styy style understanding. Extensive experiments demonstrate the superior- ity of Uni- Walker for building universal navigation agents with lifelong learning. Unil
The code is available at
INTRODUCTION
Embodied navigation has emerged as fundamental problem in robotics Liang et al. (2025): y: Dong al. (2026): Wang et al. (2026b), aiming to build agents that can follow user natural language instructions to reach destinations in visually complex scenes Liu et al. (2025). Beyond single-task
Equal contribution. Corresponding authors.

---

Published as a conference paper at ICLR 2026
70
70
20
20
70
Step hap-by--eee by-sbp fine-tuu tune
Sequential fine teeetume
80
50
Our Uni- Wai-Walker
HydraLoRA (NIPS 24)
BrainhhooRA (ACL25)
Forgefulness rate
Success Rate (%)
60
Success Rate
50
50
40
40
20
10
10
Task Index
Task index
10
(a) Navigation Tasks Forgetting
(b) Our Uni-Walker Forget-Less
Figure 2: Illustration of the LENL performance. (a) The catastrophic forgetting phenomenon under
the LENL settings. (b) Our proposed Uni-Walker has a better anti-forgetting performance.
navigation, universal embodied navigation aspires to develop a universal agent capable of under- standing diverse instruction styles and adapting to wide range of navigation scenes Gao et al.] 2202bb Zheng et al. (2025a). Such a universal agent must flexibly solve different navigation tasks, Navigation (VLN) Anderson et al. (2018)), Object-and-Languas Navigation (OLN) Qi et al. (020a)), and Dialog Understanding Navigation (DUN) Thomason et al.) 2020). demonstrating robust generalization across diverse scenes and instructions.
including Vision-and-Languaal
To develop a universal embodied navigation agent, recent methods leverage pretrained large lan- guage models (LLMs) Naveed et al. (2025) as the backbone and jointly fine-tune them on multi- ple navigation datasets. For example, NaviLLM Zheng et al. (2024), NavA3 Zhang et al. (2025)), SAME Zhou et al. (2024), and OctoNav Gao al. (2025a) use LLMs to construct multi-task em- bodied navigation agents. Although these agents undergo large-scale joint training across multiple navigation tasks, they struggle to generalize across all tasks and often lack continual adaptabil- ity to ever-changing new navigation scenarios. In contrast, humans continually learn and absorb new knowledge through lifelong learning as they grow, allowing themselves to accumulate previ- ously learned experiences over time and leverage these experiences to consecutively acquire new skills Zheng et al. 20255). Inspired by this observation, we aim for navigation agents to continually evolve, learning master diverse scenes and instruction styles for universal navigation, much like humans. When continually learning new navigation tasks, existing multi-task agents may suffer from catastrophic forgetting (see Fig. (()) of previously learned tasks if there is insufficient memory and computational resources to store the full training data of old tasks and retrain the pretrained LLMs.
To mitigate catastrophic forgetting, a straightforward strategy is employ Low-Rank Adaptation (LoRA) Hu et al. (2022) to fine-tune pretrained LLMs on each navigation task and store their task- specific low-rank weights for inference. Unfortunately, such a naive strategy fails to explore task- shared knowledge between new and old tasks, nor can it leverage the learning experiences accumu- lated from previous navigation tasks to enhance adaptation to new navigation tasks. Additionally, to capture the task-shared information across different navigation tasks, some recent studies, such as MoE-LoRA Gao et al. (2024): Chen et al. (2024); Dou et al. (2024), HydraLoRA Tian et al.) (2024), and BranchLoRA ZZang et al. (2025a), propose multi-expert structures for jointly learning all tasks. However, they assume a fixed number of experts, which limits their scalability for life- long learning and deployment in diverse real-world scenarios. To address the above challenges, we introduce Lifelong Embodied Navigation Learning (LENL) in this work, a novel problem where agents are required to continually adapt a sequence of navigation tasks while retaining previ- ously acquired navigation knowledge, ultimately developing into a universal embodied navigation agent (i.e., capable VLN, OLN, DUN), as Fig. [ Inspired by the human lifelong learning process, which summarizes and exploits acquired knowledge to continually learn new information, we argue that the crucial challenges of efficient lifelong embodied navigation are consecutively learning new navigation tasks while mitigating the catastrophic forgetting of old ones under the LENL setting.
To resolve the challenges in the LENL problem, we build a new lifelong embodied navigation bench- mark and propose a novel lifelong embodied navigation model named Uni-Walker To the best of our knowledge, this paper is a pioneer exploration for lifelong embodied navigation learning. The proposed Uni-Walker model explicitly decouples multi-navigation task knowledge into shared and specific components with a new Decoder Extension LoRA (DE-LoRA). Inheriting and exploiting the shared knowledge, It leams new tasks more effectively during continual learning. To learn the shared navigation knowledge, we design a knowledge inheritance Strategy (KIS) and an experts co-activation strategy (ECAS) facilitate task-shared knowledge transfer and refinement across
2

---

Published as a conference paper at ICLR 2026
multiple navigation tasks. To learn the specific navigation knowledge, propose an expert sub- space orthogonality constraint (ESOC) and a ravigation-specifa ch (NSCoT) reason- ing mechanism to facilitate the learning of task-specifie knowledge representations and tailor reason- ing processes according to different instruction styles (VLN, OLN, DUN). Extensive experiments performed demonstrate that Uni-Walker can learn new navigation tasks without forgetting previously learned tasks, and even produce satisfactory generalization performance in unseen tasks, thus achieving the SOTA performance in LENL. The main contributions are outlined as follows:
We introduce a novel Lifelong Embodied Navigation Learning (LENL) problem that enables navi- gation agents to continually learn new navigation tasks, including new scenes and instruction styles. We also build new lifelong embodied navigation benchmark for training and evaluating.
We propose Uni-Walker with Decoder Extension LoRA (DE-LoRA) decouple task-shared knowledge and task-specific knowledge across multiple navigation tasks (with multiple scenes and diverse instruction styles), to achieve efficient lifelong embodied navigation learning.
We propose Knowledge Inheritance Strategy (KIS) and an Experts Co-Activation Strategy (ECAS) to facilitate task-shared knowledge transfer and continual refinement across multiple navi- gation tasks, for exploring and exploiting the task-shared knowledge across multiple tasks.
We propose an Expert Subspace Orthogonality Constraint (ESOC) to constrain each expert sub- space to fully learn task-specific knowledge, and a Navigation Specific Chain of Thought (NSCoT) to provide specific chain of thought reasoning processes for each instruction style navigation, for exploring and exploiting the task-specific knowledge across multiple navigation tasks.
2 PROBLEM FORMULATION
Preliminary: In embodied navigation tasks, an agent operating in a 3D scene S is required to com- plete various tasks T described in user natural language instructions I. Following recent researches Zheng et al. (024); Zhou et al. (2024); Wei et al. (2025), there are three typical types of embod- Navigation (VLN) Anderson al. (2018a), denoted as TvLN, requires the agent to follow the user step-by-step instructions and navigate to the destination within the scene. ii). Object Localization Navigation (OLN) Qi et al. (2020a), denoted TOLN, requires the agent to localize a distant destination object according to user concise high-level instruc- tions. iii). Dialogue Understanding Navigation (DUN) Thomason et al. (2020), denoted as TDUN. requires the agent to understand the history dialogue from users and understand user requirements from it, and navigate to the destination. To construct a universal navigation agent for accomplish- ing multiple navigation tasks, following NavLLM Zheng et al. (2024). we use a pre-trained LLM, i.e. Vieuna Chiang et al. (2023), as the basic navigation agent F. And for processing the agent's s vision observation O during navigation, we use the CLIP Vision Transformer Encoder (un et al. (2023) to extract vision features VO))), and the last hidden state embeddings of its [CLS] token are served as the agent real-time observation features. The architecture of our navigation agent model is similar to the multimodal large language model LLaVA Liu et al. 2023). At each time i, the navigation agent reasons user instruction I' with the current observation OF to output the the next direction: D' F(V(O*), I'),, thus navigation step by step to the destination. However, as shown in Fig. [ (b), the agent's adaptation
led navigation tasks: 1). Vision-Languse
new navigation task T. including new navigation scenes S, or new user instruction styles [TV TOLN, . TDUN.I) } causes catastrophic forgetting of the old task (Ti T... T-1). This phenomenon limits the flexible deployment of embodied navigation agents.
To address the above continual navigation learning challenges, we introduce a new problem setting, Lifelong Embodied Navigation Learning (LENL). We define multiple sequen- tial navigation tasks as T {Ti, T2, .... Te), where t-th navigation task T, {O,, Ic} comprising multiple agent navigation vision observation O, = {or, 02, ..., oN } in a specific navigation scene S, E (S,, S2.... SM J. and specific user instruction style I. E (VV TOLN,, TD....). The navigation agent F is required to learn all the navigation tasks T consecutively, and all the tasks tested after all task learning is complete. Please note that for more practical applications of the LENL settings, the task-id is agnostic during the testing phase. And S, of each Te does not overlap with any previous tasks: S; nU Sy) = G. In summary, the proposed LENL problem aims to let a navigation agent F continually learn a sequence of new navigation tasks T while alleviating the forgetting of old tasks, thus facilitating the development of universal embodied navigation agent.
Problem Definition:
3

---

Published as a conference paper at ICLR 2026
Task:
JA.
Inputs
Turn at ints te dillly mom, cortinue shrggt sheed Ino reol icort
Task
Adaptation
Agant Ubseevation
Saved Scone Embeddingg
Setred Instruction Embedaiing
Scene
Top-K
SR
Scene Finondar
LLM (Vicuna-7B :
Task:
LLM (Vicuna-7B)
lask Adaptation
B
FFN
Atterition
To5es Relotonn
: FFN
Top-K
Attention
Top-K Aggregatton
SR
Attention
Task: t
[A, Di..... Di.
Attention
:
Task Adaptation
W
+
AW
Autrrograsatve Prociction
The (-th Trensformer Layer
Tap-K
Autorogresaive Preciction Output Activated but frozen
SR
Not activated
(b) Task-Aware Knowledge Aggregation Figure 3: Illustration of the proposed Kni-Walker pipeline. includes (a) Decoder Extension LoRA Adaptation to achieve progressive knowledge decoupled learning, which decouples naviga- tion knowledge into shared and specific parts, thereby facilitating new tasks learning using shared knowledge while avoiding forgetting. (b) Task-Aware Knowledge Aggregation to automatically aggregate the learned knowledge according to a specific navigation task for task specific inference.
(a) Decoder Extension LoRA Adaptation
3 THE PROPOSED UNI-WALKER
The overall pipeline of Uni-Walker is illustrated in Fig. It includes Decoder Extension LoRA (DE-LoRA) adaptation to learn a series of multiple navigation tasks continually ($3.1) Navigation Mfaan--ff-Thughh (NCoT) to facilitate specific navigation reasoning ($3.2). and Task- Aware Knowledge Aggregation (TAKA) to automatically aggregate the learned knowledge ($3.3).
3.1 DECODER EXTENSION LORA ARCHITECTURE
As shown in Fig. (a), in order to learn the t-th navigation task T, we extend the vanilla Lora Hu et al. (2022) and propose a new low-rank adapter named Decoder Extension LoRA (DE-LoRA) to finetune the pretrained NavLLM Fo, on each navigation task T = [O,,Ie) and then obtain an adapted updated model F' where 02 = 0 20,, A0, = (Aw) ) and AW, = B E R are learnable low-rank weight in (-th layer of total of L layers. In this case, the features z' of each transformer layer are processed as: y' W' + B/A( where the low-rank learnable weights B' E R can be regarded decoder and A R can be regarded as encoder. To address LENL, trivial solution is to train and save all low-rank adaptation weights for all navigation tasks so far, and then load the specific weights at inference time. However, multiple navigation tasks share and have specific knowledge with each other, and simple vanilla LoRA storing for a single task cannot enable the agent to evolve during learning, limiting the development of a universal navigation agent. To explore and exploit the task-shared and task-specific knowledge between navigation tasks, different from vanilla LoRA, we use subspace A to learn task-shared knowledge and B, to learn task-specific knowledge, inspired by Zhang et al. (2025a); Tian et al. (2024). Specifically, in our DE-LoRA, all navigation tasks use a shared subspace A and each navigation task uses TOP-K activated expert subspace [Br,.., Bx) } to exploit task-shares knowledge and to explore task knowledge:
K
y Wo x + AW & = W
Σ B
A.),
(1)
n=1
where B, E { Bx) is the specific activated expert subspace. Different from existing shared architecture Hy Tian al. (2024), our DE-LORA selects sparse activated experts adapt to the new navigation task and expands specific expert subspace as the task is continually learned.
3.2 EXTENSION EXPERTS INCREMENTAL LEARNING
To explore the thak-dhared and task-specific knowledge between sequential navigation tasks set T, and consolidate the DE-LORA architecture, we propose an Extension Experts Incremental Learning
4

---

Published as a conference paper at ICLR 2026
strategy. Specifically, for the initial training, use Kaiming initialization He al. (2015) to initial- ize knowledge base parameters IC = {A, B1}, ). and then train the parameters {A, B J to adapt task Ti. And then for learning the subsequent n2-th task Tn, we dynamically increase one B,, to K.
Inheritance Task-Shared Knowledge. When learning the subsequent new task T: = [S,,I,),t> 1 continually (with e-th navigation scene S. and s-th navigation instruction style Ig). propose Knowledge Inheritance Strategy (KIS) and Experts Co-Activation Strategy (ECAS) on the new in- creased expert subspace B, explore task-shared knowledge. In order inherit user instruction style knowledge, in KIS, we initialize the current expert B, using the learned experts having the same instruction knowledge, denoted as BB,.,, B,}, that have previously learned tasks with the same instruction style. We perform Principal Component Analysis (PCA) Abdi & Williams (2100) on these experts to identify the low-dimensional subspace that captures the most signifi- cant shared variations among them. Specifically, each expert parameter matrix B₃ E (B..... ..., B,) is flattened into vector 0g = vec(BR) E R and all vectors are concatenated to form an ex- pert matrix M = [8;, 11.0.+...... 0,] p()()-(+1), Then we compute the mean parameter vector 5-5-III J-HII Ei=, 0x. and obtain centered vectors 0% 02 μ.. The covariance matrix is given by C= Ex_, 0,0 then we perform SVD dccomposition on the covariance matrix = UUVT and take the top-r eigenvectors to form U = [ui, ..., u. Rdxr, We further leverage the top-r prin- cipal components to capture the dominant shared variations across previous experts. Specifically, we compute the average direction of the subspace spanned by U, and use it for knowledge inheritance:
B B mal6xr (μ
ug),
where le provides the central tendency of expert parameters, while the averaged principal directions inject additional shared variation patterns. This initialization encourages the new expert to start not only from the common mean but also aligned with the principal subspace that encodes instruction- style knowledge. addition to KIS, we also propose ECAS to explore shared knowledge between multiple navigation tasks. Specifically, as shown in Fig- 4 (a), activate TOP-K experts with the TAKA strategy ($3.3) for each task, which includes not only the specific trainable B, expert, but also the related experts (Bi ... B - that are computed in the forward pass but parameter frozen:
K-1
Wo x + AW x Wo x B A.
Σ (B AA
In addition to the exploration shared knowledge with TOP-K expert subspace (Bi,.., 11 and B, in order to progressively refine the shared subspace A' and avoid old task-specific knowledge catastrophic forgetting, we also propose shared smoothing consolidation loss for the subspace A':
Lexe,t Asse (I|FA,t-1 @ (A' A)1
where A' is current learnable shared subspace, A is the last task Ti-1 learned subspace. And Asse 0.1 is the balance typer-parameter, Fi-1 is the Fisher Information Matrix Kirkpatrick et al. (2017) which measures the importance each parameter A for the last task T--1, and can be calculated:
FA -E (0x logp(y 2: A))
where x E {O,, Ie} is the navigation input data and is the output annotation, and FA.--1 1 measures the importance of each parameter for the navigation agent F output performance, with higher value indicating greater importance. Furthermore, during the lifelong navigation learning, we also perform incremental updates for the Fisher Matrix F; in to achieve shared knowledge smooth learning:
FA.: FA.-1 (1 w) FA..,
(6)
where: w 0.9 is the exponential moving average coefficient to control the smooth update FA.t-
Exploration Task-Specific Knowledge. The exploration and exploitation of the above-mentioned task-shared knowledge summarizes old task knowledge to facilitate new task learning while avoiding old knowledge catastrophic forgetting. To further explore the independence of specific knowledge, we also propose an Expert Subspace Orthogonality Constraint (ESOC) and a Navigation Specific Chain of Thought (NSCoTT). Specifically, to consolidate the shared architecture of subspace A and facilitate task-specific knowledge learning, we propose ESOC to perform orthogonal constraint on expert subspace B, during learning the 1-th task, we prefer the expert subspace B, be orthogonal
5

---

Published as a conference paper at ICLR 2026
Dialogue Understanding Navigation
Object Localization Navigation CoT
Vision-Language Navigation CoT
You and rerigation spent You at the desorbed mom and go noom
nevigatoon to the iocallan compiiee the given baok
You neriation sgoit You rqquicd My vernution: Co frownd hhe dring tatle
User
User Uaedrn on Baa oi
User
(g Fang wilm ane unt you ner
Therking: The cunorcatioo wnffooorraate several
THinking The curent otman vilee cortare severa dreckons cibee 150 step Candicss 201 You
Uni-W Mi--Walee
Thanking
The cumee rhearation compans onwerd Yourroy diso read the Steepaaion on helery wiich Jodidoons The Hobilry Vox
Un-- WalkWa
deca ore. The haltrry oo.
You may aloo need canggriin eaor, ArEE
Walker
ANCGOOS The Hittney 101
concoponiingg pronice
compporcinng
atwn curee local or, and A0 (irelio
Thisking Sosecion observation and porey kiler dradion Toom quliti Feot
(Terking Based Hirtory oiseevatiion
(Latent progress and tier seboct the noo dircction
lurgl bcalio.. (leascting
lucution
(lassoning 1l66.3
Figure 4: Illustration of the Navigation Chain-of- Thought to design various specific LLM chains of thought for specific instruction style tasks to facilitate the embodied navigation performance.
to the previous experts avoid knowledge overlap: Ei=
(B( "B, 18)11=- =0. In addition, to avoid the expert subspace B, degenerating into the trivial solution of a zero matrix (in this case, it will not learn any knowledge, i.e., learnable weight AW = 0), we perform 1.2 normalization on the expert subspace B, to project these experts onto the unit sphere subspace B,. Thus, the ESOC loss is:
Lesoc,t = Aesoc Σ Ll (BB).))))) B B, ll
B,
B.
B.
(7)
BB l|
i=1
where Aesoc 0.1 is the balance subspace orthogonality hyper-parameter, and e 0.01 is the perturbation for avoiding the computational avalanches caused by molecules B, 11%- equal to 0.
In addition to ESOC, cesses for each specific user instruction style navigation task. Specifically, as shown in Fig. 4 (b), we provide three types of chain of thought for three types of specific user instructions (TvLN.:: TOLN, J. In VLN task, the agent is required to follow the user's detailed step- by-step instructions, and the reasoning tends to be navigation process tracking. In OLN task, the agent is required to reason about the user's 's desired destination based on observations and gradually track the navigation process to reach the destination. In DUN task, the agent is required to compre- hend user dialogues, reason user requirements, and complete navigation based on the requirements.
also propose NSCoT to perform specific navigation reasoning pro-
In summary, during the lifelong navigation learning process for the 1-th task Tr, the total adaptation loss for the LLM-based navigation agent F performing auto-regressive action generation training is:
N
L =
log
+ Lase,t + Lecoc,t,
where (A Pu)z,o) denotes the predicted probability of annotation navigation actions under the current observation IO= {I,, Ot), and the balance hyper-parameter is X = (Asse + Aewe).
3.3 TASK-AWARE KNOWLEDGE AGGREGATION
After learning a series of navigation tasks continually, WalkWe is required to recall which expert knowledge learned is valuable for the current specific navigation task and utilize this knowledge to better complete the current navigation task. We propose Twsk-Aware Knowledge Aggregation (TAKA) strategy to automatically activate the TOP-K experts most relevant to the current task. Specifically, as shown in Fig. (c), Uni- Walker adapts to each task T= (Ti, T2..... Th by learn- ing specific expert subspaces [A. B2, ..... B while also preserving the task retrieval em- bedding Re- = (Es,,, 1,, (),(),,)) (Es,2. E1.2),... (Es,H, H, where Es, are the 1-th task's scene embedding and F,, are its instruction embedding. F.s, = FVV Og.... O. ) are the embed- ding observed by the agent during navigation, using the CLIP vision encoder Radford et al. (2021)
EJ = (T] I2 ..., ...,IN are the embedding provided by user instructions in t-th task, using the CLIP text encoder. And to reduce storage space, we remove elements in (E... E with cosine similarity greater than 0.9. Under the LENL setting, the agent is agnostic about the task-id id during inference,
Re to retrieve an unknown navigation task to decide which expert subspace to use. We compute the cosine similarity for each Es., based on the agent's 's observation embeddings E. EV,OO) and each Er,t based on the embedding provided by user instruction E, ET
E. Es,t SmI, E,)
E, E1,
Smo, E.)) F., 1
t= 1,2, H. (9)
F
F
Subsequently, we set elements greater than μ in (Sm1,1, Smi, Smy,u to 1 and other elements to 0 to form mask matrix M. We apply mask M to similarity set {Smo,1, Smo,2; -... Smo,t).

---

Published as a conference paper at ICLR 2026
then select the TOP-K experts {By Bggg=1 to activate to perform inference for the test agnostic task Tg:
(B)) 9 = top-k(M @ {Smo,1 Smo,2, Smo. });
(10)
Finally, activated experts (B)) q=1 are combined with subspace A to perform inference with Eq. d1
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Lifelong Embodied Navigation Benchmark Settings: To evaluate the proposed LENL task. we follow the classic works visual navigation Anderson et al. Sa): Qi et al.
Taskr: S1
Task2: 52
Task3: 53
Taski; S4
ToskS: S5
Task8: B6
Somn To
Sorw 75k
Task7: ST
Task:: S0
Taskö: 59
Task10: S10
Task
Task12: 512
(2020a): Thomason al. 2020);ZZhen
et al. (024) using the Matterport3D Simulator Anderson et al. (2018a) navigation scene simulator, and con- struet new lifelong navigation LENL benchmark. The proposed benchmark settings are shown in Fig. 5] including eighteen different navigation scenes with three embodied navigation instruc- tion styles, i.e., VLN, OLN, DUN, described in the Problem Definition ($ 2). The first fifteen tasks used for continual learning, while the last three tasks are used for generalization testing in unseen scenes. Please note that in the LENL setting, the task-id é is only visible during continual training, but agnostic during testing. More details for the constructed LENL benchmark are provided in our Appendix CC
From Tar oF
twe 2lA33455
Soee Mideerii fee Noleaca
Sune DISGANCTIO T4 Sime tungl
Task13: $13
Task14; 514
Task 515
Teskr16 516
Task S17
Task 518
DEIAROight Seame 1200 (WAAG
Figure 5: Illustration of the lifelong navigation bench- mark. We establish a total of navigation tasks for life- long learning, including 18 navigation scenes and 3 types user instruction styles (VLN is marked in blue color, OLN is green, and DUN is purple). We the first 15 tasks for lifelong learning, and the last 3 tasks are used to evaluate unseen scene generalization performance.
Training and Evaluation Settings: To ensure fair comparisons, our methods and all comparison methods are performed with NavLLM Zheng et al. (2024) as the baseline backbone. We use eight NVIDIA RTX 6000 Ada Generation GPUs with Py Torch 2.1.2 (cu121) for training and testing. And the Adam optimizer with an initial learning rate of 3.0 x 10-5 is used for training. The low-rank r = 16 of LORA, the number of experts
2, μ 0.5, and all the other hyperparameters consistent with the MavLLM. Following previous studies Zheng et al. (2024); Wei et al. (2025); Anderson et al. (2) we report three key evaluation metrics: success rate (SR), success rate weighted by path length (SPL), and oracle success rate (OSR). In addition, to evaluate the anti- forgetting rate in lifelong learning, we propose three evaluation metrics, SR Forgetting Rate (SR-F), SPL Forgetting Rate (SPL-F), OSR Forgetting Rate (OSR-F). These metries be calculated by:
SR-F, M-SR, SR. M-SR,
SPI-F M-SPL, - SPLL OSR-F M-OSR, OSR, (11)
M-SPL,
M-OSR;
where M-SR,, M-SPIa, and M-OSR, represents the 1-th task completion status with success rate, success rate weighted by path length, and oracle success rate, under learning only the 1-st to 1-th task. Thus the larger SR-F, SPL-F, and OSR-F, the greater the degree of t-th task forgetting.
4.2 COMPARISON EXPERIMENT RESULTS
This experiment verifies the superior performance of our Uni Walker. We compare it against the SOTA L.oRA-based continual learning approaches: Seq-FT is sequential fine fine-tuning across all tasks; LwF-LORA Li & Hoiem (2017) employs knowledge distillation to preserve previously acquired tasks; EWC-LORA Xiang et al. (023) constrains important parameters of past tasks to reduce navigation forgetting: Dense MoLE Chen al. (0024) adopts dense expert routing, while Sparse MoLE Dou et al. (2024) applies sparse expert routing in MoE-LoRA: MoLA Gao et al. (2024) extends Sparse MoLE by introducing deeper-level experts; HydraLoRA Tian et al. (2024) uses a shared module A for common knowledge and multiple B modules for task-specific learning, BranchLoRA Zhang al, (2025a) further strengthens the sparse selection mechanism; O-LoRA
7

---

Published as a conference paper at ICLR 2026
Table I: Test results (task-wise SR f. %) of comparison experiment with our LENL settings.
Comparison Methods
|S
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
S16
S17
S18
Avg.
Seq Beq-FT(e (sequential fine tuning)
0
0
0
5
18
8
9
8
21
20
25
86
0
0
0
12
LwF-LoRA (Li & Hoiem
2017
0
22
12
10
30
45
36
85
15
EWC-LoRA Xlang
0
10
20
8
31
49
40
86
16
Dense MoLE (C D20 et al.
2024
10
10
4
10
10
18
16
19
17
30
56
51
85
12
10
21
Sparse MoL.E [Doi et
20244
15
12
24
15
32
58
53
23
MoLA (Gao et 22024
15
12
0
12
10
20
18
13
26
19
34
61
59
86
20
12
24
HydraLoRA [aan et al, 2024
16
14
15
19
15
16
24
17
17
21
41
68
57
86
18
14
16
27
BranchL.RRA ZZliaann et
2125aa
26
20
20
20
39
78
54
86
30
SEMA Wang al. 2025
38
34
32
34
19
32
33
38
42
30
33
51
84
61
86
39
42
36
43
NBAgemt (Lang ei ni.
A024
29
26
23
26
10
23
23
28
24
22
26
42
81
57
86
31
24
17
33
O-LoRA Wang
21258
55
79
48
56
54
88
63
86
58
SD-LoRA Wü ei al.
2025 TFTAKA
58
57
54
46
49
42
57
76
51
59
57
54
81
62
87
68
55
48
59
Uni-Walkse (ours)
67
67
75
67
57
50
67
83
50
73
62
53
88
65
86
74
61
51
66
Table 2: Test results wise SR-F 4. %) of comparison experiment with our LENL settings.
Comparison Methods
SI
$2
S3
$4
S5
$6
S7
88
59
S10
S11
S12
S13
S14
$15
S16
S17
S18
Avg.
Seq.FT (sequential fine tuniny)
100
100
100
94
93
100
93
78
90
88
88
61
77
62
0
100
100
100
85
LwF-LoRA (Li Hoiem 2017)
100
93
100
94
90
100
74
90
84
84
44
49
45
0
94
100
100
80
EWC-LoRA Xlang et al
2025
100
93
100
87
93
100
93
76
90
88
88
43
44
38
0
94
92
100
79
MoLE Clien
86
87
95
44
36
22
0
86
84
85
71
Sparse MoL.E Dou
2024
78
82
100
79
90
89
77
79
79
68
77
41
34
18
0
81
81
82
69
MoL.A (Gao al
78
82
100
85
90
82
79
85
31
9
0
76
82
80
67
HydraLoKA [ith at
2014
77
79
81
76
90
73
77
72
67
77
68
27
24
14
0
79
79
71
63
BranchLoRA Ziann et al.
20255
63
71
75
69
87
62
71
69
62
76
66
30
13
17
0
67
71
73
58
O-LoRA Wank cl 2022511111 IAKA
21
13
37
33
30
24
23
8
14
4
2
3
0
23
22
35
17
SD-LoRA Wu al.
21125 TTAKA
17
19
32
43
21
24
19
11
2
21
12
4
10
5
0
19
19
13
16
Uni-Walker (ours)
16
5
30
Comparison Results SPL
Comparison Results SPL-F (%)
Comparison Results OSR (%)
Comparison Resulss with OSR-F (%)
Figure 6: Test results (average (a) SPL 1, (b) SPL-F 4. (c) OSR t, (d) OSR-F 1) of comparison
experiment with our LENL settings. For detailed task-wise results, please refer to Appendix C
Wang et al. (003a) leverages orthogonal loss to capture task-specific knowledge; SD-LORA Wu et al. (2025) dynamically combines LoRA modules from previously learned skills. And SD-LoRA and O-LoRA are combined with our proposed TAKA. Additional comparison details are provided in the Supplementary Materials. The evaluation results with SR summarized in [ Table 2. Uni-Walker achieves a higher average success rate of 66%, surpassing the previous best (59%) by 7%, and lower forgetting rate of 5%, improving the prior best (16%) by 11% The results with SPL and OS are summarized in Fig. 6 Uni-Walker achieves a higher SPL of 61 %. surpassing the previous best (38%) by 23%, and a lower forgetting rate of 7%, improving the prior best (42%) by 35%. It achieves a higher OSR of 81%, surpassing the previous best (79%) by 2%, and lower forgetting rate of 5%, improving the prior best (7%) by 2%. Detailed results are in Appendix $C
How Does Walker Perform on Generalization in Unseen Scenes? We also provide a dedicated com- parison on the three unseen generalization tasks (S16, S17, S18). The evaluation results with SR (%) are summarized in Table Uni-Walker achieves higher average success rate of 62%, surpassing the previous best (57%) by 5%. This superior generalization ca- pability results from: Decoupled knowledge learning via DE-LORA, ESOC, and KIS, ensuring that shared
Table 3: Results (SR%) ) for Generalization.
Comparisons
S16
S17
S18
Avg
HydraLoRA
18
16
16.0
BranchLoRA
28
15
21.0
O-LoRA + TAKA
65
53
36
51.3
SD-LoRA + TAKA
68
55
48
57.0
Jni-Waleer (ours)
74
61
51
62.0
8

---

Published as a conference paper at ICLR 2026
Table 4: Ablation study results (%) for the methods with Task Shared Knowledge Exploration.
Methods
KIS
ECAS
SSC
SR
+
SR-F +
SPL f
SPL-F +
OSR t OSR-F
Baseline
X
55.7
21.1
37.0
45.0
76.7
Uni Walker w/ KIS
v
60.3
14.2
50.2
23.9
77.6
7.7
mhi-Waleer w/ SSC
V
J
x
59.7
15.1
44.7
30.6
77.9
7.3
Uhi-Walker w/o ECAS
X
v
58.1
17.4
44.7
32.3
78.3
6.9
Our Uni mil-el
/
67.3
4.3
62.3
5.7
81.3
3.5
Table 5: Ablation study results (%) for the methods with Task Specific Knowledge Exploration.
Methods
ESOC
NSCoT
SR 1
SR-F
SPL f SPL-F 4
OSR
OSR-F
Baseline
X
49.0
29.2
33,9
45.0
72.3
14.0
Uni-Walker w/o ESOC
X
/
63.5
9.8
60.6
8.2
79.7
5,3
Uni-Walker w/o NSCoT
V
x
51.1
27.3
35.5
46.3
75.3
10.5
Our Uni-Walker
/
67.3
4.3
62.3
5.7
81.3
3,5
Table 6: Ablation study results (%) for the
Knowledge Aggregation methods.
Methods
IM
OM
MM
SR
SR-F J
SPL 1
SPL-F
OSR f OSR-F
Uni Walker w IM
/
x
35.0
50.1
23.2
65.0
46.6
49.5
Uni-Walker w OM
x
x
65.1
9.6
62.7
7.5
80.1
5.5
Our Uni-Walked
x
67.3
4.3
62.3
5.7
81.3
3.5
navigation knowledge is preserved while task-specific expertise remains disentangled rather than en- tangled or overwritten; Dynamic knowledge aggregation through TAKA, which activates the most relevant experts for unseen scenes, to reuse transferable skill knowledge when encountering un- seen scenes. Together, these mechanisms allow Uni Walker to adapt to new tasks while retaining previously learned knowledge, leading to significantly better performance in unseen generalization.
4.3 ABLATION STUDIES
How Does Navigation Tasks Shared Knowledge Perform? The ablation studies results of the ex- ploration for navigation tasks shared knowledge are summarized in Table Specifically, the "KIS" denotes the proposed Knowledge Inheritance Strategy, which summarizes previously learned knowl- edge and utilizes for learning new tasks. Removing the inheritance leads to slower convergence and weaker transfer to new tasks. The "ECAS" denotes the proposed Experts Co-Activation Strategy, which directly utilizes previously acquired knowledge to accomplish the current task. Removing the co-activation leads reduction in the exploitation of previous knowledge. The "SSC" denotes the proposed shared smoothing consolidation loss for shared subspace A', which facilitates the smooth updating of shared subspaces during the continual learning process. Removing the loss leads to overfitting new knowledge and the catastrophic forgetting of old knowledge. Based on the abla- tion results, the proposed methods enable Uni-Walke explore and exploit task-shared knowledge between diverse tasks, effectively improving new task learning and reducing catastrophic forgetting.
How Does Navigation Tasks Specific Knowledge Perform? These ablation studies' results about the exploration for navigation tasks shared knowledge are summarized in the Table 5 Specifically, the "ESOC" denotes the proposed Expert Subspace Orthogonality Constraint on B. which facil- itates the independence of each expert subspace to learn task fask-specific knowledge. Removing the orthogonality constraint may lead to overlapping of the expert subspaces, resulting the failure of knowledge decoupling. The "NSCoT" denotes the proposed Navigation Specific Chain of Thought, which provides specific reasoning processes for each specific navigation task. Removing NSCoT and applying fixed reasoning template (prompting only the reasoning user instructions complete navigation tasks) leads to a decline in complex task performance. Based on the ablation results, the proposed methods enable Uni-Wer Walker to explore and exploit task-specific knowledge between diverse navigation tasks, effectively improving the new specific navigation task learning efficiency.
How Does Task-Aware Knowledge Aggregation Perform? These ablation studies' results about the Tware Knowledge Aggregation are summarized in the Table Specifically, the "IM" de- notes matching with user instruction; the "OM" denotes matching with agent observation: the "MM"

---

Published as a conference paper at ICLR 2026
denotes our proposed mixed matching method. Based on the ablation results, the proposed aggrega- tion method comprehensively considers both matching modes and achieves the best performance.
How does computational complexity scale the number of tasks increases? We analyze the resource cost of LoRA and the Fisher Information Matrix in our DE-LoRA architecture. For each additional task, Mni-Walker adds: x 2.1 MB for the LoRA expert subspace and maintains Fisher Matrix of the same size. Thus, even when the number of tasks grows beyond 100, the total storage remains modest: 100 4.2 MB 0.4 GB. Such overhead is negligible for modern LLM-based navigation systems (7B, 13B, or larger), demonstrating the practical scalability of the architecture.
RELATED WORKS
Embodied Navigation. Embodied navigation requires an agent to follow user language instructions
and reach a goal in a visual scene Liu et al. (2025); Han et al. (0255; ; Bar et al. (2255) Nie et al.
2022)) Recent research has established three representative tasks. Vision-and- and-Language Navigation
(VLN) Anderson et al. (2018a) requires user step-by-step grounding of detailed instructions, and
has been widely studied in both discrete Anderson et al. (2018)); Ku et al. (2020); Qi et al. (2020b);
Fu et al.
y Hong et al. (2020) and continuous environments Krantz et al. 2020 ; Raychaud-
huri et al. (2021); Chen et al. 2020 2022). Object Localization Navigation (OLN) Qi et al. (2020a)
requires the agent to localize distant object given concise referring expressions, emphasizing high- level semantic understanding Arnaud et al. (2025); Yu & Saniie (2025); Lei et al. (2025). Dialog- based Understanding Navigation (DUN) Thomason et al. (2020) extends this paradigm to interactive settings, where agents need to interpret multi-turn dialogues to reason user intent Majumdar et al.) 2025
; Qiao et al. (025). Building upon these advances, recent efforts aim to construct universal navigation agents by jointly training across multiple tasks, leveraging large pretrained I. LLMs/ Zheng
ct al. (024); Zhou et al. (024)); Qiao al. (2025); Gao et al. (2025a); Zhang et al. (2025b). How- ever, these methods typically assume fixed task distributions and prone catastrophic forgetting when adapting to new scenes or instruction styles. This motivates the study of lifelong navigation learning, where agents continually integrate knowledge from sequential tasks without forgetting.
Continual Learning. Continual learning seeks to enable models to acquire new skills over time without erasing previously learned ones Li et al. (2025); ; Yang et al. (2025); Wang et al. (2024)
2026a). Existing approaches can be broadly grouped into three categories. Parameter regularization
methods constrain weight updates to preserve important parameters for past tasks |Rebuffi et al.
(017)); Li & Hoiem| (2017); Derakhshani et al. (202I); ;; Douillard et al. (2020). Architecture-base
methods dedicate specific network components to different sequential tasks Jung al. (2020); Wu
cct al. (202T): Wang et al. (2022): Toldo & Ozay (2022). Replay-based methods store or generate data
from past tasks to rehearse old skills et al. (2021) ; Rebufñ et al. (2017); Sun et al, (2022); Wan
et al. (2024); et al.) (2022); Xiang et al. (2019). though such approaches can be memory- inteery-iteessve
and raise privacy concerns. In robotics, continual learning has been explored to enable agents to expand their skill repertoire through lifelong interaction Meng et al. (2025); Yao et al. (2025); Zhu cct al. (2025); Ayub et al. (2025b a). Despite progress, these existing techniques rarely address the unique challenges of embodied navigation, where both scene diversity and instruction variability necessitate balancing shared knowledge transfer and task-specific reasoning.
CONCLUSION
In this work, we introduce Lifelong Embodied Navigation Learning (LENL), a novel task that en- ables navigation agent to continually learn new multiple navigation tasks, including new scenes and with new user instruction styles. also propose Uni-Waaker, lifelong embodied navigation frame- work that decouples navigation knowledge into task- and task-specific components with De- coder Extension LoRA (DE-LoRA). For the shared knowledge, we design a Knowledge Inheritance Strategy and Experts Co-Activation Strategy to facilitate shared knowledge transfer and refinement across multiple navigation tasks. For the specific knowledge, we propose an Expert Subspace Or- thogonality Constraint together and a Aevvigation-Speciin to capture task sees-siccific knowledge and enhance specific instruction styles understanding, Extensive experiments are performed to demonstrate the effectiveness and superiority of proposed Uni-Walked.
Thhouuhhttt reasoning mechanism
10

---

Published as a conference paper at ICLR 2026
7 ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China under Grant T2596040, T2596045 and U23A20343, CAS Project for Young Scientists in Basic Research, Grant YSBR-041, Liaoning Provincial "Selecting the Best Candidates by Opening Competition Mechanism" Science and Technology Program under Grant 2023/H1/10400045, Joint Innovation Fund of DICP & SIA under Grant UN202401, Fundamental Research Project of SIA under Grant 2024JC3K01, Natural Science Foundation of Liaoning Province under Grant
REFERENCES
Hervé Abdi and Lynne Williams. Principal component analysis. Wiley interdisciplinary reviews: computational statistics, 2(4):433-459 2010.
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton van den Hengel. Vision-anddlanguagg navigation: Interpreting visually-grounddh navigation instructions in real environments. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, June 2018a.
Peter Anderson, Qi Wu, Damien Teney, Jake Bruce, Mark Johnson, Niko Sünderhauf, Ian Reid, Stephen Gould, and Anton den Hengel. Vision-and-language navigation: Interpreting visually-grounded navigation instructions in real environments. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3674-3683, 2018b. 00387.
doi: 10.1109/CVPR.2018.
Sergio Arnaud, Paul Mcy, Ada Martin, Arjun Majumdar, Krishna Murthy Jatavallabhula, Phillip Thomas, Ruslan Partsey, Daniel Dugas, Abha Gejji, Alexander Sax, et al. Locate 3d: Real-world object localization via self-supervised learning in 3d. arXiv preprint arXiv:2504.144 14151, 2025.
Ali Ayub, Zachary De Francesco, Patrick Holthaus, Chrystopher L Nehaniv, and Kerstin Dauten- hahn. Continual learning through human-robot interaction: Human perceptions a continual learning robot in repeated interactions. International Journal of Social Robotics, pp. 1-20, 2025a.
Ali Ayub, Zachary De Francesco, Patrick Holthaus, Chrystopher L Nehaniv, and Kerstin Dauten- hahn. Continual learning through human-robot interaction: Human perceptions of continual learning robot in repeated interactions. International Journal of Social Robotics, 17(2):277-296, 2025b.
Jihwan Bang, Heesu Kim, Youngloon Yoo, Jung-Woe Woo Ha, and Jonghyun Choi. Rainbow memory: Continual learning with a memory of diverse samples. In CVPR, pp. 8218-8227, 2021.
Amir Bar, Gaoyue Zhou, Danny Tran, Trevor Darrell, and Yann LeCun, Navigation world models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 15791-15801, 2025.
Cheng Chen, Junchen Zhu, Xu Luo, Hengtao Shen, Jingkuan Song, and Lianli Gao. Coin: A bench- mark of continual instruction tuning for multimodel large language models. NeurIPS, 37:57817-- 57840, 2024.
Kevin Chen, Junshen K. Chen, Jo Chuang, Marynel V'azquez, and Silvio Savarese. Topological planning with transformers for iision-anddlaanuuag navigation. JEEE/CVF Conference on Com- puter Vision and Pattern Recognition, pp. 1 11271-11 1-271-11221, 2020.
Peihao Chen, Dongyu Ji, Kunyang Lin, Runhao Zeng. Thomas Li, Mingkui Tan, and Chuang Gan. Weakly-supervise multi-granularity map learning for vision-and-language navigation. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 38149-34443 18119-38161. Curran Associates, Inc., 2022.
Wei-Lin Chiang, Zhuohan Li, Ziqing Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vieuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See htts:://vicuna, Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
11

---

Published as a conference paper at ICLR 2026
Mohammad Mahdi Derakhshani, Xiantong Zhen, Ling Shao, and Cees Snoek. Kernel continual learning. In ICML, pp. 2621-2631, 2021.
Jiahua Dong, Qi Lyu, Baichen Liu, Xudong Wang. Wenqi Liang, Duzhen Zhang, Jiahang Tu, Hongliu Li, Hanbin Zhao, Henghui Ding, Yulun Zhang, Zhi Han, Nicu Sebe, Fahad Shahbaz Khan, Salman Khan, Mubarak Shah, Philip Torr, Ming- -ssan Yang, and Dacheng Tao. Learning to model the world: A survey of world models in artificial intelligence. TechRxiv, 2026.
Shihan Dou, Enyu Zhou, Yan Liu, Songyang Gao, Wei Shen, Limao Xiong, Yuhao Zhou, Xiao Wang, Zhiheng Xi, Xiaoran Fan, et al. Loramoe: Alleviating world knowledge forgetting in large language models via moe-style plugin. In ACL, pp. 1932-1945, 2024.
Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. Podnet: Pooled outputs distillation for small-tasks incremental learning. In ECCV, pp. 86-102, 2020.
Tsu-Jui Fu, Xin Eric Wang. Matthew F. Peterson, Scott T. Grafton, Miguel Eckstein, and William Yang Wang- Counterfactual vision-and-language navigation via adversarial path sam- pler. In Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (eds.). European Conference on Computer Vision, pp. 7 71-86, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58539-6.
Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, and Si Liu. Octonav: Towards generalist embodied navigation. arXiv preprint arXiv:2506.000 09839, 2025a.
Chen Gao, Liankai Jin, Xingyu Peng, Jiazhao Zhang, Yue Deng, Annan Li, He Wang, and Si Liu. Octonav: Towards generalist embodied navigation. arXiv preprint arXiv:2506.099 2592999666999, 2025b.
Chongyang Gao, Kezhen Chen, Jinmeng Rao, Baochen Sun, Ruibo Liu, Daiyi Peng, Yawen Zhang, Xiaoyuan Guo, Jie Yang, and VS Subrahmanian. Higher layers need more lora experts. arXiv preprint wwXiv:2402.08566, 2024.
Mingfei Han, Liang Ma, Kamila Zhumakhanova, Ekaterina Radionova, Jingyi Zhang, Xiaojun Chang, Xiaodan Liang, and Ivan Laptev. Roomtour3d: Geaaetry-wware sideo-instruction tun- ing for embodied navigation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 27586-27599 2025.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance imagenet classification. In Proceedings of the IEFF international conference on computer vision, pp. 1026-1034, 2015.
Yicong Hong, Cristian Rodriguez, Yuankai Qi, Qi Wu, and Stephen Gould. Language and visual entity relationship graph for agent navigation. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp- 7685-7696. Curran Associates, Inc., 2020.
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022.
Sangwon Jung, Hongjoon Ahn, Sungmin Cha, and Taesup Moon. Continual learning with node- importance based adaptive group sparse regularization. NewrIPS, 33:3647-3658, 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barnskaaaaawiisaa,, et al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3526 2017.
Jacob Krantz, Erik Wijmans, Arjun Majumdar, Dhruv Batra, and Stefan Lee. Beyond the nav- graph: Vision-and -anguage navigation in continuous environments. In European Conference Computer Vision, pp. 104-120, Cham, 2020. Springer International Publishing. ISBN 978-3-030- 58604-1.
Alexander Ku, Peter Anderson. Roma Patel, Eugene Ie, and Jason Baldridge. Room- Rcom-accoos-room: Multilingual vision-and vision-and-language navigation with dense spatiotemporal grounding. In Conference Empirical Methods in Natural Language Processing, 2020.
grouunddin
12

---

Published as a conference paper at ICLR 2026
Xiaohan Lei, Min Wang, Wengang Zhou, and Houqiang Li. Gaussnav: Gaussian splatting for visual navigation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025.
Guodun Li, Yuchen Zhai, Qianglong Chen, Xing Gao, Ji Zhang, and Yin Zhang. Continual few-shot intent detection. In COLING, pp. 333 343, 2022.
Yichen Li, Haozhao Wang, Wenchao Xu, Tianhhe Xiao, Hong Liu, Minzhu Tu, Yuying Wang. Xin Yang, Rui Zhang, Shui Yu, et al. Unleashing the power of continual learning on non-centralized devices: A survey, IEEE Communications Surveys & Tutorials, 2025.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12):2935-294) 2017.
Wenqi Liang, Gan Sun. Qian He, Yu Ren, Jiahua Dong, and Yang Cong. Never-ending behavior- cloning agent for robotic manipulation. arXiv preprint arXiv:2403.0m 20Xx::2403.00336,, 2024.
Wenqi Liang, Gan Sun, Yao He, Jiahua Dong. Suyan Dai, Ivan Laptev, Salman Khan, and Yang Cong. Pixelvla: Advancing pixel-level understanding in vision-language-cttoon model. arXiv preprint arXiv:2511.01577 2025.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
Yunhao Liu, Li Liu, Yawen Zheng, Yunhuai Liu, Fan Dang, Ningbo Li, and Ke Ma. Embodied navigation. Science China Information Sciences, 68(4):1-39, 2025.
Swarnamouli Majumdar, Sonny E Kirkley, and Biswadip Basu Mallik. LIm-guided hybrid archi- tecture for autonomous fire response: Dialog-driven planning in space and disaster missions. In 2025 IEEE World Al loT Congress, pp. 1049-1054. IEEF, 2025.
Yuan Meng, Zhenshan Bing, Xiangtong Yao, Kejia Chen, Kai Huang, Yang Gao, Fuchun Sun, and Alois Knoll. Preserving and combining knowledge in robotic lifelong reinforcement learning. Nature Machine Intelligence. pp. 1-14, 2025.
Humza Naveed, Asad Ullah Khan, Shi Qiu, Muhammad Saqib, Saeed Anwar, Muhammad Usman, Naveed Akhtar, Nick Barnes, and Ajmal Mian. A comprehensive overview of large language models. ACM Transactions Intelligent Systems and Technology, 16(5):1-72, 2025.
Dujun Nie, Xianda Guo, Yiqun Duan, Ruijun Zhang, and Long Chen. Wmnav: Integrating vision- language models into world models for object goal navigation. arXiv preprint arX1v:2503.02247, 2025.
Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environ- ments. In Proceedings of the IEEE Conference Computer Vision and Pattern Recognition, 2020a.
Yuankai Qi, Qi Wu, Peter Anderson, Xin Wang, William Yang Wang, Chunhua Shen, and Anton van den Hengel. Reverie: Remote embodied visual referring expression in real indoor environ- ments. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9979-9988, 2020b. doi: 10. (U.11099CCVP4422002200000
Yanyuan Qiao, Haodong Hong, Wenqi Lyu, Dong An, Siqi Zhang, Yutong Xie, Xinyu Wang, and Qi Wu. Navbench: Probing multimodal large language models for embodied navigation. arXiv preprint arXiv:2506.01031, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763. PmLR, 2021.
Sonia Raychaudhuri, Saim Wani, Shivansh Patel, Unnat Jain, and Angel X. Chang. Language- aligned waypoint (law) supervision for vision-and- lnd-aauugge navigation in continuous environ- ments. In Conference on Empirical Methods in Natural Language Processing, 2021.
13

---

Published as a conference paper at ICLR 2026
Sylvestre- Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In CVPR, pp. 2001-2010, 2017.
Qing Sun, Fan Lyu, Fanhua Shang, Wei Feng, and Liang Wan. Exploring example influence in continual learning. NewrIPS, 35:27075-27086, 2022.
Quan Sun, Yuxin Fang, Ledell Wu, Xinlong Wang, and Yue Cao. Eva-clip: Improved training techniques for clip at scale. arXiv preprint vrxiv:2303./538. 2023.
Jesse Thomason, Michael Murray, Maya Cakmak, and Luke Zettlemoyer. Vision-and-dialog navi- gation. In Proceedings of the Conference on Robot Learning, pp. 394 406, 2020.
Chunlin Tian, Zhan Shi, Zhijiang Guo, Li Li, and Shteng-hhon Xu. Hydralora: An asymmetrie lora architecture for efficient fine-tuning. Advances in Neural Information Processing Systems, 37: 9565-9584, 2024.
Marco Toldo and Mete Ozay. Bring evanescent representations to life in lifelong class incremental learning. In CVPR, pp. 16732-16741, 2022.
Weikang Wan, Yifeng Zhu, Rutav Shah, and Yuke Zhu. Lotus: Continual imitation learning for robot manipulation through unsupervised skill discovery. In ICRA, pp. 537-544, 2024.
Huiyi Wang, Haodong Lu, Lina Yao, and Dong Gong. Self-expansion of pre-trained models with mixture of adapters for continual learning. Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 10087-10098, 2025.
Liyuan Wang, Xingxing Zhang, Qian Li, Jun Zhu, and Yi Zhong. Cosel: Cooperation of small continual learners stronger than a big one. In ECCV, pp. 254-271, 2022.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu, A comprehensive survey of continual learn- ing: Theory, method and application. IEEE transactions on pattern analysis and machine intelli- gence, 46(8):5362-5388 2024.
Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learning. arXiv preprint arXiv:2310.141 2023a.
Xudong Wang, Zebin Han, Zhiyu Liu, Gan Li, Jiahua Dong, Baichen Liu, Lianqing Liu, and Zhi Han. Lifelong language-condition conditioond robotic manipulation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026a.
Xudong Wang, Gan Li, Zhiyu Liu, Yao Wang, Lianqing Liu, and Zhi Han. All-day multi-scenes lifelong vision-and-laaguen language navigation with tucker adaptation. In The Fourteenth International Conference on Learning Representations, 2026b.
Zun Wang, Jialu Li, Yicong Hong. Yi Wang, Qi Wu, Mohit Bansal, Stephen Gould, Hao Tan, and Yu Qiao. Scaling data generation in vision-and-language navigation. In 2023 IEEE/CVF In- ternational Conference on Computer Vision (ICCV) pp. 11975-11986, 2023b. doi: 10.1109/ ICCV51070.2023.01100
Meng Wei, Chenyang Wan, Xiqian Yu, Tai Wang, Yuqiang Yang, Xiaohan Mao, Chenming Zhu, Wenzhe Cai, Hanqing Wang, Yilun Chen, Xihui Liu, and Jiangmiao Pang. Streamvin: Streaming vision-and-language navigation via slowfast context modeling. 2025.
Yichen Wu, Hongming Piao, Long-Kat Kai Huang, Renzhen Wang, Wanhua Li. Hanspeter Pfister, Deyu Meng, Kede Ma. and Ying Wei. S-lora: Scalable low-rank adaptation for class incremental learn- ing. arXiv preprint arXiv:2501.13/95 2025.
Ziyang Wu, Christina Baek, Chong You, and Yi Ma. Incremental learning via rate reduction. In CVPR. pp. 1125-1133, 2021.
Jiannan Xiang, Tianhua Tao, Yi Gu, Tianmin Shu, Zirui Wang, Zichao Yang, and Zhiting Hu. Lan- guage models meet world models: Embodied experiences enhance language models. NeurIPS, 36.75392-75412. 2023.
14

---

Published as a conference paper at ICLR 2026
Ye Xiang, Ying Fu, Pan Ji, and Hua Huang. networks. In ICCV, pp. 6619-6628, 2019.
Incremental learning using conditional adversarial
Yutao Yang, Jie Zhou, Xuanwen Ding, Tianyu Huai, Shunyu Liu, Qin Chen, Yuan Xie, and Liang He. Recent advances of foundation language models-bases continual learning: A survey, ACM Computing Surveys, 57(5):1-38, 2025.
Yuanqi Yao, Siao Liu, Haoming Song, Delin Qu, Qizhi Chen, Yan Ding, Bin Zhao, Zhigang Wang, Xuelong Li, and Dong Wang. Think small, act big: Primitive prompt learning for lifelong robot manipulation. In CVPR, pp. 22573-22583, 2025.
Xinrui Yu and Jafar Saniie. Visual impairment spatial awareness system for indoor navigation and daily activities. Journal of Imaging, 11(1):9, 2025.
Duzhen Zhang, Yong Ren, Zhong-Zh Zhi Li, Yahan Yu, Jiahua Dong, Chenxing Li, Zhilong Ji, and Jinfeng Bai. Enhancing multimodal continual instruction tuning with branchlora. arXiv preprint 2tt.:2506.02441 2025a.
Lingfeng Zhang. Xiaoshuai Hao, Yingho Tang, Haoxiang Fu, Xinyu Zheng, Pengwei Wang, Zhongyuan Wang, Wenbo Ding, and Shanghang Zhang. navd3: Understanding any instruction, navigating anywhere. finding anything. arXiv preprint arXiv:2508.045:: 2025b.
Duo Zheng, Shijia Huang. Lin Zhao, Yiwu Zhong, and Liwei Wang. Towards learning generalist model for embodied navigation. In 2024 IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 13624-1 13634, 2024. doi: 10.1 D0.1099CCVPSS73332200000999
Jinliang Zheng, Jianxiong Li, Dongxiu Liu, Yinan Zheng, Zhihao Wang. Zhonghong Ou, Yu Liu, Jingjing Liu, Ya-Qin Zhang, and Xianyuan Zhan. Universal actions for enhanced embodied foun- dation models. Proceedings of the Computer Vision and Patlern Recognition Conference, pp- 22508-22515 2025a.
Junhao Zheng, Shengjie Qiu, Chengming Shi, and Qianli Ma. Towards lifelong learning of large language models: A survey. ACM Computing Surveys, 57(8)11335, 2025b.
Gengze Zhou, Yicong Hong, Zun Wang, Chongyang Zhao, Mohit Bansal, and Qi Wu. Same: Learn- ing generic language-guided visual navigation with state-adaptive mixture of experts, 2024.
Ruiqi Zhu, Endong Sun, Guanhe Huang. and Oya Celiktutan. Efficient continual adaptation of pretrained robotic policy with online meta-leamed adapters. arXiv preprint arXiv:2503./8838 2025.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A EXPERIMENTS IMPLEMENTATION
Below we provide detailed experiment implementation.
Benchmark and task setup. We evaluate on the Lifelong Embodied Navigation (LENL) benchmark built on the Matterport3D simulator Anderson al. (2018a). The benchmark contains 18 distinct navigation scenes and three instruction styles: (1) VLN (step-by-step route instructions), (2) OLN (object-localization style instructions), and (3) DUN (dialogue-style instructions). Tasks are pre- sented as ordered sequence of 18 tasks: the first 15 tasks are used for continual (lifelong) learning while the final tasks are reserved for zero-shot generalization tests unseen environments. In the LENL protocol the task identifier t is available to the training procedure (to allow task-specifi adapter/cxpert creation) but not provided at test time (evaluation is aask-agnostic). The scene splits and examples are illustrated in Fig. of the main paper. The specific statistical information for the 18 tasks is shown in Table
Table 7: Statistics of the 18 sequential tasks of LENL benchmark. Each task corresponds to unique scene under one instruction style.
Task ID
Type
Scene ID
INST Style
Train INST Number
Test INST Number
S1
Continual Learning
ILXtFLiw3gL
VLN
801
99
S2
Continual Leamning
IpXnuDYAj8r
VLN
813
90
S3
Continual Leariing
pRbA3pwrgk9
OLN
336
195
S4
Continual Leurning
Sq7pvUzZiYa
VLN
738
63
S5
Continual Leaming
759xd9YjKWS
OLN
693
147
S6
Continual Leaminng
VVie2KiqLaN
DUN
24
S7
Continual Learning
82sE5b5pLXE
63
S8
Continual Learning
PSwJjjQQHXXX
OLN
1671
444
S9
Continual Learinng
B6ByNcgPMKs
VLN
801
72
S10
Continual Leaming
Le3va
OLN
291
153
S11
Continual Leaming
D7N2EKCX4S
VIN
837
63
S12
Continual Learning
DUN
228
36
S13
Continual Leamning
ULsK&CPVEIR
OLN
1848
75
$14
Continual Leaminng
eVARVeZNuST
VLN
837
63
S15
Continual Leurning
4iMM
OLN
654
141
S16
Unseen Generalization
EDJbREhghZL
VLN
72
S17
Unscen Generalization
V2XKFyX4AS
69
S18
Unseen Generalization
VzgíbhrpDEA
DUN
27
Comparisons. All methods (our method and comparisons) use the same NavLLM backbone to en- sure fair comparison. Following NavLLM (Zheng et al. (2024), visual encodings are produced by CLIP visual encoder (Radford et al. (2021)) and fused into the language model i.e., Vieuna (Chiang et al. (2023)), via the same multi-modk interface used across experiments. Implementation details of the multimodal interface follow the main paper ($.). For fair and informative compari- son re-implemented configured a collection of state-of-the-arr LoRA-based continual learning and Mixture-of-E.ae offexxrr-s-- (MoE) methods on the same NavLLM backbone, input preprocessing, train- ing schedule and evaluation pipeline. All baselines train only adapter/LoRA-sttyl parameters while keeping the backbone model weights frozen unless otherwise stated, Below we summarize each baseline, the key implementation choices used in experiments, and their inference behavior.
Seq-FT. Sequential fine-tuning of the base model using vanilla LoRA adapter per task. For each task we train standalone LoRA module (rank r 64). No explicit mechanism is used to avoid forgetting; after finishing the whole curriculum the stored LoRA module corresponding to a requested task is used for inference. Seq-FT serves as lower-bound reference for catastrophic forgetting.
LwF-LoRA Li & Hoiem (2017). Extension of the wrrtnningwtthht----ooo task apply anoweege-ddistllatte losses to frozen copy the model (logits and selected hidden states) so as to encourage the newly-trained trained LoRA preserve behavior previously tasks. We follow the distillation configura-
paradigm
to LoRA: during training
16

---

Published as a conference paper at ICLR 2026
tion and papeerrprrmmttiizttionn recommended in Li & Hotem (001 At inference time the stored LoRA corresponding to the target task is loaded (rank r 64).
EWC-LLORA Xiang et al. (023). Elastic Weight Consolidation adapted to LoRA parame- ters: after each finished task we estimate parameter importance (Fisher information) for the trained LoRA parameters and penalize future updates to parameters with high importance. We apply the EWC penalty only to LoRA adapter weights and follow the hyperparameter schedule described in Xiang al. (2023) The stored per-task LoRA adapter is used for
evaluation after training (rank r 64).
Dense MoLE Chen et al. (2024). A dense Mfxture-of-Loon LoRA -Experts design in which all experts are activated for every forward pass (i.c., dense combination of expert outputs). This design captures cross-task interactions more richly but increases per-step computation and memory usage compared to sparse routing. Expert outputs are aggregated with learnable gating coefficients; the full MOE-LoRA of adapters is retained for inference (rank r 16, K = 8).
Sparse MoLE Dou al. (2024). Sparse routing MoE applied to LoRA adapters: input is routed to only a subset of experts to reduce computation and encourage specialization. We use a top-k routing policy with k experts per instance, implement load-balancing auxiliary losses where applicable, and keep gating parameters trainable. After training the full sparse MOE-LoRA checkpoint (all expert adapters and the gating network) is used for inference (rank r = 16, K 8).
MoLA MoLAGGaoee et al. (2024). Builds on Sparse MoLE by introducing hierarchical/deeper expert layers for richer adaptation. Concretely, we instantiate a two-level expert hierarchy with 4 shallow experts and 16 deep experts (shallow experts feed into deeper expert layers). Routing is performed at both levels; during inference the trained MoLA adapter collection and gating modules are used (rank r 16.
O-LoRA Wang et al. (002a)). Per-task vanilla LoRA storage combined with an rrthogonality-proooiinee regularizer applied during training encourage disentangled task representations. Practically this requires storing one LoRA module per task. For inference we adopt the proposed TAKA selection method. (rank r 24.
HydraLoRA Tian et al. (2024). A hybrid design that shares global module A across all tasks to capture common knowledge while using multiple task-specific modules for specialization. During each task's training only the corresponding B (and optionally A) are updated according to the method specification. At inference the composed Hydral LoRA adapter (A combined with selected B) is loaded (rank r = 16, K = 8).
BranchLoRA Zhang al. 2025a Extends sparse routing with an explicit branching mechanism that allows the model to select different B branches for different input modes or instruction styles. Branch selection is learned, and branch-specific LoRA adapters are stored per branch. For evaluation, BranchLoRA uses branch selection to pick the appropri- ate B at test time (rank r = 16, K 8).
SD-LoRA Wu et al. 2025). Stores T task-specific LoRA modules and dynamically com- poses them during inference to synthesize behavior for the current input. We follow the composition rules and hyper-parameter choices recommended in Wu et al. (2025). For in- ference we adopt the proposed TAKA selection method. (rank r 24.
Evaluation metrics. Let an agent's executed trajectory within an episode be denoted by T (P1, .2..... Px), where p. is the agent's 's position at timestep i. Let g denote the goal position and d(.,.) be the distance function in the environment (c.g.. geodesic Euclidean distance depending on the simulator). We use three standard navigation metrics described below. In all experiments we 1.0 unless stated otherwise).
adopt success threshold (we
Success rate (SR). An episode is counted as successful if the final agent position is within e of the goal. Formally, the per-episode success indicator is
SRep = {
if d d(pn, g) S
(12)
otherwise.
17

---

Published as a conference paper at ICLR 2026
The reported SR is the mean of SRep over all test episodes. Oracle success rate (OSR). Oracle success measures whether the agent visited a position within e of the goal at any time during the episode. It is defined per episode as
if minisign d(p..g) Se,
ep 0, otherwise,
(13)
and the dataset-level OSR is the average of OSRep across episodes.
Success weighted by Path Length (SPL). SPL jointly measures success and path efficiency. For episode j, let Sj E {0, 1 be the success indicator (as above), let L, denote the length of the agent's s executed path, and let L; denote the shortest-path (optimal) distance from the start the goal. The standard per-episode contribution for SPL. is
L;
SPLpp = SJ: max L2)
(14)
and the reported SPL is the mean of SPLep over all episodes. SPL penalizes long inefficient trajectories while counting only successful attempts.
We use eight NVIDIA RTX 6000 Ada Generation GPUs with PyTorch 2.1.2 (cu121) for training and testing. And the Adam optimizer with an initial learning rate of 3.0 x 10-3 used for training. The low-rank 16 of LORA, the number of experts is 2, J 0.5, and all the other hyper- parameter are consistent with the NavLLM (Zheng et al. (2024)) )) We summarize some important parameter settings as shown Table
Table 8: Summary of primary hyper-parameters used in experiments.
Hlyper-paramete
Value
TOP-K activated expert
K = 2
LoRA rank
r = 16
balance hyperparameter of Lsec,t
Ascc 0.1
balance hyperparameter of Lesoe,t
Aesoc = 0.1
perturbation for avoiding the computational avalanches
0.01
Fisher smoothing
w = 0.9
elements threshold
0.5
LLM
Vicuna-7B-v0
VIT
WVA-CLIP-02-Lai (428M)
training steps
2000
batch size
64
sampling strategy with temperature
0.01
B ALGORITHM SUMMARY
For ease of readers understanding, summary of the proposed training algorithm is provided in the Algorithm I and a summary of inference algorithm is provided the Algorithm
C DETAILED COMPARISON RESULTS
Fig. presents the average results for SPL, SPL-F, OSR, and OSR-F. In this section, we provide the task- -isk- comparison results. The original comparison results (SPL, F-SPL, OSR. F-OSR) for Fig 6 are summarized in Table 9 Table 10 Table TT and Table 12 Uni-Walker achieves a higher average success rate of 66%, surpassing the previous best (59%) by 7%, and lower forgetting rate of 5%, improving the prior best (16%) by 11%. Uni-Walker achieves a higher SPL of 61 61%, surpassing the previous best (38%) by 23%, and a lower forgetting rate of 7%, improving the prior best (42%) by 35%. It achieves a higher OSR of 81%, surpassing the previous best (79%) by 2%, and lower forgetting rate of 5%, improving the prior best (7%) by 2%. Based the results, Uni Walker achieves consistent superiority across various metrics. The visualization examples of VLN, OLN, DUN are visualized in Fig.
18

---

Published as a conference paper at ICLR 2026
Algorithm Uni-Walker: Lifelong Training
Require: Frozen backbone Fög. initialize shared A, initial expert set (Bi). task sequence
[Ti....: Tr. T index R.
1: for task to 7 do
2: i 1 then
Create new expert B, Initialize B, via Knowledge Inheritance Strategy (KIS) with E.c2
4:
5:
end if
6: Add (Es,,, E1,() placeholder to retrieval index Re
7:
Compute 1-th Fisher Matrix Ft with Eq
8:
Update and save the 1-th Fisher Matrix F% with Eq
9:
for epoch = 1 to max_epochs, do for each minibatch (O,I, I, A) from task T; do E, + CLIP V(O), E, f- CLIP T(I) Activating TOP-K expert subspace K.
10:
11:
12:
TAKA(E,, E,, Re, (1) with E.
13:
Compute L (shared smoothing consolidation loss) using Fisher matrices with Eq
Compute Lesoc,t (expert subspace orthogonality constrain loss) with Eq 7
15:
Compute L, (auto-regressive action generation loss) with Eq
Back-propagert and update only and B, parameters (freeze other B,,)
17:
end for
end for
19: Save expert checkpoint B, and shared A checkpoint
end for
21:
return saved [B BTJ. final A, retrieval index Re
Algorithm Uni Walker: Inference with TAKA and Expert Co-activation
Require: Frozen backbone Fe,, shared A, saved experts [B,...., .Br), retrieval index Rey input
Ig). threshold μa, top-K
Ensure: Predicted action token sequence
Eg t CLIPPV (OO,) E, t CLIP-T(I,)
2: Compute instruction similarities $i, cosinc(E,, E,, E1,t) for all (-, E1,) E Re with Eq
3: Build instruction mask M, = ( eh
4: Compute observation similarities 8,,1 cosine( E., Es..), then 8, M2. 80,t with Eq
5: Select top-K experts K- = TopK($;,K)
6: Forward using Fe, with adapters combined over K. with Eq 3
7: Generate tokens to produce next actions using the model's policy head
8: return action token sequence
Table 9: Test results (task-wits wise SPL 1, %) of comparison experiment with our LENL settings.
Comparison Methods
S1
$2
S8
$9
S10
S11
S12
S13
S14
S15
S17
S18
Avg.
Seq-FT -T-FFT (sequential mn--tnning)
0
0
0
10
6
8
15
10
12
79
8
LwF-LoRA (Li & Hoiem
2017
0
0
0
11
5
6
21
36
24
80
11
EWC-LoRA XRinng
25
32
Dense MoLE beb
11:4
2
2
2
10
8
12
10
20
36
40
78
13
Sparse MoL.E Dou al
2124
2
2
10
13
12
15
10
24
34
42
78
14
MoLA (Gao
23
36
44
80
16
Hydral (KA Tian
20/24)
8
10
12
13
10
12
16
8
12
16
22
37
45
80
14
11
10
19
BranchLoRA' Zlliang cral.
2025a)
8
10
13
27
36
78
20
O-LoRA (Wan
TAKA
30
25
26
65
48
80
37
SD-LoRA (o Wu
2025 ITAKA
32
33
34
35
26
26
36
38
26
36
37
26
66
49
80
38
30
42
38
Uni-Walker (ours)
66
63
59
61
49
50
65
74
50
72
56
51
82
56
80
71
55
45
61
D DETAILED ABLATION RESULTS
We perform three ablation about Uni-Was Walker in the main paper. In this section, we provide the task-wise ablation results. The ablation studies results of the exploration for navigation tasks shared knowledge summarized in Table and the wise ablation results (SR) are summarized in
19

---

Published as a conference paper at ICLR 2026
VLN user instruction: Please lurn through dour with il, continue straight ahead until you reach the white bed, then turn right towards the table lamp and stop at the doonway.
Obsoraaoon
OLN user instruction: Please go to living room and help me the sofa chair.
user instruction:
flowers are not watered for long time.
Where are the llowers?
A: Go straight to the main gate, turn right, and continue straight past the table.
Oisemaatrn
Steps
Figure 7: Illustration of three visualization examples of VLN, OLN, DUN. We visualize each step of the Eni-Walker's selection process and annotate with arrows to clearly illustrate the navigation sequence.
Table 10: Test results (task-wise SPL-F J, %) of comparison experiment with LENL settings.
Comparison Methods
S1
$2
$4
S5
$9
S11
S12
S13
S14
S15
S17
Avg.
Fey-FT (sequential line uning)
100
100
100
97
96
100
97
88
88
92
86
72
88
79
100
100
100
88
LwF-LoRA [T
85
EWC-LoRA Xiang el al
20745
100
97
100
94
100
100
100
88
88
97
93
54
61
47
0
100
100
100
84
MoL.F
97
Sparse MoLE Dou el
2124
97
97
100
85
96
100
85
85
77
79
83
56
60
26
100
97
100
79
(Gao
77
HYdraLLRA Tall al.
20124) (0:24)))
88
85
81
80
96
81
82
81
85
84
73
61
56
22
0
85
84
79
72
BranchLoRA
iZhanz
70
O-LoRA Wang al
202251
56
52
48
31
50
54
51
58
53
55
41
54
23
17
0
65
53
35
44
al.
Unl- Malkea (ours)
3
11
5
19
8
Table 11: Test results (task-wise OSR 1, %)
of comparison experiment with our LENL settings.
Comparison Methods
S1
S2
S3
S8
59
S10
$11
S12
S13
$14
$15
S16
S17
S18
Avg.
Seq-FT (sequential fine-tunine)
12
12
14
16
16
19
16
18
15
35
36
36
100
16
15
24
LwF-LoRA [Li & Hoiem
2017
12
15
16
15
15
29
16
32
32
45
55
51
100
16
28
EWC-LoRA Xtang
56
52
Dense Mol.t Cheen al
4124
21
22
18
23
16
18
19
31
16
33
41
48
69
68
100
22
20
17
33
Sparse MoLE DDn et
21724
23
23
19
36
19
40
47
69
72
100
MoLA (Gao
26
21
69
73
100
36
HydraLoKA Tiain
2024)
27
25
21
25
20
25
24
36
22
27
46
48
72
68
100
29
26
22
37
BranchLoRA [Ziaing ct
2025aa)
32
36
26
46
88
73
100
O-LoRA (Wang 212.0
76
82
67
86
98
80
100
SD-LoRA L Wu 2025 TTAKA
76
82
85
66
88
84
76
85
67
72
66
85
98
81
100
79
68
71
79
Uni-Walker (ours)
82
87
88
67
90
90
72
83
63
73
62
84
100
78
100
84
76
79
81
and the task-wise ablation results (F-SR) are summarized in Table 14 and the task-wise ablation results (SPL) are summarized in Table 15 and the pask-wise ablation results (F-SPL) are summarized in Table 16 and the tisk ablation results (OSR) are summarized in Table 17 and the task-wise ablation results (F-OSR) are summarized Table 18 The ablation studies' results about the exploration for navigation tasks shared knowledge summarized in the Table 5 and
Table 13
20

---

Published as a conference paper at ICLR 2026
Table 12: Test results (task-wise OSR-F %) of comparison experiment with our LENL settings.
Comparison Methods
S1
S2
S9
S10
S11
S12
S13
S14
S15
Avg.
Seq-FT (sequential fine-tunine)
86
75
59
64
58
73
LwF-LoRA (Li Hojem
2017
86
83
82
86
84
80
66
75
57
54
48
45
40
0
86
83
81
67
EWC-LoRA Xlana et m.
2025
85
83
82
79
84
83
81
66
77
55
48
47
44
39
0
83
79
66
20
Sparse MoLE Dou et
2174
73
74
79
64
79
77
71
58
71
65
42
45
31
15
0
74
70
79
59
Mol.A (Gao al 20024
69
73
82
68
56
68
35
44
31
14
0
71
58
HydraLOKA [Tan
2024])
72
58
68
28
20
BrunchLoRA Zhang al.
2025a.)
62
67
74
63
74
72
65
58
62
59
33
41
12
14
0
65
65
64
53
O-LoRA (Waigg atal, 20250 JUIKA
11
10
2
6
SD-LoRA Wu
J+TAKA
11
5
Uni-Walker (ours)
4
2
2
7
5
2
0
8
12
12
5
Table 13: Ablation study results (SR 1. %) for the methods with task-shared navigation knowledge exploration.
Methods
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
50
56
46
51
39
38
51
71
46
60
51
48
82
60
86
56
Uni-Walker w/o KIS
61
60
51
55
43
46
56
74
48
68
59
49
86
63
86
60
Uni-Wai Shiierlker w/o SSC
60
59
49
56
42
45
56
75
48
69
58
47
84
62
86
60
Uni-Walker w/o ECAS
59
58
48
53
42
45
55
72
46
62
55
46
82
62
86
58
Uni-Walker (ours)
67
67
75
67
57
50
67
83
50
73
62
53
88
65
86
67
Table 14: Ablation study results (F-SR J, %) for the methods with task-shared navigation knowledge exploration.
Methods
$2
S3
S4
S8
S9
S10
$11
S12
S13
$14
S15
Avg.
Baseline
29
20
42
16
12
20
9
Uni-Walker w/o KIS
13
14
35
31
28
12
20
13
8
9
13
4
3
0
14
Uni-Walker w/o SSC
14
16
38
12
8
8
7
Uni-Walker w/o ECAS
16
17
39
34
30
13
21
15
12
17
15
18
9
5
17
Uni-Walker (ours)
4
4
5
16
4
4
2
4
3
5
5
2
0
4
Table 15: Ablation study results (SPL 1, %) for the methods with task-shared navigation knowledge exploration.
Methods
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
29
30
30
42
25
23
32
35
24
33
34
25
65
48
80
37
Uni- malkea w/o KIS
36
37
38
49
33
35
48
75
51
46
46
45
78
56
80
50
Uni-Walker w/o SSC
36
36
37
48
33
35
46
43
36
43
45
42
76
51
80
46
w/o ECAS
35
36
35
49
32
32
45
43
35
42
42
40
75
49
80
45
Uni- Uni-Walker (ours)
66
63
59
61
49
50
65
74
50
72
56
51
82
56
80
62
the task-wise ablation results (SR) are summarized in Table 19 and the task-wise ablation results (F-SR) are summarized in Table 20 and the task-wise ablation results (SPL.) are summarized in Table 21 and the task-wise ablation results (F-SPL) are summarized in Table 22 and the task-wiss ablation results (OSR) are summarized in Table 23 and the task-wise ablation results (F-OSR) are summarized in Table 24
E COMPARISON WITH OTHER EMBODIED NAVIGATION AGENTS
We construct universal embodied navigation agents through lifelong learning. We also perform com- parison experiments with the other state-of-the-art universal navigation agents that pre-trained
21

---

Published as a conference paper at ICLR 2026
Table 16: Ablation study results (F-SPL 4. %) for the methods with task-shared navigation knowl- edge exploration.
Methods
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Bascline
57
54
52
35
52
56
54
59
55
56
42
56
24
17
45
Uni-Walker w/o KIS
47
43
39
25
37
30
12
4
39
22
21
9
3
24
Uni-Walker w/o SSC
47
45
40
26
37
33
33
49
32
43
24
26
12
12
31
Uni-Walke w/o ECAS
49
45
44
25
38
38
35
49
34
44
29
30
13
16
32
Uni-Walker (ours)
3
3
5
13
6
4
11
5
3
Table 17: Ablation study results (OSR t, %) for the methods with task-shared navigation knowledge exploration.
Methods
S1
S2
S3
S7
S8
59
S10
S11
S12
S13
S14
S15
Avg.
Baseline
75
80
79
63
81
70
82
62
70
80
91
75
100
77
Uni-Walker w/o KIS
77
82
78
65
82
88
71
82
62
70
61
80
91
75
100
78
Uni-War Kni-Walker w/o SSC
78
81
79
82
61
71
91
75
100
Uni-Walker w/o ECAS
77
81
79
66
81
88
72
82
62
71
61
81
95
78
100
78
Hni-Walker (ours)
82
87
88
67
90
90
72
83
63
73
62
84
100
78
100
81
Table 18: Ablation study results (OSR-F J. %) for the methods with task-shared navigation knowl-
edge exploration.
Methods
S1
S2
S3
S4
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
12
10
12
12
10
4
9
7
9
12
Uni Walker w/o KIS
9
8
13
7
11
4
9
4
9
7
6
7
9
12
8
Uni-Walker w/o SSC
8
9
12
9
4
10
5
9
12
w/o ECAS
9
9
12
12
8
4
9
5
6
5
8
Mai-Walker (ours)
4
2
2
4
8
2
7
3
2
0
8
3
Table 19: Ablation study results (SR t, %) for the methods with task-specific navigation knowledge exploration.
Methods
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
45
50
42
46
42
32
42
60
41
52
48
41
76
59
71
50
Uni-Walker w/o ESOC
65
62
70
61
54
45
48
69
59
50
85
63
64
Uni-Walker w/o NSCoT
46
51
43
46
43
33
45
61
42
55
49
44
77
60
71
51
Uni-Walker (ours)
67
67
75
67
57
50
67
83
50
73
62
53
88
65
86
67
Table 20: Ablation study results (F-SR J. %) for the methods with task-specific navigation knowl-
edge exploration.
Methods
[S
S2
$3
S4
S5
S6
S7
S8
59
S10
S11
$12
S13
S14
S15
Avg.
Baseline
36
29
47
43
30
38
40
29
21
31
26
27
16
9
17
29
Uni- Walker w/o ESOC
7
11
11
24
9
11
3
10
Uni-Walker w/o NSCoT
34
27
46
43
28
37
36
28
19
27
25
21
14
8
17
27
(ours)
4
4
5
16
5
4
5
5
0
0
4
on large-scale datasets. The results summarized in Table 25 Based on the results, although these agents undergo large-scale joint training across multiple navigation tasks, they struggle to generalize across all tasks and often lack continual adaptability to ever-changing navigation scenarios. Uni-
22

---

Published as a conference paper at ICLR 2026
Table 21: Ablation study results (SPL 1. %) for the methods with task-specific navigation knowledge exploration.
Methods
SI
S2
S3
S4
S5
S6
S7
58
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline Uni- Uni-Walker w/o ESOC
27
28
29
41
22
20
30
31
22
31
30
22
60
40
75
34
65
61
55
60
45
49
64
73
48
70
55
50
81
55
78
61
Uni-Wer Walker w/o NSCoT
29
31
31
42
23
21
25
31
23
61
41
76
35
Kni-Waleer (ours)
66
63
59
61
49
50
50
56
51
82
56
80
62
Table 22: Ablation study results (F-SPL J. %) for the methods with task-specific navigation knowl- edge exploration.
Methods
S1
S2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
60
57
53
37
58
62
57
64
58
59
49
61
30
31
6
45
Mai-Welker w/o ESOC
4
6
11
7
12
5
Uni Mai-Walker w/o NSCoT
57
52
50
35
56
60
54
61
53
56
47
60
29
29
5
46
Uni-Walker (ours)
3
3
5
6
4
6
13
6
5
11
5
3
0
6
Table 23: Ablation study results (OSR 1, %) for the methods with task-specific navigation knowl- edge exploration.
Methods
SI
S2
S3
S4
S5
$10
S11
S12
S13
S14
S15
Avg.
Baseline
63
71
70
53
77
59
77
85
71
100
72
Uni-Walker w/o ESOC
80
86
79
66
89
89
71
82
63
70
62
84
100
75
100
80
Uni-Walker w/o NSCoT
67
73
72
55
62
61
81
100
78
100
75
Uni-Walle (ours)
82
87
88
67
90
90
63
62
84
100
78
100
81
Table 24: Ablation study results (F-OSR L. %) for the methods with task-specific navigation knowl-
edge exploration.
Methods
S1
$2
S3
S4
S5
S6
S7
S8
S9
S10
S11
S12
S13
S14
S15
Avg.
Baseline
24
20
22
24
16
13
15
5
12
9
10
15
16
0
14
Uni-Walker w/o ESOC
6
3
12
5
2
12
Uni- Wai-Walker w/o NSCoT
21
18
20
21
13
11
14
9
6
6
8
11
Wni-Walker (ours)
4
2
2
5
2
8
3
Walker demonstrates the feasibility of constructing universal embodied navigation agent through lifelong learning.
F DISCUSSIONS ON THE PROPOSED COMPONENTS
As shown in our ablation studies (Tables every component of Uni-Walke provides perfor- mance improvement. Below, we summarize their roles and contributions.
Knowledge Inheritance Strategy (KIS). KIS equips Uni- fni-Wrlker with human-inspiren capability to efficiently acquire new skills by leveraging previously learned shared knowledge. initializes the new expert subspace using relevant past knowledge (Eq. 2), enabling efficient adaptation. Removing KIS results in 7.0% SR degradation.
Experts Co-Activation Strategy (ECAS). ECAS is the core architectural mechanism of DE-LoRA: directly activates previously learned experts that are most relevant to the current task (Eq. 3), enabling knowledge sharing across tasks. Without ECAS, SR decreases by 9.2
23

---

Published as a conference paper at ICLR 2026
Table 25: Comparison the other state-of-the-art universal navigation agents; averaged metrics on the 18-task with our LENL settings.
Comparison Methods
Avg SR
(%)
Avg SPL (%)
Avg OSR f (%)
NaviLLLM Zheng et al. 2024
58
ScaleVI.N Wang ef al.
2023h
SAME Zhou 2024
Uni-Walker
66
61
81
Shared Smoothing Consolidation (SSC). SSC is in order to progressively refine the shared sub-
space A, mitigating catastrophic forgetting of previous tasks (Eq. 4). Removing SSC leads 7.6% SR degradation.
Expert Subspace Orthogonality Constraint (ESOC). ESOC performs orthogonal constraint across expert subspaces B, promoting decoupled representation of navigation knowledge and pre- venting subspace mixing (Eq. 7). Its removal causes 3.8% SR degradation.
NavigatioonSSpecffic Chain of Thought (NSCoT), NSCoT provides task-style-specine reasoning and guides the agent Removing NSCoT yields the largest drop (16.2% SR),
for different instruction styles. invokes relevant pretrained LLM knowledge toward more targeted showing that style- reasoning is crucial for LLM-based navigation
Task-Aware Knowledge Aggregation (TAKA). TAKA is indispensable for task-ID agagsstiic infer- ence. It selects the most relevant experts using mixed matching and enables Uni Walker to perform navigation in ID>-agnostic task. We also provide ablation results that validate the effectiveness of the mixed matching strategy used by TAKA.
In summary, ECAS, TAKA, and NSCoT are the core components for accomplishing the LENL task, while the training strategies KIS, SSC, and ESOC promote disentangled representation of embodied navigation knowledge and further improve performance. We suggest that since ESOC has a relatively low impact (only +3.8%), it can be selectively removed when users have to simplify the Uni-Walker architecture.
G DISCUSSIONS AND FUTURE WORKS
FUTURE WORKS
While Uni-Walker demonstrates improved lifelong navigation performance under the LENI, proto- col, several future works remain: Sim-to-real gap. Experiments are conducted in simulated Mat- terport3D environments; transferring learned behaviors physical robots will introduce percep- tion, dynamics, and sensor-noise challenges that the current framework does not explicitly address. Broader embodied task generalization. Extend high-order adaptation principles to other embod- ied tasks, such as object manipulation, multi-agent coordination, or language- aaguage-guiied guided manipulation, and evaluate whether the DE-LORA TAKA paradigm generalizes beyond navigation.
SOCIETAL IMPACTS
Our work aims to improve the adaptability and longevity of embodied agents, which can enable beneficial applications in assistive robotics, inspection, and disaster response. At the same time, sev- eral societal risks should be considered: Privacy and surveillance. Navigation agents operating in private or sensitive spaces may collect and store visual and spatial data. Mitigation: adopt strict data governance, on-device processing, and data minimization; use privacy-preserving techniques (c.g., differential privacy, anonymization) where appropriate. Safety and misuse. Autonomous navigation systems may cause harm if they behave unpredictably in crowded or safety-critical settings. Miti- gation: incorporate verifiable safety constraints, human-in-the-loog control, conservative fallback behaviors, and thorough scenario-based testing before deployment.
24


Reviews:
[{"review_id": "WT5guDFS0u", "reviewer": "Reviewer_Tp5K", "summary": "This paper introduces Lifelong Embodied Navigation Learning, a new problem setting where an agent must continually learn a sequence of navigation tasks across diverse scenes and instruction styles without suffering from catastrophic forgetting. To address this, the authors propose Uni-Walker, a novel framework built upon large language models. The core of Uni-Walker is a Decoder Extension LoRA architecture that decouples knowledge into a task-shared subspace and multiple task-specific \"expert\" subspaces. The framework includes specific strategies to manage this knowledge: a KIS and anECAS to learn shared knowledge, and an ESOC and NSCoT to learn distinct, task-specific skills. At inference time, a TAKA mechanism selects the most relevant experts for the current task. The authors create a new benchmark for LENL and demonstrate through extensive experiments that Uni-Walker significantly outperforms existing continual learning methods, achieving a higher success rate while drastically reducing forgetting.", "questions": "Could you comment on the scalability of the Uni-Walker framework as the number of sequential tasks grows into the hundreds? Specifically, how does the performance of the Task-Aware Knowledge Aggregation (TAKA) mechanism degrade, if at all, as the pool of retrievable experts becomes very large?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel Problem: The paper formally introduces and tackles the problem of lifelong learning for embodied navigation, a crucial challenge for developing general-purpose agents.\n- Comprehensiv Method: Uni-Walker is a well-thought-out framework that addresses the core challenges of knowledge transfer and catastrophic forgetting by explicitly decoupling and managing shared and task-specific knowledge.\n- Rigorou Evaluation: The experimental validation is a major strength. The creation of a new benchmark, comparison against a wide array of strong baselines, and thorough ablation studies for each component of the model provide convincing evidence for the method's effectiveness.\n- Significant Performance Gains: The results are not incremental. Uni-Walker shows large improvements in both task success and resistance to forgetting.", "weaknesses": "System Complexity: The final Uni-Walker model is a composite of many components (KIS, ECAS, SSC, ESOC, NSCoT, TAKA). While the ablation studies justify each part's inclusion, the overall system is quite complex. A discussion on the relative importance of these components or potential simplifications would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e90Ghk41an", "reviewer": "Reviewer_5sW8", "summary": "This paper presents a comprehensive approach to Lifelong Embodied Navigation Learning (LENL), a novel problem setting where embodied agents must continually adapt to new navigation tasks across diverse scenes and instruction styles while mitigating catastrophic forgetting of previously acquired knowledge. The authors propose Uni-Walker, a framework built upon a Decoder Extension LoRA (DE-LoRA) architecture that explicitly decouples navigation knowledge into task-shared and task-specific components.\n\nUni-Walker introduces several key strategies:\n- Knowledge Inheritance Strategy (KIS) and Experts Co-Activation Strategy (ECAS) to facilitate the transfer and refinement of task-shared knowledge.\n- Expert Subspace Orthogonality Constraint (ESOC) and Navigation-Specific Chain-of-Thought (NSCoT) to capture and enhance task-specific knowledge and instruction understanding.\n- Task-Aware Knowledge Aggregation (TAKA) for dynamically activating relevant experts during inference.\n\nThe paper also introduces a new lifelong embodied navigation benchmark consisting of 18 tasks with varying scenes and instruction styles (Vision-Language Navigation - VLN, Object Localization Navigation - OLN, Dialogue Understanding Navigation - DUN). Extensive experiments demonstrate that Uni-Walker significantly outperforms state-of-the-art LoRA-based continual learning methods and other universal navigation agents, achieving superior success rates and substantially lower forgetting rates across multiple metrics.", "questions": "1. Could the authors elaborate on the computational and memory overhead of managing the Fisher Information Matrix (FA,t) and the retrieval index (Re) as the number of tasks (T) grows very large (e.g., hundreds or thousands)? Are there strategies to mitigate this growth, or is it a known limitation for extremely long lifelong learning sequences?\n2. For the Task-Aware Knowledge Aggregation (TAKA), what is the sensitivity of Uni-Walker's performance to the choice of K (number of activated experts) and µ (similarity threshold for masking)? Have you experimented with adaptive ways to determine K or µ?\n3. The paper states that \"task-id t is agnostic during the testing phase.\" How is the instruction style (VLN, OLN, DUN) determined at inference time to apply the correct Navigation-Specific Chain-of-Thought (NSCoT)? Is it inferred from the user instruction, or is the instruction format implicitly indicating the style?\n4. For the Knowledge Inheritance Strategy (KIS), PCA is applied to learned experts having the \"same instruction knowledge.\" How is this grouping of \"same instruction knowledge\" determined? Is it hardcoded based on the benchmark's predefined instruction styles, or is there an automatic mechanism to identify similar instruction knowledge for new, previously unseen instruction styles?\n5. Could the authors provide a more focused analysis or a separate table comparing the performance of Uni-Walker and the top-performing baselines specifically on the 3 \"unseen generalization\" tasks (S16, S17, S18)? This would provide clearer evidence of the framework's ability to generalize to truly novel environments/instruction combinations.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper introduces a novel problem (LENL) and a corresponding benchmark, which is a vital contribution. The Uni-Walker framework, with its DE-LoRA and specialized strategies (KIS, ECAS, ESOC, NSCoT, TAKA), demonstrates high originality in combining and extending existing ideas for the specific challenges of embodied lifelong learning.\n2. The empirical results are robust and convincing. Uni-Walker achieves state-of-the-art performance, significantly reducing catastrophic forgetting compared to numerous strong baselines. The comprehensive ablation studies rigorously validate the effectiveness of each component, reinforcing the quality of the proposed solution.", "weaknesses": "1. While each component of Uni-Walker (DE-LoRA, KIS, ECAS, SSC, ESOC, NSCoT, TAKA) is well-justified, the overall framework is quite intricate with multiple interacting mechanisms. It might be challenging to disentangle the precise individual contributions or to simplify the architecture without performance degradation. A discussion on the trade-offs between complexity and performance, or potential avenues for simplification, could be beneficial.\n2. While the new benchmark is excellent, 18 tasks, even with diverse scenes and instruction styles, might still be considered limited for truly \"lifelong\" learning in the vast and unpredictable real world. All scenes are from the Matterport3D simulator. A discussion on how the framework might scale to hundreds or thousands of tasks, or transfer to real-world robots (as briefly mentioned in future work), would strengthen the paper.\n3. The paper lists several hyperparameters. While values are provided, a more detailed sensitivity analysis of the most critical hyperparameters could provide further insights into the robustness and general applicability of Uni-Walker.\n4. The method involves managing multiple experts, computing Fisher Information Matrices, and performing retrieval for Task-Aware Knowledge Aggregation. While LoRA is parameter-efficient, the runtime and memory overhead for a very large number of tasks (e.g., hundreds or thousands) could become substantial. A more explicit discussion of the computational scalability with increasing task numbers would be valuable.\n5. The paper mentions 3 tasks (S16, S17, S18) for unseen generalization. While these are included in the average results, a dedicated discussion or a separate table comparing Uni-Walker's performance specifically on these unseen tasks against baselines would highlight its true generalization capabilities more clearly.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cOCLLTItaA", "reviewer": "Reviewer_AqL1", "summary": "In this work, the authors introduce a lifelong embodied navigation learning task, where an agent is required to continually acquire new navigation skills, without catastrophic forgetting on the previously learned knowledge. To achieve this goal, they design a Uni-Walker to decouple navigation knowledge into task-shared and task-specific components. Then they design different strategies to learn shared and specific knowledge for life-long learning.", "questions": "Please see the weakness section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* Novelty\n\nIn this work, the author propose a kind of new task in navigation, namely, lifelong embodied navigation learning. This task itself is interesting and practical for embodied navigation in the real world. \n\n* Clarity\n\nThe paper is with good structure. Hence, the clarity is basically good.\n\n* Significance\n\nThis paper focuses on a new task of lifelong embodied navigation learning. It would possibly bring new research works for navigation. Hence, the significance is basically OK for this community.", "weaknesses": "* Method\n\nEven though the task is kind of new, the techniques to address this task are relatively straightforward. It basically leverages LORA adaption in the decoder for decoupling navigation knowledge into shared and specific parts. Then, they integrate the existing strategies to learn these two types of knowledge for life long learning. Please futher clarify the key novel design in the proposed framework, instead of adapting and integrating the straightforward strategies together on a newly-defined task.\n\n* Figure Issue\n\n The figures (e.g., Fig 1-2) should be further re-arranged. The words are too small to see. The whole figure looks very crowded.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nuuwzbVad0", "reviewer": "Reviewer_Fhsy", "summary": "This paper addresses the challenge of lifelong embodied navigation (LENL), where an agent must continually adapt to new navigation tasks without catastrophic forgetting. The authors propose Uni-Walker, a novel framework that combines LoRA-based expert decomposition with task-aware knowledge aggregation (TAKA) to effectively manage task-specific and shared knowledge across sequential navigation tasks. Uni-Walker is evaluated on a newly introduced LENL benchmark, comprising 18 task sequences with varying instruction styles and environments. Results show significant improvements over state-of-the-art continual learning methods, achieving an average SPL of 61% compared to 38% for the strongest baseline (SD-LoRA+TAKA). The work highlights the importance of efficient knowledge reuse and dynamic adaptation in lifelong learning scenarios.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- **Originality**:  The paper tackles a timely and important problem—how to enable agents to continuously learn new navigation tasks while avoiding catastrophic forgetting. The proposed Uni-Walker framework introduces a novel combination of LoRA experts and CLIP-based retrieval for task-aware knowledge aggregation, offering a fresh perspective on how to balance shared and task-specific knowledge in lifelong learning settings.\n\n- **Quality**: The experimental design is rigorous and comprehensive. The authors introduce a new LENL benchmark consisting of 18 task sequences, covering various instruction styles and environments. They report detailed results and demonstrate consistent performance improvements across early and late tasks. Extensive ablation studies validate the effectiveness of each component (DE-LoRA, TAKA) and provide insights into their contributions.\n\n- **Clarity**:  The paper is well-written and clearly structured. Key concepts are explained with clear diagrams and pseudocode. The overall framework is well illustrated and the evaluation setup helps readers understand the technical details. The comparison with prior work is fair and balanced, acknowledging both the strengths and limitations of existing approaches.", "weaknesses": "- **Limited evaluation**: While the experiments show strong performance on the LENL benchmark, it would be beneficial to evaluate Uni-Walker on additional datasets or more diverse environments and benchmarks to assess its generalizability. The current focus on indoor navigation tasks may limit the broader applicability of the findings.\n\n- **Insufficient comparison with non-LoRA methods**: The paper primarily compares Uni-Walker with other LoRA-based continual learning methods (e.g., MoLA, BranchLoRA). Including comparisons with other continual learning methods would provide a more comprehensive understanding of Uni-Walker’s relative advantages and limitations.\n\n- **Real World Applications**: The success of TAKA depends heavily on the quality of CLIP embeddings for aligning instructions and observations. If there is significant semantic drift or if instructions vary widely in style, the retrieval mechanism might degrade. Furthermore, the paper emphasizes “training-free” inference, but it still requires training a new expert for each incoming task. Clarifying the assumptions about task boundaries and the feasibility of applying Uni-Walker in truly online settings would strengthen the claims.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)