Policy Compatible Skill Incremental Learning via Lazy Learning Interface
Authors: Daehee Lee
Dongsu Lee
TaeYoon Kwack
Wonje Choi
Honguk Woo
Venue: NeurIPS 2025 spotlight ()
Keywords: Skill Incremental Learning
Imitation Learning
Primary area: reinforcement_learning
TL;DR: Policy Compatible Skill Incremental Learning via Lazy Learning Interface

Abstract: Skill Incremental Learning (SIL) is the process by which an embodied agent expands and refines its skill set over time by leveraging experience gained through interaction with its environment or by the integration of additional data. SIL facilitates efficient acquisition of hierarchical policies grounded in reusable skills for downstream tasks. However, as the skill repertoire evolves, it can disrupt compatibility with existing skill-based policies, limiting their reusability and generalization. In this work, we propose SIL-C, a novel framework that ensures skill-policy compatibility, allowing improvements in incrementally learned skills to enhance the performance of downstream policies without requiring policy re-training or structural adaptation. SIL-C employs a bilateral lazy learning-based mapping technique to dynamically align the subtask space referenced by policies with the skill space decoded into agent behaviors. This enables each subtask, derived from the policy's decomposition of a complex task, to be executed by selecting an appropriate skill based on trajectory distribution similarity. We evaluate SIL-C across diverse SIL scenarios and demonstrate that it maintains compatibility between evolving skills and downstream policies while ensuring efficiency throughout the learning process.

Policy Compatible Skill Incremental Learning via Lazy Learning Interface
Daehee Lee Dongsu Lee TaeYoon Kwack Wonje Choi Honguk Woo
Sungkkyukwan University dulgi7245@skkuu.er edu, hwoodskku. edu
University of Texas at Austin
Abstract
Skill Incremental Learning (SIL) is the process by which an embodied agent ex- pands and refines its skill set over time by leveraging experience gained through interaction with its environment or by the integration additional data. SIL facili- tates efficient acquisition of hierarchical policies grounded in reusable skills for downstream tasks. However, as the skill repertoire evolves, it can disrupt compati- biliity with existing
policies, limiting their reusability and generalization. In this work, we propose SIL-C, a novel framework that ensures skill-policy com- patibility, allowing improvements in incrementally learned skills to enhance the performance downstream policies without requiring policy re-training or struc- tural adaptation. SIL-C employs bilateral lazy learning-basee mapping technique to dynamically align the subtask space referenced by policies with the skill space decoded into agent behaviors. This enables each subtask, derived from the policy's s decomposition of a complex task, to be executed by selecting an appropriate skill based on trajectory distribution similarity. We evaluate SIL-C across diverse SIL scenarios and demonstrate that it maintains compatibility between evolving skills and downstream policies while ensuring efficiency throughout the learning process. Source code: htttps//gittubb com cww 2ttmm/L2lllg/SS---0
1 Introduction
Lifelong embodied agents must continuously integrate novel knowledge from unceasing streams of data into their evolving skill library while simultaneously leveraging previously acquired skills [14 Skill Incremental Learning (SIL) supports this process by enabling agents to expand and refine their skill sets time through continual interaction with the environment integration of new data. Accordingly, SIL often aims to facilitate the development of hierarchical policies grounded in reusable skills for downstream tasks (577.
Recent research explores diverse formulations of SIL, such as modular skill composition (8-I]. continual adaptation [4], and hierarchical policy learning [15] [6] to support scalable embodied agents. Yet, a critical challenge remains underexplored; as skills evolve over time, maintaining compatibility with downstream policies that depend those skills becomes increasingly difficult [17] 20 Without proper aligmment, updated skills may invalidate previously learned policies, limiting their reusability and generalization 24]
Figure Dillustrates policy-compatible SIL scenario involving two types of skill-policy compatibility: (i) Forward Skill Computibility (FwSC), which ensures that newly added skill can be effectively utilized during the training of future downstream policies, and (ii) Backward Skill Compatibility (BwSC), which ensures that existing downstream policies continue skills without requiring re-training, and, when applicable, benefit from improved policy performance.
newly added or updated
*oorrespndingg author: Honguk Woo (hwoo@skku.eduu)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Policy Compatible Skill Incremental Learning
Skill update
Policy update
Policy
Policy V1
Policy V2
Policy
Env.
Stade
Policy
Lazy Learning
Interfaca
Skill
Phase
Phass
Phaae
Phass
Decoder
Autiirs
Actions
SIL-C framework
Agent
Decoder
Skill Decoder V1.1
Decoder
Skill Decoder V1.3
Hierarchical Agent Evaluation
A policcy-ooppatibee Skill Incremental Learning scenario one which the skill library evolves incrementally while meintaining compasbiity with both previously learned and future downssteam policies, without requiring policy re-trainee re-training or structural adapiation.
Forward Skill Compatibility (FwSC)
Initial Skills
Updated Skills
Policy V2
space evaluation
Skill
Policy
space
Updates
Update
Skill Decoder V1.3
ensures currently updated skill (e.g. on top burner) utlizee by future downstream policies (egg, Policy V2).
Backward Skill Compatibility (BwSC)
Policy V1
Policy V1
Model
Skill hooking
Skill valliaaiion
Traj, Diet. Similrity
I
Skill Updates
Decoder
Solve
Decoder V1.33
by remapping misaligned
approprtate
BwSC: ensures that currently updated skills (e.g.. turn an top burner) can be utilized by existing downstream policies (e.g.. Policy V1).
Figure 1: Overview of policy-compatibl skill incremental learning and SIL-C framework
To address skill-policy compatibility challenges, we introduce SIL-C, novel SIL framework with an interface layer, enabling hierarchical skill composition to seamlessly support downstream policy learning. For FwSC, inspired by append-only file systems, we design the framework to support non-destructive skill updates that mitigate forgetting and to introduce clear abstractions between task-specific policy learning and the task-agnostic evolving skill library, thereby enabling practical lifelong learning. For BwSC, further design the interface within the framework to provide skill validation, which ensures that subtasks proposed by the high-level policy are contextually appropriate, and skill hooking, which intercepts misaligned subtasks and remaps them to the most suitable skill based on trajectory distribution similarity.
We implement both compatibility mechanisms through lazy learning- based interface that maintains a consistent structure, enabling seamless integration with existing SIL approaches. To achieve FwSC and BwSC without requiring policy re- re-training structural adaptation, reformulate alignment between subtask space and skill space through trajectory distribution similarity as an instance-based classification problem with append-only prototype memories. By deferring decisions to inference time and offloading only the mapping function, this approach preserves existing training procedures while supporting the decoupled evolution of skills and policies.
With diverse SII. scenarios regarding compatibility, sample efficiency, and modularity, we evaluate SII--C and other SII. baselines. SIL-C improves the normalized reward of downstream policies learned during the initial phase by an average of 18.6pp to 42.5pp across full scenarios, while the baselines only maintain setting in SIL, where each task is provided with only a single demonstration and 20% of its transitions, SIL-C achieves nearly double the overall performance of the baseline within the BwSC evaluation.
degrade in performance. In particular, under the few-shot imitation
Our contributions are summarized as follows: (1) We propose SIL-C, a novel framework for SIL that maintains compatibility between evolving skills and downstream hierarchical policies. (2) We introduce a lazy learning- based mapping technique that aligns the subtask space from policies with the skill space based on trajectory distribution similarity. (3) We demonstrate the effectiveness of SIL-- across diverse SII. scenarios, showing that maintains skill-policy compatibility while supporting efficient and scalable skill integration.

---

2
Related Work
Skill incremental learning. To achieve pre-trained behavior models for efficiently learning down- stream tasks, prior work has explored unsupervised skill discovery [25 26] and exploited task-agnostic data to build skill priors that can accelerate learning [27-30] Contrastive objectives are also used to improve representation quality [31434]. Several methods have been extended with new expert demonstrations refine or expand priors for downstream adaptation 33] Building on pre-trained models, recent work has investigated continual task adaptation in offline settings, aiming to handle streams of diverse tasks without full re-training proaches aim to incrementally acquire and organize skills at finer abstraction levels [39] by leveraging subgoal information [ often via hierarchical formulations [12] [3]. Yet, these SIL approaches typically assume synchronous updates between a skill library and downstream policies, limiting
38 Beyond task-level adaptation, several ap-
scalability when adding or removing skills.
Continual updates for pre-trained models. Expansion-based continual learning methods [40]
maintain task compatibility via explicit task identifiers, while recent adapter-based approaches 42
45 enable parameter-efficient updates, However, these typically rely on frozen backbones, limiting their ability revise pre-trained knowledge 446] Such issue arises in sequential decision-making [
38 where pre-trained policies must adapt over time. Recently, a continual pre-training approach has been proposed to improve forward compatibility [47. but its integration into hierarchical settings remains underexplored. We explore this direction in the context of lifelong skill acquisition.
Lazy learning for behavior learning. Lazy learning techniques reduce the cost of online adaptation and mitigate forgetting by leveraging instance-basee retrieval and local updates [50]. They have been applied in behavior learning through prototype memories and nearest-neighhor matching (31 and in robotics applications, particularly for adapting to novel tasks from limited demonstrations (54- [59] While effective for adaptation, these methods typically do not support long-term skill reuse or accumulation without additional training. We address this by extending lazy learning into the SIL setting, reformulating trajectory siirry-immlaait-bbaaaae skill validation and hooking as an instanc-bbseed classification mechanism that enables skill reuse and composition without policy re- re-training.
Problem Formulation
We formulate each task is the state space, A is the action space, P is the transition probability of the environment, R is the reward function, nr is the initial state distribution, and is the discount factor. The objective is to learn a policy ग (a|s; (x; 8) that maximizes the expected return:
Markov decision process (MDP) M (S,A, Mo, 7), where S
((0) = E ET(a s:0)
y rR(ss, at)
(I)
To address long-horizon tasks, we adopt a hierarchical policy. A high-level policy Tn (2,zlss; 0%) selects a subtask zn. which is mapped to a skill 2 = (22)) and executed by low-level decoder maas,, 21; f.). In our setup, the mapping &e may be the identity map or more general transformation. with both 24, and 2 lying shared latent space Z. The overall objective denoted J(),, 0,.) [60]
In Skill Incremental Learning (SIL), we observe a stream of datasets {DD where each Dp contains demonstrations for updating the skill decoder ı at phase p. The goal is to continually update the decoder with D,, while maintaining or improving performance across a set of evaluation tasks T, presented at each phase. We assume access to task-level expert demonstrations D, of T for training the high-level policy тhe enabling supervised learning and facilitating adaptation to evolving skill sets. The high-level policy for each task is updated independently of the phase and may be replaced or fixed. Our objective is to find the collections of high- and low-level optimal parameters (0;, 07
over all phases, that maximize performance across all observed tasks:
(07, O7) arg max Esp
Σ Or (h) FET,
-
O- -
(2)
0,00,
Here, of and 0g denote the parameters of the skill decoder u. at phase p and the high-level policy =/ for task 7, respectively.
3

---

Bilateral Lazy Learning Interface (5 4.1) & Update (8 4.2)
Policy Integration (9 4.3)
Palicy (f s)
Subtask space update Ea11))
SIL Policy Training
Policy inference
(HG
Exp
High-level policy
Sublask space X, (92)
P Strwan of dataset
Seill decoder
subgoal prediction
R2 Expert dennstration
Subtask Seill
Interface (8.
skill Increnental learning
high-level policy 4 - x2())
Skill space 25 (9f, 972
Stoeed in task-specffc memory X twhtich represents Xa)
for range(P):
skill validaation hookinng
Skill space update Eu18)
Skill decoder of update
Skill space update tn
SIL-C interface ts subgoul pradiction())
Skill decoder nf (ol) X. (t))
Do
(Ex)
(H [g) + (x
if skill, validation(ga)))
dovnstrean task learning for In 3
Instance-bused classifier
multimodal Gausslan prototypes
Subtask space update Eq19 Compute prices Updace task policy a.
bookinggg))))))))
z- (oe Ca
Be lnw-lavel seill decoder af (9,22)
Miject
Staeed in sec
x .x/ (sdecct regnoere)
Figure 2: Overview of the SIL-C framework: components, updates, and integration
4 Our Framework
We introduce the SIL-CC framework, a hierarchical architecture with a lazy learning interface designed to address skill-policy compatibility in SII.. It consists of high-level policy xf, a low-level skill decoder xf.. and their respective learning algorithms A,, and A₁. The interface layer I connects the two levels by mapping each subtask yn to a skill z;. Both subtask and skill are drawn from the shared index set 2p = [I....,,y, zp}. which expands over phases by appending new indices 2p 1Zp-1t1..... Zp}. Given a state s, execution in SIL-C follows:
2h (- {), 21 I(s. 2n), a ~ (- 8, 21), where 21, E Zp.
(3)
The core component of SIL-C is the bilateral lazy learning interface I. which operates across two spaces: subtask space Xr, and skill space Xi. We implement this interface as a two-stage instance- based classifier that performs trajectory similarity matching between subtasks and skills. In our design, the subtask space Xr is represented by prototypes of subtask initial states used to predict the corresponding subgoal, while the skill space X; is represented by prototypes that capture the distributions of initial states and subgoals for each skill. Together, these representations allow trajectory similarity to be compared across skills and subtasks. Building on this, interface operates in two stages: skill validation first checks whether the current subtask can achieve the task-specific subgoal predicted by X4. given the current state; if not, skill hooking remaps the subtask to the most feasible skill based on the current state. Figure Lillustrates the components and their integration.
4.1 Bilateral Lazy Learning Interface
Bilateral modules ( Vy. Tt)). To enable efficient trajectory distribution similarity matching under SIL, the interface I estimates similarity through bilateral lazy learning modules; the task-side module Vi operating on the subtask space X₁ and the skill-side modules D. V operating on the skill space Xi The two sides communicate via abstracted trajectory distributions represented by current state and subgoal g pairs, where a subgoal represents a desired future state reachable from the current state within m steps. To map subtask 2n, to an executable skill 21, the task-side module V first predicts the target subgoal from s. Then, the skill-side module V assesses the direct exccutability of 2n toward g. for skill validation. If validation fails, skill-side modules V v identify an appropriate skill via two-stage matching conditioned on (s, g). for skill hooking.
sestance-based classifier (V). To implement lazy learning in the interface, we adopt an instance- based classifier for all modules Each label E C, representing concrete prediction target (e.g.. a subgoal g or a skill index 2), is modeled by a multimodal Gaussian prototype X. = {Mo, E.,4) K._, where K2 denotes the number of Gaussian components for label and Σe, is diagonal for computational efficiency. For query instance x, the module computes distances to all prototypes:
dx(x) min
(x (x--i) Mc.k)
E T
Pc,))-
k.KK,
This Mahalanobis distance- based multimodal approach effectively captures complex distributions for both accurate prediction and reliable validation. Specifically, it enables two operations: (i) classification via V ();;; C) arg min,ec (a) for nearest prototype selection, and (ii) validation
via (xx c) (x) S 5%] for out-r--dist detection, where 8, is the square root of 99% chi-square quantile [61].

---

4.2 Interface Update
We parameterize the interface modules with append-only prototype memories. On the task-side, classifier V relies on task-specific memory Xx to represent subtask space Xr for each task On the skill side, classifiers V and v use task-agnostic shared memories 27 and x to represent skill space X; at phase p. The subtask space X₁ is updated during downstream policy learning, while the skill space X; is updated during SIL.
Skill space update (X2). At SIL phase SIL-C updates Xi by generating skill prototypes from the streamed dataset Pp. An unsupervised skill clustering algorithm 62 60 segments D, into distinet skill groups {E=] where 2, is the index of newly discovered skills phase p. The number of discovered skills 2, can either be fixed or automatically determined by the clustering algorithm. For each group &g. we apply K K-means separately to the state and subgoal spaces capture the multimodal distribution within the group. The number of sub-clusters K2 is selected either manually or automatically via silhouette score 663 This process yields sub-clusters { g, H 1 for each
K2
skill 2. Then, we compute je.k and Σz,: to create prototype for each skill, denoted as X and x². respectively, corresponding to skill label The overall process follows:
D
Skill Clustering
{E
K-means
(H2 H. H..kk H
ke(l:Ka] S...
(X2 xx)
(5)
segment
8.9
skill dataset
skill groups
sub-clusters
skill protolypes
The resulting skill prototypes are stored in skill memory X and XPO.P
Subtask space update (22)). At downstream policy learning for task T. SIL-C constructs X, by generating subtask prototypes from the most recent expert demonstrations Dr. to the discretized subgoal groups G.. To efficiently represent demonstrations, we apply the same skill clustering algorithm [60] to derive G,, followed by the same sub-clustering procedure as in the skill space
update:
Subinsk Clustering
(Eg
K-means
kk(l:Kg) (H ( geGr
Pa...Eu.k
(xilaEs.
segment
expert dataset
subtask groups
sub-elusters
subtask prototypes
After prototype construction, subtask prototypes are stored in the task-specific memory N,
4.3 Policy Integration
Policy learning on the interface. We utilize an energy- based prior [64] to guide the learning of the high-level policy Ag. This prior is induced by the skill decoder: given state s, the decoder evaluates all candidate skill pairs [1) 2,zz, and assigns the subtask lahel whose decoded action best matches the expert action a". Specifically,, compute the subtask label as:
2h arg minl à a"
zEZZ,
where à ~ TT(- 8, z1) denotes the decoded action from the skill decoder using skill zy. The decoder is continually updated by the skill incremental learning algorithm A!-
Policy inference via the interface. The policy inference process follows the sequence defined in Eq. (3). Given a sampled subtask 2/, ~ x s) from the high-level policy, the interface first predicts the subgoal
V (s; G-) using the task-side module, conditioned on the task-specific memory x Using this inferred subgoal g. the interface then performs skill validation by evaluating the distance of sabgoal-subtasse pair (g. 2n) by skill-side module 4 2x). ). If the skill validation passes, the interface returns ž = Z₁. Otherwise, initiates skill hooking, seeking an alternative skill better aligned with the inferred subgoal y. Formally, the decision rule for interface I follows:
2/:
Vg 2n)
I(s, 2n)
(g, = 0 where g (s:
(8)
(s;
Here, candidate skill set 2' (=' 22 =10==)==u[ff) (2,)) where 24. originally inferred subtask from the high-level policy, is included default fallback when no valid candidate skill exists.
5

---

Emergent SIL
Dataset Wer
Datawet
Dabanet
Datazet
Stream task-agnostic demonstrations
Ver.
Ver
Ver.
Explicit SIL
Stream of segmented skill demonstrations
Close Drawer
Open
Door
Slide Puck
Push Button
Backward Skill Compatibiliy (BwSC)
Skill-Policy Compatibllity (Overaf)
Forward Skill Compatibility (FwSC)
Policy
Skill
Skill update
Policy update
Alignment without policy re-training
Figure 3: (Top) SIL scenario types and (Bottom) evaluation groups
Experiments
5.1 Experiment Settings
Environments and SIL scenarios. To evaluate skill-policy compatibility, construct various SIL
scenarios using two simulation enviromments: Franka Kitchen [65] 66] and Meta- Meta-World [T]. In experiments, have four SIL phases where we add new skills by training the skill decoder, and train 24 downstream task policies using the updated skills for each phase.
As illustrated in the top of Figure we define two scenario types to evaluate SIL under different levels of skill supervision. The Emergent SIL setting follows skill-based pretraining approaches [28] (7].
introduces only task-agnostic demonstrations, reflecting an unsupervised SIL setup. In contrast, the Explicil SIL setting adopts the setup from prior SIL works [12 and uses demonstrations that exhibit clearly chunked behaviors with predefined semantics.
As shown in the bottom of Figure we categorize each scenario into three evaluation groups based on the combinations of skill decoders and policies used for task evaluation. The BwSC group pairs the skill decoder from each phase with the policy initially trained in the first phase. The FwSC group pairs the skill decoder from each phase with its synced policy trained using that decoder. The Overall group aggregates both BwSC and FwSC groups measure comprehensive compatibility.
Baselines. We evaluate three baseline groups Type combines skiel-based pre- pre-training with continual learning strategies for SIL, adopting simple skill appending scheme. We extend BUDS 662 [Q]
and PTGM [60], which discretize the skill space during pre-training. As continual learning methods, apply fine-tuning (FT), experience replay (ER) 669], and adapter append (AA) (38 which trains an adapter after the initial phase and stores it for use in subsequent phases. Type II includes SIL approaches that use semantic goals skill labels, relying on pre-defined supervision. They include prototype-based skill retrieval [ and temporal replay with model expansion [12]. Type III corresponds SIL-C, which uses the Type configuration as its base architecture. We also report joint-training performance, obtained by training single decoder on the union of all skill datasets.
All baselines adopt the subgoals hashed by skills, and use a diffusion-basee model (70 71 for decoding. For policy learning, they follow the same training objective, behavior cloning with prior, defined in Eq. (
conl-connitioned skill decoder architecture [60], which conditions
Metrics. We measure skill-policy compatibility by adapting continual learning metrics from lifelong robot learning [4] I 38). All task evaluation results normalized to 100% based the maximum reward defined in each environment. Forward Transfer (FWT) measures the performance newly trained policies after skill updates, quantifying how updated skills facilitate learning new tasks. We report FWT at the Initial and Final phases of SIL. Backward Transfer (BWT) measures the change in performance from initial evaluation, assessing how skill updates affect previously evaluated tasks. Area Under the Curve (AUC) averages all phase performances within a scenario. All metrics are measured per task and reported averages across tasks for each evaluation group in each scenario.
For comprehensive experiment settings including implementation details, hyperparameters, and additional configurations, please refer to the Appendix.

---

Table 1: Performance evaluation of skill-policy compatibility in two SIL scenarios of the Kitchen environment with four seeds. Each row corresponds to baseline, categorized by the skill interface configuration, hierarchical agent structure, and the SIL algorithm used for skill decoder updates. The symbol denotes methods that require pre-defined semantic skill labels for both policy and decoder training. The left side reports BwSCC results, the right side reports FwSC results, and the center shows the overall performance considering both. Best results are shown in bold. In this table, SIL-C uses the Type I (PTGM+ + AA) setting as base architecture.
Kitchen: Emergent SII.
Baselines
Initia!
BwSC (Initial)
Overall performance
FWSCC (Synced)
Final
Type
Skill Interface
SI1. Algo.
(WT
BWT 199
AUC
BWT
AUC
BWT (9)
AUC (s)
FWT
Skill Segments
62
10
FT
23.5
1664
11.2.
18.3.12
4,3
265
18.7
(BLDS)
ER
23.5+38
-10.8e51
15.5=23
3.6420
26.6-28
17,9.65
37,0:39
44,5+70
AA
21.9+41
1,3-44
22,9=40
15.7+35
35.3-29
30.1=39
44.4
57.1475
Subgual Bins
FT
45.0
-36.9
17.3
24.3
24.1
-11.7:11
36.2
24.7:30
(PTGM)
ER
46.8+25
-19.7:44
32.1=29
-5.0-11
42.5
9,7-33
54,0=15
57.7=33
AA
45.0
2,6,00
46.9
15
58.2
28.1
66.1
83.6
II
Instrctiooss
Skcillprotocynes
P2
M
AA
ER
55. 15.1
54.0.27
18.5
0,5-31
67.8
55.5-27
19.1
0.5228
70.3
55.7
19.7.20
0.9.23
55.8:24
68.7.11
77.4
56.9s43
III
SIL.CC (w/ PTGM)
AA
$2.9129
18.6111
66.8 26
22.0 14
71.8 1.3
25.4
71.9 (n
87.2
Subgoel Bins
Joint
86.9
Kitchen: Explicit SIL
Baselines
Initia!
BwSC' (Jnitia))
Overall performance
FWSCC (Synced)
Finul
Type
Skill Interface
S11. Algo.
FWT cv)
BWT 199
AUC (5)
BWT
AUC
BWT (s)
AUC (s)
FWT (5)
Skill Segments (BUDS)
62
ER
14.6200
14.6:00
-13.6
0.2405
14.7-04
4,480s
14.1a13
-9.5-06
26.7a11
6,5-05
28.0=22
-5.400
10.5a04
35.6.11
37.0
3,0=1.
14.6200
0.0=00
14.6=00
20.9
32.5=10
41,9-23
46.0a17
62.8=30
Subgoal Bins 60
FT
14.6100
114.4102
3.8
.8.9
n.
-1.7199
12.1
1.7
(PTGM)
14.6203
-1.5eLi
13.5-0s
17.2:11
29.3+11
35,9-13
41.5:15
52.0
AA
14,5a02
0.0=01
14.6-04
25.8=11
36.7-08
51.6-19
53,3+14
79.8
Skill protolynes
14.6100
0.0
14.6 m00
18.4
33.6
:4.
47.8
a5.0
Instrettoons [2]
ER
14.6100
-0.6
14.1
19.1
30.9
38.7
43.6.15
60.2gk
III
SIL-C (w/ PTGM)
AA
14.6200
42.5=30
46.5-23
46.2+15
54.1-21
49.8-24
51.9
80.6
Subgual Bins (0)
Joint
86.9
5.2
Skill-Policy Compatibility: Backward and Forward
Table the poill-policy compatibility evaluation for the Franka Kitchen environment, covering both Emergent and Explicit skill incremental scenarios. It jointly reports BwSC and FwSCC and the overall performance shows that SIL-C achieves the highest AUC among all baselines.
Backward Skill Compatibility (BwSC). The left side of Table I reports the performance of the Initial phase policy after subsequent skill updates, evaluated across all phases. Compared to Type I baselines, SIL-C achieves a higher initial performance (Initial FWT) and closely matches the performance of methods with predefined skill labels. This suggests that SIL-C can robustly replace lower-confidence skills via skill validation and hooking. Furthermore, SIL-C matches the BWT and AUC scores of Type II methods. However, this is partly due to the nature of the Emergent scenario in Franka Kitchen, where future skills are already available and fully observable from the start, creating nearl setting. In Explicit scenarios with incrementally revealed skill labels, SIL-C consistently improves performance, whereas most Type II methods show near-zero BWT, merely preserving initial performance without adaptation. Meanwhile, Type I baselines with limited replay suffer from skill forgetting, with only the AA variant maintaining backward compatibility but failing to improve it.
Forward Skill Compatibility (FwSC). The right side of Table I reports performance when policies are retrained at each phase to align with updated skills. SIL-C and PTGM with append-only skill learning achieve comparable final performance (Final FWT) to joint training, highlighting effective utilization of accumulated skills. It also achieves the highest AUC, indicating strong forward com- patibility. Among Type I methods, ER and AA yield positive BWT, suggesting that newly acquired skills contribute to subsequent policy learning, although less effectively than SIL-C.
In Appendix C we provide additional overviews and extend the experiments with diverse configura- tions of SII algorithms to further motivate our design choices, and present experiments with varying SIL phase orderings, showing that SIL-C consistently maintains skill-policy compatibility.
7

---

Table 2: Few-shot imitation results on Kitchen Emergent SIL. Shots denotes expert demonstrations per task for high-level policy training. Ratio indicates the proportion of transitions uniformly sampled from each demonstration. All baselines use SIL algorithm AA for skill decoder. For SIL-C at Ratio 50% and 20%, each sampled transition is treated as an individual subtask prototype.
Kitchen: Emergent SIL
Baselines
Initial
RuSC (huitia))
Overall performance
FwSC (Synced)
Final
Type
Skill Interface
Shots
Ratio
FWT (9)
BWT (S)
(UC
BWT (a
AUC
BWT (s)
AUC
FWT (9)
100%
40.5
04424
40.8
10.0
49.1
19.6=20
55.3+24
67.6a50
Subgoul Bins
60
100%
34.7a21
0,7
35.1425
5,944
38.6-20
11.0.56
40.2
43.7a36
(PTGM)
100%
26.5
1,7.33
27.7
6.1
31.7
10.5
34.3
37.8
50%
28.6.57
-0.8+44
27.9
1.0-51
29.5
2.9.60
30.8
36.5
20%
25.7
-2,91
23.5
1.5
27.0
5.9174
30.1
30.5
100%
153
12
17.5
62.4
23.6141
65.1
75.8
SIL-C (w/ PTGM)
100%
44.4:18
11.2a25
$2.8
14,9
$7.2
18.6
58.4112
68.5
100%
43.1
11.9-28
52.0s5a
15.7
56.5
19.4
57,6
65.7+33
50%
41.6
11.7a23
50.4a47
14,9
54.3
18.0=47
15.1
67.2
20%
37.
12.0aa3
46.4a50
14.4
49.7=2m
16.8
58.5 49.9
Franka-Kitchen Emergent
Franka-Kitchen Explicit
Meta-world Emergent
Meta-world Explicit
Normalizzdd Reward
60
60
60
40
40
20
20
20
20
p-3
p-
2=
p-3
7-6
p=2
2-3 p-
2=
p-
p-4
Inttaal policy (darker)
Synced Policy (brighter)
BUBS
PTGM
Baselines
Baselines SEL-C
Figure 4: Results on Kitchen and Meta-World tasks under Emergent and Explicit skill incremental scenarios. Each group of bars represents a skill update phase (x-axis), and the y-axis shows the scaled reward. Darker bars represent evaluation with the initial policies. Lighter bars show performance after re-training with updated skills. Diamonds ( (O) indicate the application of SIL-C to each baseline. The skill decoder uses the AA strategy for Kitchen and ER for Dert-World.
5.3 Sample Efficiency: Downstream Few-shot Imitation Learning
Effect of number of demonstrations. Table shows outperforms the baseline, highlighting the benefit of skill validation and hooking via lazy learning under limited supervision. With 5 demonstrations per task (5-shots), SIL-C improves overall per- formance AUC from 49.1% % 62.4% and Final FWT from 67.6% to 75.8%, indicating utility even with ample supervision. When using only demonstration per task (1-shot), the gains become more significant, with AUC increasing from 31.7% to 56.5% and Final FWT from 37.8% to 65.7%.
shows that across all demonstration budgets, SIL-C
Effect of transition sampling ratio. Reducing the number of transitions extracted from demonstra- tions (from 100% 20%) has only moderate effect on SIL--, which still maintains positive BWT and achieves an AUC above 46% on BwSC. In contrast, the baseline degrades substantially, with AUC dropping below 24%. This indicates that the skill-conditioned interface provides robustness to sparse supervision by substituting missing transitions with reusable behaviors. These results suggest that the interface enables strong generalization by leveraging prior skills in low-data regimes.
5.4 Modularity: Under Varying Design Choices for Hierarchical Architecture
Figure shows that across all scenarios, SIL--C consistently improves Initial Policy performance by enhancing BwSC, regardless of environment, skill clustering strategy, or SIL algorithm for low-level decoder. In the Franka Kitchen environment, as shown in Table [ SIL-C enables both BUDS and PTGM with AA to make more effective use of their accumulated skills, leading to more efficient reuse. This effect is clearly reflected in the visualized performance, In the Meta-World environment, unlike in Kitchen, both BUDS and PTGM use ER train a shared skill decoder. While this promotes generalization and enables reuse of new skills without policy re- re-training, the actual gains remain limited. When SIL--C is applied, we observe a clear improvement in evaluation with the initial policy, suggesting that skill validation and hooking are effective in filtering unreliable skills. Although SIL-C relies less on broadly generalized skills from ER, tends to reuse verified skills more precisely, which may contribute to safer and more stable policy exccution.
8

---

Franka kitchen Emergent
SIL [Random policy)
SIL/C (Pollay + Interface)
Fer
100
80
Subtaste (r)
60
40
Skile In)
20
Initial Policy (darkker)
Synced Policy (brighter)
Policy only
SIL-C random policy)
SIL-C (wi learned policy)
Figure 5: Ablation of the SIL-C lazy learning interface in the Kitchen Emergent SIL scenario using PTGM with AA configuration. (Left) Per-phase performance across three settings. (Right) Evaluation trajectories example illustrating selected subtasks and corresponding skills used to solve the task: open mi crowave move kettle + turn on top burner + open hinge cabinet.
Table 3: Performance under varying input noise levels Kitchen Emergent SIL scenario. All baselines use the same SIL algorithm AA for the decoder.
Kitchen Emergent SIL
Baselines
Initial
BwSC (Jnitiaf
Overall performance
FwSC (Synced)
Final
Type
Skill Interface
Noise
FWT (8)
BWT
AUC
BWT
AUC
BWT
AUC
FWT
x1
45.0
2.6209
46.9
15.4
58.2-12
28.1
66.1206
83.6
Subgoal Bins (PTGM)
60
x2
44.4
0.8-20
45.0a34
10.7
53.5-22
20.5
59.7+19
68.623.
x3
35,8
0,5
36.1
3,2456
38.5
5,9777
40.2
40.5
x5
19.7
-1,2%%4
18.8
-1.8
18.2:09
-2.4
17.9
15.9
x1
$2.9
18.6
66.8
22.0:224
71.8
25.4
71.9:00
87.2
SIL-C (w/ PTGM)
x2
49.7
11.3:11
58.2
16.7
64.0=13
22.1419
66.3
76.4-22
III
x3
35.6333
13.6242
45.8
15.4
48.8
17.3
48.5
$2.9
x5
19.6
4.3
22.8
3.8
22,9
3.3
22.1
23.1 24.1
Ablation and Analysis
6.1
Lazy Learning Interface Ablation
Figure aablates lazy learning interface and policy integration contributions across three settings: policy-only, SIL-C with random policy, and SIL-C with a learned policy. While the policy-only configuration cannot leverage newly added skills without retraining. In contrast, the SIL-C w/ random policy shows that skill validation and hooking mechanisms alone offer minimal decision- making capabilitt via trajectory distribution similarity matching, but still underperform due to limited generalization ability compared to using learned high-level policy. The SIL-C w/ learned policy further improves performance by using skill validation with out-of -f-isrrrbbuttion detection to correct subtask-skill mismatches, which is crucial for error-sensitive long-horizon tasks. Evaluation trajectories show that while the random policy fails immediately from inconsistent subtask selection and the policy-only configuration struggles with the not-yet-acquired skill move kettle, SII--C completes tasks through the interface remapping subtasks to utilize newly added skills.
6.2 Robustness
Table evaluates robustness in the Kitchen environment by scaling input noise during evalua- tion, where the noise parameter [65] is increased from x to x5. SIL-C consistently outperforms PTGM+AA across all noise levels. Notably, under x5 noise, SIL-C maintains a positive BWT of 4.3%. The performance gap further increases with larger skill library sizes. For example, under x3 noise, the FWT difference grows from a marginal -0.2% Initial FWT to 12.4% Final FWT at convergence with 80 skills, where SIL-C achieves 52.9% compared 40.5% for PTGM+AA.
This robustness stems from architectural differences in handling noisy observations. When noise corrupts high-level policy outputs, the bilateral interface in SIL-C performs skill validation to detect out-of-distribution subtasks and skill hooking to remap them based on trajectory similarity. In contrast, the static skill mapping in PTGM+AA cannot adapt distributional shifts, leading to error accumulation in long-hod long-horizon tasks. The widening performance gap suggests that trajectory-baset matching becomes increasingly effective as the skill repertoire expands.

---

BwSC (AUC)
Overall Performance (AUC)
FWSC (AUC)
10
20
40
10
40
10
40
AUG(N)
75.0
10
54.5
53.5
56.9
61.5
6.8
61.4
10
06.3
57.1
05.5
67.5
20
60.0
00.0
61.0
20
0771
72.0
70.4
20
00.9
71.9
71.7
60.0
40
40
40
Figure 6: Analysis with varying 2pl and | of SIL-C (w/ PTGM, AA) in Kitchen Emergent SIL scenario. Rows represent different values of Z,l while columns represent different values of [9-|- AUC results shown for (Left) BwSC, (Center) Overall performance, and (Right) FwSC.
6.3 Skill and Subtask Space Resolution
how the resolution of skill and subtask spaces influences FwSC and BwSC. We vary the number of skill groups per phase (lZpl) and subtask groups per task ((g)) in SIL-C (with PTGM and AA). Starting from the default setting in Table 1 2 = 20, | = 20), we test doubled and halved values to assess sensitivity to space resolution, which is determined by the granularity and quality unsupervised clustering. Full results are reported Appendix [D.T]
Skill space resolution (l Increasing Zpl from 10 to 40 raises overall AUC from 61.5% to 75.5%
when |G,| = 40. This improvement is consistent across all subtask settings, with 2,1 = 40 yielding the best performance in both BwSC (70.3%) and FwSC (75.8%). Higher skill space resolution also reduces dependence on subtask space resolution. With | 40, BwSC remains stable across subtask configurations (69.2%-70.3%) indicating that sufficient skill diversity compensates for coarse subtask space.
Subtask space resolution (Gr The effect of subtask space resolution depends skill space resolution. With coarse skill space (lZ,1 = 10), increasing [G-] from 10 to 40 changes performance only marginally (61.5% 61.4% in Overull Performance).
contrast, with finer skill space ((Z,) = 40), the same increase yields gains (73.8% to 7..5%). This shows that sufficient skill diversity is necessary for benefiting from finer subtask space, as coarse skill space limits the effect additional subtask prototypes.
7 Conclusion
We present SIL-C, a framework for policy-compatbbe skill incremental learning (SIL) that introduces bilateral lazy learning interface preserve FwSC and BwSCC without requiring full re-training or structural adaptation of downstream policies. Empirical results demonstrate that, by ensuring skill-policy compatibility, SIL-C consistently improves performance, efficiency, and modularity.
Future Directions. While our approach performs well in the simulation- based environments with well-defined state representations, future work may explore settings with more diverse noisy skill distributions, where unsupervised clustering alone becomes less effective. Enhancing robustness in such conditions and leveraging minimal goal information for exploration efficiency. Additionally, monitoring skill reliability and distribution shifts during deployment may help enforce expected behaviors and detect anomalies, contributing to safer execution.
improve sample
Impact Statement. Our framework aims support large-scale deployment in practical settings where new skills must be integrated without disrupting existing behaviors. By maintaining policy compatibility and enabling decentralized skill sharing, SIL-C provides a foundation for scalable robotic systems that evolve over time. This line of research has potential to assist the development of adaptive, modular agents in applied domains.
10

---

Acknowledgement
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT), (RS-2022-00143911, AI Excellence Global Innovative Leader Education Program, 2022-11220043 (2022-0- -002240-00 Adaptive Personal- ity for Intelligent Agents, 1S-2002-II12211 (2022-0-01045), Self-directed multi-modal Intelligence for solving unknown, open domain problems, RS- RSS-0025-02188768 Accelerated Insight Reasoning via Continual Learning, and RSS-225--2544259, Al Star Fellowship Support Program(Sungkyunkye Univ.) RSS-001--1119421,, Artificial Intelligence Graduate School Program (Sungkyunkwan Uni- versity)), the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. RS-2023-002131 18)). LTT-ITTCC (Information Technology Research Center) grant funded by the Korea government (MIST) ---20--R---222-00477777 10%), IITP-ICT Creative Consilience Program grant funded by the Korea government (MSIT) (UTP-2002-RSS-2200-122) 10%). and by Samsung Electronics.
References
[1] David Abel, Andre Barreto, Benjamin Van Roy, Doina Precup, Hado van Hasselt, and Satinder Singh. A definition of continual reinforcement learning. In Advances Neural Information Processing Systems 37 (NeurIPS), 2023.
[2] Edward Hughes, Michael D Dennis, Jack Parker-Holder, Feryal Behbahani, Aditi Mavalankar. Yuge Shi, Tom Schaul, and Tim Rocktäschel. Position: Open-endedness is essential for artificial superhuman intelligence. Forty-first International Conference on Machine Learning (ICML), 2024.
[3] Yuan Meng, Zhenshan Bing, Xiangtong Yao, Kejia Chen, Kai Huang, Yang Gao, Fuchun Sun, and Alois Knoll. Preserving and combining knowledge in robotic lifelong reinforeement learning, Nature Machine Intelligence, pages 1-14, 2025.
[4] Bo Liu, Yifeng Zhu, Chongkai Gao, Yihao Feng, qiang liu, Yuke Zhu, and Peter Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. In Thirrty-svvnnth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS), 2023.
[5] Richard S Sutton, Doina Precup, and Satinder Singh. Between mdps and semi monimmdps:AA A framework for temporal abstraction in reinforcement learning, Artificial intelligence, 1999.
[6] Alexander Li, Carlos Florensa, Ignasi Clavera, and Pieter Abbeel. Sub-palicy adaptation for hierarchical reinforcement learning. International Conference on Learning Representations (ICLR), 2020.
[7] Eleftherios Triantafy liidis, Fernando Acero, Zhaocheng Liu, and Zhibin Li. Hybrid hierarchical learning for solving complex sequential tasks using the robotic manipulation network roman. Nature Machine Intelligence, 2023.
[8] Mikel Malagon, Josu Ceberio, and Jose A Lozano. Self- colf-conp policies for scalable continual reinforcement learning. In Forty-first International Conference on Machine Learning (ICML), 2024.
[9] Thomas Schmied, Markus Hofmarcher, Fabian Paischer, Razvan Pascanu, and Sepp Hochreiter. Learning to modulate pre-trained models in rl. In Advances in Neural Information Processing Systems (NeurlPS), 2024.
[10] Weikang Wan. Yifeng Zhu. Rutay Shah, and Yuke Zhu. Lotus: Continual imitation learning robot manipulation through unsupervised skill discovery. In 2024 IEEE International Conference on Robotics and Automation (ICRA), 2024.
[11] Dachee Lec, Minjong Yoo, Woo Kyung Kim, Wonje Choi, and Honguk Woo. Incremental learning of retrievable skills for efficient continual task adaptation. The Thirtt-eighth Annual Conference on Neural Information Processing Systems (NeurIPS), 2024.
[12] Zexin Zheng, Jia-Feng Cai, Xiao-Ming Wu, Yi-Lin Wei, Yu-Ming Tang, and Wei-Shi Zheng. imanip: Skill incremental learning robotic manipulation. arXiv preprint arXiv:2503.070 2025.
[13] Jingkai Xu and Xiangli Nie. Speci: Skill prompts based hierarchical continual imitation learning for robot manipulation. arXiv preprint arxiv:2044.10 15561, 2025.

---

[14] Yixiao Wang, Yifei Zhang, Mingxiao Huo, Thomas Tian, Xiang Zhang, Yichen Xie, Chenfeng Xu, Pengliang Ji, Wei Zhan, Mingyu Ding, and Masayoshi Tomizuka. Sparse diffusion policy: A sparse. reusable, and flexible policy for robot learning. Proceedings of The 8th Conference on Robot Learning (CoRL), 2025.
[15] George Konidaris and Andrew Barto. Skill discovery in continuous reinforcement learning domains using skill chaining. Advances in neural information processing systems (NeurIPS), 2009.
[16] Nur Muhammad Mahi Shafiullah and Lerrel Pinto. One after another: Learning incremental skills for a changing world. In International Conference on Learning Representations (ICLR), 2022.
[17] Nico Gürtler, Dieter Büchler, and Georg Martius. Hicrarchical reinforcement learning with timed subgoals. Advances in Neural Information Processing Systems (NeurIPS), 34, 2021.
[18] Hao Ju. Rongshun Juan, Randy Gomez, Keisuke Nakamura, and Guangliang Li. Transferring policy of deep reinforcement learning from simulation to reality for robotics. Nature Machine Intelligence, 2022.
[19] Matthias Hutsebaut- Buysse, Kevin Mets, and Steven Latré. Hicrarchical reinforcement learning: A survey and open research challenges. Machine Learning and Knowledge Extraction, 2022.
[20] Hongjoon Ahn, Jinu Hyeon, Youngmin Oh, Bosun Hwang, and Taesup Moon. Prevalence of negative trans- fer in continual reinforcement learning: Analyses and simple baseline. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
[21] Jakob Fnerster, Joannis Alexandros Assael, Nando De Freitas, and Shimon Whiteson. Learning to communicate with deep multi-agent reinforcement Jeamnigg Advances in neural information processing systems (NeurIPS), 2016. 2016.
[22] Jiechuan Jiang and Zongging Lu. Learning attentional communication for multi-agent cooperation. Advances in neural information processing systems (NeurIPS), 2018.
[23] Amanpreet Singh, Tushar Jain, and Sainbayar Sukhbaatar. Individualized controlled contimuous communi- cation model for multiagent cooperative and competitive tasks. In International Conference on Learning Representations (ICLR), 2019.
[24] Ziluo Ding. Tiejun Huang, and Zongqing Lu. Learning individually inferred communication for multi-agent cooperation. Advances neural information processing systems (NeurlPS), 33:22069-223 220799, 2020.
[25] Scohong Park, Jongwook Choi, Jackyeom Kim, Honglak Lec, and Gunhee Kim. unsupervised skill discovery. In International Conference on Learning Representations (ICLR), 2021.
constrained
[26] Seohong Park. Kimin Lee. Youngwoon Lee, and Pieter Abbeel. Controllability-wwerr unsupervised skill discovery. In Proceedings of the 40th International Conference Machine Learning (ICML), 2023.
[27] Karl Pertsch, Youngwoon Lee, and Joseph Lim. Accelerating reinforcement learning with learned skill priors. In Conference on robot learning (CoRL), 2021.
[28] Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. arXiv preprint arXiv: 1802.06070, 2018.
[29] Archit Sharma, Shixiang Gu, Sergey Levine, Vikash Kumar, and Karol Hausman. Dynamics-aware unsupervised discovery of skills. In International Conference on Learning Representations(((II))) 2020.
[30] Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. arXiv preprint arXiv: 1el:::1117.000 1.05007, 2016.
[31] Ruijic Zheng, Yongyuan Liang, Xiyao Wang, Shuang Ma, Hal Daumé III, Huazhe Xu, John Langford, Praveen Palanisamy, Kalyan Shankar Basu, and Furong Huang. Premier-taco is a few-shot policy learner: pretraining multitask representation via temporal action-driven contrastive loss. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024.
[32] Benjamin Eysenbach, Tianjun Zhang, Sergey Levine, and Russ R Salakhutdinov. Contrastive learning as goal-condittoned reinforcement learning. Advances in Neural Information Processing Systems (NeurIPS). 2022.
[33] Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, and Pieter Abbeel. Unsuper- vised reinforcement learning with contrastive intrinsic control. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems (NeurIPS), 2022.
12

---

[34] Michael Laskin. Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In International conference on machine learning (ICML), 2020.
[35] Karl Pertsch, Youngwoon Lee, Yue Wu, and Joseph Lim. Demonstration-guis guided reinforcement learning with learned skills. 5th Conference on Robot Learning (CoRL), 2021.
[36] Qiyang Li, Jason Zhang, Dibya Ghosh, Amy Zhang, and Sergey Levine. Accelcrating exploration with unlabeled prior data. Advances in Neural Information Processing Systems(NeurIPS), 36, 2023.
[37] Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson. Sergey Levine, and Pierre Sermanet. Learning latent plans from play. In Conference on robot learning (CoRL). 2020.
[38] Zuxin Liu, Jesse Zhang, Kavosh Asadi, Yao Liu, Ding Zhao, Shoham Sabach, and Rasool Fakoor. TAIL: Task-specific adapters for imitation learning with large pretrained models. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
[39] Yijie Guo, Bingjie Tang. Iretiayo Akinola. Dieter Fox, Abhishek Gupta, and Yashraj Narang. SRSA: Skill retrieval and adaptation for robotic assembly tasks. In The Thirteenth International Conference on Learning Reprexentations (ICLR), 2025.
[40] Arun Mallya and Svetlana Lazebnik. Packnet: Adding multiple tasks to a single network by itcrative pruning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR), 2018.
[41] Jachong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations (ICLR), 2018.
[42] James Seale Smith, Yen- Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion: Continual customization of text-to- tomagee--geeeee diffusion with c-IoRA. Transactions on Machine Learning Research (TMI.R), 2024.
[43] Xiao Wang, Tianze Chen, Qiming Ge. Han Xia, Rong Bao, Rui Zheng, Qi Zhang. Tao Gui, and Xuanjing Huang. Orthogonal subspace learning for language model continual learning. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023.
[44] Yan-Shuo Liang and Wu-Jun Li. Inflora: Interference free low-rank adaptation for continual learning. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[45] Yichen Wu, Hongming Piao, Long-Kai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, and Ying Wei. SD-IoRA: Scalable decoupled low-rank adaptation for class incremental learning. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
[46] Da- Wei Zhou, Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained models: survey. In Proceedings of the Thirty-Thine International Joint Conference on Artificial Intelligence, IJCAI-24, 2024.
[47] Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Beltagy. Doug Downey, and Noah A. Smith. Don't stop pretraining: Adapt language models to domains and tasks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL), 2020.
[48] Yujia Qin. Cheng Qian. Xu Han, et al. Recyclable tuning for continual pre-training. In Findings of the Association for Computational Linguistics (ACL), 2023.
[49] Haizhou Shi, Zihao Xu, Hengyi Wang, Weiyi Qin, Wenyuan Wang. Yibin Wang, and Hao Wang. Continual learning of large language models: A comprehensive survey. arXiv preprint arXiv:2404. 16789, 2024.
[50] W. Aha, Dennis Kibler, and Mare Albert. Instance-based learning algorithms. Machine Learning, 1991.
[51] Alexander Pritzel, Benigno Uria. Sriram Srinivasan, Adria Puigdomenech Badia, Oriol Vinyals, Demis Hassabis, Daan Wierstra, and Charles Blundell. Neural episodic control. In International conference on machine learning (ICMI.), 2017.
[52] Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph Lampert, icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Puttern Recognition (CVPR), 2017.
[53] Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. Meta-learning with augmented neural networks. In International conference machine learning (ICML), 2016.
13

---

[54] Kourosh Hakhamaneshi, Ruihan Zhao, Albert Zhan, Pieter Abbeel, and Michael Laskin. Hierarchical few-shot imitation with skill transition models. In International Conference Learning Reprexentations (ICLR), 2022.
[55] Soroush Nasiriany, Tian Gao, Ajay Mandlekar, and Yuke Zhu. Learning and retrieval from prior data for skill-based imitation learning. In 6th Annual Conference on Robot Learning (CoRL), 2022.
[56] Maximilian Du, Suraj Nair, Dorsa Sadigh, and Chelsea Finn. Behavior Retrieval: Few-Shot Imitation Learning by Querying Unlabeled Datasets. In Proceedings of Robotics: Science and Systems (RSS), 2023.
[57] Li- Heng Lin, Yuchen Cui, Amber Xic, Tianyu Hua, and Dorsa Sadigh. Flowretrieval: Flow-guided data retrieval for few-shot imitation learning. In 8th Anmual Conference on Robot Learning (CoRL), 2024.
[58] Marius Memmel, Jacob Berg, Bingqing Chen, Abhishek Gupta, and Jonathan Francis. STRAP: Robot sub-trajectory retrieval for augmented policy learning. In The Thirteenth International Conference on Learning Representations (ICLR), 2025.
[59] Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lce. REGENT: retrieval-auu sarreval-augmennted generalist agent that can act in-context in new environments. In The Thirteenth International Conference on Leurning Reprexentations (ICLR), 2025.
[60] Haoqi Yuan, Zhancun Mu, Feiyang Xie, and Zongging Lu, Pre-training goal-based models for sample- efficient reinforcement learning. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
[61] Prasanta Chandra Mahalanobis. On the generalized distance in statistics. Proceedings of the National Institute of Sciences of India, 1936.
[62] Yifeng Zhu, Peter Stone, and Yuke Zhu. Bottom-up skill discovery from unsegmented demonstrations for long horizon robot manipulation. JEEE Robotics and Automation Letters, 2022.
[63] Peter Rousseeuw. Rousseeuw, p.j.: Silhouettes: graphical aid to the interpretation and validation of cluster analysis. comput. appl. math. 20, 53-65. Journal of Computational and Applied Mathematics. 20:53-65, 11 1987.
[64] Pete Florence, Corey Lynch, Andy Zeng, Oscar A Ramirez, Ayzaan Wahid, Laura Downs, Adrian Wong, Johnny Lee. Igor Mordatch, and Jonathan Tompson. Implicit behavioral cloning. In 6th Conference on robot learning (CoRL), 2022.
[65] Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4rl: Datasets for deep data-driven reinforcement learning. arXiv preprint arXiv:2004.07219, 2020.
[66] Abhishek Gupta, Vikash Kumar, Corey Lynch, Sergey Levine, and Karol Hausman. Relay policy learning: Solving horizon tasks via imitation and reinforcement learning. arXiv preprint arXiv:1910.1 1956. 2019.
[67] Tianhe Yu, Deirdre Quillen, Zhanpeng He, Ryan Julian, Karol Hausman, Chelsea Finn, and Sergey Levine. Mcta-world: A benchmark and evaluation for multi-task and meta reinforcement learning. In Conference on robot learning (CoRL), 2020.
[68] Byeonghwi Kim, Minhyuk Seo, and Jonghyun Choi. Online continual learning for interactive instruction following agents. In The Twelfth International Conference on Learning Representations (ICLR), 2024.
[69] Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet K Dokania, Philip HS Torr, and Marc' Aurelio Ranzato. On tiny episodic memories in continual learning. arXiv preprint arXiv:1902. 10w.199020006..... 2019.
[70] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in neural information processing systems (NeurIPS), 2020.
[71] Zhendong Wang, Jonathan J Hunt, and Mingyuan Zhou. Diffusion policies as an expressive policy class for offline reinforcement learning. In International Conference on Learning Representations (ICLR), 2023.
[72] Jesse Zhang, Minho Heo, Zuxin Liu, Erdem Biyik, Joseph Lim, Yao Liu, and Rasool Fakoor. Ex- tract: Efficient policy learning by extracting transferable robot skills from offline data. arXiv preprint arXiv:2406. 17768, 2024.
[73] Kyowoon Lee, Seongun Kim, and Jaesik Choi. Adaptive and explainable deployment of navigation skills via hierarchical deep reinforcement learning. In 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.
14

---

[74] Hongjoon Ahn, Sungmin Cha, Donggyu Lee, and Taesup Moon. Uncertainty- based continual learning with adaptive regularization. Advances in neural information processing systems (NeurIPSS) 32, 2019.
[75] Wonje Choi, Jinwoo Park, Sanghyun Ahn, Daehee Lee, and Honguk Woo. Nesyc: A neuro-symbolic continual learner for complex embodied tasks in open domains, In The Thirteenth International Conference on Learning Representations (NeurlPS), 2025.
[76] Dongsu Lee and Minhae Kwon. Temporal distance-aware transition augmentation for offline model-based reinforcement learning. Forty-second International Conference Machine Learning (ICML), 2025.
[77] Sangwoo Shin. Dachee Lee, Minjong Yoo, Woo Kyung Kim, and Honguk Woo. One-shot imitation in non-stationary environment multi-modal skill. In Fortieth International Conference on Machine Learning (JCML), 2023.
[78] Edward Hu, yelong shen, Phillip Wallis, Zeyuan Allen Zhu, Yuanzhi Li, Shean Wang, Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Reprexentations (ICLR), 2022.
[79] James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Mondderan-Milngg and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018.
15

---

Appendix
Contents
A
SIL--C Implementation
17
A.1 Notations
17
A.2 Algorithms
18
A.3 Pseudocode
20
A.4
Hyperparameters
21
Experimental Settings
B.
Environments
21
B.2
SIL Scenario
21
3
Baseline Details
................................
23
4
Metrics
25
B.5
Training Details
25
B.6
Evaluation Detail
26
Additional Experiments
27
Motivating Examples
27 ................................
Skill-Policy Compatibility Overview and Ordering Effects
................................
D Additional Analysis
29
D.I Extended Skill and Subtask Space Resolution 
29
D.2
Subtask Space Analysis on Few-shot Imitation Learning
................................
30
D3
Robustness to Observation Noise
31
D.4 Analysis Distance Metrics for Instance- basedd Classification
31 ................................
D.5 Analysis on Instance-based Classifier Thresholding
D.6
Memory and Time Analysis
32
16

---

A SIL-C Implementation
This section details key notations in A.1 and operational steps in A.2 and throughout the method. We
aim to facilitate understanding of concepts described in the main text and supplementary explanations provided in this appendix.
A.1 Notations
This subsection summarizes all notations used throughout the paper.
Table 4: Notation used in the paper
Symbol
Meaning
Symbol
Meaning
MDP fundamentals
M
AS P, R. Ho.
State space
A
Action space
Transition probability
R
Reward function
Mo
Initial-state distribution State
Discount factor (0 <
Action
Step reward; episode Task identifier
(Stochastic) policy
Hierarchical policy
High-level policy
Low-level skill decoder Low-level skill index Parameters m
Z4
High-level subtask index
Parameters of Fy
2
Discrete subtask/skill index space
Skill Incremental Learning (SII.)
SIL phase index
P
Total # phases
(Dp K
Datastream (task-agnostic trajectories)
Tp
Evaluation task set at phase
skilt index set at phase p
D.
task Params of my at plase p
Expert demonstrations
High-level policy task
Skill decoder phase
Params of n. for task
0
Lazy-Learning Interface (bilateral modules & instance based classifier)
X,
Subtask space
X;
skill space
Subgoal state
Steps ahead to define g
V (o;
Task side subgoal predictor: returns
X
Task memory of 7 (subtask prototypes)
(g.9)) 2)
Skill-side validator: iff d-(g)
Skill memory at phase p (subgoal-based prototypes)
(x::; 2")
Skill- side classifier (restricted 2'): returns 2t Interface (Eq. (8)
Skill memory phase p (state based prototypes)
[().
Candidate skills s.t. (g. [g.=")=1
&
Distance threshold for class Prototype for class of sub-clusters (modes) for
d,(x))
Mahalanobis distance class-c prototype
X5 K.
Mak. E..
Mean and (diagonal) covariance k-th Gaussian
Query instance (e.g.. g)
Prototype construction (skill & subfask spaces)
Eg Hi R2
Clustered skill group
Zp
Index set of skills discovered at phase of cluhecusters for skill Mean. covariance of skill-2 sub-cluster
Sub-clusters
(state/subgoal)
K2
Xa. x3
Prototypes for skill (state/subgool)
E
Clustered sabtask group
G.
Subtask label set (from demos)
Sub-clusters for subtusk of subgoal (state)
K₂
# of sub-clusters for subtask of subgoal 8
X5
Subtask prototype (state)
Ha.k Eg.
Mean, covariance of subtask sub-cluster subgoal
17

---

A.2 Algorithms
We provide phase-level training and run-time inference procedures that instantiate the SIL-C frame- work using the append-only prototype memories and the bilateral lazy-learning interface described in Sec. 4.
Algorithm SIL-C on SIL & Policy Learning with Interface at phase p
Require: Phase index p. streamed dataset Dp. set of evaluation tasks Tp. demonstrations {D+) FET, previous decoder di (P-11), previous memories X, x,<(o=1),, x29((=11) previous index set 2p-1 Ensure: Updated decoder = updated memories X, 2P updated index set 2p. trained high- level policies [Tī]T task memories (xx JEET,
SIL AT PHASE-P
SKILL SPACE UPDATE (EQ. (5))
Skill clustering: segment Dp into {Es}EEZ,
Initialize X/ + 0, APp +-
for all 2p do Sub-clustering: apply K-means to & to obtain (H2 - kk
Prototype creation: compute {Ms,A, De and form x2,
Add prototypes: X² X. U (x3). X/ -2 p U {xx]
Memory append: X² + X, (, U XPP,
X. c- 2: x- U X7.
LOW-LEVEL SKILL DECODER UPDATE
Train with A, on D, to obtain x! Index growth: Zp & Zp-1 U Zp
# DOWNSTREAM POLICY TRAINING AT PHASE-P
for all E TTp do
# SUBTASK SPACE UPDATE (BQ. (6))
Initialize X (-0
Subtask clustering: segment D, into {Eg
for all E G, do
Sub-clustering: apply K-means to Ey to obtain (H K
k=1
Prototype creation: compute {Mg Eg, 1 and form x
Kgg
Add prototypes: N, +- - U (x,)
LABEL ASSIGNMENT WITH UPDATED DECODER (EQ.
for all transition (s, a²) E D do ||a a² 11
2/ f- arg min 21EZ,
à â~ x
(1)
LEGEL POLICY OPTIMIZATION
Update मí, by minimizing E(.,s,)l log कं ( s)]
return m
X...P X.P.P Zp, ( (M TTeT,)
Phase training (Alg. D.. (A) Skill space update. At the beginning of phase the streamed dataset Dp segmented into skill groups {Es)se,, (5)) For each group, we perform K-means in the state and subgoal spaces to obtain sub-clusters {He H²,)) and form Gaussian prototypes x² x with diagonal covariance. Newly created prototypes are appended to the skill memories A X without altering past entries, and the skill decoder is updated to Ti: the index set grows append-only as Zp <- Zp-1 Z,.. (B) Policy learning on the interface. For each task т, we build the subtask memory by clustering D, into {Eg]aeg and constructing state-side prototypes (x,) (6)) Labels for behavior cloning are assigned by energy-basee prior that selects 2 minimizing the decoder mismatch (7) ), and Tī is trained with cross-entropy over (s, 27,)
18

---

Algorithm 2 SIL-C Evaluation with Interface
Decoder of. policy The interface modules VrP, V. " task memory G-, skill index set 2p. initial state so
Initialize R 4 0, 8 6 50
while episode not terminated do
HIGH-LEVEL POLICY
2h 7 (zh 8)
sample subtask
INTERFACE INFERENCE (SUBTASK SPACE)
9 +- W,""" (s; 9,)
b predict subgoal
# INTERFACE INFERENCE (SKILL SPACE) (EQ. (8)
if 1/ 2h) = 1 then
skill validation
21 + 2/
else
skill hooking
3' {2 2p (9 V/ (8; 2¹)
LOW-LEVEL SKILL DECODER
Tp(a 8, 21)
b sample action
Execute a; observe reward and next state s'; R R + 5
return R
Policy n (Z s)
Updated
S, 2z
4/ (s)
(i) Skill validation
(ii) Skill hooking ep
9. nh.
Accept
p
Reject
221
4 (g)
4/ (s)
( (F((s
Accept
Accept
Skill decoder πÝ (a) z1)
Figure 7: Visualization of agent inference procedure with SIL-C
Agent Inference (Algoritm 2 Figure Given the current state s, the high-level policy samples subtask on < 8), and the task-side module predicts subgoal performs skill validation with the skill-side validator V/: if 1 [g. 24 = the subtask is accepted (2) = 2h); otherwise, skill hooking restricts the candidate set to Z' { Zp 1 (g
V Gr). The interface first
[ and chooses 2 = (s; Z') (restricted classification). Equivalently, validation can be written as (g) 8- using the Mahalanobis distance in Eq. (4). This two-stage eestacee-based procedure operationalizes Eq. (8) and realizes compatibility at inference without policy re-training.
19

---

A.3 Pseudocode
## Initialization
from collections import defaultdict
from scipy. stats import chi2
Interface
"skills": defaultdict (lambda: ("subgoal"; D, "state": ()))
"subtasks"; defaultdict (lambda: defaultdict(list)),,
} Gaussian component (mean, precision)
pi_l, pi_h = None, 0
def label_distxx, comps): Eq. (4)
def (x,a(x, comp): mu, Sigma_ing inv = comp
= mu Sigma_ inv return min (maha (x, c) for с comps)
def chi2_thr (comps, conf=0.99 .9) dim comps [0] [0] shape [0] return chi2.ppf (conf, dim)
## Policy compatible skill incremental learning
for in (1nge + 1):
Skill space update Eq. (5)
chi_gz, chi_sz
for z in chi_gz:
Interface ("skills""" [z] ["aabaaaa)_xtttmmccl_eelll
extend(chiiggzzz) [x])
Interface ("skille"" [2] ["state"] exxend exxtend(hii_ssz())
Skill incremental learning
pi_l train_lox_level_ deeoeeeedeeeeee1 (D_p, Interface["skillle pi_1)
# Downstream policy learning
for tau in T_p:
Subtask space update Eq. (6)
chi_g wreat__sbraakk.pottoypee (cluster_subtasken (D_
for in chi_g:
(iterface[ssubtaske [tau] [g] extendcchi_pe
Policy learning on the interface Eq.(7)
labels aesign_subtaskk_aabllee (D_tau, Interface ["skills"), pi_1)
pi_h[tau] Leve
labels)
## Policy inference the interface
infer infer_actionee, (a, tau, conf=0.99):
# High-level policy: subtask proposal
z_h pi_h[tau] (s)
g predict_subboale
Interface ["subtasks"] [tau])
delta chi2 thi (Interface ["skills"]
("stiiil [z_h] ["subgoal"], conf)
# Interface: skill validation and skill hooking Eq. (8)
if Label_dist(e Interface ("sxil1sss [z_h] ["subgoal"]) <= delta:
z_1 z_h
else:
cand = [z for z in Interface["atee ["skills"]
if label_dist(ce Interface "arrfacce""kiiii ["skills"] [z] ["subgoal"]) <= delta] [z_h]
min (cand, key=lambda
label_dist Interface ("skills") [z] ("state"]))
Lon-level decoder: action execution
return ps_ z_1)
Figure 8: Python style pseudo code of SIL-C
20

---

We provide Python-style pseudo code in Figure 8 to illustrate the SIL-C implementation. The code shows how the append-only prototype memories are constructed and updated during each phase, and how the bilateral interface performs skill validation and hooking at inference time using Mahalanobis distance matching.
A.4 Hyperparameters
We apply a consistent set of hyperparameters to update each space in the SIL-C interface for both the Kitchen and Meta- World environments. The configuration, used in Table T is summarized in Table 5
Table 5: Default hyperparameter configuration for SIL-C
Skill-Side (per phase)
Task-Side (per task)
Sub-clusters per skill (K²) Goal offset (m)
Sub-clusters per sub-task (Kg)
4
20
Goal offset (m)
20
B Experimental Settings
B.1 Environments
Franka Kitchen 665 is long-horizon manipulation environment where robot must complete multi-stage tasks by interacting with various kitchen objects such as a microwave, kettle, burners, light switches, and cabinet doors. We use the kitchen sexcchen-mixed-v0 dataset for experiment. Each demonstration consists solely of state-action transitions, without any re- ward signals task labels. The state space is 60-dimensional, encompassing both the robot arm state and the states the manipulated objects. The action space is 9-dimensional, corre- sponding to the 9 degrees of freedom (DoF) of the Franka robotic arm. Each task requires the robot to complete a sequence of subtasks(stage) selected from a fixed set of predefined sub- tasks: open microwave, move kettle, turn top burner, turn light switch, open hinge cabinet, open slide cabinet. A reward of 1.00 is given for each subtask successfully completed in the correct order, yielding maximum of 4.00 when all subtasks are completed. For evaluation, we normalize the reward such that the maximum score is 100.
bottom burner,
Multi-stage Meta-World extends the original Meta- World benchmark (67. which includes 50 diverse single-step robot tasks, by composing multiple tasks into sequential stages within a single episode. Each multi-stage task requires an agent to complete subtasks a fixed order, mimicking realistic long-hot long-horizon objectives puck, drawer, button, door, box, handle, lever, stick (77). We use the Easy dataset introduced by [11], which consists of composed tasks involving objects selected from subset of: puck, drawer, , button, door, with varied subtask sequences. Each demonstration consists solely of state-action transitions, without any reward signals or task labels. The space is 140-dimensional, capturing both the robot state and the states of the relevant objects. The action space is 4-dimensional, corresponding to the 4 degrees of freedom the robotic arm. Each task consists of 4 sequential subtasks selected from the predefined set: slide puck, close drawer, push button, open door, close box, press handle, pull lever, and insert stick into red box. As in the Franka Kitchen environment, reward of 1.00 is given for each subtask completed in the correct order, with maximum cumulative reward of 4.00 per task. For evaluation, we normalize the reward so that the maximum possible score is 100.
B.2 SIL Scenario
In each phase p. new skill dataset D, is provided to train the skill decoder. Subsequently, based on the trained low-level skill decoder, 24 task-specific high-level policies are individually trained.
21

---

Table Kitchen: Emergent SIL
Table 7: Meta-World: Emergent SIL
Kitchen: : Emergent SIL.
Meta-World: Emergent SIL
Skill Dataset
Phase
Skill Datasets
bottom humer
microwave
bottom burner
top hurner
light switch
light switch
slide cahinet
slide cabinet
puck
drawer
button
door
microwave
bottom
top
puck
drawer
door
button
microwave
bostom burner
slide cahinet
hinge cabinet
puck
button
drawer
door
microwave
bottom burner
top burner
slide cabinet
puck
button
door drawer
burner
top
slide cabinei
hinge cabimet
puck
door
drawer
button
kettle
bottom burner
top burner
slide cabinet
puck
door
button
drawer
microwave
bottom burner
top burner
binge cabinet
microwave
kettle
bottom burner
hinge cabinet
drawer
puck
button
door
microwave
kettle
top burner
hinge cabinet
drawer
puck
door
button
kettle
boitom burner
lop burner
bings vabinet
drawer
button
puck
door
micnowave
kettle
top
switch
drawer
button
door
puck
micnowave
top burner
light swichh
hinge cabinet
drawer
door
puck
button
microoaae
kettle
light switch
binge cabinet
drawer
door
button
puck
microwave
switch
slide cabinet
binge cabinet
micnowave
kettle
slide cabinet
hings cabinet
button
puck
drawer
door
microwave
kettle
bottoen hurner
slide cahinet
button
puck
door
drawer
microwave
kettle
light swisch
slide cabinet
button
drawer
puck
door
kettle
top burner
light switch
slide cabinet
button
drawer
door
puck
kettle
bottom burner
slide cabinet
hinge cabinet
hutton
door
puck
drawer
kettle
bottom burner
light switch
hinge cabinet
button
door
drawer
puck
kestle
botiom burner
top burner
light switch
light
cabinet
hinge cabinet
door puck
drawer
button
kettle
bostom burner
light swith
slide cabinet
door puck
button
drawer
door drawer
puck
button
4
door drawer
button
puck
door
button
puck
drawer
door
button
drawer
puck
Table 8: Kitchen: Explicit SIL
Table 9: Meta-World: Explicit SIL
Kitchen: Explicit SIL
Meta-World Explicit SIL
Phase
Skill Datasets
Phase
Skill Datasets
microwave
kettle bottom bumer
puck
top humer// light switch
2
drawer
4
slide cabinet / hinge cabinet
3
button
4
door
B.2.1 Datastream Types: Emergent and Explicit SIL
Emergent SIL. In the Emergent SIL scenario, we divide each dataset into four datastreams by grouping full aggosticc demonstrations, resulting in datastreams that contain a mixture of tasks without explicit task identifiers. To facilitate scenario construction and explanation, we partition the dataset using task information. To maintain the task-agnostic nature of skill learning, this information is not exposed during skill-incremental learning, and the learning process remains fully task-agnostic. In the case of Franka Kitchen, we collect task demonstrations based on the predefined subtask goal information provided by the environment. Failed or truncated trajectories are discarded, and we identify 24 distinct tasks. The resulting datastream composition is summarized Table 6 For Meta-Woor World, follow similar approach, segmenting the dataset into 24 tasks, corresponding to all possible subtask sequence permutations, by verifying subtask goal completion using the environment's 's built-in success conditions. The datastream configuration is shown in Table 7
Explicit SIL. In the Explicit SIL scenario, we segment each dataset into shorter demonstrations based on predefined skill boundaries specified by the environment. These skill segments are then grouped into four datastreams. To facilitate scenario construction and explanation, we use skill labels provided by the environment to organize the data. However, this information is not exposed to the agent during training, and the learning process remains fully skill-agnostic. For Franka Kitchen, we segment trajectories by identifying transitions that correspond to each predefined subtask. These are clustered
22

---

by subtask type to form spill-specific datastreams, as shown in Table For Meta- World, we apply similar segmentation strategy, isolating the trajectory from the beginning of each subtask to the point where the agent returns to a neutral position. These segments are then clustered according to subtask identity, resulting in the skill incremental setup summarized Table 9
B.2.2 Evaluation Groups for SIL Scenarios
At each phase p, a new skill dataset D, is used to train the skill decoder. Using this updated decoder, then train 24 task-specific high-level policies, one for each evaluation task. Across both environments, we evaluate all 24 tasks listed in Table and Table ( by training a dedicated policy for each task. This setup enables us to evaluate bidirectional compatibility. Specifically, we measure Backward Skill Compatibility (BwSC) by checking whether policies trained in earlier phases remain functional with updated skills. In parallel, we assess Forward Skill Compatibility (FwSC) by determining whether new policies can effectively leverage skills learned in previous phases.
Formally, the evaluation groups for BwSC, and FwSC, of task are defined as:
BwSC, = (aTx T.(L) (1) ). "i (T "
T.(1) (2)
(P)) (0))
(9)
FwSC, ((m (1)) (1
T2) (2)
(10)
Here, n 7.(1) denotes the initial high-level policy trained with the skill decoder from the first phase I. For any phase 2 1, Tr T.P)) refers to the updated high-level policy synchronized with the corresponding skill decoder u/ phase p.
B.3 Baseline Details
We categorize our baselines into four distinct types based on their methodology and compatibility with the Skill Incremental Learning (SIL) scenario.
Type I: Skill-based approaches with continual learning, Type combines skill-based pre-training with continual learning strategies using simple skill-space appending scheme. We adopt two hierarchical skill pre-training methods:
BUDS 62 10: BUDS [62] segments trajectories into fixed-length intervals and merges them bottom-up up based on trajectory similarity, and was later adopted as the foundation in LOTUS [10] for continual imitation learning. It discovers skills by clustering these segments. In our implementation, we base our hyperparameter choices on those reported in BUDS. To support coal-conditioned skill decoder learning, we annotate each trajectory with subgoal states prior to skill discovery. For each discovered skill, we select its goal representation as the subgoal state closest to the average of the subgoals observed across its transitions. Each phase can generate up to 10 skills, but the actual number is determined automatically using the silhouette score. In the Kitchen environment, this resulted in an average of approximately 8 skills per phase.
PTGM 160]: PTGM leverages subgoal state information during skill decoding by discretiz- ing the subgoal state space and treating each subgoal bin as a distinct skill. This approach supports pre-training in open domains and demonstrates strong performance tasks, particularly in complex environments, often outperforming skill mre-training methods based continuous skill representations. In our experiments, adopt the hyperparameters from the PTGM paper for the Kitchen domain. Each phase produces a total of 20 skill clusters.
downstream
In both methods, we define the subgoal state for each transition as the state reached after m = 20 steps. To support the SIL scenario, we expand the skill set at each phase by appending newly discovered skills to those from previous phases. This expansion is strictly append-only and does not modify previously learned skills.
These pre-trained skills are integrated with continual learning or adaptation strategies as follows:
Fine- Tuning (FT): The simplest approach. The skill decoder is incrementally trained on each new skill without access to prior data, memory or replay is used.
23

---

Experience Replay (ER) 69: In our implementation, a replay buffer stores 10% of the data from each previous phase. During training in the next phase, these stored samples are interleaved with new data at 1:1 sampling ratio to mitigate forgetting.
Adapter Append (AA) [3: In the first phase, we pre-train the entire model. Starting from the second phase, freeze the pre-trained model and train separate LoRA adapter [78] for each new phase, using rank 4 for the Emergent scenario and rank 16 for the Explicit scenario. During evaluation, we use the adapter corresponding to the phase in which the skill was introduced. This strategy, introduced in TAIL [38], preserves skill-policy compatibility by preventing forgetting in both the base model and its adapted parameters. However, limits the ability to transfer or incorporate newly acquired skills across different phases.
Type II: Semantic representation- based skill incremental learning approaches. These methods rely on predefined semantic subgoals as skill labels. They incorporate prototype-based skill retrieval and model expansion with temporal replay.
Skill protolpptttypes + (AA) Mi Performs skill retrieval using a prototype memory, where each prototype is linked with adapter parameters and indexed by persistent semantic skill labels. We set the number of skill prototype bases 50. Following the original setup, tasks are learned using semantic subgoal labels provided in fixed sequence.
Instructions + (ER) [1] Defines and learns skills incrementally from semantic representa- tions. We adopt trajectory-basee temporal replay support skill incremental learning.
To ensure fair comparison, these methods are restricted using only the semantically labeled skills available for policy training at the corresponding SIL phase Trajectories without semantic skill labels are merged with the skill from the previous transition.
Type III: SIL-C. Our method builds on the Type structure and its configuration. In our SIL setup, assign four sub-clusters to each skill prototype. For subtask prototype construction, we follow the PTGM algorithm, setting 20 prototypes per task and assigning four sub-clusters to each prototype as well.
High-level policy and skill decoder implementation. For all baselines, we implement the high-level policy as four-layer MLP that acts as classifier for sub-task prediction. To enable behavior cloning in SIL-C, we generate supervision signals using Eq. 7 and train the classifier with cross-entropy loss.
The skill decoder, used across all baselines, is goal-conditioned policy consisting of two components. The first maps each skill to a hashed embedding. Following the skill decoder architecture proposed in 660. we compute each skill embedding as the subgoal state closest to the centroid of the subgoal states comprising that skill. We then feed this subgoal state, along with the current state, into conditional denoising diffusion model with four conditioned denoising blocks. The model denoises from Gaussian noise to reconstruct the corresponding action.
For Type II methods, where skills are defined semantically (e.g., using natural language descriptions), we embed the descriptions using the text text-embedding- -eriiingn--lan 3-large model from OpenAl and use these vectors directly skill representations.
24

---

Table 11: Default skill decoder configuration
Table 10: Default policy configuration
Hyperparameter
Value
Hyperparameter
Value
Diffusion Model
DDPM
Denoising step
64
Model
MLP
Sanpleriaforenee)
Deterministic (no noise added)
Hidden size
512
Schedule
Linear
Hidden layers
4
Lincar start
le-4
Lincar end
2c-2
Dropout
0.1
Block
MLP
State input dim
60
Hidden dimension
512
Layers
Dropout
0.0
Optimizer
Adam
denoised
True
Learning rate B1
1 x 10-4
Optimizer
Adam
0.9
Learning
Learning rate (LoRA)
x 10
0.9
B.4 Metrics
For each scenario, we evaluate the FwSC and BwSC groups using three metries: Forward Transfer (FWT), Backward Transfer (BWT), and Area Under the Curve (AUC). Overall performance is reported by computing each metric over the union of both groups. Let rTPP denote the normalized reward for task phase p, evaluated using the corresponding high-level policy and skill decoder.
For example, TT = (m T.((),, .T") for the BwSC group, and TTPPP (T 7.P mT²) for the FwSC group. This score reflects task performance under the given high-level and low-level policy configuration. We assume a fixed evaluation task set T1, which is used across all phases, i.e., Tp = Ti for all p E P).
Forward Transfer (FWT) measures the performance of newly trained policies after skill updates, quantifying how updated skills facilitate learning new tasks.
FWTT-P T.P
(11)
We report both the Initial FWT, computed at p and the Final FWT, computed at the final phase P.
Backward Transfer (BWT) quantifies the influence of skill incremental learning on previously acquired task performance. It defined as the average change in performance across phases, relative to phase 1:
BWT
T.P (2))
(12)
P-
2
Area Under the Curve (AUC) captures the overall performance trend across all skill learning phases, computed as the mean normalized reward:
P AUCT
Σ
TT
(13)
P
We report the final values for each metric by averaging over all evaluated tasks T E T.
B.5
Training Details
Compute Resources We conducted our experiments the following computing environments:
AMD Ryzen 9 7950X3D 16-Core Processor with a single RTX 4090 GPU. OS: Ubuntu 22.04, CUDA Version: 12.4, Driver Version: 550.144.03
AMD Ryzen Threadripper PRO 5975WX 32-Core CPU with 2x RTX 4090 GPUs. OS: Ubuntu 22.04, CUDA Version: 12.4, Driver Version: 140.03
25

---

AMD Ryzen Threadripper PRO 5975WX 32-Core CPU with 2x RTX 4090 GPUs. OS: Ubuntu 22.04, CUDA Version: 12.2, Driver Version: 535.230.02
Training Framework Details We implemented our framework using JAX 79 to efficiently ac- celerate training and handle the continual phase's s alternating training and evaluation process. software dependencies are included with the released code for full reproducibility. The versions of core libraries are:
jax: 0.4.34
jaxlib; 0.4.34
flax: 0.10.2
optax: 0.1.9
Experiment Compute Estimates For experiments Kitchen and Meta-World, ran 4 seeds in parallel (4 processes on a single GPU). Each run took approximately 4 hours and 30 minutes, broken down follows:
Training: hour for training 4 skill update phases, covering a total of 96 policy training
runs (24 tasks per phase).
Evaluation: 3 hours and 30 minutes for 168 task evaluations (24 tasks FwSCJ).
[3 BwSC
Total Compute Usage
Main experiments only: Approximately 225 GPU hours (RTX 4090), including core experiments and additional SIL scenario evaluations.
Experiments including Appendix: Approximately 500 GPU hours (RTX 4090), covering extended experiments and additional SII. evaluations.
All experiments, including preliminary and discarded runs: An estimated 600 GPU hours (RTX 4090), accounting for exploratory and failed runs not included in the final results.
B.6 Evaluation Details
To report performance metrics, we used random seeds and report the mean and standard deviation across them. For each scenario, the normalized reward of given task was computed by running the evaluation three times and averaging the results. The averaged value was recorded as the reward for that task.
26

---

Table 12: Performance evaluation of skill-poliey compatibility in two SIL scenarios of the Franka- Kitchen environment with four seeds. Each row corresponds to a baseline, categorized by the skill interface configuration, hierarchical agent structure, and the SIL algorithm used for skill decoder updates. The symbol denotes methods that require pre-defined semantic skill labels for both policy and decoder training. The left side reports BwSCC results, the right side reports FwSC results, and the center shows the overall performance considering both.
Kitchen: Emergent SIL.
Baselines
Initial
BuSCC (nitial)
Overall performance
FinSC (Synced)
Final
Type
Skill Interface
Algo.
FWT
BWT (9)
BWT (9)
(s)
AUC
FWT (9)
(BUDS)
Skill Segments
N2
10
ER
FT
23.5
23.5 .s
-16.4 Lss
-1088
11
3.6
26.6:22
18.3
17.9
4.3143
37.0
26.7
44.5
18,7
AA
21.9
1,3:44
22,9
13.7=35
35.3
30.1
44.4
57.1
FT
25
.36.9 L13
17.
-2.3111
21.
-11.7.21
24.7 16.2
ER (10%)
25.8-25
-19.7
32.1
-5.0:28
42.5
9.7433
54.0
$7.7
(PTGM)
Subgoal Bins
60
ER (50%)
48.2-19
-9.2+22
41.3=20
5.8-14
53.1+18
20.8-18
63.8
74.7=12
MT (100%)
48.4-14
-3.1818
46, 100
11.6=n9
58.4
26.2
68.1 1
81.3
45.0=31
2.6
46.9
15:4:13
58.2
28.1
66.1
83.6
Skiitt-rotettpes 10
AA
15.
0.5
55,5
0,5
55.7.97
0.9
56.9 15.8
ER
54.0
18.5
67.8
19.1
70.3
19.7
68.7 119
77.4
MT (100%)
52.7-25
15.2
64.1
19.7
69.6221
24.3
20.9
82, 42.7
III
SIL-C
52.9
18.611
66.
22.0
21.8
25.4
21.9.09
87.2g13
Sub-goal Bins
60
Joint
86.9
C
Additional Experiments
C.1 Motivating Examples
Table 12 extends our main results with additional baselines to demonstrate a critical challenge in skill incremental learning: even multi-task learning, often considered an oracle baseline in continual learning, fails to achieve backward skill compatibility (BwSC).
Multi-task learning fails to enable compositional skill reuse. We evaluate PTGM with varying replay ratios (10% 50%, 100%) to understand the limits of experience replay in SIL. Even with 100% replay, where the model has access to all previous data, PTGM achieves only -3. % BWT in the BwSC setting. This negative transfer indicates that despite having full access to historical data, the method cannot leverage newly acquired skills to improve existing policies without complete retraining. The pattern persists across all replay configurations: ER (10%) shows 199.7% BWT, ER (50%) shows -9.2% BWT, and even multi-task learning (MT 100%) exhibits negative transfer. These results reveal a fundamental architectural limitation: naive combinations of continual learning approaches with skill-based pre-training fail to support dynamic skill composition.
SIL-C enables true compositional learning without retraining. In contrast, SIL-C achieves substantial positive backward transfer across both continual learning strategies: 15.2% BWT with MT (100%) and 18.6% BWT with AA. This improvement is not merely incremental but represents a qualitative difference in capability. While baseline methods require complete policy retraining to utilize new skills (as evidenced by their improved FwSC but poor BwSC performance), SIL--Css lazy learning interface enables existing policies to dynamically compose newly acquired skills at inference time. This compositional property particularly evident the overall performance metrics, where SIL-C maintains competitive or superior AUC scores on BwSC (64. 1% with MT. 66.8% with AA) while simultaneously achieving the highest final performance (82.7% and 87.2% FWT respectively).
The significance of these results extends beyond numerical improvements: they demonstrate that SIL-C fundamentally changes how skills and policies interact in hierarchical reinforcement learning, enabling true lifelong learning where each new skill enhances the agent's 's entire behavioral repertoire without the computational burden of retraining.
C.2 Skill-Policy Compatibility Overview and Ordering Effects
Figure 9 presents phase-wise normalized rewards across four SIL datastreams, each defined by different permutation of skill datasets from T 1). The corresponding evaluation metrics for each stream are summarized in Table 13 These experiments assess both backward skill compatibility
27

---

Emergent (1-23-4)
Emergent Seq, (4-3-24)
80
60
40
40
==3
p-
Franka Kitchen Emergent
Franka Kitchan (2-1--3)
80
80
40
40
2
Initial policy (darker)
BUDS ER
BUDS +AA
PTOM + ER
PTGM AA
Synced (brighter)
pkil-protootpes
(Ours;
Figure 9: Phase-wise normalized rewards across four Kitchen Emergent Skill Incremental datas- treams, each corresponding a different permutation of skill execution order. Each bar group shows performance for different methods under both initial and retrained (Synced) policies.
(BwSC), which captures the evolution of the initial policy over phases, and forward skill compatibility (FwSC), measured after the skill policies have been resynchronized and retrained. In the BwSC setting, SIL-C consistently achieves the highest normalized rewards or matches the performance of methods that pre-defined semantic skills. After retraining (Synced Policy) in the FwSC setting, SIL-C continues to show the highest performance across all phases.
Each sequence corresponds to permutation of phase orders from Table In Seq. 2, the dataset of first phase does not include demonstrations for open microwave, even methods that rely on pre-defined semantic skills fail to execute this task in early phases. Once policies are retrained, SIL--C, SIL with semantic skills, and PTGM+AA reach similar levels of performance. similar issue appears in Seq. 3, where demonstrations for turn top burner are absent in the initial phase. Only SIL-C maintains high performance throughout, showing strong skill-policy compatibility despite delayed exposure this skill. Seq. follows the same pattern for open slide cabinet, where again only SIL-C maintains high rewards both before and after retraining.
Additionally, methods that combine semantic retrieval with skill prototypes exhibit decreasing performance over successive phases, particularly the Emergent skill incremental setting. As the diversity of skills increases, these methods are more susceptible to incorrect skill retrieval, which in some cases results in performance degradation relative earlier phases. Overall, SIL-- is the only method that consistently maintains skill-policy compatibility without requiring prior knowledge of future skills. This property is essential for scalable, SIL.
28

---

Table 13: Additional experiments for testing robustness to incremental ordering. Experiments were conducted on the Franka Kitchen environment using various combinations of phase orderings. Each row represents baseline method, organized according to the skill interface setting, hierarchical agent architecture, and the specific SIL algorithm applied for skill decoder updates. The symbol indicates methods requiring pre-specified semantic skill labels during both policy and decoder training phases. The left columns present the BwSCC results, the right columns show the FwSCC results, while the central columns report the overall combined performance. The best results are indicated in bold. In this table, SIL-C utilizes the Type I (PTGM + AA) approach as its foundational architecture.
Kitchen: Emergent SIT. Seq. (1-2-3-4)
Baselines
bitial
RwSC (Initial)
Overall performance
FwSC (Synced)
Final
Type
Skill Interface
SIL Algo.
(W
BWT (8)
(UC
BWT (8)
AUC (8)
BWT (8)
AUC (e)
FWT (8)
Skill Segments 10:00 (RUDS)
23.5435
-10.8
15.5a33
3,6370
266613
179965
37.0=30
44,5
AA
21.
1.3g44
22
15.7
35.3
30.1
44.4
57.1
1
Subgoal Bins 60 (PTGM)
46.8=25
-19.7+44
32.1+29
-50-25
42.5
9,7433
54.0=1a
57.7
AA
45.0a31
2.6ens
46.9a37
154
58.2+17
18.1
83.6
Skill-prototyye pes
AA
55.1
0.5
55.5
0.5
58.7
0.9.3
55.8
56.9144
Instructions
ER
54.0
18.5.44
67.8 11.
19.1
70.3
19.7.10
68.7
17.4
III
CIL-C
AA
$2,9.29
18.6
66.8 126
22.0 14
71.8 :3
25.4
71.9
87.2
Subgoal Bins
M
Joint
86.9.30
Kitchen: Emergent SIT Seq. (4-3-2-1)
Basclines
Initial
BwSC Unitial)
Overall performance
FwSC (Synced)
Final
Type
Skill Interface
SIL Algo.
FWT
BWT 1so
AUC
BWT
AUC
BWT os
AUC
FWT (5)
Skill Segments 62
10
BR
17.9
-4,4:20
14.611.
9.9
26.4
24.2
36.1
42.1
(BUDS)
AA
19.5
-0.2+21
19.4509
14.0=22
31.5
28.2+14
40.7
55.2+44
1
Subgnal Bins (PTGM)
6D
ER
26.111
-10.3135
18.4
9.3133
10.1
28.8114
47.7
62.8.15
AA
25.2
0.6217
25.6
19.7
38.9.15
54.3
75.5
Skill-prototype pes 11
31,9202
-3.1s02
29.5
13.7811
43.6
30.5a10
54.7
60.5a53
Instructions
12
ER
29.8
12.3
25.8
14.7 k13
42.3 1o0
34.6.11
55.7
75.7
III
SIL-C
AA
25.2
28.9=16
46.9
35.7
55.8:1
42.5
57.0
79.3
Subgnal Rins 60
Joint
86.9.70
Kitchen: Emergent SIL Seq. 3 (3-4-1-2)
Baselines
Initial
BwSC Unitial)
Overall performance
FwSC (Synced)
Final
Type
Skill Interface
Algo.
FWT
BWT es)
AUC
BWT ow)
AUC
BWT ex:
AUC
FWT (9)
Skill Sogments (
ER
32,5
-13.2a21
22.6a13
-2.6.5k
30.3
8.0
38.5
40.4
(BUDS)
AA
31.8=12
1,2+20
32.6#08
11.7
41.8+12
22.3
48.5
57,2+12
Subgnal Bins 6D)
ER
29.7
...9
24.5
9:8
38.1
26.4
49.5
65.2
(PIGM)
AA
29.6
1.1515
30.4
18.3-14
45.3
35.4112
56.2
73.3
Skill-prototyps pes
11
30.7=1.
19.7=05
45.5
25.6:84
$2.7
31.5-00
54.4a1a
56.7504
Instructions [2]
ER
30.1=21
22,1
46.7a13
29.7=13
55.6+16
37,3=12
1830
79.1
III
SIL-C
37.2
26.5
57.1
28.9
62.0:24
31.2
60.6
79.5
Subgoal Bins 60
Joint
86.9
Kitchen: Emergent SIL Seq. (2-1-4-3)
Baselines
Initial
BWSCC (Initial)
Overall performance
FuSC (Synceuf)
Final
Type
Skill Interface
SII. Algo.
FWT cs;
BWT (s)
AUCC
BWT cw)
AUC (9)
BWT cs)
AUC (s)
FWT (5)
Skill Segments 62 (BUDS)
31,9119
-20.2+36
16.7a10
-81
24.9:16
4.0542
34.8:10
46.4a25
1
AA
34.0:29
-0.1
32.9
72
40.2
14.6
44.9
61.0
Subgoal Bins (PTGM)
48.3
-20.3440
33.0azs
-5.
43.6432
9.4
55.3
59.7.44
AA
1.0.20
11.
114.11
56.2.os
21.8137
62.7
72.8
I1 Skill protolypes pes
AA
56.0
142.1
46.9
9,1
48:2
-60
51.5
51.7
Instructions
12
ER
57,9:29
4.9=22
61.6:16
9.7538
66.2
14.5
68.8:10
76.3
III SIL-C
AA
528824
17.2-10
65.7=30
20.0
69.9
22.9-1a
69.9
80.4
Subgnal Bins
Joint
86.9
D
Additional Analysis
D.1 Extended Skill and Subtask Space Resolution Analysis
Table 14 reports the full results corresponding to Figure 6 showing the performance of SIL-C (with PTGM and AA) under different skill and subtask space resolutions. Following the setup in Section 6.3, we test both halved and doubled values of 12pl and |G, | relative to the default configuration.
29

---

Table 14: Performance under varying skill and subtask clustering configurations in Kitchen Emergent SIL scenario. All baselines use the same SIL algorithm AA for the decoder. Analysis with varying [ and 2pp in Kitchen Emergent SIL scenario. Here, |9-| denotes the number of subtasks into which expert demonstration data D, segmented and |pl indicates the number of skills derived from the datastream Dp at each phase p.
Kitchen: Emergent SIL
Baselines
Initial
BwSC Unitial)
Overall performance
FW (Synced)
Final
Type
Skill Interface
12pl
[G.|
FWT 19a
BWT e8)
AUC
BWT (s)
(
BWT (s)
AUC
FWT
Sub-poal Bins (PTGM)
60
10
46.7
-0,2a21
46,2-18
10.5
55.3
21.1a17
62,2
76.0
20
45.0aa.
2.6
46.9
15.4
58.2
28.1
66.1408
83.6
40
52.8a16
-0.5.00
52.4
12.9
63.8
26.2+1a
72.4s12
86.4
10
51.6#2.1
3.9
54,5=00
11.615
61.5
19,4
66. 16.1=20
76.8=21
10
49.8
5.0=23
53,5
14.0
61.8-12
23.0
67.1
78.1
40
51.7
4.2
54,9
11.3
61.4
18.4122
65,5 121
74.0
SIL-C
10
48.6
16.0-33
60.6
27.8 14.1
67.1
27.1
68.9
81.3
III
(w/ PTGM)
42.9
18.6
66.8
22.0
71.8
24,4
71.9
87.2
40
52.3
16.1=12
64,6216
20.8
70.4
25.6a05
71.7
86.2
10
55.1
18.8-23
69.2=1.
21.5
73.8
24.2+22
73.3
88.0
40
20
54.0.12
18.8
21.5
73.3
26.1
73.6.11
88.8.:1
40
55.5 0
19.7
10.3
23.4
25
27.1
75.8
90.5
First, we analyze Type (Subgoal Bins). At 2pp = 10, the skill space resolution is too coarse, leading limited overall performance (55.3% AUC). The BwSC' evaluation group shows weak results (46.2%), and the BWT score remains negative. Increasing to 1zpl 20 improves overall AUC to 58. 28,2%% and FwSC evaluation results rise to 66. %, but BWT still indicates poor transfer. At [Zzl = 40, Type I gains further, with 63.8% AUC and 72.4% in FwSC evaluation, yet BwSC remains low (52.4%). These results show that while higher skill space resolution helps Type I, static subgoal binning as the interface is unable to leverage newly added skills without retraining the policy, which reflected by persistently low BWT in the BwSC evaluation group.
When 2,1 = 10, Type III shows clear improvements over Type I. It achieves higher Initial FWT 51.6-51.7%) and stronger AUC in the BwSC evaluation group (up 54.9%). Importantly, the BWT score is positive, indicating that SIL-C enables backward transfer even at coarse resolution, which is not observed in Type I.
When 1,1 = 20, Type III shows consistent improvements over Type I. achieves higher Initial FWT (48.6-52.3%) and stronger results in both the BwSC (up to 68.6%) and FwSC (up to 71.9%) evaluation groups. Importantly, the BWT score increases significantly, confirming that SIL-C leverages moderate resolution more effectively than static binning.
When Ip 40, Type achieves the strongest improvements. It achieves higher Initial FWT (up to 55.5%) and consistently high results in both the BwSC (70.3%) and FwSC (75.8%) evaluation groups. Most notably, the BWT score peaks at 19.7, the best among all configurations. contrast, Type I also benefits from higher resolution but remains far behind, with 63.8% overall AUC, 72.4% in FwSC, 52.4% in BwSC, and negative BWT.
In summary, increasing skill resolution improves both overall performance (AUC) and evaluation results across BwSC and FwSC groups. Moreover, SIL-C (Type III) consistently leverages this advantage to achieve strong skill-policy compatibility, which is not observed in the static subgoal binning bascline (Type I).
D.2 Subtask Space Analysis on Few-shot Imitation Learning
Table 15 expands Table by replacing the prototype-based representation in the SIL-C subtask space with an clement-wise retrieval strategy across 1-5 shot scenarios. Element-wise retrieval consistently yields stronger performance than prototype-bbsed representations.
In the 5-shot setting, for example, it achieves a +4.2% absolute gain in overall AUC (66.6% vs. 62.9%) and +3.3% in final FWT (79.1% vs. 75.8%). This performance gap persists across shot counts, highlighting the benefit of fine- fine-grained memory when the budget allows. Notably, SIL-C supports both representations and can incorporate clement-wise memory when additional storage is available.
30

---

Table 15: Comparison between Gaussian prototypes (Prototype) and simple instance accumulation (Element) the subtask space the Franka Kitchen Emergent skill incremental scenario under varying shot settings (1, 3. 5). Memory per task indicates the number of embedding vectors stored in the subtask space for single task (e.g.. μ or E), with relative usage at 100% corresponding to the configuration in Table
Kitchen: Emergent SII.
Ablations
Initial
BuSC (Juitial)
Overall performance
FwSC (Synced)
Final
Instance
Shots
Memory per Task
FWT (3)
BWT (9)
AUC (9)
BWT (3)
AUC (5)
BWT (3)
AUC (9)
FWT (5)
160 (100%)
_32
55.9
17.5
a2.4
23.6142
63.
75.8
Prototype
3
160 (100%)
44.4-18
11.2425
52.8
14,9
17.2=15
18.6433
58.4112
68.5a44
(100%)
_3
11.9
52.0.51
15.7
26.5
19.4
57.6239
65.7236
5
1000 (625%)
50.3
13.9
60.7.66
20.611A
66.6144
24.3111
68.5
79.1
Element-wwie wise
600 (375%%)
48.9 -00
15.6
60.61:25
19.3
65.4
23.0
66.1
77.3
200 (125%)
42.3
8,3
48.5144
14.6124
21.0118
58.0.19
64.6112
D.3
Robustness to Observation Noise
We extend the analysis in Table Bl which compares SIL-C against PTGM+AA across different noise levels with Gaussian noise scaled by factors of x to x5 in Franka Kitchen [65].
SIL-C maintains compatibility under noise. As shown in Table 3 SIL-C consistently outperforms PTGM+AA across all noise levels and evaluation metrics. Most notably, SIL-C maintains positive backward transfer (BWT) even under extreme noise (x5: 4.3% vs. -1 -1.2%), while PTGM+AA suffers from catastrophic forgetting with negative BWT. This gap becomes more pronounced in the final phase with approximately 80 skills, where SIL-C achieves 23.1% 1 FWT compared to 15.9% for PTGM+AA under x5 noise.
Trajectory matching provides noise resilience. The robustness gap stems from fundamental ar- chitectural differences. Under noise, high-level policies produce increasingly unreliable subtask selections. The bilateral lazy learning interface in SIL-C handles this through: (i) skill validation that detects -iitt--diitiiuution subtasks, and (ii) skill hooking that remaps to appropriate skills based on trajectory similarity. The static skill mapping of PTGM+AA, however, cannot adapt to noise-induced distributional shifts, leading to compounding errors long-horizon tasks.
Performance scales with skill repertoire size. The advantage of SIL-C grows as training progresses. Under x3 noise, the performance gap increases from 0% (Initial FWT: 35.6% vs. 35.8%) to 12.4% (Final FWT: 52.9% vs. ). This scaling effect suggests that skill validation and hooking become more valuable as the skill library expands, opening the possibility for more robust lifelong learning systems that can maintain performance even in challenging real-world like conditions.
D.4 Analysis Distance Metrics for Instance-based Classification
Table 16: Comparison across distance metrics (Mahalanobis, Euclidean) and sub-cluster selection strategies (Fixed, Auto) the Franka Kitchen Emergent Scenario. We define 100% as the configura- tion matching the number and memory usage of sub-clusters per prototype used in Table Fixed uses a predetermined number of sub-clusters, while Auto determines the number of sub-clusters using the silhouette score. The left columns report initial forward transfer (Initial) and backward skill compatibility (BwSC), which evaluate existing policies with updated skills. The right columns report forward skill compatibility (FwSC), assessing the effectiveness of new policies with learned skills, and final forward transfer (Final). The central columns indicate overall performance, combining Bw:CC and FwSC.
Kitchen: : Emergentt SH.
Baselines
Intriat
RwSC (Juitial)
Overall perfomance
PwSC (Symeed)
Type
Distance Type
Sub-clasters prototype
FWT 99
BWT c9
AUC())
BWT (9)
AUC (9)
BWT (9)
AUC
FWT (R)
Mahulanobis
Fixed K., K, 4.0 (100%)
52.9.39
18.6132
66.8124
22.0
71.8113
71.9
87.2
III
Auto Kg, Kg ss 2.5 (63,5%)
52.7
166e25
65.1229
19.6234
69.5
22.7
69.7a14
80.6
Eaclidean
Fixed Ks, K2 4.0 (50%)
52.2
5.5323
56,3551
14.3
64,5an1
23.2
69.6
81.0136
K, (100%)
54.0=26
5.6=18
14.4119
23.1
82.9-2
We ablate the choice of distance metric in the costance-based classifier, comparing Mahalanobis and Euclidean distances with varying memory configurations shown in Table 16 Memory usage
31

---

is measured relative to storing 4 sub-clusters with both mean and covariance per prototype (100% baseline).
Mahalanobis distance enables better trajectory matching, Across all configurations, Mahalanobis distance consistently outperforms Fuclidean distance. With the baseline configuration, Mahalanobis achieves 66.8% AUC in BwSC while Euclidean only reaches 56.3%, 10.5% gap. This difference is most pronounced in backward compatibility, where Mahalanobis maintains 18.6% BWT compared to 5.5% for Euclidean, demonstrating superior ability to align evolving skill distributions with existing policies.
trade-off with automatic clustering. When using automatic sub-cluster selection via silhouette score, the Mahalanobis variant reduces memory to 63.5% while maintaining competitive performance (65.1 % vs. 66.8% AUC in BwSC). This 3.2% performance drop for 36.5% % memory savings presents viable option for essurce-constraing significantly outperforms Euclidean variants even with reduced memory.
deployments, the SIL.-C still
Richer skill distribution modeling improves compatibility. The performance gap between distance metrics demonstrates the importance capturing skill distributions accurately. Euclidean distance with 4 sub-clusters (50% memory) achieves only 56.3% AUC in BwSC, and even doubling to 8 sub-clusters (100% memory) reaches just 58.2% AUC. In contrast, Mahalanobis distance with 4 sub- clusters achieves 66.8% AUC despite using diagonal covariance for efficiency. This 8.6%-10.55 gap shows that modeling variance, even in simplified diagonal form, is more effective than simply adding more isotropic prototypes. The Euclidean approach scales poorly because increasing prototypes cannot compensate for the lack of directional information, while Mahalanobis captures essential trajectory variations that distinguish functionally different skills. This enables more accurate skill validation and hooking, particularly critical for backward compatibility where precise distribution matching determines whether existing policies can leverage new skills.
D.5 Analysis on Instance-based Classifier Thresholding
Figure shosw the stability of confidence intervals used for threshold selection in the Emergent SIL sce- nario. Overall, varying the threshold between 80% and 99% results in minimal differences in both the confidence intervals and performance. This indicates that our bounding method does not simply adjust the threshold for skill validation, but instead serves as an effective mechanism for filtering out out-of- distribution cases.
Synced
72.5
70.0
Overall
67.5
Initial
62.5
80
85
90
95
99
Confidence interval
Figure 10: Ablation on SIL-C threshold
D.6 Memory and Time Analysis
Memory. Each instance-based classifier stores Gaussian prototypes in both skill and subtask spaces. The memory cost for skill prototypes is dim x |z,l x Kg x 2 float values, and for subtask prototypes it is dim x 19-1 K2 x 2. Here, dim is the dimension of the state or subgoal embedding. Each (p. Σ) pair is stored using diagonal covariance matrix. K₂ and K2 denote the average number of sub-clusters per skill and subtask, respectively.
In the Kitchen Emergent and Explicit skill incremental settings with dim = 60, skill prototypes occupy 150KB and subtask prototypes occupy 37.5KB, shown in Table [ In the Meta-Worla benchmark with dim = 140, the memory usage increases to 350KB for skill prototypes and 87,5KB for subtask prototypes.
Time. Each Mahalanobis distance computation with diagonal covariance requires x dim floating point operations: dim subtractions, dim squarings, dim divisions, dim 1 additions, and one square root. For dim = 60, the total 240 FLOPs per (p, E) pair.
In the best case, proposed is directly accepted as the skill during the skill validation process using a selected subgoal from the subtask space. This process requires (19-) x K2 + 1) x 240 FLOPSs In the worst case, 2n is rejected during skill validation, and skill booking requires selecting candidate skills 2' and scoring them to choose the appropriate skill 2t. The total computational cost is up to
{(G))) x Kg + 1) (2 x Zpp x K₃ + 1)} 240 FLOPs.
32

---

Table 17: Mean and variance (ms) of evaluation function timing in the Kitchen Emergent Skill Incremental scenario. kbls denotes the task with the following sequence: move kettle, turn on bottom burner, light switch, open slide cabinet, and bils denotes: turn on bottom burner, turn on top burner, light switch, open slide cabinet.
Baselines
kbls-Initial
btls-Initial
kbls-Final
btls-Final
Type
Method
Mean
Var
Mean
Var
Mean
Var
Mean
Var
PTGM + AA
28.089
2.042
28.117
1.784
27.693
1.978
28.052
2.191
SIL-C+ (PTGM + AA)
29.042
2.341
27.782
2.324
28.787
2.732
28.538
2.320
Table 17 reports the inference time (mean and variance) of the hierarchical model, consisting of the policy, interface, and skill decoder, in the Kitchen Emergent Skill Incremental scenario. All timings were measured on an AMD Ryzen 9 7950X3D CPU with single NVIDIA RTX 4090 GPU, running Ubuntu 22.04, CUDA 12.4, and driver version 150.04 The Initial phase (p = 1) corresponds reflects
the evaluation after 20 skill prototypes were generated. while the Final phase (p evaluation after the final phase with a total of 80 skills. Compared to Type I, the mean inference time of Type increased by 0.953 ms and 1.094 ms in the kbls-Initian and kbls-Final respectively. In the brls-Initial, the mean time decreased by 0.335 ms, while in the btls- Final, it increased by 0.486 ms. While greater number of skills generally leads longer retrieval time, the additional overhead was minimal compared to overall policy and skill decoder inference time, and variation due to runtime system state was more prominent practice.
33

---

NeurIPS Paper Checklist
Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction explicitly state the motivation and objective of SIL--C and outline three core contributions corresponding to the methods and experiments.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We outline future work that aims to address current limitations by incorporating external feedback, particularly in light the trade-off observed between generalizability and compatibility.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
Theory assumptions and prool's
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
34

---

Answer:
[NA]
Justification:
We support our method by showing experimental results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Ys]]
Justification: Implementation details, scenarios, metrics and datasets described in Sec- tions 5-6 and Appendix A-D; also abstract provide the Git repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, descríbing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
the contribution is new model (e-g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
Open access to data and code
35

---

reproducible, they Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: A public anonymized Git repository is provided in abstract which contains code and instructions. Experimental settings are also discussed throughout section 5-6 and appendix.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/tips.cccpuulicggu...d
www.subiiissoooooooooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permirted.
Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Section 5 and Appendix A give datasets, hyper- paper-pprameters, continual learning methods and evaluation metrics.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, or as supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Ys]]
Justification:
We report our results with variance, using least four seeds for each experi-
ment.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
36

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time execution) needed reproduce the experiments?
Answer: [Yes]
Justification:
We provide them in appendix
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
We adhere to the NeurIPS Code of Ethics in every respect.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: Yes]
Justification:
We discuss the the potential impacts in the conclusion section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
37

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e-g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
Safeguards
Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (c.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
This work is not expected to pose any significant risk of negative societal impact thus explicit safeguards for responsible data or model release were considered unnecessary.
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All baselines and datasets are cited in the References section. Additional information, including license and version details, will be provided the appendix.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
38

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, latasets,papersstt pwp.pppppssiiaco
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: No new assets were introduced in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
Not applicable to this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Not applicable to this paper.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
39

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: Not applicable to this paper.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: /ttp:::///eurps... http///rrrssscc/ooofeeeeee////// Cc/Connferences/2225//LI. for what should or should be described.
40


Reviews:
[{"review_id": "7cmpH6VOun", "reviewer": "Reviewer_PQiK", "summary": "The paper presents a method of Skill Incremental Learning by introducing a lazy learning interface that connects subtasks to skills and a policy integration mechanism that enables seamless integration of learned skills to existing downstream policies. The key problem that this method tries to tackle is the forgetting of past skills as well as the usefulness of newly added skills for downstream tasks. New skills at different phases is determined using clustering. The paper evaluates this method on both backwards and forwards compatibility; it outperforms the baselines in overall performance and the forward compatibility, and performs closely to methods with predefined skill labels.", "questions": "- Line 95: In what cases would $\\psi$ not be the identity map and why?\n- Line 99-100: Would $\\mathcal{T}_p$ depend on the dataset $\\mathcal{D}_p$ that is presented to the agent? I.e., would the set of evaluation tasks be setup to best use the dataset, or would it be based on some other criterion (if any)?\n- Line 101 \"... assume access to task-level expert demonstrations...\" -> Does this mean that the agent gets examples of how to incorporate the newly acquired skills, rather than learning whether they are useful or not itself?\n- Line 125: Is there a difference between a \"subgoal\" and a \"subtask\" in the context of the paper?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- Generally well-written and with reasonably clearly written motivations and method. See minor comments below for improvements with the exposition. \n- Good literature review.\n- The method is an interesting idea.\n- A good range of experiments were carried out with a good number of baselines.\n- Good ablation study on removing the lazy learning interface.\n## Weaknesses\n- I think that the paper could use another small iteration where some improvements to clarity are made (see suggestions below and Questions).\n#### Typos and suggestions\n- Line 27: \"Yet, a critical challenge remain**s**...\" (remain -> remains).\n- Line 95: \"... may be the identity...\" -> \"... may be the identity **map** ...\"; I think adding map makes it clearer.\n- Line 100: Please define what $\\tau$ is right next to $\\mathcal{D}\\_\\tau$ rather in line 102. E.g., \"Let $\\tau$ denote a task; we assume access to task-level expert demonstrations $\\mathcal{D}_\\tau$ ...\".\n\n- Line 103: Could you add \"Our objective is to find the collections of high- and low-level optimal parameters over all phases, $(\\Theta\\_h\\^\\*, \\Theta\\_l\\^\\*)$ ...\". This is so that it immediately clear what $(\\Theta_h^\\*, \\Theta_l^\\*)$ are without having to wait till the end of Equation 2.\n\n- Section 4 paragraph 1: Could the appropriate mathematical notation for terms such as \"subtask space\", \"skill space\", \"task space\", etc.\n\t- The same in Section 4.2 paragraph 1. \n- Section 5: Can the number of repetitions be added to the captions of tables with $\\pm$ values.", "was_revised": true, "final_justification": "I am happy with maintaining my score provided that some of the responses to my questions are discussed in later revisions of the paper (if it proceeds). I believe that this is a strong, interesting paper.", "rebuttal": "We thank the reviewer for the clear summary and for recognizing the main contributions of our work. We also appreciate the positive comments on our method, writing, and experiments.\nIn addition, we are grateful for the questions, which reflect a solid understanding of our approach and raise important points about its broader applicability. These help us clarify the motivation and design choices behind our method.\n\n---\n> W1.  Clarity improvements.\n\nWe will incorporate the proposed suggestions across Sections 1 to 5 to improve clarity and precision throughout the paper.\n\n* Specifically, we will clarify the notation for \"subtask space\" and \"skill space\" as $\\mathcal{X}\\_{h}$ and $\\mathcal{X}\\_{l}$, respectively.\n* In addition, we will revise the term \"task space\" in line 112 to explicitly denote *subtask space*, to avoid ambiguity.\n* Regarding the reviewer's suggestion on table captions with $\\pm$ values, we confirm that the reported values reflect the mean and standard deviation across 4 different random seeds (repetitions). We will revise the main table captions to explicitly indicate this for clarity, with this information currently stated in Appendix B.5.\n\n---\n> Q1. In what cases would $\\psi$ not be the identity map and why?\n\nThe mapping function $\\psi$ is introduced to reconcile the representational discrepancy between the high-level policy, which operates over a subtask and the low-level skill decoder, which consumes inputs from a skill.\n\nAlthough $\\psi$ reduces to the identity function when both modules are co-designed to share an ideally aligned latent space, such alignment is not guaranteed in practical settings. This is especially true in lifelong learning scenarios, where policies must continually adapt to evolving conditions. Even in co-designed systems, factors such as distributional shifts, task-specific embedding variations, or implementation-level inconsistencies can still require a non-trivial mapping.\n\nThe need for a non-identity $\\psi$ becomes particularly evident when the high-level and low-level modules are trained asynchronously or with different algorithms. In such cases, their internal vocabularies may drift apart. Two common situations where this occurs are:\n* In related works such as PTGM, the high-level and low-level policies are trained independently, often using distinct objectives and datasets. The high-level policy typically outputs a symbolic subtask (e.g., a behavior embedding), while the low-level policy expects a subgoal state as input. A hash-based lookup function $\\psi$ is commonly used to bridge this semantic gap. \n* In lifelong learning scenarios, the alignment between the high-level output space and the low-level input space may shift over time. These shifts can occur even when task semantics are preserved, as a result of ongoing learning or environmental changes. In such cases, assuming $\\psi$ is the identity function becomes invalid, and updating the mapping is necessary to maintain effective coordination between components.\n\n---\n> Q2. Would $\\mathcal{T}_p$ depend on the dataset $\\mathcal{D}_p$ that is presented to the agent?\nI.e., would the set of evaluation tasks be setup to best use the dataset, or would it be based on some other criterion (if any)?\n\nFor, problem formulation, the set of evaluation tasks $\\mathcal{T}_p$ is independent of the dataset $\\mathcal{D}_p$ used for skill incremental learning. Most incremental learning settings assume that future tasks are unknown in advance, and our problem formulation reflects this assumption. \n* For skill-policy compatibility evaluation, as noted in Appendix B.1.2, in our experiments we selected 24 tasks from the base tasks included in the Franka-Kitchen mixed dataset. This setup assumes that all skills have been sufficiently learned by the final phase of SIL, such that successful execution of these tasks is reasonably expected. This design choice is intended to evaluate skill-policy compatibility , under the assumption that learned skills can be appropriately applied to the evaluation tasks. \n* Including unseen tasks or tasks that are inherently more difficult to learn could introduce confounding factors, making it unclear whether performance degradation is due to limitations in skill-policy compatibility or simply due to task difficulty. \n* Therefore, structuring the evaluation tasks based on the dataset is a deliberate choice to ensure clarity in evaluation results.\n\nAs a possible real-world use case that can arise in the producer-consumer deployment configuration for these hierarchical agents:\n* Here, the producer provides a task-agnostic pre-trained model, i.e., trains the skills, while the consumer fine-tunes a modular policy to make it task-specific.\n* In such scenarios, building on our evaluation protocols for FwSC and BwSC, the construction of $\\mathcal{T}_p$ could incorporate objectives such as maximizing consumer rewards in task composition.\n* Likewise, clearly formulating the problem of skill-policy compatibility and designing $\\mathcal{T}_p$ accordingly is essential for ensuring a robust and meaningful evaluation.\n\nIn practical skill incremental learning scenarios, factors like domain shift or noisy skill distributions may arise. Extending our evaluation framework to assess generalization under such conditions, such as evaluating performance on tasks with perturbed state-action transitions, would be valuable. We provide an example of such experimental scenarios in our response to `JDZP Q2`, which illustrates the model’s performance under these challenging conditions.\n\n---\n> Q3. \"... assume access to task-level expert demonstrations...\" -> Does this mean that the agent gets examples of how to incorporate the newly acquired skills, rather than learning whether they are useful or not itself?\n\nYes. In our work, we assume that the agent learns from training data that contains objective about the task which to be accomplished. The provided expert demonstrations implicitly contain examples of how to compose the relevant skills, either in a dense or sparse manner. Importantly, these demonstrations are not used directly during evaluation or after the policy and subtask prototypes have been learned.\n\nTo relax and generalize this assumption, we designed the few-shot imitation learning experiments in Table 2. Specifically, in addition to full demonstration trajectories, we include sparse examples obtained by downsampling the trajectories to partial versions (e.g., 50% and 20% of the original trajectory). These partial trajectories are often insufficient for full behavior cloning. Despite the limited information, the results in Table 2 clearly show that SIL-C performs effectively under these constrained conditions.\n\nTo address the reviewer's insightful question regarding the generality of our framework, we provide an extended discussion in the Future Directions section. Based on our current formulation, we believe several meaningful extensions are possible, opening up exciting avenues for future research:\n* **Minimal task supervision**: SIL-C could be extended to more weakly supervised settings, potentially even to scenarios where only a high-level goal is provided. In such cases, the agent could learn to chain prototypes within the interface to infer plausible skill execution paths without explicit demonstrations.\n* **Skill usage analysis for practical deployment**: SIL-C could also be leveraged as an analysis tool in large-scale deployment scenarios. By tracking how often and in what contexts certain skills are referenced, it could serve as a guide for data collection or for continual adaptation of skill libraries.\n* **Safety-oriented skill validation**: The Gaussian prototype structure also allows for potential safety extensions. For example, it may help prevent the execution of unsafe or unvalidated skills in real-world environments by identifying anomalies in prototype activations.\n* **Reward-based skill hooking without task demonstrations**: Similar to other skill-based learning methods, reinforcement learning acceleration can be indirectly observed through improved FwSC. However, as discussed earlier, achieving BwSC generally requires access to task-level information, since the agent must know what it is trying to improve. That said, it would be interesting to explore reward-driven skill hooking, where the agent hooks into relevant skills purely from learning signals, even without any demonstration data. This direction may prove valuable for developing more autonomous lifelong embodied agents.\n\nFurther clarifications on the motivation behind our architectural design and its implementation can be found in our responses to the following questions:\n1. Motivation for Architecture Design - `JDZP W1 Q1`\n2. Connection between motivations and implementation - `39nR W2`\n\n---\n> Q4. Is there a difference between a \"subgoal\" and a \"subtask\" in the context of the paper?\n\nYes, there is a difference. A subtask refers to an abstract unit of behavior selected by the policy, whereas a subgoal is a concrete intermediate state that the agent aims to reach in order to carry out that subtask."}, {"review_id": "Tj9Iir59dx", "reviewer": "Reviewer_39nR", "summary": "In this paper, the authors introduce SIL-C, a novel framework for Skill Incremental Learning (SIL) that maintains compatibility between evolving skills and downstream hierarchical policies. That is, SIL-C addresses the challenge of ensuring that updates to low-level skills do not invalidate or require retraining of high-level policies. The proposed approach decouples policy learning from skill updates using a bilateral lazy learning interface, which dynamically maps high-level subtasks to low-level skills based on trajectory similarity. This interface allows for both forward and backward skill compatibility; that is, (1) the newly added skills will be used effectively when learning future policies, and (2) existing policies can, when applicable, use the newly added skills without re-training. The proposed method is evaluated on two different environments and compared to relevant existing techniques.", "questions": "1. What are possible alternatives to the lazy learning interface proposed in SIL-C? For example, could a learned (parametric) subtask-to-skill mapping perform similarly or better in practice? It would be helpful to understand the motivation for using a lazy, instance-based approach over other alternatives, and whether this choice contributes significantly to performance.\n2. How sensitive is SIL-C to the quality of the unsupervised clustering used? The performance of the method seems highly dependent on the clustering results, but there is little discussion or ablation on this. How were hyperparameters (e.g., number of clusters, clustering algorithm) selected, and how robust is SIL-C to suboptimal clustering outcomes?\n3. How many random seeds were used for the experiments presented in Tables 1 and 2? The reported standard deviations suggest significant variance, so knowing the number of seeds is important to assess statistical significance. Were statistical tests performed to validate the significance of SIL-C’s improvements over baselines?", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper addresses an important and underexplored challenge in lifelong learning: how to maintain compatibility between evolving low-level skills and higher-level task policies. The ability to incrementally integrate skills without retraining existing policies is an important contribution with practical implications for robotics and hierarchical RL.\n\nThe paper is generally well-written, and the motivation is clearly laid out. Figure 2 is helpful in conveying the structure of the proposed system, but it is dense. I believe the paper could benefit from a high-level pseudocode or flow diagram that lays out the full execution and learning process (e.g., when are classifiers updated? how is skill hooking triggered?)\n\nThe architecture introduces significant complexity (e.g., bilateral lazy-learning interface, trajectory similarity computations, prototype memories, etc.), but the design choices are not clearly justified. There is very little intuition/high-level discussion and description of the elements, and there are no ablations or comparisons to simpler or parametric alternatives, making it unclear whether the full complexity is necessary or beneficial.", "was_revised": true, "final_justification": "Most of my concerns were addressed, and I believe this paper would be a nice contribution to the field. I increased my score, and I expect the authors to include some of the discussions from the rebuttal into the final version of the paper (if accepted).", "rebuttal": "We sincerely appreciate the reviewer’s time and effort in evaluating our work, particularly their recognition of the practical importance of maintaining skill-policy compatibility in lifelong learning agents, along with the constructive suggestions to improve clarity and the thoughtful remarks that highlight the significance of our contribution.\n\n---\n> W1. Paper could benefit from a high-level pseudocode or flow diagram that lays out the full execution and learning process (e.g., classifier updates, skill hooking), and Figure 2 is dense.\n\nIn the revision, we will add a high-level pseudocode before the current Algorithm in Appendix A.2 to clarify the overall execution and learning process, including the flow of inputs and outputs across modules. We will also simplify Figure 2 into a flow diagram aligned with the pseudocode to improve clarity. \n\nThe pseudocode is as follows:\n```python\n## Initialization\nfrom collections import defaultdict\nfrom scipy.stats import chi2\n\nInterface = {\n\t\"skills\":   defaultdict(lambda: {\"subgoal\": [], \"state\": []}),\n\t\"subtasks\": defaultdict(lambda: defaultdict(list)),\n} # Gaussian component = (mean, precision)\nπ_l, π_h = None, {}\n\ndef label_dist(x, comps): # Eq.(5)\n\tdef maha(x, comp):\n\t\tμ, Σ_inv = comp\n\t\td = x - μ\n\t\treturn d.T @ Σ_inv @ d\n\treturn min(maha(x, c) for c in comps)\n\ndef χ2_thr(comps, conf=0.99):\n\tdim = comps[0][0].shape[0]\n\treturn chi2.ppf(conf, dim)\n\n## Policy compatible skill incremental learning\nfor p in range(1, P + 1):\n\t# Skill space update Eq.(6)\n\tχ_gz, χ_sz = create_skill_prototypes(cluster_skills(𝓓_p))\n\tfor z in χ_gz:\n\t\tInterface[\"skills\"][z][\"subgoal\"].extend(χ_gz[z])\n\t\tInterface[\"skills\"][z][\"state\"].extend(χ_sz[z])\n\t# Skill incremental learning\n\tπ_l = train_low_level_decoder(𝓓_p, Interface[\"skills\"], π_l)\n\n\t# Downstream policy learning\n\tfor τ in 𝓣_p:\n\t\t# Subtask space update Eq.(7)\n\t\tχ_g = create_subtask_prototypes(cluster_subtasks(𝓓_τ))\n\t\tfor g in χ_g:\n\t\t\tInterface[\"subtasks\"][τ][g].extend(χ_g[g])\n\n\t\t# Policy learning on the interface Eq.(8)\n\t\tlabels = assign_subtask_labels(𝓓_τ, Interface[\"skills\"], π_l)\n\t\tπ_h[τ] = train_high_level_policy(𝓓_τ, labels)\n\n## Policy inference via the interface \ndef infer_action(s, τ, conf=0.99):\n\t# High-level policy: subtask proposal\n\tz_h = π_h[τ](s)\n\tg   = predict_subgoal(s, Interface[\"subtasks\"][τ])\n\tδ   = χ2_thr(Interface[\"skills\"][z_h][\"subgoal\"], conf)\n\n\t# Interface: skill validation and skill hooking Eq.(9)\n\tif label_dist(g, Interface[\"skills\"][z_h][\"subgoal\"]) <= δ: \n\t\tz_l = z_h\n\telse: \n\t\tcand = [z for z in Interface[\"skills\"]\n\t\t\tif label_dist(g, Interface[\"skills\"][z][\"subgoal\"]) <= δ] + [z_h]\n\t\tz_l  = min(cand, key=lambda z:\n\t\t\tlabel_dist(s, Interface[\"skills\"][z][\"state\"]))\n\t\t\n\t# Low-level decoder: action execution\n\treturn π_l(s, z_l)\n```\n---\n> W2. The architecture introduces significant complexity (e.g., bilateral lazy-learning interface, trajectory similarity computations, prototype memories, etc.), but the design choices are not clearly justified.\n\nWe clarify the detailed connection between the motivations and the implementation in the SIL-C design, aiming to support modularity, compatibility, and efficiency as follows:\n* **Motivations**\n  * To ensure **modularity** between pre-trained and adapted modules for long-term maintainability (e.g., in a 1:N producer-consumer configuration) in lifelong agents operating in complex decision-making scenarios (e.g., long-horizon tasks).\n  * To ensure **compatibility** and efficiency in the interaction between pre-trained and adapted modules during deployment.\n  * To explore mechanisms for improving agent performance **without retraining**, beyond merely maintaining the performance of the existing policy.\n\n* **Architecture design** \n  * **Hierarchical skill-based agent**: This structure supports both the reuse of pre-trained knowledge and the composition of new behaviors, aligning with practices in robotic learning for complex task solving.\n  * **Bilateral interface module**: This component connects high-level subtask specifications to low-level skill modules (both adapted and pre-trained), ensuring compatibility and controlled information exchange.\n  * **Lifelong learning interface**: This design supports efficient incremental adaptation in scenarios where future data distributions are not fully known, providing flexibility for unforeseen tasks. \n\n* **Implementation** \n  * **Skill representation**: Skills are represented using trajectory information from the streamed dataset, segmented into distinct skill groups via unsupervised clustering and encoded as skill prototypes.\n  * **Trajectory matching**: To match a given subtask with a suitable skill, we compute trajectory similarity through an instance-based classification process that aligns subtask prototypes with skill prototypes, determining executability via distance metrics and out-of-distribution rejection.\n  * **Instance-based classifiers for lazy learning interface**: Due to the unpredictability of future tasks and the high variability of behavior, learning a generalizable mapping function is challenging. To address this, we avoid irreversible updates and instead use a lazy learning approach with bilateral instance-based classifiers. For efficiency, each skill's trajectory is compressed into skill prototypes derived from sub-clusters (e.g., selecting subgoal states closest to the average across transitions), enabling the similarity estimation to be reformulated as instance-based classification with dynamic alignment.\n\nTo address the reformulated classification task and enhance performance, we refine the instance-based classifier as follows:\n* **Prototype memories**: Each prototype comprises multiple sub-clusters to support fine-grained decision boundaries, as motivated by prior works [1,2,3]. Performance improvements are shown in Table 12 (Appendix C.3).\n* **Gaussian mixture and sub-clusters**: Prototypes are modeled as Gaussian mixtures, with retrieval based on Mahalanobis distance to capture dense and expressive distributions. This enables distribution-aware similarity, enhances interpretability, and supports out-of-distribution detection crucial for scalable matching in lifelong agents.\n\n---\n> Q1. What are possible alternatives to the lazy learning interface proposed in SIL-C? Could a learned (parametric) subtask-to-skill mapping perform similarly or better in practice?\n\nA key requirement of a subtask-to-skill mapping for lifelong learning agents is common contract, which ensures the validity of selected subtask and supports dynamic relabeling at inference time. A simpler implementation of such an interface is a supervised-learned parametric mapping, which involves training on labeled subtask-skill pairs. However, this approach presents several limitations:\n* Parametric subtask-to-skill mapping\n  * Infeasible as ground-truth subtask-skill pairs are rarely available.\n  * Even if such labels are available, training requires coordination between high-level and low-level modules, undermining their modularity.\n  * As adding new skills typically requires aligning with subtasks, reintroducing retraining that the interface is intended to avoid.\n* Parametric classifier for SIL-C framework.\n  * Additional parametric classifiers for skill validation and hooking introduce extra complexity, particularly for class incremental learning on the skill incremental learning.\n  * Parametric knowledge is difficult to decouple once learned, which hinders maintainability and efficiency in tasks like creating, updating, and deleting skills over time.\n\n---\n> Q2. Provide discussions of dependency of SIL-C on unsupervised clustering quality and analysis and hyperparameter selection justifications.\n\nAs shown in Figure 8 (Appendix C.2), we observe that performance generally improves as the number of clusters increases, particularly during both the SIL and policy learning stages. This suggests that finer clustering leads to a more diverse and expressive skill set, contributing to improved policy learning and task completion.\n\nOur hyperparameter selection process followed principled steps based on both prior literature and empirical findings:\n* We initially adopted skill clustering setting from the PTGM [4] baseline (Type 1), which uses 50 clusters for entire dataset. For phase-wise skill learning, we divided these into 4 temporal segments, yielding 12 skills per phase.\n* However, this configuration produced suboptimal results. Based on observations from prior work [4] that performance improves with larger cluster sizes, we empirically increased the number of clusters to 20 skills per phase.\n* For fair comparison, SIL-C adopted the same underlying clustering configuration and algorithmic hyperparameters with Type 1 baselines.\n\n---\n> Q3. How many seeds were used for Tables 1-2, and were statistical tests conducted to confirm SIL-C's improvements?\n\nWe used four random seeds for the experiments presented in Tables 1 and 2, and this detail will be added to each table caption for clarity.\n\nWe performed p-value tests to validate SIL-C's gains.\n* SIL-C matches or outperforms baselines without using skill labels, even in settings where future skills are unknown.\n* Emergent SIL scenario in Table 1 shows statistically significant improvements over all baselines except those using explicit labels, where the difference is less significant (p > 0.05).\n\n[1] Wistuba, Martin, et al. \"Continual learning with low rank adaptation.\" arXiv preprint. 2023.\n\n[2] Lee, Daehee, et al. \"Incremental learning of retrievable skills for efficient continual task adaptation.\" NeurIPS. 2024.\n\n[3] Wang, Yanjie, et al. \"High-dimension Prototype is a Better Incremental Object Detection Learner.\" ICLR. 2025.\n\n[4] Yuan, Haoqi, et al. \"Pre-training goal-based models for sample-efficient reinforcement learning.\" ICLR. 2024."}, {"review_id": "b2g7x5YiCI", "reviewer": "Reviewer_UFGb", "summary": "Build up an incremental skill library by applying append-only skill updates and versioning, as inspired by filesystems. This involves a high-level policy passing subgoals to a low-level policy according to a temporal distance estimator, where the low-level policy selects between skills. Each module uses memory as the parameterization, and consists of classifiers that output scores for different selections of skills and goals. The skills are added to by clustering the new data, and described using an energy-based function over the primitive actions. Recovering policies involves selecting the lowest-scoring low-level skill. This method is then evaluated on Franka Kitchen and meta-world environments.", "questions": "The lack of clarity for the policy hooking, lazy learning, memory usage, classifiers and other components makes it hard to assess this work properly.", "limitations": "See weaknesses", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\nThe challenge of building up skills without losing current capabilities is fundamental to lifelong learning.\n\nThe method draws interesting connections and evokes interesting ideas related to saving skills.\n\nThe evaluation environments are challenging\n\n**Weaknesses:**\n\nThere appears to be a typo in the intro line 48.\n\nWhile the method is a fairly sizable system, Figure 2 does not clearly articulate the core premise of the system, that is, what data it expects to receive and how it should transform that data into an appropriate set of skills. While this may be contained in the figure, it is obfuscated by the layers of complexity. \n\nEither the problem statement for Figure 2 is inadequate for describing how certain components are being used, especially how the memory system for both states and skills works, and what the inputs and outputs of the modules are. While it is possible to make assumptions from other work, it is not clear because of the way these components are expressed.\nIt is also not clear what is done if a query-label pair is rejected (line 140). This is because the concept of skill hooking has not been introduced yet. \n\nThroughout the paper, the ordering is confusing, since many of the terms used are not clearly explained, nor are they common practice. For example, lazy learning and skill hooking are terms used but not explained, even intuitively, let alone formally, and without any reference to when they will be described in detail.\n\nSeveral key issues with adding skills incrementally are not directly addressed: What happens to the skill space when there are too many skills, so that clustering and classification become infeasible? How do the classifiers both scale and handle errors? Is this method able to improve existing skills or transfer them, since this is another one of the core promises of incremental skill learning, and if not, is this a core issue with the philosophy of keeping old data? How does this method apply to settings where some of the data may be suboptimal? \nThe description of lower score being better is confusing because score is typically higher is better. \n\nIn like 182, it does not make sense to introduce new skills by training the skill decoder, since the skills are introduced by modifying the environment. There must be some sequence of skills to describe when the values will come out. \n\nAs far as I am aware, the baseline methods are much simpler. It seems important to guarantee that the core principles behind SIL-C are the source of its advantage, and not just the presence of more hyperparameter tuning. However, there are no clear descriptions of what the core principles of this method might be, and the ablative experiments removing different components are terse and do not provide much insight into how the results imply that the component adds benefit. For policy hooking, this is because it is not clear why random should do better than a policy, and varying the classifier does not help suggest why the classifier itself is useful.", "was_revised": true, "final_justification": "The writing is improved with the author's clarifications", "rebuttal": "We thank the reviewer for the time and effort invested in carefully reading not only the main manuscript but also the appendix. We truly appreciate the detailed and constructive feedback, which clearly identifies points for improvement and gives us the opportunity to address them. The reviewer’s thoughtful comments on clarity and presentation are especially valuable in enhancing the overall quality of the paper, and we have carefully reflected on these suggestions with a strong commitment to improving the paper.\n\n---\n> Core claims and motivating experiment of SIL-C\n\nTo enable lifelong learning in a modularized agent with a hierarchical architecture capable of complex decision making, we identify the following key challenges:\n* **Forward Skill Compatibility (FwSC)**: ensures that future downstream policies can effectively utilize newly added skills.\n* **Backward Skill Compatibility (BwSC)**: ensures that existing downstream policies can continue to use newly added skills **without requiring re-training of each module** and, when applicable, benefit from improved policy performance.\n\nTogether, these two principles not only aim to preserve the agent’s existing capabilities during Skill Incremental Learning (SIL) but also **enhance the performance of existing policies** by enabling the effective integration of newly added skills.\n\nTo motivate our problem setting, we highlight that even multi-task learning, often considered an oracle in lifelong learning, fails to resolve this issue through the extended Kitchen Emergent SIL scenario results in Table 1.\n|Baselines|||Initial|BwSC||Overall||FwSC||Final|\n|-|-|-|-|-|-|-|-|-|-|-|\n|Skill Interface|SIL Algo.|Replay Ratio|FWT|BWT|AUC|BWT|AUC|BWT|AUC|FWT|\n|PTGM|ER|10%|46.8±2.5|-19.7±4.4|32.1±2.9|-5.0±2.8|42.5±1.4|9.7±3.3|54.0±1.8|57.7±3.6|\n||ER|50%|48.2±1.9|-9.2±2.2|41.3±2.0|5.8±1.8|53.1±1.8|20.8±1.9|63.8±1.7|74.7±2.2|\n||ER (multi-task)|100%|48.4±1.4|-3.1±1.8|46.1±0.9|11.6±0.9|58.4±1.2|26.2±0.9|68.1±1.7|81.3±4.5|\n|SIL-C (ours)|ER (multi-task)|100%|52.7±2.8|15.2±2.8|64.1±2.6|19.7±2.1|69.6±2.1|24.3±1.7|70.9±1.7|82.7±0.7|\n|\n|PTGM|AA|0%|45.0±3.1|2.6±0.9|46.9±3.7|15.4±1.7|58.2±1.7|28.1±4.1|66.1±0.6|83.6±3.1|\n|SIL-C (ours)|AA|0%|52.9±2.9|18.6±1.2|66.8±2.6|22.0±2.4|71.8±1.3|25.4±4.0|71.9±0.9|87.2±3.2|\n\nWe evaluate simple baselines (PTGM with ER or AA) across varying replay ratios (10%, 50%, 100%) to assess FwSC and BwSC. Notably, even multi-task learning, often considered an upper bound of SIL, fails to achieve BwSC, showing near-zero BWT and thus limited reuse of new skills for improving existing policies. \nIn contrast, SIL-C with the same configuration as the simple baselines, achieves positive BWT in the BwSC evaluation, indicating that newly acquired skills enhance existing policies without retraining and highlighting SIL-C’s key strength in enabling scalable and compositional lifelong learning through both FwSC and BwSC.\n\n---\n> W1 - W3. The paper contains some typos, and the explanations in Section 4 and Figure 2 lack clarity. \n\nTo improve clarity as suggested, we summarize the key revisions.\n* We correct the percentage point notation and revise the previously misreported value of 18.0 to the correct value of 18.6 in line 48.\n* We will simplify Figure 2.\n* We provide high-level pseudocode in Section 4 for clarity, with details addressed in our response to `39nR W1`. We also clarify and explicitly reference the Appendix A.2 algorithm in the main manuscript.\n\n---\n> W4. Furthermore, the overall ordering of concepts is difficult to follow.\n\nWe acknowledge that certain details may not have been fully elaborated due to space constraints.\n* Revise the introduction to include a brief paragraph that motivates and defines key SIL-C functionalities.\n* We will provide a summary of the architectural motivations, which connects with the discussions presented in our responses to `JDZP W1` and `39nR W2`, thereby offering clearer context prior to the detailed method description in Section 4.\n\nWe also provide clarification for key terms used throughout the paper:\n* Skill validation: The process of determining whether a subtask is suitable or effective to be used as a skill under the task context.\n* Skill hooking: A mechanism for intercepting and altering skill outputs at runtime between modules. The term is borrowed from software engineering, where \"hooking\" refers to dynamically modifying program behavior.\n* Lazy learning [1]: In our work, this refers to an instance-based retrieval approach in which computation is deferred until it is needed.\n\n---\n> W5. Concerns regarding scalability and skill addition.\n\nAlthough our primary focus is on achieving skill-policy compatibility, SIL-C offers a robust and easily extensible framework for integrating new skills.\n\n5-1. Framework scalability\n* SIL-C is reformulated as an instance-based classification problem, and we observe that its performance improves proportionally with classification accuracy (our response on `39nR Q2`).\n* Regarding scalability with respect to the number of skills, the instance-based classification in SIL-C is inherently scalable and can be efficiently implemented using vector database optimization tools such as Faiss [2].\n\n5-2. Skill improvement and transfer \n* As shown in Figure 4 of Section 5.4, SIL-C can be flexibly applied to various combinations of skill clustering methods and SIL algorithms.\n* In the append-only setting, improving and transferring existing skills essentially involves adding a modified version of the skill while preserving the original, allowing both to coexist.\n* When generating a new skill based on existing ones, the explicit improvement and transfer process is governed by the skill decoder’s learning algorithm. Our method supports two such approaches:\n  * Experience Replay-based approaches, where the quality of a skill is directly and forcibly updated. This is demonstrated in Figure 4 and the Core Claim experiments.\n  * Adapter Append approaches, where learning new skills is facilitated by adapter-retrieval-based initialization [3,4], leveraging existing knowledge.\n* Additionally, implicit skill improvement and transfer can be easily achieved by appending new Gaussian components to each prototype.\n* In these cases, obsolete or infrequently used skills can be managed similarly to a file system, using either soft deletion (e.g., deprioritizing retrieval) or hard deletion (e.g., removing from memory).\n\n5-3. suboptimal data \nWe further continue the discussion of SIL-C’s robustness to noise and suboptimal data in `JDZP W2`. \n\n---\n> W6. Concerns about terminology and phrasing that may cause confusion in context.\n\nWe sincerely thank the reviewer for giving us the opportunity to clarify and improve the terminology.\n* To avoid confusion, we now use \"distance\" instead of the previously used term \"score\", and we clarify that a lower distance indicates better performance in the context of our method.\n* We replace \"introduce\" with \"add\" in line 182 to better reflect the process, since skills are added through training in response to changes in the environment, rather than being introduced independently.\n\n---\n> W7. What are the core principles behind SIL-C, and how do the ablation results demonstrate their contributions beyond potential effects of hyperparameter tuning? (\"As far as I am aware...\")\n\nThe core principle of SIL-C lies in adopting a lazy learning paradigm that modularizes prototype memory at each level of the hierarchy, enabling efficient trajectory distribution similarity matching through memorization. This follows up on our responses in  `Core Claim`, `JDZP W1`, `39nR W2`.\n* Type 1 baselines, which represent simple combinations of skill learning and continual learning, retain individual policy performance. However, since they lack modular integration for skill reuse, they fail to achieve BwSC, which is the main objective of our work. \n* Type 2 methods benefit from predefined labels for future skills, but we demonstrate that SIL-C achieves comparable performance without such foresight. As shown in the Kitchen Explicit scenario of Table 1, these methods degrade sharply when future skills are not known in advance, underscoring their limitations in realistic lifelong learning settings.\n* Therefore, the core strength of SIL-C lies in its ability to maintain past capabilities while seamlessly integrating future skills, without requiring prior knowledge of future tasks. This is made possible through its append-only modular updates and memory-based lazy learning, which together enable robust and scalable backward-compatible policy composition.\n\nTo justify the ablation study in Figure 5, we will include the following clarification in the revised version:\n* The \"SIL-C w/ random policy\" setting is designed to evaluate how accurately decision-making can be achieved using only the skill hooking mechanism(trajectory similarity matching), since most subtask outputs from the high-level policy are not directly compatible in this case.\n* The results suggest that lazy learning alone offers minimal decision-making capability via skill matching, but still underperforms compared to using a synchronized high-level policy. \n* \"SIL-C w/ learned policy\" further improves performance by using out-of-distribution detection to identify and correct mismatches between the subtask and selected skills, which is crucial in long-horizon tasks where errors are costly.\n\n---\n> Formatting concerns.\n\nWe followed the official format, which allows flexible citation styles as long as they are used consistently.\n\n---\n[1] Atkeson, Christopher G., Andrew W. Moore, and Stefan Schaal. \"Locally weighted learning.\" Artificial intelligence review 11.1. 1997.\n\n[2] Douze, Matthijs, et al. \"The faiss library.\" arXiv preprint. 2024.\n\n[3] Lee, Daehee, et al. \"Incremental learning of retrievable skills for efficient continual task adaptation.\" NeurIPS. 2024.\n\n[4] Guo, Yijie, et al. \"Srsa: Skill retrieval and adaptation for robotic assembly tasks.\" ICLR 2025."}, {"review_id": "hgFVdOCY86", "reviewer": "Reviewer_JDZP", "summary": "This paper addresses the problem of Skill Incremental Learning, where an agent incrementally expands and refines its skill set over time through interaction or the addition of new data. The focus is on a particular challenge in this setting: skill compatibility—specifically within hierarchical architectures where the skill module and policy module are updated at different rates, leading to potential inconsistencies. The authors propose two mechanisms to maintain compatibility:\n- BwSC Ensures that newly updated skills remain usable by previously trained policies.\n- FwSC Allows previously trained skills and policies to utilize newly added skills effectively.", "questions": "- Clarify the motivation behind the assumed architectural setup. Why is a hierarchical skill-policy architecture necessary, and why are updates asynchronous? Is this common in practice or a design choice unique to this work?\n- Add evaluation or discussion on how the method would handle noisy inputs, domain shifts, or other real-world challenges. Even a small-scale real-world test or robustness analysis would significantly strengthen the contribution.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n- The paper clearly identifies a practical challenge in skill-based policy architectures and provides a thoughtful solution.\n- The proposed compatibility mechanisms are well-motivated and show promising results.\n- Extensive experiments in simulated environments are used to validate the effectiveness of the method.\n\n\nWeaknesses\n- The issue of skill-policy incompatibility appears to arise from the design choice of using a hierarchical architecture with asynchronous updates. However, the paper does not sufficiently motivate this design. Why are skills and policies trained separately or at different rates? Establishing this necessity upfront would better justify the need for compatibility mechanisms.\n\n-  There is no evaluation in more realistic settings, particularly under noisy or imperfect state/action representations. This limits confidence in the generalizability of the approach beyond clean simulations.", "was_revised": true, "final_justification": "Thanks, author, for answering my questions and providing additional experiments. I remain positive on this submission; however, I still hope to see more real-world experiments.", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback. We deeply appreciate the time and care taken to evaluate our work, and especially value the insightful suggestions that highlight both the strengths of our contribution and areas for refinement. \n\n---\n> W1. Q1. Clarify the motivation behind the assumed architectural setup. Why is a hierarchical skill-policy architecture necessary, and why are updates asynchronous? Is this common in practice or a design choice unique to this work?\n\nHere, we clarify the motivation behind the Policy Compatible Skill Incremental Learning scenario, which forms the basis of our architectural setup.\n* Embodied agents for complex decision-making (hierarchical skill-policy architecture): (Section 2, paragraph 1) [1,2,3]\n  * Hierarchical architectures are widely adopted to address complex, long-horizon tasks (such as those in the Franka Kitchen environment) by decomposing them into manageable subtasks. \n  * This decomposition facilitates more tractable decision-making and enables the modular reuse of pre-trained skills, which is a standard practice in robotic learning.\n  * By decoupling high-level decision-making from low-level control, the hierarchical structure improves learning efficiency and execution flexibility across diverse tasks.\n\n* Lifelong learning for modular architecture deployment configurations. (asynchronous updates): (Section 2, paragraph 2) [4,5] \n  * Asynchronous updates are common in lifelong learning scenarios where modularized components are trained on distinct data sources with different objectives. For example, in producer-consumer deployment configurations, the producer pre-trains and deploys a general-knowledge model, while the consumer uses or customizes it for specific tasks.\n  * Differing update frequencies naturally arise across modules due to variations in how frequently their respective data sources are updated. This makes it essential to maintain compatibility between evolving skills and policies.\n  * Modularization thus facilitates asynchronous updates, enabling efficient, task-specific learning without the need to retrain the entire agent system.\n\nTo the best of our knowledge, our work is the first to integrate asynchronous modular updates into a hierarchical policy-skill architecture for lifelong robotic learning.\n\nHere, we clarify compatibility problems and solutions for the architectural design process.\n* Forward Skill Compatibility (FwSC)\n  * To ensure FwSC, it is crucial to maintain existing skills without forgetting while allowing for the addition of new ones. \n  * We propose building a common and expandable embedding space for skills and subtasks, using a model expansion paradigm that effectively prevents the problem of explicit forgetting.\n  * Lightweight adapters like LoRA offer an efficient and robust approach to lifelong learning. Rather than modifying existing components, new functionalities are appended modularly.\n  * This strategy is effective in preventing forgetting by structurally isolating newly added knowledge from older skills.\n  \n* Backward Skill Compatibility (BwSC)\n  * To ensure BwSC, the policy's high-level decision-making process must be made \"lazy\". In other words, the agent should defer integrating any newly appended skill until it actually observes a state distribution that makes that skill beneficial, rather than eagerly re‑optimizing the whole hierarchy in advance. \n  * Our core insight is that not every decision-making function needs to be lazy; rather, only specific components should be lazily offloaded when appropriate.\n  * To implement this, each modular model must establish a shared protocol regarding its behavior (e.g., pre-defined skills, subtask labels) and make decisions accordingly.\n  * Specifically, only the \"validate subtask and reassign skill\" step is lazily offloaded within the shared space (i.e., state space), leveraging a prototype memory to compute trajectory distribution similarity.\n\nCollectively, these design choices enable lifelong learning in complex decision-making agents. We will incorporate this explanation into the introduction to clarify the design choices and their relevance to lifelong embodied agents. We continue the discussion of the connection between the motivation and the implementation in our response to `39nR W2`.\n\n---\n> W2. Q2. Add evaluation or discussion on how the method would handle noisy inputs, domain shifts, or other real-world challenges. Even a small-scale real-world test or robustness analysis would significantly strengthen the contribution.\n\nTo evaluate the robustness of SIL-C to observation noise, we used a model trained on the Kitchen Emergent Skill Incremental scenario in the Franka Kitchen simulator [1], where Gaussian observation noise is present by default as part of the robot's specification. During evaluation, we varied the intensity of this built-in noise by applying different noise scales, specifically scaling the default noise level by factors of ×1, ×2, ×3, and ×5. SIL-C was compared against the Type 1 baseline PTGM+AA under these noise conditions. The results are summarized in the following table.\n|Kitchen Emergent scenario||Initial|BwSC||Overall||FwSC||Final|\n|-|-|-|-|-|-|-|-|-|-|\n|Noise level|Baselines|FWT|BWT|AUC|BWT|AUC|BWT|AUC|FWT|\n|×1|PTGM (AA)|45.0±3.1|2.6±0.9|46.9±3.7|15.4±1.7|58.2±1.7|28.1±4.1|66.1±0.6|83.6±3.1|\n||SIL-C (AA)|52.9±2.9|18.6±1.2|66.8±2.6|22.0±2.4|71.8±1.3|25.4±4.0|71.9±0.9|87.2±3.2|\n|\n|×2|PTGM (AA)|44.4±4.5|0.8±2.0|45.0±3.4|10.7±2.7|53.5±2.2|20.5±3.9|59.7±1.9|68.6±3.5|\n||SIL-C (AA)|49.7±0.6|11.3±1.8|58.2±1.5|16.7±1.1|64.0±1.3|22.1±1.9|66.3±1.8|76.4±2.6|\n|\n|×3|PTGM (AA)|35.8±5.3|0.5±3.7|36.1±2.9|3.2±5.6|38.5±1.5|5.9±7.7|40.2±1.7|40.5±4.6|\n||SIL-C (AA)|35.6±2.2|13.6±4.2|45.8±3.1|15.4±2.9|48.8±1.5|17.3±3.4|48.5±1.0|52.9±3.9|\n|\n|×5|PTGM (AA)|19.7±1.7|-1.2±3.4|18.8±1.7|-1.8±2.4|18.2±0.9|-2.4±2.5|17.9±1.1|15.9±1.6|\n||SIL-C (AA)|19.6±2.3|4.3±1.3|22.8±2.4|3.8±2.4|22.9±1.4|3.3±4.1|22.1±1.4|23.1±4.1|\n\nSummary of Results:\n* **Overall Performance:**\n  SIL-C consistently outperforms PTGM+AA across all noise levels, demonstrating superior robustness in noisy environments.\n* **Initial Performance:**\n  At the beginning of training (Initial FWT), SIL-C exhibits stronger performance than PTGM+AA. However, under extreme noise conditions (×5), both methods experience a performance drop, and their scores converge and become comparable.\n* **BwSC:**\n  SIL-C maintains a positive Backward Transfer (BWT) across all noise levels, indicating stable retention of previously learned skills. In contrast, PTGM+AA suffers from negative BWT under higher noise, suggesting degradation of earlier knowledge.\n* **FwSC:**\n  SIL-C consistently achieves higher AUC, reflecting better integration of newly learned skills.\n* **Final Performance:**\n  As training progresses and more skills are added (Final FWT), the performance gap between SIL-C and PTGM+AA widens compared to Initial FWT. This divergence is especially pronounced under high-noise conditions.\n\nThis robustness becomes particularly important as the agent accumulates a larger repertoire of skills, up to approximately 80 in the final FWT phase. As noise levels increase, SIL-C exhibits a more gradual degradation in performance, reflecting stronger generalization capabilities in noisy environments.\n\nWe attribute this robustness to the distinct ways each method handles noisy subtask distributions. In extreme noise conditions, where the high-level policy’s subtask selection becomes less reliable, SIL-C increasingly leverages a prototype memory-based skill validation mechanism, combined with a trajectory distribution similarity-based skill hooking process. These components enable SIL-C to retrieve and deploy skills in a more adaptive and noise-tolerant manner.\n\nIn contrast, PTGM+AA relies solely on static subtask identifiers, which fail to account for distributional shift caused by observation noise. As a result, its performance deteriorates more sharply as environmental noise increases. This disparity becomes especially pronounced in long-horizon environments like Franka Kitchen, where even a single misstep can lead to substantial reductions in cumulative reward. Such environments amplify the impact of noise, further underscoring the robustness of SIL-C.\n\n[1] Gupta, Abhishek, et al. \"Relay policy learning: Solving long-horizon tasks via imitation and reinforcement learning.\" CoRL. 2019.\n\n[2] Pertsch, Karl, Youngwoon Lee, and Joseph Lim. \"Accelerating reinforcement learning with learned skill priors.\" CoRL. 2021.\n\n[3] Yuan, Haoqi, et al. \"Pre-training goal-based models for sample-efficient reinforcement learning.\" ICLR. 2024.\n\n[4] Mendez, Jorge A., Harm van Seijen, and Eric Eaton. \"Modular lifelong reinforcement learning via neural composition.\" arXiv preprint. 2022.\n\n[5] Sukhbaatar, Sainbayar, et al. \"Branch-train-mix: Mixing expert llms into a mixture-of-experts llm.\" COLM. 2024."}]

Author rebuttal:

Meta-review: The authors propose a novel framework for Skill Incremental Learning (SIL) that maintains compatibility between evolving skills and downstream hierarchical policies. It addresses the challenge of ensuring that updates to low-level skills do not invalidate or require retraining of high-level policies. The proposed approach decouples policy learning from skill updates using a bilateral lazy learning interface, which dynamically maps high-level subtasks to low-level skills based on trajectory similarity. The proposed method outperforms the baselines in overall performance and forward compatibility, and performs closely to methods with predefined skill labels. 
Most concerns are addressed through clarification and additional experiments. The final recommendations are two acceptances and two borderline acceptances. One reviewer would like to see more real-world experiments. However, the reviewer still gave borderline acceptance. Hence, the AC recommends acceptance.
Decision: Accept (spotlight)