Continual Slow-and-Fast Adaptation of Latent Neural Dynamics (CoSFan): Meta-Learning What-How & When to Adapt
Authors: Ryan Missel
Linwei Wang
Venue: ICLR 2025 Poster (2025)
Keywords: continual meta-learning
latent dynamics forecasting
time-series
Primary area: learning on time series and dynamical systems
TL;DR: In this work, we present a new continual meta-learning (CML) framework to realize continual slow-and fast adaptation of latent dynamics (CoSFan) by meta-learning what-how & when to adapt.

Abstract: An increasing interest in learning to forecast for time-series of high-dimensional observations is the ability to adapt to systems with diverse underlying dynamics. Access to observations that define a stationary distribution of these systems is often unattainable, as the underlying dynamics may change over time. Naively training or retraining models at each shift may lead to catastrophic forgetting about previously-seen systems. We present a new continual meta-learning (CML) framework to realize continual slow-and fast adaptation of latent dynamics (CoSFan). We leverage a feed-forward meta-model to infer *what* the current system is and *how* to adapt a latent dynamics function to it, enabling *fast adaptation* to specific dynamics. We then develop novel strategies to automatically detect *when* a shift of data distribution occurs, with which to identify its underlying dynamics and its relation with previously-seen dynamics. In combination with fixed-memory experience replay mechanisms, this enables continual *slow update* of the *what-how* meta-model. Empirical studies demonstrated that both the meta- and continual-learning component was critical for learning to forecast across non-stationary distributions of diverse dynamics systems, and the feed-forward meta-model combined with task-aware/-relational continual learning strategies significantly outperformed existing CML alternatives.

Published as a conference paper at ICLR 2025
CONTINUAL SLOW-AND-FAST ADAPTATION OF LATENT NEURAL DYNAMICS (COSFAN): WHAT-HOW & WHEN TO ADAPT
Ryan Missel, Linwei Wang (rxm7244, 1xwast]erit. Rochester Institute of Technology
ABSTRACT
An increasing interest in learning to forecast for time-series of high-dimensional observations is the ability to adapt to systems with diverse underlying dynamics. Access to observations that define stationary distribution of these systems is time. Naively
often unattainable. the underlying dynamics may change training or retraining models at each shift may lead to catastrophic forgetting about previously systems. We present new continual meta-learning (CML) framework to realize continual slow-and fast adaptation of latent dynamics (CoS- Fan). We leverage feed-forward meta-model to infer what the current system is and how to adapt a latent dynamics function to it, enabling fast adaptation specific dynamics. We then develop novel strategies to automatically detect when a shift of data distribution occurs, with which to identify its underlying dynamics and its relation with previously-seen dynamics. In combination with fixed-memory experience replay mechanisms, this enables continual slow update of the whar-how meta-model. Empirical studies demonstrated that both the meta- and continull-eeaming component was critical for learning to forecast across non- stationary distributions of diverse dynamics systems, and the feed-forward meta- model combined with lask-aware/-relation continual learning strategies signifi- cantly outperformed existing CML alternatives.
INTRODUCTION
Learning to forecast from time-series observations of high-dimenssonal data, such as series of im- ages, is becoming importantly crucial in various applications. Among recent advances, one repre- sentative approach is to learn the dynamics function governing these observations in abstracted latent space as a means to forgo the need for direct supervision (and by extension, direct knowl- edge) on the system's 's latent variables (Chung et al.,, [015)] Krishnan et al.) 2015 Karl et al.) 2017]] Yildiz et al. 2019 Fraccaro et al.) 2017] Botev et al.) 2021). This is often realized as a sequential latent variable model (sLVM) with a latent dynamics function 2, f(zzz 0) and the latent states'
emission back to observation space X2 = g(z,) Yildiz et al. 2019 Jiang et al. 2023)
Recent advances have moved towards modeling similar-yet-distine dynamics that arise from hetero- geneous systems (Jiang et al. 2023). One approach that has garnered increasing attention involves learning-to-leann latent dynamics function via meta-models capable of rapid adaptation. The suc- cess of these methods rely on two key fundamental assumptions: 1) training samples are available from all systems of interest and this distribution does not change test time, known as a stationary task distribution, and 2) each sample has an identifier linking it to specific system. These assump- tions often break down time-series forecasting, where data samples arrive as a stream over time such that neither stationary distribution of systems nor the incoming samples' system identifiers is guaranteed, This represents a scenario of non-stationary task distributions with unknown task boundaries or identifiers, where traditional meta-learning approaches have limited applicability.
The emerging research area of continual meta-learning (CML) relevant to lifting these restric- tions. CML elevates continual learning (CL) originally designed to manage non-stationary data distributions (Van de Ven & Tolias 2019) to meta-learner, over non-statin non-stationary task distributions
(Riemer et al. 2018 Joseph & Gul 2021 Examples include weight regularization meta-models

---

Published as a conference paper at ICLR 2025
(He al., 2019) or sample-storing reservoir to approximate task stationarity (Joseph & Gu 2021
Unfortunately, existing CML works have predominantly focused on supervised image classifica- tion (Joseph & Gu 2021: Riemer et al., 2018; Caccia et al.) 2020) or low-dimensional regression tasks (He et al. 2019). leaving open challenges towards its adoption in latent dynamics forecasting. Two potential challenges stand out. First, current CML research largely employs gradient-based meta-lcarners, such as model-agnostic meta-learning (MAML) (Finn et al.) (017]) which, although effective for image-based tasks, can fail to generalize that focuses on short-term forecasting of action
broader range of problem domains. This
limitation was noted in meta- wetaa--eiiioooee--eeeeeeee sequences (Mishra et al., 2017), Second, many CML approaches lack a isssiidetiificttion mecha- nism (Joseph & Gul Riemer et al. 2018), which prevents them from fully utilizing the potential of bi-level optimization for the meta-learners to learn to adapt to all previously seen tasks.
Our contributions. This work introduces a new what-how & when framework for continual slow- and-fast adaptation of latent neural dynamics (CoSFan) across sequentially presented non-stationary distributions of dynamics systems (i.e., tasks.) all without prior knowledge of task boundaries or identifiers. It has two major contributions. First, we propose a what-how metal-odel to infer what the current system is and how to adapt a dynamics function to it (Jiang et al. (023), in feed-forward manner to realize fast adaptation to specific dynamics systems. We show that this feed-forwart meta-model provides stronger algorithmic prior than gradient-based optimization (e.g., MAML) for adaptation, offering significantly faster adaptation and improved predictive performance.
Second, propose novel strategies automatically detect when shift of data distribution occurs and, based on which, to identify the underlying dynamics systems and their relations. In combination with fixed-memory experience replay mechanisms, this enables pairing of context-query samples to fully utilize bi-level optimization for continual slow update of the what-how meta-models. Specifi- cally, we propose two alternative strategies: a simple boundary- -baed lseudo-labeling mechanism, paired with standard reservoir sampling (Viterr 1985), to maintain a stationary approximation of the observed samples and their tasks IDs over time; and cluster-based labeling mechanism, employ- ing continual Bayesian Gaussian mixture models (GMM) on the what-embedding extracted by the meta-model, to approximate task distribution via a distilled representation of task relations.
We evaluated CoSFan in time-series forecasting in two comparative studies. First, we compared CoSFan with state-of-the-ar latent dynamics models and their continual extensions, evaluating the necessity of both the meta- and task distributions. Second, we compared with its alternative formulations based on existing CML works, evaluating the benefits of the proposed what-how meta-models and their continual learning strategies. In series of increasingly complex continual learning settings with high-dimensional rep- resentations of Hamiltonian systems, we quantitatively and qualitatively demonstrated C's capacity to detect and learn new tasks while preventing catastrophic forgetting in sequential tasks. Models and experimental codes are available at attttps: /t com/ qu- CoSCan
in learning to adapt across non-stationary
RELATED WORKS
Learning-to-learn latent dynamics. Recent advances have focused on modeling similar-yet- distinct dynamics manifested as heterogeneous samples, particularly through meta-learning (Jiagg ct al. 2023 Wang et al. 20222; Kirchmeyer et al. (002)) and multi-cnvironment learning (Yin et al. 2021 Zintgraf et al. 2019). One effective approach of increasing interest extends latent dynamics functions via meta- learning (Wang et al. 2022 Jiang et al. 2023). where each dynamic system is treated as a task, and meta-models learn-to-leas
to rapidly adapt the latent dynamies function to dif- ferent tasks. Both MAML- (Kirchmeyer et al. (022) and feed-forward based meta-learners (Jiang let al. 2023) have shown success in real-world applications, such as cross-subject clinical forecasting Jiang et al.
2022 and cross-buoyancy turbulent flow forecasting (Wang et al. 2022).
Unfortunately, meta-learning fundamentally assumes tasks are independent and identically dis- tributed (Khoee et al. 2024), which can lead to poor generalization for significantly different tasks at meta-test time (Hospedales et al. 2021). Furthermore, relies on a stationary training distribution, assuming all relevant tasks are present and labeled during training to pair context and query samples for bi-level optimization (Hospedales et al. 202T). In time-series forecasting, these assumptions hinder the continuous aggregation of knowledge unique dynamic systems may phase in and out
2

---

Published as a conference paper at ICLR 2025
Lettmnnn Framework
Identification
What:
How
fg(z)
Reservoir Buffer of Previous Samples
Nhor
Continual Stream of Dynamic Environments
B
Table Related continual meta-lcaming work, noting if they consider certain modeling choices.
Method
Forgetting
Algoritomic Prior
Task Inference
Roundary Detection
Regularization
MER (Riemer et 2018)
Reservoir Sumpling
Gradient-Bare Baed
La-AMM. (Jsseph & Gu., 2021)
Reservoi Sampling
Gadicui-Baso Based
OSAKA
(Caceua et 2020)
Not Addreessdd
Gradiest Based
CML-BGNN (Las al. 2019)
Repularizaion
Generic
MOCA (Hanison
Reservair Sampling
Generic
Proposed
Figure I: A) Overview of CoSFan, showing the what-how meta-model continually aggregate heterogeneous data stream with a reservoir that identifies tasks via Gaussian mixture models. B) Comparison of the proposed model to relevant CML work.
without prior knowledge. Identifying appropriate auxiliary mechanisms to continually learn how to adapt latent dynamic functions in non-stationary task settings remains an open challenge.
Continual meta-learning. Recent developments at the intersection of continual learning and
meta-learning, known continual meta-learning (CML), offer inspiration. By leveraging estab- lished continual methods, such as reservoir sampling (Riemer et al. 2018 Joseph & Gu 2021 or Bayesian Gradient Descent (He et al. 2019). on the meta-learner, allows the meta-leamer to be continually trained on sequence of tasks. However, existing studies focus primarily on supervised image classification tasks (Joseph & Gu 2021 Riemer et al., 2018 Caccia al., 2020) or low- dimensional regression problems (He et al. 2019). The potential of CML in the space of time-series or latent dynamics forecasting remains unexplored, leaving two primary open questions.
First, most prior CML works leverage pradient-based meta-learners specifically MAML (Finn et al.) 2017) and its variants. While effective in image and classification domains, MAML's algorithmic prior of gradient descent has been shown to be ill-fit for adapting the task landscape of other domains (Mishra et al. 2017). In wetarrrrrnoorreee-eeaaaaae collapse into one mode, for instance, the shared initialization that MAML uses often results in sub- optimal adaptive performance on isolated tasks (Mishra et al., 2017: Vuorio et al. 2019 Fu et al.) 2023). Moreover, gradient- -based meta-learners require fine-tuning for every incoming task, even for known tasks, which is sensitive to optimization syper-parammters and computationally inefficient (Nguyen al. 2021 Mishra al. (017). Meta-models with alternative algorithmic priors, such as leed-forward hetersn remains unexplored in the broader scope of CML.
where the task distributions cannot neatly
Second, meta-learning's signature bi-level optimization depends on known task identifiers to pair context and query samples. How to achieve this in task agnosiic settings - where neither task bound- ary nor identifier known is unresolved. Existing CML approaches bypass this through either reg- ularization techniques (He et al. 2019 Harrison et al.) 2020 Caccia et al.) 2020) or MAML-based strategies to align gradients between new and previous tasks memory buffer (Riemer et al.) 2018 Joseph & Gul (222I) Limitations of the latter are discussed above, while the former often faces capacity saturation and instability in high-dimensional settings (Joseph & Gu 2021).
CoSFan breaks through the reliance of mainstream CML methods on MAML-variants. demonstrat- ing the effectiveness of
meta-models, combined with automatic task identification and task-relation modeling, in fully leveraging bi-level meta-optimization over non-stationary task dis- tributions. We position our work against prior CML works in Fig. B.. Discussion of other related and other algorithmic priors is included in Appendix A
areas, e.g., CL. and meta-continual
3

---

Published as a conference paper at ICLR 2025
3 PROBLEM FORMULATION
Consider the problem of learning to forecast the trajectory of a data sequence Xa:r when only given access / initial frames of observations Xp: (1 < T), where X; E RD at time is high in dimension. Latent dynamics forecasting approaches this by learning the dynamic function governing some low- dimensional latent representations 2 E R2 (d < D) and its mapping to X,. We learn do so from many observed samples of Xo.r- Depending on the diversity and distribution of the observed data, below we introduce the setting for this problem in layers, building up from the simplest setting of learning from single systems to learning from son-stationary distributions of multiple systems.
Single-System Forecasting. Let us start with the simplest setting where all observed samples of Xo.7 has shared generative low-dimensional dynamic system T. Leveraging the commonly used sLVM framework (Botev et al. 2021: Jiang et al. (023), the modeling consists of three components:
i) An encoder Zo (xo.), parameterized by o, to infer Zo from the sub- sequence Xo:1
ti) A dynamics function fe parameterized by 0, to describe the evolution of Z; over time and comes in many forms- from linear combinations of transition matrices (Karl et al. 2017 7 Fraccaro al. (017) to neural ordinary differential equations (NODE) (Chen et al. 2018 Yildiz et al. (019)) We opt for the latter in this work, which describes the continuous evolution of Ze as Z, = Zo + So fe(z, )dT.
tii) An emission function x, = dec,(zz), parameterized by p. to map 2, 's to x; 's in parallel.
Given the final forecasted sequence Xu:r from the few-frame input Xo:1, the difference in the forecast and the ground truth [(xo:T; X).r) can be used as a loss signal to optimize the parameters {Ø, 0, p).
Heterogeneous-Dynnniee Extension. Now consider extension where multiple dynamics sys- tems, { exist, each with an unknown system variable c(Ti). Examples include varying sets of parameters for a predator-prey model (Kirchmeyer et al. 2022) or image sequences of objects affected by various Hamiltonian systems (Jiang et al.) (023). Under this setting, fundamental lim- itation of fe as desdribed above emerges: learns a single parameter set global to all training sequences; as such, it lacks the ability to adapt to unknown dynamics at test-time. It is easy to see then, when only given the partial sequence to forecast from, we need mechanism infer what the current system is and to know how to adapt to this system in order to perform across all dynamics si- multancously. Formally, this means that fe should now be conditioned on a changing context c(T,) as: z, fz ((T))) in order to change forecasting given the same input of past frames.
Nta-Snat Extension. Lastly, consider an additional condition in which the set of dynamics M
systems, ie., tasks {T) are presented in sequential order (Ti, T2, TM ). We consider the continual-learning scenario where neither the boundary between tasks nor the label of any task is known (Van de Ven & Tolias (2019). We also assume the presence of local stationarity where data from each task are presented for some period of time. This allows the forecasting method to optimize sufficiently and stably before new task has the potential of appearing.
Goal of the Learner. Overall, we aim to learn a model which can continually adapt the latent dynamics function z; = fo(z e(T,)) to different dynamics systems over a non-stationary stream of such systems- without the availability of known boundary or identifier for these systems.
4 METHODOLOGY
We present what-how & when CML framework as illustrated in Fig. DA where fast-adaptation meta-model infers what the current task is and how to adapt the dynamics function accordingly (Section 4.1) while slow- alw-addaptation of the meta-model occurs on an approximated stationary task distribution enriched with task identifiers and relations when a task shift occurs (Section 4.2p
4.1 What-Howw FAST ADAPTATION BY INFERRING & USING A DYNAMICS' CONTEXT
The concept of what & how is broadly applicable across meta-learner algorithms: the what compo- nent extracts the context c(T,) from lt sequences
k { TT,", called the context set for task
4

---

Published as a conference paper at ICLR 2025
Tj; the how component maps c c(Ty) task-specific predictive function fo(z<t; c(T,)) to forecast on unknown query sequences x 's. In the commonly used MAML techniques in CML, a shared parameter set vnit is adapted into a task-specific parameter set ftosk (what) by taking few gradi- ent optimization steps (how) over 7, While successful in image classification and low-dimensional regression, they have been shown unsuitable in broader domains, notably in meta-reinforcement learning which act action sequences of action (Mishra et al. (017).
We propose a more general what & how method for CML that learns the adaptation algorithm in a feed-forward manner, eliminating the need for gradient-based updates at test-time. We use linear encoder, termed the context encoder, to embeds and aggregate the context data into a lower- dimensional representation (what), along with a feed-forward hyper-network that transforms this representation into task-specific latent dynamic functions (how). We contrast these two what-how models from the lens of algorithmic prior (Section 4.1.2 and adaptation efficiencies (Section
4.1.1 FEED-FORWARD WHAT-HOW MEDA-M FOR LATENT DYNAMICS FORECASTING
a. What. Given the context set 7, we encode each sequence x XIT to its individual embedding cf through the use of a learnable encoding function ev- We then use averaging function to extract the shared knowledge over the set where is the size of the set:
c(T,)
cj where cj'" =x((x.T))) XOTS TJ.
How. To use the inferred e(T,) to adapt the latent dynamics function, we employ a hyper-network Ha et al. 2016) b, to map c(T,) to the task-specific parameters of the latent dynamics function as:
0 = (c(T))
(2)
For the hyper-network architecture, we consider a simple linear transformation as the what-encoder already provides sufficiently complex non-linear mapping to the task context variable.
Meta-Objecte Objectvbe, The meta-learner's parameters are optimized by minimizing the error between the forecasted and ground-truth query sequences across all tasks, with each task evaluated individ- ually in parallel and their errors aggregated. The loss function for a given task is expressed as:
CT Σ
x where Khrr dec, (ec.
(3)
10 o TT,
As such, across the tasks, the meta-objective becomes:
min ΣT CT, 1.1-77.10
(4)
which jointly optimizes the sLVM's parameters (Φ) p) and the what-how meta-parametes (7, v)
4.1.2 HYPER-NETWORKS AS ALGORITIMIC PRIORS
Recent work in meta-learning has formalized the concept of algorithmic priors, the underlying al- gorithms used for adaptation, ranging from gradient-based methods to general approaches that learn the adaptation algorithm itself (Mishra et al. 2017). The latter, often realized using recurrent archi- tectures, is sensitive to context set permutations and entangles input representations with the learning algorithm. These factors have historically constrained general meta-learner in complex domains, leading to the prevalent use of gradient- based techniques (Huisman et al., (023).
In comparison, hyper-networks have seen success in adapting latent dynamics functions in station- ary settings (Kirchmeyer al. 2022). They also have inductive biases that promote weight transfer across tasks while minimizing gradient interference (Jayakumar et al.] 2020), which make them suitable general algorithmic prior for addressing the transfer-inteeffernce trade-off in non-stationary tasks (Riemer et al. 2018). Meanwhile, unlike gradient-based methods that rely on global initial- ization vector for all tasks, feed-forward what-how meta-man models learn complex non-lineat mappings that structure tasks in embedding space suited for diverse adaptations. Thus, hyper-networks over- come the limitations of general algorithmic priors while avoiding the constraints of gradient-based methods in diverse task landscapes (Mishra al. 2017
Vuorio et al. 2019 Fu et al. 2023).
5

---

Published as a conference paper at ICLR 2025
4.1.3 FEED-FORWARD ADAPTATION EFFICIENCY
Feed-forward approaches are more efficient than gradient-based approaches in both meta-training and meta-testing for two key reasons. First, hyper-networks require only single forward pass to adapt, avoiding the costly, often repeated, backpropagation steps. Second, gradient- based methods typically require sequential loop over tasks aggregate losses and update meta-weights, as most automatic differentiation libraries lack support for parallelized computation branches (Utkarsh al.) 2024). This limits the use of architectures like NODE, where model evaluation involves sequential integration over dynamic functions. In contrast, cerward methods can easily parallelize per-task weights (e.g., using vectorized maps) and are unaffected by task quantity.
Dedicated works have resulted in first-order gradient methods for computational efficiency NNchol et al. 2018) and subsequent CML methods that eliminate the sequential loop (Joseph & Gu 2021)
In Section 5.3 we evaluate these CML approaches in terms of adaptation speed and training times.
4.2 When CONTINUAL FAST& & SLOW ADAPTATION OF LATENT DYNAMICS
The learning objective as defined in Equations and 4 is optimized over a stationary distribution of tasks, as in standard meta- veta-learning practice. As such, despite its ability to infer context and adapt its dynamics, its meta-components become susceptible to catastrophic forgetting in non-stationary task setrings. Additionally, the absence of task identifiers complicates meta-learning, as pairing context-query data becomes infeasible. Therefore, it is necessary to incorporate mechanisms to automatically detect task boundaries (4.2.1). combat catastrophic forgetting (4.2.2), and identify tasks and their relations to accurately match query samples to their relevant context (42.2).
4.2.1 TASK BOUNDARY DETECTION
Similar to many other CML works (Caccia et al. 2020 He et al. 2019). we choose the most recent observations to function as the context set for the current task. This is a sound choice given the local stationarity assumption. When task shift occurs, the tasks underlying the context and query sequences differ, resulting in a noticeable dip in prediction performance. We leverage this, via a simple yet effective threshold mechanism proposed in (Caccia et al. 22200), to flag task boundary via ||C,,. - -il > y where C3 is the current task loss, n is the global batch index, and v is determined based on the dataset. Further rationale and details for determining are in Appendix C.7
4.2.2 RESERVOIR SAMPLING BASED ON TASK IDS AND TASK RELATIONS
In the non-stationary setting where only the data of one task is actively streaming in given time and no task identifiers are available, two challenges arise in applying meta-learning: i) how to obtain and aggregate errors from prior tasks update the meta-weights and ii) how to accurately pair context and query samples from the same task, especially for previous tasks without active data.
Existing works approach this by standard reservoir sampling method, a simple algorithm that tracks the number of samples (N) seen and, for each incoming sample, overwrites an existing buffer sample with probability M/N where M is the size of the reservoir. To realize meta-learning in this task- agnostic reservoir, recent work leverages the approximate equivalence between meta-learning and continual learning objectives in aligning the current task's s gradient T, with the average gradient of previous tasks To (Riemer et al. 2018 Joseph & Gu 2021). This alignment can then be achieved by using the current task's s samples as context to obtain parameters 0T,, and evaluate the meta-loss using 07, on all past tasks' samples in the reservoir (Joseph & Gul 2021). We refer to this approach as Task-Agnostic Reservoir Sampling and demonstrate in Section 5.3 that this is less effective in CML compared to full bi-level oettaoppii that uses context-query pairing.
Task-Aware Reservoir Sampling. To make the reservoir task-aware, we leverage the boundary detection mechanism, assuming each boundary to represent a unique dynamics system and assigning pseudo-labels with boundary counter. When sampling from the reservoir, context-query pairs are easily matched by using each sample's lseudo-label. In situations where the data stream contains numerous task boundaries unbalanced task representations, this approach can lead to an increas- ingly fragmented buffer, resulting in overfitting to specific context-query pairs or quickly forgetting rare tasks redreseepeeeented in the buffer. We experimentally validate these issues in Section 5.4

---

Published as a conference paper at ICLR 2025
Task-Relational Experience Replay. To make the reservoir be aware of task relations, lever- age a Bayesian Gaussian mixture model (GMM) that automatically identifies clusters within the reservoir over time and determines sample's ' relation to them. At detected task boundaries T; - T we the GMM on context-embeddings from the reservoir's samples Creservoir along with an auxiliary memory buffer Cactive from the active task Tj. To maintain continuity, we reuse the previous GMM components and their weights to initialize the current clustering. New samples are integrated by initializing a new mixture component based on the mean and covariance Cualive The weight of this component after refitting determines if these samples represent new task (high weight) or a re-emerged task (low weight, covered by an existing component). To avoid the number of components from exploding, remove inactive components with weights below 0.05.
2
Given the fit mixture components Q and the set of context- -entexiitembeddings {Creservoir, Cactive), the reservoir Rj is re-balanced in a novel way by allocating M/ 2 samples to each component, where Ssize is the number of components. Components with more samples than their allocated size have excess samples discarded, ensuring the reservoir remains within its memory limit.
M2, R3j M Caetive Hmg og);
Tq =1
where Ty is the weight of a component. The resulting sample cluster assignments .M, pseudo-labels enabling context-quer, pairing when sampling the reservoir. In Section 5.4
serve as we show
that this approach addresses frequent task transitions and the presence of rare tasks.
4.3 CONTINUAL META-OBJECTIVE AND OPTIMIZATION
Continual Approximation of the Meta-Objective. We now continually approximate the true stationary meta-objective in Equation 4 as below, bounded by the extent to which the sample reser- voir R, approximates the true data distribution of task Tj and to which the approximated task identifiers Mj approximate the true task assignments Ij:
min
E E(RR-TT, LR,, (M₂)).
(6)
0..P.T..1
Cluster Embedding Regularization. To guide meta-moder optimization to structured clusters, regularization from deep clustering (Manduchi al. 2021) MSE between the meta- emheddings ey of the reservoir samples and their closest cluster mean fz (by Fuclidean distance):
Celuster C3
Σ ||o(T,) -1-112
5 EXPERIMENTS
We evaluate CoSFan as follows. In Section dynamic models with continual extensions, examine the need of both meta- and continual- formulations in this domain. In Section 5.3 we explored the benefits of CoSFan's what-how and when components existing CML strategies. In Section itations of task-aware and task-relational replay strategies. Appendix (reendii/Ccontainss further ablations.
5.2
we compared CoSFan to representative latent
analyzed the strengths and lim-
5.1
EXPERIMENTAL SETUP
Data. First. we considered data where the underlying dynamics governed by the same equation but with different parameters. We used benchmark image-based time-series of balls bouncing in box (Fraccaro et al., 2017] Jiang et al., 2023), influenced by 6 evenly-spaced directions of gravity and sampled 4500 sequences from each with varying initial positions and velocities. The magnitude of gravitational forces was kept constant. We refer to this as gravity-6. Next, we explored a broader setting involving a collection of different dynamics, each with range of heterogeneity within the system. In addition to bouncing balls (Fraccaro et al., 2017). we included Hamiltonian pendulums, Hamiltonian mass-spring systems, and two-body systems (Botev al. 2021). For each dynamics, we selected 3 parameter configurations, resulting in a total 12 dynamics with 1500 samples each. We refer this mixed-physics. Additional data details and visualizations are in Appendix D.O.
7

---

Published as a conference paper at ICLR 2025
Learning Performance
Retained Performance
Backwards-Transter Interference
-0.5
-1.0
ML.
TAgnostic T-Awar
ER
NL
T-Agnostic T-Aurre
ER
NL
T-Agnostic T-Aware
ER
RGNRes
MetaRGNRas
Figure 2: DST performance comparison on mixed-physie over existing latent dynamics models and their continual extensions. All methods were run over random seeds.
Metries. We evaluated two quantitative metrics for latent dynamics forecasting: 1) the MSE of the forecasted images the pixel level; and 2) a distance metric (DST) proposed by (Jiang et al. 2023) to measure the Euclidean distance between the ground-truth and forecasted objects. To evaluate the ability of a model to learn continually, we followed (Riemer et al. 2018) and presented the above forecasting metrics in two perspectives: Retained Performance (RP) and Learning Performance (LP). RP metrics represent the average performance across all tasks after they are sequentially con- sidered, emphasizing model's ability retain performance average performance on a task immediately after it is learned, measuring a model's 's effectiveness in incorporating information. To assess the extent of catastrophic forgetting, also reported the average difference between IP and RP metrics, referred to as Backward Transfer and Interference (BTI), with negative values indicating increased forgetting (Riemer et al. 2018).
older tasks. LP metrics reflect the
When comparing the choice of the what-how meta-models in Section 5.3 we include two additional metrics related computational efficiency. Time-to-Adapt-N (TTA-N) represents the average time it takes, in seconds, for model to adapt to its context data in the presence of N tasks. Time-to-Ti to-Train (TTT) represents the average time it takes, in minutes, for model train over sequence of tasks.
Implementation Details. Discussion on the implementation- speccffics are included in Ap- pendix D.0.3 detailing reservoir sizes, sequence lengths, and hardware considerations.
5.2 THE NEED OF WHAT-HOW AND WHEN: COMPARISON WITH EXISTING LATENT DYNAMIC MODELS AND THEIR CONTINUAL EXTENSIONS
Models. We first compared CoSFan with representative latent dynamic models and their continual extensions. The baseline latent dynamic models included Deep Kalman Filters (DKF) (Krishnan et al. 2015), Variational Recurrent Neural Networks (VRNN) (Chung et al.) 2015). a Residual Recurrent Generative Network (RGNRes) (Botev al. 2021). and meta- learning realization of meta-learning
the RGNRes (MetaRGNRes) using the proposed what-how meta-model. We evaluated these four models under four experience replay settings: i) Naive Learning (NL) where no past samples replayed, iî) Exact Replay (ER) where the reservoir can accommodate all past samples, iii) Task- Agnostic Reservoir Sampling, and iv) Task-Aware Reservoir Sampling described in Section 4.2.2
Results. We present results the DST metric on mixed-physies in Fig. 2 with gravity-6 results and complete metrics in Appendix B.1 The gray and blue shaded results respectively represented worst- and best-case scenarios, where the models had (NL) full (ER) access to examples from previous tasks. In ER, the addition of the meta-model significantly improved all performance met- rics, clearly demonstrating the benefits of the what-how model for learning across dynamic systems.
Looking at LP, the effect of experience replay on model's ability to learn the "current" task var- across datasets: while performance on mixed-physics was relatively unaffected, the absence of experience replay had greater impact on gravity-6. Looking at RP, all models struggled to retain performance on previous tasks when experience replay was absent (NL). The inclusion a memory component, whether aask-tgi task-aware proposed, effectively addressed forgetting and stabilized all models, achieving results comparable to exact replay (ER), However, is the combi- nation of the whwt-how modal and the task-aware reservoir sampling strategy that delivered
8

---

Published as a conference paper at ICLR 2025
Task-Aware Reservoir Sampling
Naive Learning
0.12
0.32
0.08
0.08
MSE1
MSE2
0.04
0.02
0.02
10 13 14
12 13
10 17
Gravity
Gravity SE
Pendulum 3G
Mass Spring 1K
Mass Spring 3K Body
Two Body 2G
Gravity
Pendulum
Pendulum
Spring
Body
Figure 3: GeteRGNRes with lask-aware reservoir (left) overcomes alleviate catastrophic forgetting
the continual setting compared to MetaRGNRes without any experience replay (right).
Learning Performance 4
Retained Performance
Backwards-transfer Interference
0.025
0,025
-0.002
0.015
0.015
00,094
0.085
0.005
-0.006
0.000
0.000
860
T-Agnaatic TAlarr
BGD
TAgnostic T-Awars
8GD
Kdgnostic T-Awars
MAML [1-Step]
MAML [5-Step)
Feed-Forward
Figure 4: MSE comparison on gravity-6 over existing CML formulations, averaged over 5 seeds.
significant improvements over all alternatives. This demonstrates the importance of both the con- tinual and meta-learning components, as proposed, for learning forecast non-stationary dynamics systems, Fig. B shows the per-task performance of MetaRGNRes when continually learning over sequentially presented tasks, with (left) and without (right) task-aware experience replay strategies.
5.3 DESIGN CHOICES FOR WHAT-HOW & WHEN: COMPARISON WITH EXISTING CML
Models. We now considered alternative formulations for each of the what-how and when compo- nents in CoSFan using existing CML methods (by adopting them from their original implementa- tions). The initial state encoder and decoder backbones, as well as the architecture of the dynamics function, are identical across these baselines. Differences arise from 1) the design of the what-how meta-models, and 2) the continual learning strategy. More specifically:
Choice of meta-model (what-how): We considered the proposed feed-forward what-how meta- model in comparison to the MAML-based meta-model most commonly used in existing CML works (Riemer et al. 2018 Joseph Gu 2021 Caccia et 2020 Harrison et 2020 He al. 2019)
We evaluate the MAML method with 1- and 5-inner gradient steps for dynamics adaptation.
Choice of muntinual-stratese (when): We considered the proposed Task-Award Reservoir Sampling in comparison to two continual-learning strategies used in existing CML works: standard Task- Agnostic Reservoir Sampling from Section 4.2.2 (Riemer et al.] 2018 Joseph Gu 2021) and
uncertainty-basel weight regularization via Bayesian Gradient Descent (BGD) (He et al. 2019).
Results. Fig. present the MSE results gravity-6, with results mixed-physics and full metries in Appendix B.2 Among the three continual learning strategies, BGD--bsed weight regu- larization (leftmost group) resulted in the lowest performance in both LP and RP. Compared to task- agnostic reservoir, task-aware reservoir showed significant (gravity-6) to moderate umixed-physier improvements in retaining performance and reducing negative interference. While the task task-agnostic approach (middle group) showed better LP for MAML (blue) at the cost of catastrophic forgetting, the task-awar approach (rightmost group) showed little negative to positive transfer.
Comparing meta-moded choices, the feed-forward approach with a task-aware reservoir outper- formed all other combinations of meta-models and continual strategies. Interestingly, in task-

---

Published as a conference paper at ICLR 2025
A
Task Rolationl Experience Replay
Task-Aware Recervoir Sampling
T
Tosk 17
GreeRt
Grovity
Pendutuee
Sprine
Paaee Sprins
Bady
Figure 5: A) Telaioeeass reservoir's meta-embeddings throughout mixed- Black stars represent GMM mixture means. B) Ress-Relational vs. Task-Aware strategics on imbalanced tasks where the first task appears once while others cycle. Considered dynamics are highlighted in gray.
agnostic setting, the feed-forward approach performed similarly or worse than MAML. This con- firmed our suspicion that while the task-agnostic setting works reasonably well for MAML-based strategies to learn a shared initialization, it limits the potential of general meta-learners. Overcoming this limitation via CoSFan has the potential to advance the state-of-the-ar in CML.
Comparing computational efficiency in T the feed-forward method significantly outper- formed MAML in both training and adapta- tion times across all continual strategies. While task-aware MAML had linear slowdown in adaptation per task present, the feed-forward's parallelized forward pass was unaffected.
Table CML adaptation efficiency compari- son. All methods were adapted on 1200 batches. MAML-X refers to the X number of inner-steps.
Model
Metric
T-Agnostic
T-Aware
TTA-1 [s]
000148(0.0009
0.0150(0. 0080)
MAML-1
TTA-12 [s]
0.0149X00 0069)
0.1602(0. 0058)
TTT [min]
38.8(0.2)
116.3(11.0)
TTA-1 [s]
0.0618(0.0077)
1.0670X0.0096
5.4 TASK-AWARE
MAML-5
TTA-12 [s]
0.0636(0.0074)
0.7475(0.0413)
TTT [min]
91.5(0.7)
302. ((2.7)
vs. TASK-REA RELATIONAL BUFFERS
TTA-1 [s]
0.0018(0.0044)
0.0017(0.0043)
TIA-12 [8]
0.0018(0.0043)
0.0017(0.0005)
CoSFan trained with the Task-Relational strat- egy showed a small dip in RP (averaging 20.7%) compared to the Task-Aware approach, remaining significantly better than all other base- lines (full numerical results are in Appendix B.3). The performance drop may be due to some samples being misclassified into the wrong dynamics cluster, leading incorrect context sets. This came with benefit of automatically inferring relationships between all encountered tasks. Fig. 5A illustrates the reservoir's context embeddings and clusters over time, showing both aligned well with the underlying dynamics. Two Hamiltonian systems have their three parameter sets collapse into 1-2 clusters, suggesting the optimization found them insufficiently distinct to warrant separate clusters.
TTT [min]
30.0(0.3)
85.2(5.5)
Effeet of Rare Tasks. We expect task relation distillation to scale more effectively with numerous boundaries or recurring tasks. 53 shows results from an example where the first task appeared once, and three other tasks cycled through 40 boundaries, with the reservoir limited to 500 samples. The Aware approach discarded most of the first task's ' data, retaining only 15 samples by the end, leading to deteriorating performance this task. In contrast, the Tassk-Relaiional approach retained an even distribution of 125 samples per task, retaining stable rare-task performance.
CONCLUSIONS & DISCUSSION
We introduced CoSFan, CML. framework of what-how when, which adapts latent neural dy- namics to non-stationary distributions of dynamical systems without task boundaries or identifiers. Additional ablation results show robustness to the size of k-shot context set (Appendix C.4). learn-
ing rate modulation (Appendix C.5). and increasingly restricted reservoir sizes (Appendix C.3 We
show a resilience to gradual task shifts (Appendix C.8 We also justify the task boundary threshold (Appendix C.7) and use of cluster embedding regularization (Appendix C.6). Limitations. Test- ing on real-world datasets and on broader CML tasks (e.g.. meta- meta-reinforcemmett learning) is needed
to support our findings. The assumption of local stationarity may limit some applications.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Aleksandar Botev, Andrew Jaegle, Peter Wirnsherger, Daniel Hennes, and Irina Higgins. Which priors matter? benchmarking models for learning latent dynamies. (arXiv:2111.05458) Nov 2021. doi: 10.48500/rxiv.2111..555
arXiv:2111 T.w:2111.05455 1.05458 [cs, stat].
Massimo Caccia, Pau Rodriguez, Oleksiy Ostapenko, Fabrice Normandin, Min Lin, Lucas Page- Caccia, Issam Hadj Laradji, Irina Rish, Alexandre Lacoste, David Vázquez, al. Online fast adaptation and knowledge accumulation (osaka): a new approach to continual learning. Advances in Neural Information Processing Systems, 33:16532 16545, 2020.
Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. Advances in neural information processing systems, 31, 2018.
Yinbo Chen and Xiaolong Wang. Transformers as meta-learners for implicit neural representations. In European Conference on Computer Vision, pp. 170-187. Springer. 2022.
Junyoung Chung, Kyle Kastner, Laurent Dinh, Kratarth Goel, Aaron C Courville, and Yoshua Ben- gio. recurrent latent variable model for sequential data. In Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference machine learning, pp. 1126-1135, PMLR, 2017.
Marco Fraccaro, Simon Kamronn, Ulrich Paquet, and Ole Winther. A disentangled recognition and nonlinear dynamics model for unsupervised learning. Advances in neural information processing systems, 30, 2017.
Qiming Fu, Zhechao Wang, Nengwei Fang, Bin Xing, Xiao Zhang, and Jianping Chen. Maml2: meta reinforcement learning via meta-learning for task categories. Frontiers of Computer Science, 17(4):174325, 2023.
David Ha, Andrew Dai, and Quoe V Le. Hypernetworks. arXiv preprint
2016.
James Harrison, Apoorva Sharma, Chelsea Finn, and Marco Pavone. Continuous meta-learning without tasks. Advances in neural information processing systems, 33:17571-17581, 2020.
Xu He, Jakub Sygnowski, Alexandre Galashov, Andrei A Rusu, Yee Whye Teh, and Razvan Pas- canu. Task agnostic continual learning via meta learning. arXiv preprint arXiv:1906.05201 2019.
Timothy Hospedales, Antreas Antoniou, Paul Micaclli, and Amos Storkey. Meta-lcarning in neural networks: A survey. IEEE transactions pattern analysis and machine intelligence, 44(9):
5149-5169, 2021.
Mike Huisman, Thomas M Moerland, Aske Plaat, and Jan N van Rijn. Are Istms good few-shot learners? Machine Learning, 112(11):4635 4662, 2023.
Siddhant M Jayakumar, Wojciech M Czarnecki, Jacob Menick, Jonathan Schwarz, Jack Rae, Simon Osindero, Yee Whye Teh, Tim Harley, and Razvan Pascanu. Multiplicative interactions and where to find them. 2020.
Xiajun Jiang, Zhiyuan Li, Ryan Missel, Md Shakil Zaman, Brian Zenger, Wilson W Good, Rob S MacLeod, John Sapp, and Linwei Wang. Few-shot generation personalized neural surrogates for cardiac simulation via bayesian meta-learning, In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 46-56. Springer, 2022.
Xiajun Jiang, Ryan Missel, Zhiyuan Li, and Linwei Wang. Sequential latent variable models for few-shot high- nigh-dimensional time-series forecasting. In The Eleventh International Conference on Learning Representations, 2023.
Joel Joseph and Alex Gu. La-maml: Look-ahead meta learning for continual learning, ml repro- 2021.
ducibility challenge 2020. arXiv preprint

---

Published as a conference paper at ICLR 2025
Maximilian Karl, Maximilian Soelch, Justin Bayer, and Patrick van der Smagt. Deep variational bayes filters: Unsupervised learning of state space models from raw data. In International Con- ference Learning Representations, 2017.
Arsham Gholamzadeh Khoee, Yinan Yu, and Robert Feldt. Domain generalization through meta- learning: a survey. Artificial Intelligence Review, 57(10):285, 2024.
Matthieu Kirchmeyer, Yuan Yin, Jérémie Donå, Nicolas Baskiotis, Alain Rakotomamonjy, and Patrick Gallinari. Generalizing new physical systems via context-informed dynamics model. In International Conference on Machine Learning, pp. 11283-1 1128-11301 11301. PMLR, 2022.
Rahul G. Krishnan, Uri Shalit, and David Sontag. Deep kalman filters. (arXiv:1511.05121), Nov 2015. arXiv:1511.051 [cs, stat].
Laura Manduchi, Matthias Hüser, Martin Faltys, Julia Vogt, Gunnar Rätsch, and Vincent Fortuin. T-dpsom: An interpretable clustering method for unsupervised learning of patient health states. In Proceedings of the Conference on Health, Inference, and Learning, pp. 236-245, 2021.
Nikhil Mishra, Mostafa Rohaninejad, Xi Chen, and Pieter Abbeel. A simple neural attentive meta- 2017.
learner. arXiv preprint arXiv:
Thanh Nguyen. Tung Luu, Trung Pham, Sanzhar Rakhimkul, and Chang Yoo. Robust maml: Prioritization task buffer with adaptive learning process for model-agnostic meta-learning. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 3460-3464. IEEE, 2021.
Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms. arXiv preprint wrXiv:/803.02990 2018.
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint 1rXiv:/810.119/// 2018.
Anthony Robins. Catastrophic forgetting, rehearsal and pseudorehearsal. Connection Science, 7(2):
123-146, 1995.
Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint 2016.
Utkarsh Utkarsh, Valentin Churavy, Yingbo Ma, Tim Besard, Prakitr Srisuma, Tim Gymnich, Adam R Gerlach, Alan Edelman, George Barbastathis, Richard D Braatz, et al. Automated trans- lation and accelerated solving of differential equations on multiple gpu platforms. Computer Methods in Applied Mechanics and Engineering, 16591 2024.
Gido M Van de Ven and Andreas S Tolias. Three scenarios for continual learning. arXiv preprint arXiv: 2019.
A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Jeffrey S Vitter. Random sampling with reservoir. ACM Transactions on Mathematical Software (TOMS), 11(1):37-57, 1985.
Max Vladymyrov, Andrey Zhmoginov, and Mark Sandler. Continual hypertransformer: A meta- learner for continual few-shot learning. Transactions on Machine Learning Research.
Risto Vuorio, Shao-Hua Sun, Hexiang Hu, and Joseph J Lim. Multimodal aggostii meta- learning via task-aware modulation. Advances in neural information processing systems, 32, 2019.
Rui Wang, Robin Walters, and Rose Yu. Meta-learning dynamics forecasting using task inference. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 21640-21653 Curran Associates, Inc., 2022.
12

---

Published as a conference paper at ICLR 2025
Omry Yadan. Hydra a framework for elegantly configuring complex applications. Github, 2019.
URL http:///h....ccccceeeeeeeeerrraa
Cagatay Yildiz, Markus Heinonen, and Harri Lahdesmaki. Ode2vae: Deep generative second order odes with bayesian neural networks. Advances in Neural Information Processing Systems, 32, 2019.
Yuan Yin, Ibrahim Ayed, Emmanuel de Bézenac, Nicolas Baskiotis, and Patrick Gallinari. Leads: Learning dynamical systems that generalize across environments. Advances in Neural Informa- lion Processing Systems, 34:7561-7573, 2021.
Luisa Zintgraf, Kyriacos Shiarli, Vitaly Kurin, Katja Hofmann, and Shimon Whiteson. Fast context adaptation via meta-learning. In International Conference on Machine Learning, pp. 7693-7702.
PMLR, 2019.
13

---

Published as a conference paper at ICLR 2025
A RELATED WORKS
In addition to learning-to-lest latent dynamics and CML., the proposed work is also relevant to the areas of research in continual learning and meta-continual learning.
Continual-learninn Relevant to these limitations is the research area of continual learning, in which the fundamental goal is to handle data non-stationarity as the ability to simultaneously pre- serve old performance while accelerating the aggregation of new knowledge. Under this framework, incoming data is seen as a sequence of distinct tasks expressing unique properties that differentiate themselves (Van de Ven & Tolias 2019). A multitude of approaches have been proposed under continual learning to alleviate catastrophic forgetting, with the most representative approaches cat- egorized as: i) rehearsal- -aasd methods that approximate the stationary training distribution via storing or generating samples over time (Robins, 1995). ii) structural methods that preserve sub- networks of neural networks over time to maintain per-task performance (Rusu al., 2016), and iii) regularization-busel methods that leverage the sequentially learned Bayesian posteriors as priors that constrain learning on new tasks (He et al. 2019).
However, an underlying assumption of these continual methods is that, regardless of what tasks have been are being learned, the network is able learn a set of parameters that can perform well across all tasks. In heterogeneous time-series forecasting, this is often an invalid assumption as the input- output mapping can conflict as the same observation can provide different forecasts depending on the (potentially unknown) system context. As such, naïve continual learning is not directly applicable to this setting as it lacks both task-inference and context-aware adaptation mechanisms.
Meta-continual learning. The recent field of meta-continual learning (MCL), though similar in name and approaches considered, is distinct from CML in its goal and optimization. CML uses continual learning techniques to enable the training of meta-learning models over distribution of tasks that appears sequentially over time. Its goal is to preserve the performance of the meta-model on the previous tasks, even when little-to-no data for those tasks are available, while also aggregating knowledge from the current task. Conversely, MCL aims to frame the continual learning problem as a meta-learning problem, in which each meta-task is some variation on the order that the continual learning tasks are presented in. Its goal is to meta-learn the continual learning algorithm such that on a new sequence of tasks, can adapt quickly. Unlike CML, which has context-dependent targets, MCL assumes fixed target distribution of some x-y mapping. Due to differences in the goal of the meta-models and the data settings considered, we do not consider MCL relevant to our evaluation and omit comparisons against their methods.
Summary of related work. The terminology and goals of meta-continual learning and continual meta-learning are nuanced and often challenging to distinguish, as both adapt the same fundamental methods to different ends. We defer to the breakdown in (Caccia et al. (2020) for a formalized review of MCL and CML, and how they relate to meta-learning and continual- leammigg, respectively.
Alternative algorithmic priors. Beyond gradient- based and feed-forward hyper-networks, se- quence learners such as Transformers [Vaswani 2017) are powerful architectures for extracting knowledge in high-dimensional settings. In standard meta-learning, Transformers have been used as algorithmic priors (Chen & Wang 2022) by combining an initial token set of parameters with to- kenized context data to adapt the weights, conceptually similar approach to gradient-based based meta- learners' initial parameter set. A recent CMI. work (Vladymyrov et al.) further explores using Transformer architectures as hyper-networks to generate target network weights based on the con- text set. In their framework, the weights generated for previous task are reused as parameter tokens to update the current task's weights alongside active task samples. Notably, this method omits the use of a replay buffer, relying instead on weights updated iteratively through active samples. While this approach primarily targets image classification tasks, adapting it to latent dynamics settings offers promising direction for future exploration.
In this work, we focus on applying feed-forward algorithmic priors, specifically in the form of a hyper-network, and comparing their performance to the standard CMI approach of gradient-based meta-leasners. However, exploring other algorithmic priors represents an intriguing avenue for fu- ture research.
14

---

Published as a conference paper at ICLR 2025
B FULL QUANTITATIVE RESULTS
B.1 COMPLETE SECTION 5.2 RESULTS
Here we include the full numerical experimental results for the comparison existing latent dy- namics baselines on gravity-6 and mixed-physie in Table and Table 3 respectively. The numer- ical results of these methods trained under the stationary distribution, where all tasks are available simultaneously and have known task identifiers, are shown in Table
As well, we include similar visualization as Fig. but for MSE on mixed sexxed-shysicsti in Fig. and for DST and MSE gravity-6 in Fig. and Fig. respectively.
Table 2: Performance comparison gravity-6 over existing latent dynamics models and their con- tinual extensions. All methods were run over 5 seeds.
Naive
Exact Replay
Boundary Reservoir
Model
Metric
DST
MSE
DST
MSE
DST
MSE:
LP
9.13(107)
0.0234(0.0003
6.39(0.07)
0.0219(0. 0001)
7.98(1.06)
0.0228(0.0004
DKF
8.80(0.77)
0.0234(0.0005)
6.79(0.11)
0.0222(0.000))
7.13(0.47)
0.0226(0.0003)
BTI
0.33(0.30)
0.0000(0.000M
-0.39(0.15)
-0.0003(0.000))
0.85(0.97)
0.0002(0.0003)
I.P
9.17(0.77)
0.0221(0.0000)
4.69(0.18)
0.015000.000)
7.49(0.71)
0.0190X0.0011 1)
VRNN
RP
10.07(0.51)
0.0215(0.0000)
8.99(0.17)
0.0214(0.0005) 0002)
9.08(0.78)
0.0225(0.00188
BTI
-0.9(00.77)
0.0006(0.0000) (0.0000)
1.30(0.19)
0.0065(0.0003))
-1.59(0.57)
0..035(00.0015)
LP 9.23(0.52)
0.0195(0.00022
2.73(0.16)
0.0129(0.
3.82(0.53)
0.0146(0.0009)
RGNRes
12.63(1.30)
0.0203(0.00 (001)
7.830.17)
0.0199(0 0005)
7.48(0.20)
0.0214(0. 0008)
BTI
-3.40(1.45)
-0.0009(00000))
-5.10(0.15)
-0.00710000005)
-3.66(0.34)
-0.0068(0.0010)
LP
9.92(0.32)
2.0199(0.0022
1.6200.88)
0.0111(0.0000
1.67(0.09)
0.0114(0.000)
MetaRGNRes
RP
13.04(1.22)
0,0207(0.000)
1.57(0.03)
0.0108(0.0001 00001)
1.72(0.08)
0.0120X0.000 0002)
B11
-3.12(1.11)
-0.0008(0.0001)
0.05(0.06)
0.0002(0.0002)
-0.05(0.07)
-0.0006(0.0001
Table 3: Performance comparison mixed-physics over existing latent dynamics models and their continual extensions. All methods were run over seeds.
Naive
Exact Replay
Boundary Reservoir
Model
Metric
DST
MSE
DST
MSE
DST
MSE
2.73(0.39)
0.0228(0.0013)
2.52(0.24)
0.0220(0.0000
2.93(0.41)
0.0231
DKF
4.49(0.33)
0.0445(0.01
2.78(0.31)
0.0223(0.0008)
2.84(0.31)
0.0226(0.0007)
BTI
-1.76(0.66)
10.0217(000155)
-0.15(0.03)
0.0003(0.0003
0.09(0.12)
0.0005(0.0005)
I.P
2.48(0.23)
0.0164(0.0010)
2.71(0.38)
0.0169(0.0007)
2.97(0.51)
0.0182(0.0009)
VRNN
RP
4.87(1.01)
0.0434(0.0034
3,70(0.45)
0.0214
3.78(0.52)
0.0234(0.0005)
BTI
-2.50(0.84)
-0.0270(0.0034)
0,99(0.19)
0.0045(0.0008)
-0.81(0.21)
-0.0052(0.001
LP 2.43(0.82)
0.016200.0022)
1.56(0.00)
0.0143(0.0000
1.92(0.33)
0.0162(0.0010)
RGNRes
4.75(0.32)
0.0578(0.0141)
2.94(0.00)
0.0188/0. (000)
2.70(0.30)
0.0208(0. 0008)
BTT
-1.87(0.91)
50.0415(000120)
-1.38(0.00)
-0.004((000000)
-0.78(0.07)
-0.0046(100055)
LP
1.32(0.00)
0.0128(0.0000
1.54(0.07)
0.013%0000000
1.56(0.14)
0.014100.0007)
MetaRGNRes
RP
4.52(0.00)
0.0736(0.0000)
1.59(0.08)
0.0140(0.0009)
1.61(0.13)
(0.008
BTI
-3.19(0.00)
-0.0608(0.0000)
-0.05(0.09)
-0.0001(0.0003)
-0.05(0.02)
.0.0013(0.000)
Table 4: Performance comparison gravity-6 and mixed-physics over existing latent dynamics models trained on the true stationary distribution. All methods were run over 5 seeds.
gravity-6
mixed-physie
Model
DST
MSE
DST
MSE
DKF
6.41(0.00)
0.0220(0.0000)
4.14(0.03)
0.0220(0.0001)
VRNN
1.86(0.14) 0.0192(0.0000)
4.37(0.05)
0.0201(0.000))
RGNRes
7.37(0.12) 0.0185(0.0001)
2.37(0.02)
0.0203(0.0001)
MeteRGNRes
3.51(0.45) 0.0197(0.0011) 1)
2.34(0.23)
0.0194(0.0011)
15

---

Published as a conference paper at ICLR 2025
Learning Performance
Retained Performance J
Backwards-Transter Interference
0.00
-0.01
0.05
0.05
0.04
0.82
0.02
-0.05
0.01
-0.06
ML
EAgnostic T-Aware
ER
ML
T-Agnostic T-Aware
ML
TAgnostic TAware
Figure 6: MSE performance comparison on mixed-physies over existing latent dynamics models and their continual extensions. All methods were run seeds.
Learning Performance
Retained Performance
Backwards-Transfrr Interference
12
12
DST
Tehgnsstic
T-Agnsttcc Trhwarn
T-ggnaatic
RGNRes
MetaRGNRes
Figure 7: DST performance comparison on gravity6 over existing latent dynamics models and their continual extensions. All methods over seeds.
Learning Performance
Retained Performance
Backwards-Transfer Interference
0.025
6.225
MSE
0.015
MSE
0.015
MSE
0.004
0.030
0.010
-0.086
0.065
0.005
0.000
0.000
T-Agnottic T-Awarn
NL.
T-Agnoetic T-Awar
NL T-AgnorticT-inne T-Aware
DKF
VRNN
RGNRes
MetaRGNRes
Figure 8: MSE performance comparison gravity-6 over existing latent dynamics models and their continual extensions. All methods were run over seeds.
16

---

Published as a conference paper at ICLR 2025
B.2 COMPLETE SECTION 5.3 RESULTS
Here we include the full numerical experimental results for the comparison to existing CML for- mulations on gravity-6 and mixed-physics in Table 6 and Table respectively. complete efficiency comparison, including BGD.
Table shows the
As well, we include similar visualization as in Fig. but for DST and MSE on mixed-physics in Fig. 9 and Fig. I0, respectively, and for DST on gravity-6 in Fig. I
Table 5: Performance comparison on mixed-physics over existing CML formulations and the feed- forward approach. All methods were run over 5 seeds.
BGD
Task-Agnostic
Task-Aware
Model
Metric
MSE
DST
MSE
MSE
10.92(0.00)
0.0563(0.0000)
1.76(0.21)
0.0162(0.0014)
1.56(0.14)
0.0141(0.0007)
Feed-Forward
RP
8.50(0.00)
0.056330.0000 0000)
2.70(0.20)
0.0219(0.0012)
1.61(0.13)
0.0154(0.000M
B11
2,42(0.00)
-0.000000.0000)
-0.94(0.01)
-0.005700.0000 00000))))
-0.0500.02)
.0.0013(0.00))))
LP
6.66(1.18)
0.0505(0.0083
1.83(0.20)
0.0164(0.0009)
2.68(0.13)
0.0182(0.000) (0082))
MAML [1-Step]
5,90(0.59)
0.0516(0.0037)
2.67(0.34)
0.0208(0.00L))
2.62(0.14)
0.0185(0.0010)
BTI
0.77(0.99)
-0.001
-0.84(0.17)
-0.0044(0.0007)
0.06(0.07)
-0.0003
LP
9.60(4.38)
0.0503(0.00k4
2.18(0.35)
0.0176(0.0009)
3.07(0.58)
0.0192(0.0008
MAML [5-Step]
RP
10.70(6.25
0.0515(0.00300
2.81(0.40)
0.0207(0.0014
2,92(0.43)
0.0186(0 0007)
-1.10(1.96)
-0.0012(0.00000
-0.63(0.11)
-0.003100.0008)
0.15(0.22)
0.0006(0.0004)
Table 6: Performance comparison on gravity-6 over existing CML formulations and the feed-
forward approach. All methods were run over 5 seeds.
BGD
Task Agnostic
Task-Aware
Model
Metric
DST
MSE
DST
MSE
DST
MSE
LP
8.22(0.54)
0.0235(0.0004)
4.47(3.39)
0.0143(0.0019)
1.67(0.09)
0.0114(0.0001)
Feed-Forward
RP
(1.11(0.33)
0.0243(0.0001)
8.81(2.65)
0.0206(0.00017)
1.72(0.08)
0.012010.0002
-2.90(0.72)
-0.0009(0.0004
4.35(0..79)
.0.006300.002))
-0.05(0.07)
-0.0006(0.000))
LP
8.57(1.27)
0.0236(0.0000 0.007)
2.12(0.05)
0.0129
5.34(0.24)
0.0165(0.0005)
MAML [1-Step]
RP
10.68(0.53)
0.0244(0.0000
1.24(0.18)
0.019)(0.0003)
5.94(0.44)
0.0174(0.0004)
BTI
-2.11(1.37)
0..00080000007
-5.12(0.16)
-0.0070(0 0000)
-0.60(0.41)
-0.0009
LP
7.70(0.76)
0.0232(0 (006)
2.56(0.48)
0.0137(0.0006)
4.67(1.89)
0.0156(0.0018)
[5-Seet
10.19(0.87)
0.0242(0.0005,
6.81(0.14)
0.0198(0.0007))
5.31(243)
0.0156(0.0017)
BTI
-2.49(0.79)
-0.001000.0066)
-4.25(0.43)
-0.0661000002)
-0.6410.76)
-0.0000(0.0005)
Table 7: Complete adaptation efficiency comparison the choice of meta-model and continual- strategy. All methods were adapted over 100 batches of data across the 12 mixed mixed-physicss dynamics, timing each batch independently. Bold and italic indicate 1st- and 2nd-best metries, respectively.
Model
Metric
BGD
Task Task- Agnostic Reservoir
Task-Aware Reservoir
TTA [1] (seconds)
0.0155(0.0011)
0.0148(0.0009) 0.0149(0.0069)
0.0150(0.0088) 000000))) 0.166((0.0008))
MAMI. [1-Step]
TTA [12] (seconds)
0.0152(0.0013)
TTT (minutes)
126.3(4.7)
38.8(0.2)
116.3(1 1.33(110)) 1.0)
TTA [1] (seconds)
0.0637(0.0025)
0.0618(0.0077)
0.0670(0.009) 00.6))
MAML [5-Step]
TTA [12] (seconds)
0.0623(0.0015)
0.0636(0.0074)
(..477((0.0433))
TTT (minutes)
315.8(6.5)
91.5(0.7)
302.8(22.7)
TTA |]] (seconds)
0.0017(0.0005)
0.0018(0.0048)
0.0017(0.0045)
Feed-Forward
TTA [12] (seconds)
0.0018(0.000))
0.0018(0.0043)
0.0017(0.0005
TTT (minutes)
122.1(3.4)
30.0(0.3)
85.2(5.5)
17

---

Published as a conference paper at ICLR 2025
Learning Performance
Retained Performance
Backwards-Transfer Interference
10
10
DST
T-Agosticc
T-Agnstiic
[1-Stap]
Feed-Forward
Figure 9: DST performance comparison on mixed-physics over existing CML formulations and the ferward approach. All methods seeds.
Learning Performance
Retained Performance 1
Backwards-Transfer Interference
0.05
0.05
0.64
0.000
MSE
-0.092
0.02
0.01
0.03
8G0
T-ggnettic RS T-tware R5
BGD
T-Agnostic RS T-Auare RS
BGD
TAgnostic RS T-Aware RS
MAML [1-Stop]
MAML [5-Stop]
Foed-Forward
Figure 10: MSE performance comparison mixed-physics existing CML formulations and the feed-forward approach. All methods were rum over seeds.
Learning Performance
Retained Performance
Backwards-Transfer Interference
12
DST
T-Agnastic T-lwarn
TrAgnostic
T-Awaru
T-Agnsticc
RS
[1-5tep)
Food-Forward
Figure 11: DST performance comparison gravity-6 over existing CML formulations and the beed-forward approach. All methods were run over seeds.
18

---

Published as a conference paper at ICLR 2025
B.3 COMPLETE SECTION 5.4 RESULTS
Here we include the full numerical experimental results for the comparison between the Task-Aware Reservoir Sampling methods and the Task-Relational Experience Replay gravity-6 and mixed- physics in Table g As well, Fig. 12 and Fig. 13 show the per-task performance both continual strategies when continually learning over the sequentially presented tasks, on mixed-physics and gravity-6 respectively.
Best Practice. We suggest using the Reas-Relational approach under limited memory scenar- ios, when there is prior knowledge about frequent switching among recurring tasks under- representation of rare tasks. In settings where is assumed only novel tasks will show up over time, the Task-Aware method is sufficient to identify the tasks and comes with better training efficiency.
Table 8: Comparison between Task-Aware Reservoir Sampling and Task-Relationa. Experience Re-
play. seeds were tested. Bold and italic indicate 1st- and 2nd-best metrics, respectively.
Task-Aware Reservoir Sampling
lask-Relational Experience Replay
Model
Metric
DST
MSE
DST
MSE
LP
1.56(0.14)
0.0141(0.0007)
1.61(0.12)
0.0143(0.0008)
mixed -hxescs
RP
1.61(0.13)
0.0154
1.91(0.22)
0.0172(0.0012)
BTI
-0.0((0.02)
-0.0013(0.0002)
-0.29(0.14)
000029(000007)
LP
1.67(0.09)
0.0114(0.0001)
1.67(0.07)
0.0115(0.0002)
gravity-6
RP
1.72(0.08)
0.0120(0.0002)
2.11(0.26)
0.0128 00066)
BTI
-0.05(0.07)
-.000060000033)
-0.44(0.31)
-00003(000000))
Task-Aware Reservoir Sampling
Task-Relational Experience Replay
0.040
0.040
0.030
0.030
0.020
0.020
0.015
0.010
0.010
6.085
0.000
0.000
15
10
Gravity
Gravity SE
Pendulum 3G
Mass Spring 1K
Mass Spring 3K
Two Body 2G
Gravity
Pendulum
Pendulum
Spring
Two Body
Two Body 3G
Figure 12: Performance comparison per-task over the continual sequence between Twsk-Aware Reservoir Sampling and Task-Relational Experience Replay evaluated on mixed-physics.
Task-Aware Reservoir Sampling
lask-Relational Experience Replay
0.0175
0.0175
0.0350
0.0150
MSE
0.03.0
0.0180
0.0675
0.0075
0.0025
0.0025
0.0000
0.0000
Gravity
Gravity
Gravity
Gravity
Gravity
Figure 13: Performance comparison per-task over the continual sequence between Task-Aware
Reservoir Sampling and Task-Relational Experience Replay evaluated on gravity-6.
19

---

Published as a conference paper at ICLR 2025
C ADDITIONAL ABLATION RESULTS
C.I ADDITIVE VS. MUL.TIPLIAATIV INTERACTIONS
The original meta-sLVM formulation proposed by TTang al. 2023 considers additive condi- tioning mechanism, where the derived context variable c(T;) conditions the latent dynamics func- tion by concatenating with the latent state, represented as 2 co,)) before propagating the dynamics. In this work, we choose the hyper-network conditioning mechanism instead, as prior eval- uations of conditioning interactions suggest that multiplicative interactions, such as those enabled by hyper-networks, are super-set of additive interactions and provide stronger inductive biases for a broader range of function families (Jayakumar et al. 2020).
Table compares the performance of the additive mechanism with the proposed hyper-network con- ditioning method, both evaluated using Twsre Reservoir Sampling on mixed-physics. Fig. [4] highlights the progression of the tasks' metries over the course of training between both methods. The results indicate comparable performance between the two methods, suggesting that the consid- ered baselines do require the additional flexibility of multiplicative interactions, and that this flexibility does not hinder convergence.
Table 9: Performance comparison between the additive conditioning mechanism from (Jiang et al. 2023 and the proposed hyper-network conditioning mechanism evaluated on mixed sexxedphysiccs
Learned
Retained
Interference
Method
DST
MSEJ
DST
MSE
DST
MSEJ
Hypernet Additive
1.56(0.14)
0.014100.000))
1.61(0.13)
0.0154(0.0008)
-0.05(0.02)
-..0013(0.0002)
1.6((0.08
0.0145(0.0006)
1.62(0.10) 0.0155(0.0009)
-0.01(0.08)
-..001(0000004)
Additive Conditioning
Hyper-network Conditioning
DST
1STO
Gravity
Gravity SE
Pendulum 3G
Mass Spring 1K
Mass Spring 3K
Two Body 2G
Pendulum
Pendulum
Spring
Body
Body
Figure 14: Performance comparison per-task over the continual sequence between the additive con- ditioning mechanism from (Tang al. 2023) and the proposed hyper-network conditioning mech- anism evaluated on mixed-physics.
20

---

Published as a conference paper at ICLR 2025
C.2 COMPUTATIONAL REQUIREMENTS OF RELATIOAAIIO EXPERIENCE REPLAY
In this section, we evaluate the computational and memory requirements of Task-Relational Expe- rience Replay during training. Notably, this replay strategy does not impact adaptation efficiency at test time and introduces additional computational demands only at detected task boundaries. For fitting the Bayesian Gaussian Mixture Model, we utilize the scikit-learn library due to its robust implementation and feature set. This process is performed on the CPU, requiring data transfer be- tween the GPU and CPU for the meta-embeddiings, which may introduce a minor slowdown in the presented results here. While GPU-based implementation would eliminate this transfer and pro- vide computational speed-ups, we note that the current CPU-based fitting remains efficient, with execution times within seconds for the datasets evaluated.
To assess the scalability the Bayesian Gaussian Mixture Model with respect to task numbers and reservoir size, varied the reservoir size from 500 to 4500 samples in increments of 1000. For each configuration, over range 12 unique dynamics, we measured the average mixture model fitting time (in seconds), the average memory usage during fitting (in MB), and the average memory usage after fitting (in MB). Fig. 15 presents these metrics when applied over seeds. The results indicate favorable scalability in both memory and speed requirements the reservoir size and number of unique dynamics increase.
We measured the Time-to-Train metric for the Task-Relational Experience Replay mechanism and compared it to the Task-Aware Reservoir Sampler on the mixed-physics dataset, using a reservoir size of 4500 samples and averaging results over 5 seeds. The Task-Aware mechanism required 83.04 + 0.67 minutes to train, while the Task-Relational mechanism required 85. + 2.25 minutes, indicating a relatively minor increase of 2.5% training time for the additional benefits provided by task-relational modeling.
21

---

Published as a conference paper at ICLR 2025
Model Fitting Time
0.04
Time
0.03
0.02
Peak Memory Usage During Fit
2.00
1.75
Memory (MB)
1.00
0.75
0.25
Memory Usage After Fitting
0.040
Model Memory (MB)
0.035
0.025
0.020
0.010
12
Number Unique Dynamics
Reservoir Size
500
1500
2500
3500
4500
Figure 15: Memory and speed performance ablation the mechanism over increasing reservoir sizes and number of unique dynamics.
Experience Replay
22

---

Published as a conference paper at ICLR 2025
C.3 EFFECT OF RESERVOIR SIZE
We investigated the task-aware feed-forward method across increasing reservoir sizes on mixed- physies understand the extent which can preserve performance less and less data becomes available for prior tasks. The results are shown in Fig. T6 for MSE and Fig. T7 for DST It can be seen that while Learning Performance remains similar across the reservoir sizes, the Retained Performance and catastrophic forgetting exhibited lessens up to certain point before beginning to stabilize at the higher numbers.
Learning Performance
Retained Performance
Backwards-Transfer Interference
0.0175
0.0175
-0.001
0.0125
0.0125
-0.002
0.0100
0.0100
-0.003
0.0675
0.0075
0,0050
0.0050
-0.008
0,0000
0.0000
560
Size
Reservair
Figure 16: MSE performance of the task-aware feed-forward meta-model on varying reservoir sizes for mixed-physice
Learning Performance
Retained Performance
Backwards-Transfer Interference
2.5
2.5
0.0
-9.4
0.5
0.3
-0.8
-1.0
500
1500
2500
3500
4500
500
1560
2500
3500
4500
500
1500
2500
3660
4506
Reservoir Size
Reservoir Size
Reservoir Size
Figure 17: DST performance of the task-aware feed-forward meta-model on varying reservoir sizes for mixed -hysiccs
23

---

Published as a conference paper at ICLR 2025
C.4 EFFECT OF CONTEXT-SET lt
We performed an ablation for the feed-forward model with task-aware reservoir sampling over in- creasing k-shot context set sizes mixed-physics, shown in Fig. [8 for MSE and Fig. 19 for DST A slight trend of improvement can be seen in the Retained Performance as the number of samples in the context increases, though notably the variance exhibited in the Learning Performance and Backwards- Transffr Inference of DST that decreases at higher k-shots.
Learning Performance
Retained Performance
Backwards-Transfer Interference
0.0200
0.4266
0.60000
0.0150
0.0199
-0.00075
0.0100
0.1109
-0.00190
0.0150
0.0050
-0.00175
-0.00200
0.0000
0.0000
Figure 18: MSE performance of the task-aware feed-forward meta-model varying k-shot sizes for mixed-physics.
Learning Performance
Retained Performance
Backwards-Transfer Interference
2.0
2.0
DST
0.5
0.0
0.0
K-Shot Size
K-Shot Size
K-Shot Size
Figure 19: DST performance of the task-aware feed-forward meta-model varying k-shot sizes for mixed-physee physscs
24

---

Published as a conference paper at ICLR 2025
C.5 EFFECT OF UPDATE MODULATION
Learning rate schedulers are often essential for achieving good convergence in deep learning (Botev al. ).2T). However, this presents unique challenge in settings where task boundaries are known. While local stationarity could guide scheduling, available strategies (i.e., fully reducing the learning rate by the end of window or maintaining higher learning rate) pose potential op- timization risks. Inspired by the update modulation schema described in (Caccia et al. 2020), we tested using a dynamic modulation coefficient that scales the learning rate based on loss curred at each timestep. This coefficient is calculated using the equation um = 1 e ve, where the hyper-parameter adjusts the modulation's likelihood scale from 0 to
Results of ablation over varying values for and DST, respectively. Interestingly, we found that neither the presence nor value of the update modulator had much of an effect overall convergence. We attribute this to the datasets having generally stable likelihood range from batch to batch as well as the choice of an adaptive learning rate may provide enough of stabilization that additional component top shows little effect.
presented in Fig. 20 and Fig. 21 for MSE
Learning Performance
Retained Performance
Backwards-Transfer Interference
0.012
6.012
0.0000
0.068
0.008
-0.0002
MSE
MIS
0.006
6.006
-0.0004
0.004
0.052
0.002
-0.0006
0.1
0.15
0.5
1.0
Nome
0.1
0.15
0.5
3.0
None
0.1
0.15
0.5
3.0
None
Update Modulation Value
Update Modulation Value
Update Modulation Value
Figure 20: MSE performance of the task-aware feed-forward meta-model on varying update modu- lation um values for gravity-6.
Learning Performance
Retained Performance 1
Backwards-Transfer Interference
1.75
1.90
1:50
0.10
1.25
0.05
0.00
0.75
-0.05
-0.10
0.00
0.00
0.5
Update Modulation
Update Modulation Value
Update Modulation Value
Figure 21: DST performance of the task-aware feed-forward meta-model varying update modu- lation um values for gravity-6.
25

---

Published as a conference paper at ICLR 2025
C.6 EFFECT OF CLUSTER REGULARIZATION
We performed an ablation study on the cluster regularization term for the feed-forward model with task-relational experience replay, evaluating its effect across range of values in the loss function. Fig. [2 and Fig. 23 presents the numerical results for MSE and DST, respectively, on the mixed- physies task. The inclusion the cluster regularization term stabilized the BTI of the meta-model across different seeds though left LP and RP mostly unaffected as the weight increased.
Fig. visualizes the final reservoir context-embedding space for models with no regularization and with regularization strength of le-2. The inclusion of the regularization term improved the separation of the Two-Body equations into more distinct clusters and, notably, disentangled the Gravity dynamics from the Pendulum system.
Learning Performance
Retained Performance
Backwards-Transfer Interference
0.0200
0,6200
0.8175
0.0005
0.0130
0.0150
0.0300
0,0100
0.0875
0.0075
0.0050
-0.0029
0,0925
0.0025
1e-4
1e:3
1e-2
1e:1
1e.6
1e.3
1e.2
1e-1
10:4
1e-3
1e-2
1e-1
Regularization Beta
Regularization Beta
Regularization Beta
Figure 22: MSE performance of the task-relational feed-forward meta-model methods on varying cluster regularization values for mixed mhixes-physics.
Learning Performance
Retained Performance 4
Backwards-Transfer Interference
0.2
3.5
1.5
DST
-6.2
0.0
0.0
Regularization
Regularization
Regularization
Figure 23: DST performance of the task-relational feed-forward meta-model methods varying cluster regularization values for mixed-physiee
No Regularization
1e-2 Regularization
Gravity Gravity SW
Pendulum
Spring Mass Spring 2K
Pendulum 3G
Two Body 2G
Gravity
Pendulum 4G
Mass Spring 3K
Gody
Figure 24: t-SNE visualization of the reservoir's resulting context-embedding space for model with no cluster regularization (left) and with a 3 of 1e-2 for cluster regularization (right).
26

---

Published as a conference paper at ICLR 2025
C.7 TASK BOUNDARY THRESHOLD
We investigated the task boundary detection mechanism and justify the use of the threshold param- eter, v. Fig. 25 and Fig. 26 show the progression of the task-aware cerd-forward model's training likelihood across the tasks for both gravity-6 and mixed-physies across the seeds, respectively. Markers show the flagged performance dip on the task boundary where context and query sets stem from different origins. For both datasets, it is clear that significant dip occurs at the task switch, making feasible to use task boundary detection method. We note that the choice of dependent on the data distribution and expected likelihood ranges.
Rather than a static threshold value, it may be possible instead maintain an online estimate of the training likelihood mean μ.C, and standard deviation o,,, flagging new task whenever a likelihood comes that out-of-distribution via:
[|C3nn Lin--ll MC, + (2.5 o2,)
(8)
20-
Likellhood
10
5000
10000
25000
20000
23000
Iteration
Figure 25: Progression of the 's training likelihood over tasks on gravity-6, highlight- ing via markers the performance dip exhibited at task boundaries across the seeds. Shaded areas represent one standard deviation.
70-
Likelihood
5000
10000
15000
20000
25000
30000
33000
Iteration
Figure 26: Progression of the feed-forward's 's training likelihood over tasks on mixed-physics. high- lighting via markers the performance dip exhibited at task boundaries across the seeds. Shaded areas represent one standard deviation.
27

---

Published as a conference paper at ICLR 2025
C.8 PERFORMANCE UNDER GRADUAL TASK BOUNDARIES
Our task boundary detection mechanism relies on identifying performance dips during task shifts when context and query samples originate from disjoint tasks. A potential limitation of this approach arises when task shifts are gradual, resulting in mixture of samples from the current and next tasks. In this section, we present an ablation study on increasingly mixed task boundaries, evaluating the detection accuracy of the task boundary mechanism and the overall training performance under these conditions.
To evaluate this, we tested the Task-Aware Reservoir Sampler (noting that the task boundary detec- tion mechanism is identical in the Task-Relational setting) under a scenario where mixed batches of current and next-task samples composition, with the percentage of new-task samples incrementally decreasing from 100% to 20% of the total batch size. Following the initial mixed batch, the subsequent batch consisted entirely of new-task samples to assess whether the task boundary mechanism could correct to new the samples or if it failed due to the sliding window of likelihoods used for detection.
introduced task boundaries. The mixed batches varied in
Fig. 27 and Fig. 28 present the 5-seed results of the boundary detection experiments on mixed- physies and gravity-6 datasets, respectively. These figures show the percentage of identified task boundaries across the mixture percentages for the fixed (mixed) batch as well as the percent of addi- tional boundaries identified on the subsequent (pure) batch. Note that the second hatch percentages additive to the mixed batch percent, so the difference between them is the performance of the correction. They additionally include the mean and standard deviation of the recorded likelihood differences. Note that, even at higher percentages of new data, the identification rate may be below 100%. This can occur in cases where technically different dynamics, based on parameter configu- rations, are optimized under single cluster because the model does not find their realizations to be sufficiently distinct to justify separate clusters. This phenomenon is illustrated in Fig. 5 where two of the Hamiltonian equations (Pendulum and Mass-Spring) with unique parameter configurations collapse into 1-2 clusters each.
Table 10 and Table II summarize the overall performance metrics and the number of successfully identified task boundaries across varying mixture percentages. For both datasets, the total number of task boundaries differs slightly from the expected 18 (mixed-physics) and 12 (bouncing-ball) due to omitting cases where the same task appears
For mixed-physies as the percentage of new samples decreased, the number of tasks identified during the mixed batch dropped significantly. However, the subsequent correction batch managed to capture the majority of boundaries. The mean of the mean likelihood differences steadily decreased, overlapping with the threshold syper-parameter likelihood difference remained stable across all percentages. For the overall performance metrics, steady though modest decline in all metrics was observed. Despite the decline, the method still performed significantly better than the baseline gradient- musie meta-learners.
around 40%. The standard deviation of the mean
In contrast, for gravity-6, which features harder-to- distiiiggg dynamics from the high- diheeeeinnii observations, performance at gradual task boundaries was notably worse. Even the correction batch succeeded in only 45% of cases the percentage decreased. The mean likelihood difference began overlapping with v earlier, at 60%, and its standard deviation showed a sharp drop around 40%. In the performance metrics, significant decrease in the performance occurred across all metrics. However, we note that in this dataset still, the method performed better than the baseline gradient- based meta-learners.
The boundary detection mechanism demonstrates some resilience to gradual task boundaries, but noticeable decline in identification success occurs as the signal produced by mismatched sam- ples weakens. For datasets with more diverse dynamics, such as mixed-physics, this effect is less pronounced. However, for datasets with handte-to-distingiish dynamics, such as bouncing-ball, this limitation becomes more significant. Given this, auxiliary components to handle increasingly gradual task boundaries is required to extend this framework to a broader set of complex continual meta-learning settings. We note that despite the decrease in identification success, that the method still outperformed related Task-Agnostic versions. We posit that the Task-Aware setting only fully degenerates to the Task-Agnostt setting when all task boundaries are missed and that the inclu- sion of even some losk-idntiificttion can still provide significant benefit to the metaaoptimizztion. Further study of this result with additional ablation is needed, which we leave future work.
28

---

Published as a conference paper at ICLR 2025
A straightforward modification the detection mechanism is to incorporate recent buffer of like- lihoods over steps and compare the mean of that buffer to the new likelihood. While this adjustment may not resolve issues with identifying mixed batches, it could enhance the resilience of the cor- rection steps that follow. The current boundary detection mechanism relies solely on the likelihood from the previous step, which allows gradual task boundaries to slowly shift the distribution of like- lihoods into the range of the new task without producing significant enough difference to surpass the detection threshold v.
Percent of Boundaries Identified
Moan of Likelihood Differences
Std of Likelihood Differences
160
Mixed Batch Banch
Thresheld'
Standard Deviation
Percent (%)
60-
20
Percent Samples
Samples
Percent Samples (%)
Figure 27: Gradual task boundary ablation for the Task-Aware Reservoir Sampling mechanism
mixed-physics. Left) Percent of the total boundaries identified between the mixed task boundary step and the subsequent pure batch of only new samples. Middle) Mean of the mean likelihood differences as calculated by boundary detection mechanism in Sec. 4.2.1] Dotted black line refers the considered threshold hyper-parameter in experiments. Right) Standard deviation of the mean likelihood differences.
Percent Boundarles Identifled
Likellhood Differences
Likellhood Differences
Batch
Standard Deviation
Percent (1)
Mean
15
10
Percent New Samples (%)
Percent New Samples (%)
Percent New Samples (%)
Figure 28: Gradual task boundary ablation for the Task-Aware Reservoir Sampling mechanism on gravity-6. Left) Percent of the total boundaries identified between the mixed task boundary step and the subsequent pure batch of only new samples. Middle) Mean of the mean likelihood differences as calculated by boundary detection mechanism in Sec. 4.2.1 Dotted black line refers the consid- ered threshold hyper-parameter in experiments. Right) Standard deviation of the mean likelihood differences.
29

---

Published as a conference paper at ICLR 2025
Table 10: Performance metrics for the gradual task boundary ablation for the Task-Aware Reservoir Sampling mechanism on mixed-physics. % New Data represents how much of the batch at the task boundary composed of new samples, with 20% meaning only 20% of the given batch is new sam- ples while the remaining 80% is the current task. # Tasks Identified represents how many boundaries were successfully flagged by the boundary mechanism out of the average total boundaries.
Learned
Retained
Interference
Tasks Identified
DST
MSE
MSEL
DST
MSEJ
100%
16.00.9917..4
1.59(0.11)
0.0143(0.0005)
1.58(0.06)
0.01 500.000))
0.01(0.07)
-0.0012(0.0004)
80%
15.8(0.7117.4
0.0145(0.0006) 1.72(0.22)
1.680.122 12)
0.0159(00.00))
0.04(0.11)
0.0015(0.0002))
60%
15.000.9/17.4
1.74(0.24)
0.0146(0 0007)
(.)
0.0155(0.0008)
0.04(0.04)
-0.0099(0.0033)
40%
15.0(133/17.4
1.75(0.20)
0.0146(0.0008)
1. 16690..66
0.0155(0.0009)
0.06(0.08)
-0.000900.0003)
20%
13.6(L5V17.4
1.79(0.19)
0.0149(0.0005)
L82(0.21)
0.0155(0.0008)
-0.03(0.14)
-0.0006(0.0004)
Table 11: Performance metrics for the gradual task boundary ablation for the Task-Aware Reser- voir Sampling mechanism on gravity-6. % New Data represents how much of the batch the task boundary is composed of new samples, with 20% meaning only 20% of the given batch is new sam- ples while the remaining 80% is the current task. # Tasks Identified represents how many boundaries were successfully flagged by the boundary mechanism out of the average total boundaries.
Learned
Retained
Interference
56 New Data
Tasks Identified
DST/
MSEJ
DST
MSR
DST
MSR
100%
9.0(0.6)1144
L.72(0.09)
0.0117(0.0002)
1.6((0.06)
0.0118(0.0000
0.08(0.09)
-0.0000X0.
9.0(0.6)/11
1.71(0.04)
0.0117(0.0001)
1.6600.08)
(..118(0.0002))
0.05(0.08)
-0.000100.0002
60%
7.2
3.13(2.41)
0.0135(0.0030)
2.80(1.75)
0.0142(0.0036)
0.32(0.67)
-0.0008(0.0000
40%
5,2(2.6)/144
3.61(2
0.0147(0.0130)
2.87(1533)
0.0143(0.0035)
0.75(0.81)
0.0004(0.0012 2)
20%
5.410.5)/114
2.14(0.81)
0.0122(0.0012)
1.70(0.07)
0.0115(0.0003)
0.44(0.82)
0.0007(0.0012)
30

---

Published as a conference paper at ICLR 2025
D IMPLEMENTATION DETAILS
In this section, we give the specific resources and considerations for each. All experiments were NVIDIA RTX3090 GPUs with 24GB memory in instanced cloud systems to control hardware purity. We used Py PyTocch 1.13.1 and scikit-leam 1.4.2 for deep learning optimization and GMM fitting, respectively.
on each experiment over all models, as well as
D.0.1 DYNAMICS SCHEMATIC VISUALIZATIONS
We present visualizations of the schematics of the considered dynamics and their governing equa- tions in Fig. 29
Fig. 30
we present ground truth sequence visualizations for each of the governing
equations.
Bouncing Ball
Pendulum
Mass Spring
Two Body
2² 2m
mgy + 80
mlg(1 cos ())
2m
P 2m0
gNO,AR 10:
200
Figure 29: sehematic-baseer visualizations of the varying dynamics along with their underlying equations. Context variables that differentiate tasks within the same dynamic are highlighted in blue
Bouncing Ball
Pendulum
Mass-Spring
Two-Body
Figure 30: Image-based visualizations of the varying dynamics.
D.0.2 DATA GENERATION DETAILS
Bouncing Ball dynamics were simulated through the PyMunk Physics Engine (www. pymunk. org), , following the procedure of (Jiang et al. 2023). For gravity-6, we choose gravitation forces of 160" 120% 180 240 300 360 with time simulation At 0.25 and initial velocity angle lim- its between f0 180%). For the 3 Bouncing Ball dynamics of mixed-physics, we choose gravitation forces [90", 210 330%). keeping the other settings the same.
The Hamiltonian equations were simulated through the Deepmind Hamiltonian Suite (https: // deepmind/ httpi/ddddmmmamaa dmm_hamil ton an wttttonnnnanniisssssssess as www.ddcaaanssiittteooe satsssssueee.eeeee following
github. ccm/ procedures and suggested initial values described in (Botev al. 2021). Notably, we took the red color channel from the simulations to turn them from RGB to grayscale. No friction values were considered. For the Pendulum equation, considered three gravitational values of [2,3, 4). For the Swo-Body equation, we considered three gravitational values [1, 2, 3). For the lass-Spring equation, considered three spring constants of 2, 2..
31

---

Published as a conference paper at ICLR 2025
D.0.3 EXPERIMENT IMPLEMENTATION DETAILS
All models were trained forecast 20 timesteps using only the first 3 frames, with the exception of DKF and VRNN, which were given 8 frames for fairness. For both datasets, we set the window of local stationarity 1500 iterations, where each iteration composed of 32 active task samples. For CL, we used reservoir size that can accommodate approximately 25% of the total data at any given time, resulting in a reservoir size of 4500 for both gravity-6 and mixed-physics. Ablations on reservoir sizes and their impact are provided in Appendix C.3 The size of the k-shot context set was fixed at A 15 for the main experiments, although we include an ablation over varying sizes in Appendix C.4 We used public implementations of DKF and VRNN for training. To ensure fairness, comparable model components to the meta-models were scaled to maintain consistent total parameter counts of 1 million trainable parameters. Shared backbone components had identical hyper-parree paramettass,rss while model-specific seperrparaamterss were tuned.
To test LP and RP metries, considered held-out testing sets of every dynamics, for both gravity-6 and mixxed-hysicc. To test IP,, whenever task boundary was detected, the task that just finished was evaluated its testing set for its performance. At the end of the task sequence, on the resulting model, we evaluated it on every task independently and averaged their performances to get the RP metric. BTI, then, was the average difference between all task's first LP (in the event that task re-emerged throughout the sequence) and their respective RP.
D.0.4 ARCHITECTURE FOR META MODELS
The implementation of the considered meta-models is here: TBD handle configuration setup, in which every model's configuration file is available in the configs/ folder. The specific models, memory, and datasets to use can be changed via command-ling argu- ments (e.g., model=metargnres memory=boundary dataset=mp). The 5 seeds considered are 1 1111, 2222, 3333, 4444, 5555. Detailed hyperparameter values are shown below for the feed-forward and MAML models, as well as the existing latent dynamics models. These settings are shared across memory settings and datasets.
We use Hydra (Yadan) 2019)
Feed-Forward Architecture
Domain Input: 20 observation timesteps of 32 32 dimensions
Initialization Input: 3 observation timesteps of 32 32 dimensions
Optimizer: Adam W, 5 10-3 learning rate
Gradient Norm Clipping: 5
Transition: Recurrent Generative Network (RGN-res)
Batch size: 32 active, 32 reservoir
2e Latent Size:
Transition Network: [SiLU(Linear(8, 64)), SiLU(Linear(64,, 64)), Tanh(Linear(64,, 8))]
Domain Encoder Filters: [32, 32]
Domain Time Units: [10, 5, 1]
Initial Encoder Filters: [32, 64, 128]
Emission Filters: [128, 64, 32, 1]
Zo KL Beta: A = 10-2
KL Beta: A2 10-3
Cluster Loss Beta: A 1
MAML Architecture
Domain Input: 20 observation timesteps of 32 x 32 dimensions
Initialization Input: observation timesteps 32 32 dimensions
Inner Optimizer: SGD, 10-4 learning rate
32

---

Published as a conference paper at ICLR 2025
Outer Optimizer: AdamW, 5 x 10-3 learning rate
Gradient Norm Clipping:
Transition: Recurrent Generative Network (RGN-res)
Batch size: 32 active, 32 reservoir
Latent Size: 8
Transition Network: [SiL.U(Linear(8, 64)), SiLU(Lineear(64, 64)), Tanh(Linear(644, 8))
Domain Encoder Filters: [32, 64, 32]
Domain Time Units: [10, I]
Initial Encoder Filters: [32, 64, 128]
Emission Filters: [128, 64, 32, I]
KL Beta: A1 = 10
KL Beta: A2 10 10-3
VRNN Architecture
Input: observation and 12 prediction timesteps of 32 x 32 dimensions
Optimizer: AdamW, 1 10-3 learning rate
Gradient Norm Clipping:
Batch size: 32 active, 32 reservoir
Latent state dim ze: 32
Dropout probability: 0.2
Dense X Size: 512
Dense Z Size: 512
Dense H(X)-Z Size: 256
Dense H(Z)-X Size: 256
Dense H(Z) Size: 256
RNN Layers: 2
RNN Dim: 128
Beta coefficient: 1.0
Activation: LUL..R)
DKF Architecture
Input: observation and 12 prediction timesteps 32 x 32 dimensions
Optimizer: AdamW, 1 x 10-3 learning rate
Gradient Norm Clipping: 5
Batch size: 32 active, 32 reservoir
Transition: RNN Unit
Encoder Units: [1024, 512, 256, 256]
RNN Units: 256
Correction Units: 256
Transition Units: 256
Emission Filters: [128, 64, 32,
33

---

Published as a conference paper at ICLR 2025
RGNRes Architecture
Initialization Input: 3 observation timesteps of 32 x 32 dimensions
Optimizer: AdamW, 5 x 10-3 learning rate
Gradient Norm Clipping:
Transition: Recurrent Generative Network (RGN-res)
Batch size: 32 active, 32 reservoir
Latent Size: 32
Transition Network: [SiLU(Linear(32, 256)), SiLU(Linear(256, 256)), Tanb(Lincar(256, 32))]
Initial Encoder Filters: [32, 64, 128]
Emission Filters: [128, 64, 32, I]
KL Beta: A1 = 10
34


Reviews:
[{"review_id": "2bu6squwZC", "reviewer": "Reviewer_GEod", "summary": "This paper introduces CoSFan, a method for meta continual learning that the paper evaluates on sequences of dynamical systems tasks. The approach consists of a hyper network based approach where the parameters of the current task being adapted to are generated using a forward pass of the hypernetwork on a context vector created using the average encoding of the samples in a context set. The generated parameters are used in a dynamics model that predicts the latent of the next element in the sequence and is then decoded. The meta-parameters and the encoder/decoder are optimized using the MSE between predicted and ground truth query sequences. The paper also explores how to detect task boundaries to manage/balance the number of samples in a replay buffer being used to rehearse on previous tasks in the sequence. It proposes two different mechanisms: one where it looks at any spike in loss as a task change, and one where it use a gaussian mixture model to cluster examples and detect whether any new clusters have formed.\n\nThe paper evaluates on a series of image based dynamical systems tasks where the physics/gravity constants are changed in each task, and the model must predict the next state. The paper shows an improvement in a slight improvement in learning performance over other meta learning approaches, but shows a clear improvement in forgetting over other approaches when using both their meta learning approach and the task detection based replay.", "questions": "- For the experiments depicted in Figures 2 and 4, it seems each task is trained for the same number of samples. In this setting, why is there a big difference in the retained performance of specifically the meta learned methods between task agnostic and task aware replay? Wouldn’t the number of samples of each task still be approximately the same given that the replay size per task is still in the hundreds? Is the difference of a few samples really making that big of a difference?\n- How does the hypernetwork based method scale with the number of heterogeneous tasks?\n- I am a bit unclear about the setting. Is there a different context set sampled for each query example? Or is it just that the previous k sequences is treated as the context set?\n- It would be interesting to see an adaptation based baseline which was also given the context vector as input.\n- Do you have a version of Figure 3 comparing the Task aware replay with the task relational replay?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Figure 5 shows a very clear use case for the task relational replay over even a task aware (and presumably task agnostic) replay.\n- Figure 4 shows that the meta learning approach proposed by the paper helps mitigate forgetting when task detection is used.\n- The method proposed can adapt to new sequences/new samples much faster than gradient based adaptation approaches, since they don’t need to take any gradient steps.", "weaknesses": "- The way Figures 2 and 4 are presented is a bit confusing. For one, they use different metrics, and it’s unclear what the insight from using one over the other is. For figure 2, the red bar is the only meta-learned method, so the other 3 methods were presumably evaluated in a different setting. Were they given the “context” information? It seems the more direct comparison with baselines is in Figure 4. \n- It’s unclear if the hypernetwork based approach can scale better than an adaptation based approach when the number of heterogeneous tasks increases. \n- As the authors do mention, the task shift detection seems to rely on abrupt task shifts, and would likely fail on gradual task shifts. In the case where the task shift detection fails, and the default is to task agnostic reservoir sampling, this method seems to do slightly worse than other meta learning based approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DWdZbThQ4p", "reviewer": "Reviewer_U6Tv", "summary": "This paper proposed a new method for continual meta-learning (CML). Different from previous methods (e.g. MAML variants), the proposed method uses a single feed-forward network. Using the contex encoder and hyper network, the support samples are encoded to context vector, and the context vector is used to produce the parameters for encoded query samples using the hyper network. Furthermore, the authors also proposed task-aware reservoir sampling approach by using the gaussian mixture model to identify the tasks. In the experiment section, the proposed methods outperforms the gradient-based meta learners, and also show the effectiveness of using task-aware sampling strategy.", "questions": "Already mentioned in weaknesses section", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Strengths\n\n1. Different from the gradient-based methods which needs a number of gradient steps to adapt, the proposed methods can adapt to novel tasks using a sinlge feed-forward network.", "weaknesses": "Weaknesses\n\n1. I think the motivation behind each component is weak. For example, for adapting to the query samples, the CosFan uses the hyper network to produce the parameters. However, why we should use the hyper network framework in this feed-forward network? Isn't it possible to use other trainalbe embedding networks for the query samples?\n\n2. The mechanism for detecting the task boundary is not novel. The detection mechanism simply comes from the methods in [1]. However, though the authors show that the advantage of CosFan over other baselines lies on the ability for detecting the task boundary without any assumptions on the task identifiers, I think other baselines can adopt the task boundary detection mechanism used in this paper with simple modification. Therefore, I don't think the ability on detecting the task boundary in CosFan is advantage compared to other methods\n\n3. The overall notation is confusing. For example, in line 217~220, what is the measing of superscript s in $T_j$? Is it support set? I think it is not clear. Furthermore, in the definition of $T_j$, the internal sequences are not dependent on $j$. I think the authors should clarify all the notations to prevent the confusion.\n\n\n[1] Caccia et. al., Online Fast Adaptation and Knowledge Accumulation (osaka): A New Approach to Continual Learning, NeurIPS, 2020", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bTOFze9bFs", "reviewer": "Reviewer_i4xk", "summary": "This paper proposes a continual meta-learning framework (CoSFan) for forecasting high-dimensional time-series data generated by non-stationary distributions of dynamic systems. CoSFan addresses the limitations of traditional meta-learning approaches that assume stationary task distributions and known task identifiers. It proposes a feed-forward \"what-how\" meta-model to quickly infer the current dynamic system and adapt a latent dynamics function accordingly. Furthermore, it introduces mechanisms to detect task boundaries and employs task-aware and task-relational experience replay for continual adaptation of the model, mitigating catastrophic forgetting. Experiments on simulated physical systems demonstrate CoSFan's ability to learn and adapt to new tasks while retaining performance on previously seen ones, outperforming existing CML alternatives and standard latent dynamic models with continual extensions.", "questions": "1. Line 215, what is s in $T_j^s$? Could you add an explaination of what s is in the text?\n2. Line 216, what's z? Could you add an explaination of what z is in the text?\n3. Eq. 3, what's $l$? Is it summing over $l$?\n4. Line 367, typo \"withou\".\n5. Figure 2, why is it some method task aware replay performs better than full ER?\n6. Table 1, could you add a description to what number represent in the caption? Could you add the unit in the table?\n7. For task-relational buffers, why is the wrong cluster assigned? Is it because of the context embedding is not well-represented? If so, is it due to the meta-learning objective? What could be done to make it better?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is clearly written. Different components of the system and how they work together to form the final solution are clearly described. \n\nThe experiment section provides lots of insight, with studies of individual components in the framework and comparisons with other alternative solutions. These ablations are especially important for a complex system.", "weaknesses": "The paper opts for using hypernet as the meta-learner. Although it ablates the hypernet solution to alternatives such as MAML and Bayesian gradient descent, it lacks the comparison with a sequence leaner, such as Transformer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7zwgaKvfu5", "reviewer": "Reviewer_JYok", "summary": "The paper introduces CoSFan, a continual meta-learning framework that adapts to changing dynamics in high-dimensional time-series data. By using a novel \"what-how & when\" model, it detects task shifts and quickly adjusts to new tasks, minimizing catastrophic forgetting. The claim is that CoSFan outperforms existing methods in accuracy and adaptability across non-stationary data distributions.\n\n### Contributions\n- Novel Continual Meta-Learning (CML) Framework: CoSFan, designed for continual adaptation of latent dynamics in time-series forecasting. It combines slow and fast adaptation mechanisms.\n\n- What-How & When Framework:\n  - What-How Meta-Model: Quickly adapts to specific tasks by identifying system dynamics (what) and generating task-specific parameters (how) using a feed-forward hyper-network.\n  - Automatic Task Shift Detection: Identifies task boundaries to update the model for non-stationary distributions.\n\n- Experience Replay Mechanisms:\n  - Task-Aware Reservoir Sampling: Uses boundary detection to pseudo-label tasks for better context-query pairing.\n  - Task-Relational Experience Replay: Clusters tasks using Bayesian Gaussian Mixture Models to handle frequent transitions and maintain rare tasks.", "questions": "See weaknesses", "limitations": "", "rating": 10, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Well written paper.\n- Effectively combines feed-forward meta-learning with task-aware adaptation, addressing limitations in prior CML approaches reliant on gradient-based updates.\n- Uses task-relational experience replay and GMM clustering to manage task transitions, a creative extension for handling non-stationary data.\n- Robust empirical results validate CoSFan’s advantages, demonstrating reduced catastrophic forgetting and better adaptability than existing methods.\n- Methodology is rigorous with well-defined metrics and detailed evaluation, covering key aspects of adaptation speed, memory usage, and performance retention.\n- Generally clear presentation of the framework and methodology, with a structured breakdown of components (\"what-how & when\") aiding comprehension.\n- Potential to influence further work in both CML frameworks and high-dimensional latent dynamics forecasting, with broader implications for adaptive AI in dynamic settings.\n- The use of 5 seeds with average and standard deviation for all experimental results is greatly appreciated and confirms that these results are not simply a lucky seed; although the std remains very high on certain experiments.", "weaknesses": "- There is work to be done on the presentation of the figures (too small, need better explanations/more exhaustive captions, not clear/to many overlapping lines...)\n---\n- Limited to synthetic datasets (e.g., bouncing balls, Hamiltonian systems); adding real-world datasets (e.g., financial time series, climate data) would strengthen claims of applicability.\n(Weakness acknowledged by the authors in limitations)\n---\n- The comparison with prior CML methods, especially MAML-based and task-agnostic approaches, lacks deeper discussion on specific performance trade-offs (e.g., speed vs. accuracy in adaptation).\n- The added computational cost of GMM clustering and task-relational replay is not well-documented; detailing memory and processing requirements would help assess scalability.\n---\n- Assumes detectable task shifts and local stationarity; the model’s robustness to gradual or overlapping task shifts is not explored, limiting the generalizability to more fluid non-stationary environments.\n- A quantification of how slow is to slow for the task shift to be picked up by the model would be interesting.\n- Suggested improvements: include experiments with blurred task boundaries or transitions to assess flexibility in ambiguous scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)