Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks
Authors: Susmit Agrawal
Krishn Vishwas Kher
Saksham Mittal
Swarnim Maheshwari
Vineeth N. Balasubramanian
Venue: NeurIPS 2025 poster ()
Keywords: Associative Memories
Domain Generalization
Continual Learning
Primary area: deep_learning
TL;DR: We propose Memory-Integrated Reconfigurable Adapters, a unified ML framework integrating Hopfield-style associative memories atop a shared backbone, demonstrating remarkable flexibility across domain shifts and sequential task exposures.

Abstract: Organisms constantly pivot between tasks such as evading predators, foraging, traversing rugged terrain, and socializing, often within milliseconds. Remarkably, they preserve knowledge of once-learned environments sans catastrophic forgetting, a phenomenon neuroscientists hypothesize, is due to a singular neural circuitry dynamically overlayed by neuromodulatory agents such as dopamine and acetylcholine.
In parallel, deep learning research addresses analogous challenges via domain generalization ($\textbf{DG}$) and continual learning ($\textbf{CL}$), yet these methods remain siloed, despite the brain’s ability to perform them seamlessly. In particular, prior work has not explored architectures involving associative memories ($\textbf{AM}$s), which are an integral part of biological systems, to jointly address these tasks. We propose Memory-Integrated Reconfigurable Adapters ($\textbf{MIRA}$), a unified framework that integrates Hopfield-style associative memory modules atop a shared backbone. These memory modules store adapter-weight updates as values and retrieve them via learned keys. Associative memory keys are learned post-hoc to index and retrieve an affine combination of stored adapter updates for any given task or domain on a per-sample basis. By varying only the task-specific objectives, we demonstrate that $\textbf{MIRA}$ seamlessly accommodates domain shifts and sequential task exposures under one roof. Empirical evaluations on standard benchmarks confirm that our $\textbf{AM}$-augmented architecture significantly enhances adaptability and retention: in $\textbf{DG}$, $\textbf{MIRA}$ achieves SoTA out-of-distribution accuracy, and in incremental learning settings, it outperforms architectures explicitly designed to handle catastrophic forgetting using generic $\textbf{CL}$ algorithms. Extensive ablation studies validate the necessity of both associative memory storage and post-hoc key learning for robust interpolated retrieval of adapters. By unifying adapter-based modulation with biologically inspired associative memory, $\textbf{MIRA}$ delivers rapid task switching and enduring knowledge retention in a single extensible architecture, charting a path toward more versatile and memory-augmented AI systems.

Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks
Susmit Agrawal 12,3 12.34+
Krishn Vishwas Kher
susmit. waw.:at..gg.aaa@@eehheee....
cs cs19b23300001111thhooo ac. in
Saksham M
Swarnim Maheshwari cs2 cs2mmtechh20@@ithh.......
ai22stech1102401thh ac. in
Vineeth N. Balasubramanian 1,4
vineethnb@cse with. ac.in www.ethoottbmccicccoooooooo
IIT Hyderabad
Universityy of Tübingen
Tübingen AI Center
Microsoof Research, India
Equaa Contribution Majority of work done at IIT Hyderabad
Abstract
Organisms constantly pivot between tasks such evading predators, foraging, traversing rugged terrain, and socializing, often within milliseconds. Remarkably, they preserve knowledge of once-learned environments a phenomenon neuroscientists hypothesize, due to singular neural circuitry dynamically overlayed by ncuromodulatory agents such as dopamine and acetyl- choline. In parallel, deep learning research addresses analogous challenges via domain generalization (DG) and continual learning (CL), yet these methods remain siloed, despite the brain's 's ability to perform them seamlessly. In particular, prior work has not explored architectures involving associative memories (AMs), which are an integral part of biological systems, to jointly address these tasks. We propose eeemory-Interaaee Reconfigurable Adapters (MIRA), unified framework that integrates Hopfield-style associative memory modules atop shared backbone. These memory modules store adapter-weight updates as values and retrieve them via learned keys. Associative memory keys are learned post-hoe to index and retrieve an afline combination of stored adapter updates for any given task or domain per-sample basis. By varying only the task-specific objectives, we demonstrate that MIRA seamlessly accommodates domain shifts and sequential task exposures under roof. Empirical evaluations standard benchmarks confirm that our AM-augmented architecture significantly enhances adaptability and retention: in DG, MIRA achieves SoTA -istof--iitiibutiin accuracy, and in incremental learning settings, outperforms architectures explicitly designed handle catastrophic forgetting using generic CL. algorithms. Extensive ablation studies validate the necessity of both associative memory storage and post-hoc key learning for robust interpolated retrieval of adapters. By unifying adapter-based modulation with biologically inspired associative memory, MIRA delivers rapid task switching and enduring knowledge retention in single extensible architecture, charting a path toward more versatile and memory-augmented Al systems.
catastrophic forgetting,
Project Page: htte://snimm.iihubee cattuubiio/mrrr_whhe
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
Organisms across the animal kingdom navigate myriad environments and behavioral demands, flexibly switching between survival tasks (such foraging for food evading predators) and complex social interactions, within fractions of a second. Concrete examples include: echolocating bats, which adjust their sonar pulse rates from 20 to 200 Hz in milliseconds when tracking evasive prey, while simultaneously computing three-dimensional flight paths to avoid obstacles [25] 73 or jazz pianists among humans who instantaneously transition between playing memorized composition and spontaneous improvisation, a cognitive shift marked by distinct prefrontal activation patterns 8 61 Similarly, many animals (including humans) learn navigate a particular environment, such as intricate pathways of dense forest, or subtle acoustic cues of predator's 's approach, and retain that knowledge indefinitely, without the catastrophic forgetting that plagues AI systems [95] (2]. Such phenomena are commonly attributed to the brain's 's capability to rapidly repurpose the same circuitry Some neuroscientific observations
for multiple tasks without dismantling its core wiring 18 62
indicate the presence of overlapping sets of neurons that encode multiple task rules simultaneously with neuromodulatory signals that regulate the active rule at a given time [5].
62
From another perspective, the field of deep learning has developed a rich taxonomy of paradigms that echo these natural behaviors. Domain generalization (DG) methods ensure robustness to distribution shifts [9]; for example, a driver-assistance model that has learned to drive during daytime adapts to driving night.
learning (DIL) 38 seeks learn and identify the same objects in new orientations, abstractions, and settings; for example, a model learns objects from cliparts and then learns to identify the same objects in anime or real world. Class- cusss-incremenall learning (CIL) 38 aims to accumulate knowledge of newer classes as they arrive over time without forgetting; for example, a model that learns to identify flora and fauna being introduced to data from new continent. Although these paradigms differ in terms of data availability, distributional shifts, and forgetting dynamics (and are treated so most commonly in literature), they share a common thread: adapting efficiently to new tasks environments. Efforts in these paradigms have largely progressed in isolation, unlike the brain where such adaptation tasks are handled conjointly [62 71 99]. We seek to address this gap in this work. An ancillary line of work on parameter-effiiien fine-tuning (PEFT) has attempted adaptation with an objective of parameter efficiency by freezing a given base model and adapting it to new tasks domains via small, task-specific "adapters". These adapters overlaid over the base model to allow switching between different tasks. Techniques such as LoRA instantiate this idea.
(36], VeRA [77, and FourierFT 23
However, despite a conceptual resem- blance to neural task-switching, ex- isting work has predominantly over- looked explicit memory-based mech- anisms that biology suggests are fun- damental to rapid and efficient adapta- tion [44]. Motivated by this observa- tion, we propose novel architecture and learning methodology that explic- itly integrates biologically plausible associative memory models into deep learning frameworks, as illustrated in Figure Diverging from contempo- rary work in associative memories that stores raw data or their representa- tions, our architecture stores weight adapters as values within an associative memory. Furthermore, instead of using fixed keys, we learn retrieval keys post hoc to optimally recall adapters for task-specific modulation of the substrate network. These keys facilitate accurate and context-sensitive retrieval through Hopfield network, effectively generating affine combinations of adapter adjustments required for task sess--pecific modula- tion at inference time, thus enhancing knowledge retention and out-of-domoe generalization. Our proposed neuro-inspired framework thus establishes a common umbrella architecture capable of simultaneously and effectively addressing DG, CIL,, and DIL scenarios. Our approach differentiates itself by serving all these settings through only minor adjustments task-specific objective functions. Our key message in this work is to expound the utility of integration of associative memories into DL
Spec-Specific Networks
Adapters
Detect objects
night time
Associative Memory
Classity flora and fauna that arrive over time without catastrophic forgetting
Base
Network
Recognize objects clipart and anime formats
Figure 1: Associative memories can enable networks quickly adapt diverse tasks, by storing and recalling task-specific weights on-demand. MIRA proposes framework for such an approach.

---

architectures for improved efficacy on multiple tasks, rather than any specific heuristic or method to outperform baselines on one given setting, Our main contributions comprise:
Unified Framework: We introduce MIRA. framework that leverages biologically inspppeee
associative memories to propose unified architecture for DG, CIL, and DIL.
Key Refinement of Hopfield Networks: Our core technical novelty lies in embedding Hopfield networks in every ViT layer, dynamically aligning their keys preceding layer activations instead of using static indexing keys, thus allowing the model to learn appropriate indexing rules.
Comprehensive empirical evaluation: We study MIRA on standard benchmarks across multi- ple settings, attaining state-of-the-ae (SoTA) accuracy in multiple settings, outperforming task- specialized architectures by as much as 10% in some cases.
2 Background and Related Work
Memory in Deep Learning: Traditional Hopfield networks +0 pioneered the computational models of associative memories by allowing a set of stored binary patterns to be retrieved via energy minimization. Recent variants, including Modern Continuous Hopfield Networks (MCHN) [69] and Universal Hopfield Networks (UHN) [63] improved upon the original to achieve exponentially greater storage capacity in addition to storing and retrieving real-valued vectors. Other forms of explicit memory have also been previously used in various architectures [100] 28 29), integrating explicit read/write operations an external memory module to support long-range dependency handling. albeit each having its own distinct formulation. Recent works have also studied memory networks that can operate via Predictive Coding better emulate biological memories [T66] 82).
Recent advancements have explored the integration of associative memory mechanisms into diverse machine learning paradigms. Kaliency-Guidde Hidden Associative Replay (SHARC) [5] framework utilizes associative memory to store and replay salient data representations, enhancing retention of prior knowledge. (43 present a spiking neural network model that emulates associative memory functions for classifying neuroimaging data. [103] develop a neuromorphic computing framework that integrates global and local learning mechanisms, drawing inspiration from the brain's 's associative memory processes. These works only consider memories as storage medium for data, independent of the main forward pass. (2] postulates that associative memories can store and retrieve neuromodu- latory signals given the input context, achieving performance comparable to storage and retrieval of model weights from disk. However, it considers the AM as disjoint module from the main neural storage medium. We instead use AMs as an
network, posing it as akin to integral part of the forward pass, adjustable via backpropagation.
Adapters in Contemporary Models:
The surge in large pre-trained models has led to methods that minimize the computational and storage overhead associated with fine-tuning. Techniques like LoRA 440 105)
19] factorize weight updates into low-rank matrices, while Prefix Tuning (55] Adapter Layers [35,, LayerNorm Tuning [188), and BitFit 77 similarly constrain the number of trainable parameters. These methods significantly reduce computational and storage overhead by constraining trainable parameters to minimal subsets or low-rank structures.
(36] and its variants 47 24 107
Unified Frameworks for Multiple ML Settings: Machine Learning literature tackled multiple paradigms like Domain Generalization (DG) [109) Continual Learning (CL) 190), and multi-task Our work focuses on a unified framework driven by the practical necessity of robust models that generalize across new and previously unseen tasks, or learning new tasks and domains without forgetting prior knowledge. Unlike existing methods, our MIRA framework brings fresh approach, providing both high adaptability and robust knowledge retention in single extensible architecture, marking a significant innovation in handling diverse and evolving ML challenges.
learning (MTL) [1]] largely in isolation with few exceptions 44 65
3 Method
Preliminaries and Notation. Formally, a task t e [T'] is defined by a dataset, D, = ((x ,, 1
sampled from probability distribution p(0) (x((1) x y(0)). Here, T denotes the total number of tasks, and x, y(t), the domains of features and labels for task respectively.
3

---

01.1
FCN
FCN
adapter 2 adletter
LN
LN
Transformer Blocks LoRA adapters
M Menury M
AWW
MHAA
MILA
adayter
0LL
LN
LN
MIRA: Adaptation
MIRA: Consolidation
MIRA Adptattin
MIRA Consolidatio
MIRA: Adaptation
MIRA: Consolidation
MIRA: Adaplation
MIRA: Consoldation
MIRA: Adactation
MIRA Conseldation
D2
D2
D
D2
D
Time
Domain Generalization
Domain/Class Incremental Learning
Figure 2: Overview of MIRA for Domain Generalization and Continual Learning scenarios. In DG, all training tasks are provided together to both the Adaptation and the Consolidation stages. In the CL scenarios, the dataset for each task arrives sequentially, and each dataset is passed to both stages, The Adaptation stage trains adapters for each task, while the Consolidation stage learns the associated keys for the stored adapters.
Our method is demonstrated each layer e E L, we attach rank-r LoRA adapters to the Query and Value matrices of the transformer, jointly represented by a flattened vector of²) E Rdd.
frozen substrate network F (e.g., a ViT) consisting of T. layers. At
Adapters are stored in UHN memory units M, attached to each layer. Each memory unit is equipped with write operation, denoted by W(M,, k 0), and a read operation R(Me, q). A write operation the default UHN writes a key-value pair (k, (in that order) to M, and a read operation retrieves weighted combination of values stored in Mr when prompted with query q. Concretely, if K, dRd XN and O &d represent keys-value pairs respectively:
R(Me, q)
O sep(sim ,
where sep is separation function such as Softmax, and sim similarity function like the Euclidean inner product.
The Modern Hopfield Network 668 (MHN) proposed with the Softmax function to achieve superlinear storage capacity with respect to the query embedding dimension. (63] generalized the notion proposed by the MHN to accommodate functions besides Softmax, dubbed "Separation functions" as these functions are responsible for assigning weights to stored memories during recall. Separation functions facilitate accurate retrieval of specific memory by assigning very high weight to the concerned memory, or allow for recall of superposition of memories by assigning diffused weights. We primarily use an affine function compute the weights of the combination.
We illustrate the efficacy of our approach on three learning paradigms comprising DG, DIL, CIL. The primary differences between these settings lies in how data is made available (summarized in Table [). while the overarching objective in all these settings to achieve high accuracy on the test distribution using optimally trained parameters &o across all training tasks, i.e.:
min (( [xx;; sp) # yi]-
Conceptual Framework. The chosen settings underscore the need for the trained parameters yp to consolidate knowledge adapted from diverse tasks, enabling effective use of relevant information

---

Data Availability
DG
DIL
CIL
Tasks arrive sequentiall??
X
Same label sets across tasks?
/
X
Task identifier available at inference?
X
X
X
Test distribution seen during training?
X
Table 1: Comparison of data availability across ML settings.
during inference. This is usually done via multiple "expert" models, each learning subset of the tasks at hand, making their individual predictions [4] 14 72]. Heuristics then dictate which output is considered at the end. At their core, however, all these methods rely the parameters learned per task, which serve as reservoirs of task-specific knowledge acquired during training. Memory systems capable of storing and retrieving such parameters based on (possibly transformed) input query thus a natural in such settings, as they allow retrieval of sspput-ppccficc weight combinations on demand. The central question, then, is how to efficiently retrieve appropriate set ensemble of parameters on a per-sample basis at test time. Formally, for an input dataset D, this can be posed as an optimizatiin problem over the weights of the combination, a.k.a.
|
(ai.) = arg min E (a.g))D and
#
t=1==
In general the above problem is highly non-convex and may involve first/second order methods to solve, which is computationally demanding at a per-sample level. We critically observe that, for an arbitrary key matrix M, Euclidean inner product as the similarity function, and values θ₂ the associative memory retrieves consolidated parameters for F if at = sep((M, (x))) for some function (z)... The following lemma formalizes this intuition:
Let Hy denote the reproducing- assume an optimal solution to Eqn. { (x))²² admits representation in a finite eigenbasis of the integral operator associated with k. Then, for any dataset 'D drawn from distribution encountered during training, AM retrieval allows obtaining the minimum characterized by Eqn. 3
Lemma 1.
Hilbert space induced by the kernel k(.,)) and
Proof. By Mercer's theorem on compact spaces [64], Hy always has orthonormal eigenbasis.
Since we assume {ar,, (=) to admit a representation via some finite number of eigenfunctions
f;, 1 < d < we can pick the function Je to be the function that outputs [fiz), f2(1)... ... fg(x))] (x(:)...f((x))
the key matrix M as the corresponding eigenvalues. Since such a choice is subsumed under the AM retrieval dynamics elucidated in Equation] 1 the assertion in the lemma follows.
O
Importantly, this retrieval operates per-sample at inference without requiring gradient computation. Thus, motivated by AM systems, and specifically UHNs, propose simple and general mechanism for retrieving adapter weights. Assuming the availability of appropriate keys indexing into the adapters, retrieving an effective ensemble reduces to computing inner products between the query Zrest (or its processed representation) and the stored keys. Consequently, the adapter weight combination is dictated by the geometry of the inner product space in which the UHN operates. This strategy yields a single, aank-agn model that dynamically composes per- per-task adapters via an AM. Conceptually. training proceeds in two distinct stages. The first involves standard training of independent adapters per task using any suitable method for the setting of interest, followed by storing these adapters into the associative memory. This storage requires may be either fixed or randomly initialized. The second stage aims to ensure that the consolidated adapter weights retrieved via associative memory, when loaded into the backbone network J. perform effectively on their corresponding tasks. If performance is suboptimal, the retrieval keys are refined to improve alignment between the retrieved adapter ensemble and the task was originally trained for. Conceptually, this constitutes constrained variant of the first stage, where updates are restricted to lie in the subspace spanned by the -eiilartty-wiighted combination of adapters learned earlier. Viewed differently, this stage attempts to solve Equation 3 over a training set where ytrain is known. Crucially, it facilitates implicit consolidation of cross-task knowledge via retrieval dynamics of the shared A.M. Unlike prior uses of associative memories for storing raw content such as images or feature representations 5 106 74), our formulation embeds the Hopfield network directly into the training loop. Morcover, standard strategies mitigate catastrophic forgetting can be readily
of keys capable of indexing the adapters, which
5

---

incorporated, as the adapter combinations themselves are treated as trainable parameters. Finally, whereas the first stage trains separate adapters per task and layer É, the refinement of retrieval keys in stage two depends on the specific learning scenario: for DG, all domain data is jointly accessible and thus the partition the whole dataset itself, whereas for continual learning settings, tasks are discarded sequentially, and so the partitions in this case are the individual tasks themselves.
Algorithm 1: MIRA: Training
Require: Tasks (Dt) frozen backbone F, AM models U Me
for 1 to I do 2: A. F.U
[ Train adapters 0,, via method specific loss. */
1/x Write 0,,,, M via placeholder keys.
3: if Setting == CL then
4: Consoliddation(P,,..... My):=
[/+ Finotune only koys via second pass over data. x/
1/+ Apply eL heuristics to handle catastrophic forgetting. +/
5: end if
6: end for
7: if Setting = DG then
8: for = 1 to 7 do
9: Consolidation FF, U Mp)
end for end if
11:
This two-stage blueprint formally outlined in Algorithm !
Method Design. We now delve into specific design choices we make as we instantiate Algorithm Note that both subroutines described below assume the task given as input, as allocating the right task to each subroutine is handled in Algorithm
Two-Stage Training. The first of the two stages, dubbed Adaptation, involves training the base network to adapt to a single new task. We perform this adaptation by training LoRA-style adapters for each layer, using the provided dataset. Once trained, these adapters are then stored in the associative memory. Since such storage requires data in the form of key-value pairs, we initially randomly choose keys for each adapter by sampling from N(0, o² f). Notably, our method not constrained to storing LoRA adapters; infact, entire weight matrices can be stored into the associative memory.
Algorithm 2: MIRA: Adaptation
Input: Dataset D, frozen backbone F. AM models U Mr, hyperparameter o²
1: for (z, v) E DD do
2: Train memory adapters 0g on D with Cross-Entropy loss, Ve E T
3: end for
4: for E L do
5: Sample ke o0,
6: W(Me, kig, 0z)
7: end for
We choose LoRA adapters for simplicity and ease of implementation. To produce consolidated
parameters for given input, we explicitly train the randomly initialized keys using backpropagation in the second stage dubbed Consolidation, to yield adapter ensembles that minimize cross-entropy over the training set.

---

Algorithm 3: MIRA: Consolidation
Input: Dataset D, backbone F, AMS U Me. frozen U O₂ initial keys U K
1:
for (z) E: D do
2: for l E do
Compute layer inputs he-1 using 0--1 // ho 4: qe - ( // ge :query module for layer &
3:
5: Read 02 ITM,,TT.)
6: end for
7: Compute Cross-Entropy loss and back pnok-propagate
8: Update only U K, and U
9: end for
For a fixed value stored the UHN, the keys pointing to the values are continuously updated across sequential task exposures. Specifically, a query imput to the VIT backbone FF passes sequentially through a stack of layers, each prepended with a lightweight query module Jr TR + Rd This module transforms the output he-1 E Th of the previous layer into query vector for layer f. The module can be instantiated as an identity map, a linear transformation, or a small neural network.
The transformed query @r (h is then matched against the keys at layer to compute similarity scores, which are normalized using their sum (as opposed to the norm of the sum) and used to weight the corresponding adapters in the associative memory. This weighted ensemble is loaded onto the current layer, and the modulated layer output becomes the input for the next layer, hę+1-
Both the keys and the query modules are updated via backpropagation. The query module serves to align the geometry of the layer outputs with the key space, as these may naturally lie in different representational domains. Ultimately, the joint training of keys and query modules aims to produce the appropriate adapter combinations for inputs sampled from the task distribution encountered during training. In continual learning settings, where catastrophic forgetting is concern, we incorporate standard mitigation techniques such as DualGPM (59] within this framework. During inference, the architecture follows the procedure in Algorithm 3 except that no parameter updates are performed.
4 Experiments and Results
We rigorously evaluate MIRA across the three scenarios- Class-incrementt Learning (CIL), Domain- incremental Learning (DIL), and Domain Generalization (DG), on several benchmark datasets. Details regarding training protocols and hyperparameter tuning are provided comprehensively in the
Appendix. Following PEGO (37], we make use of the ViT-B/16 architecture (16) initialized with
CLIP 69
weights, and use LoRA with rank 4 for adapting downstream tasks.
Datasets. For the CIL and DIL scenarios, we adhere to the established setup from [65]. benchmarking primarily the widely-used datasets: iDigits, CORe50, and DomainNet [66]. For CIL, we partition the dataset classes into five sequential tasks, each consisting of a mixture from all available domains. In contrast, the DIL setup constructs sequential tasks, each encompassing all classes from single domain. Additionally, we expand our benchmarking to incorporate more recent and challenging datasets: ImageNet-R (32) split into both 5-task and 10-task scenarios in CIL, and the popular CDDB (53 dataset along with the recent DN4IL dataset [26 for the DIL setting. For the DG scenario, our evaluation covers four prominent datasets: PACS
54 VLCS 85), OfficeHome [87. and Net
Baselines. We benchmark MIRA against an extensive suite of SoTA baselines tailored specifically to each scenario. For the CIL and DIL settings, we include classical regularization--asee approaches, such as Elastic Weight Consolidation (EWC) 46) and Learning without Forgetting (LwF) [57]. More- over, compare against cutting-edge parameter-efficient fine-tuning (PEFT) methods, including S-Prompts [39]. L2P 998, DualPrompt [98], CODA- Prommpt (93), and LAE 94). We further compare to ICON (65]. designed specifically for unified handling of CIL and DIL scenarios, though lacking inherent DG capabilities. For the DG scenario, we benchmark against state-of of-the-art methods including popular methods like SWAD 50 and CoOP (110), and more recent works like PEGO (37).
Evaluation Metries. We employ two principal metrics extensively utilized incremental learning literature: Average Accuracy (Avg. Acct), where higher values indicate superior overall performance,
7

---

Table 2: Comparison with SoTA CIL and DIL methods on three standard datasets. Baseline numbers have been taken from prior work. In the CIL setting, we divide all classes into 5 distinct tasks, while the DIL setting, every domain serves as a separate task. MIRA outperforms all baselines on average accuracy, with minimal forgetting. Best results are highlighted in bold, and results within 2%
of the best are underlined.
Dataset
Mothod
CIL
DIL
Avg. Ace
Avg. Acc-t
Forgetting
Avg. Acc.
Forgetting
Finc-tuning
30.3222.777
48.01=10.72
35.04=0.899
23.23:+0:74
31.68
EWC 45
34.16=0.32
38.72:::0.99
68.62-00.92
25.9 05.94
$1.39
33.35:10.52
69.61:10.33
25.81:::..99
1.2P 97
63.17:10.88
28.53:10..81
73.83:10.26
23.43:1:0.65
68.50
S-Prompes 9]]
55.09:43.27
25.61+1.66
75.11
25.66::6.23
65.
iDigits
DualPrompt 96
68.82.10.97
11.81:11.77
76.42.10.46
26.33...0.22
72.62
CODA-P [
69.974:
19.8342.28
77.42:40.71
22.20:::0.18
73:70
LAE 22
65.77=0.83
28.47:40.77
79.09=
21.86=:0:40
72.43
1CON 65
71.53:00.68
19.36:1
84.8
12.67:400.61
78.18
Ours (MIRA)
83.00=1..29
10.62-+-.80
82.46=0.12
8.49=0.43
82.73
Fine-tuning
1..54
74.05-11.31
23.52:10.26
3.09=0.11
22.53
EWC 45
50.18:10.30
73.86 73.86
1.09
53.88
LxF 50
04,5
41.05+0.30
74.35=0.522
0.81=0.27
54.44
L2P 97
70.03:10.51
6.51
80.72:10.39
0.51 0.226
75.38
CORe50
S- S-Prompts 01
68.2743.92
11.79::0.24
86.50=
0.92=0.31
77.39
DualPrompt
71.96+0.37
5.04=0.71
81,41
0.21
CODA-P 30
77.85=0.44
0.78=0.37
84,36= 1.04
0.64= 14
81.11
LAF 22
0..1
18.38:11677
83.09
0.51
80. 10
ICON
80.85+00.23
7.68=0.52
89.01-0.33
0.17=
84.93
(MIRA)
81.99-1.24
7.99
88.64
Fine-tuning
35.4320.58
47. 47.79--0.28
39 09.52:00.32
28.81 0.64
37 97.48
EWC
$3.0410.53
24.41:1:0.48
01.5
06.79:
LwFF
56
53.79:10.61
19.11100.11
43.74
18.23:10.100
48.77
8.23-1..0
19.71
DomainNet
S- S-Prompts
91
39.78
19.29:11.04
50.80:00.63
4.20=0.53
45.29
DualPrompi
96
62.55 0..92
1.07
51.33
9.60 1.41
56.94
CODA-P
0..21
15.01:10.21
49.13=0083
25.96:11113
LAE 22
65.06 15.06110.18
9.68.10.84
44.67 14.67.00.62
28.99.10.64
$4.87
ICON 65
65.43+0.155
9.72-0046
54,44=0.21
13.32:40.46
59.94
Ours (MIRA)
07.
7.60=11066
69.18:10.10
4.07=20.15
68.24
and Forgetting, where lower values imply better retention of previously learned tasks. We follow standard protocol [65
96 80 for reporting these metrics, emphasizing the final task accuracy after
completing all incremental tasks.
Main Results. Table 2 summarizes the extensive experimental results in DIL and CIL scenarios. We utilize numbers reported from prior work where available to ensure fair comparison. MIRA consistently outperforms the state-of-the-art methods by a significant margin in both Avg. Accuracy and Forgetting metrics across these scenarios. For instance, on the iDigits dataset, MIRA achieves notable Avg. Accuracy of 83% and remarkably low Forgetting of just 10.69 10.62%. clearly surpassing the next best ICON by significant margin. It should be noted that ICON was designed to handle both CIL and DIL tasks MIRA achieves SoTA on both these settings. in addition being suitable for DG. Similar trends are evident CORe50 and DomainNer datasets, highlighting the robustness and effectiveness of our approach. We see similar trends the case of the DG setting, with MIRA achieving SoTA performance on three out of four benchmark datasets, and being comparable to SoTA in the remaining dataset. The margin achieved by MIRA over bascline methods on the harder OfficeHome and DomainNet datasets are particularly significant. As in the case of CIL and DIL settings, the baseline methods were specifically proposed for DG, and are not directly applicable to Continual Learning settings.
Additional Datasets.
To evaluate MIRA extensively, we employ additional benchmark datasets used by recent CIL and DIL works. In particular, in the DIL setting, we use the challenging CDDB- dataset, achieving SoTA performance as shown in Table We also benchmark on the recently proposed DN4IL. (26] dataset (Table which currently lacks widespread use. We compare against methods that evaluate on and outperform them by a significant margin, setting new SoTA baseline. Notably, outperform methods that use replay buffer with 200 exemplars, without using Exemplar

---

Table 3: Comparison with SoTA DG methods on 4 standard DG datasets. Bold = best; underlined = within
Table 4: DN4IL (DIL, 200- exemplar buffer): single- vs multi-model methods.
2% of best.
Mothod
PACS
VLCS
OfficcHome
DomainNet
Avg
Method
Last Aoc.
SWAD E
91.30+0.1
79.40+0.4
76.90-a.1
51.70+0.8
74.33
ER
20
27. 27.45-0.94
CLIP 67]
20.1
20.70
82.00-0.1
37.50g0.1
79.85
DER+ 10
35.71
SMA
92. 20.2
78.10=0.1
76.95 35.90
DARE
12
40.59
ERM 186
9
1
78.50_0.1
53.8010.1
77.65 68
CLS-ER
41.70 411
CoOp 0
96.20+0..1
77.60
83.90=0.1
39.80
79. 88
DUCA
2+
44.45-0.18
MIRO 12
40.6
82. 2..2
82.50-0.1
54.00
78.58
DARE+
12
g4.11
SEDGE 68
96. 10
82.20
80.70_0.2
54.7010.1
78.43
GESTUR 52
96.00
g2.
80.20_o.1
58.9010.1
80 48
Ours (MIRA) 78. 78.40
PEGO 3
96.50+0..
83 o0.
84.20=0.1
37.30
80.30
(MIRA) 07.01 22.10
87.36
61.19
881.92
Table 5; Comparison of recent SoTA methods on the Imagenet-R dataset in 5-task and 10-task CIL settings.
Table 6:
Comparison of recent SoTA DIL methods on the CDDB dataset. 'Joint' refers to training on all experiences at once in a static setting instead of continually train- ing, and serves as an upper bound on performance.
Tasks
5
Method
ACCs (1)
ACC10 (1)
Joint
81.14 0.34
81.14 1. 0.34
Sequential
58.74 + 1.28
46.07 + 1.15
Method
Average Acc (?)
L2P 97]
64.13 + 0.78
02.54 + 0.24
DualPrompt
96
67.88 0.17
65.41
+ 0.52
EWC
[55
50.59
CODA-P 80
T3.09 0:21
71.47
1 0.35
LwF
60.94
C-LoRA
79
75.85 + 0.31
71.89
+ 0.45
DyTox 77
$1.27
22
73.84 0.14
71.70
- 0.39
1,2 97
61.28
S-iPrompts 13
74.51
Ours (MIRA)
78.06 0.76
73.08
1: 0.46
Ours (MIRA)
77 37 : 0.21
Joint
85.50
Replay ourselves. To evaluate MIRA over longer learning timeframes, we also evaluate in 5-task and 10-task splits of the ImageNet-R dataset, outperforming recent baselines as seen in Table 5
Effectiveness of Separation Functions. In addition to Softmax, other memory models have proposed the use of various separation functions - Identity (classical Hopfield Networks (34)), Polynomial
(Dense Associative Memories (48).. ReLU (proposed in (33) and Linear (Tolman-Eichenbaum Machine (TAM) [0O [02), which attempts to model the Hippocampus).
We investigate the influence of different separation functions in model architecture. Specifically, we explore variants employing affine transformation, Softmax normalization. and ReLU and Tanh activations. Note that affine and Tanh functions allow "removing" destructive information from the output representations by allowing negative weights to be assigned to certain adapters, while Softmax and ReLU can at most "mask" such information with zero very low coefficients.
The affine variant (used in our primary experiments) intuitively allows flexible linear transformations, capturing nuanced task relationships, in addition to reflecting the TAM model. Alternatively, the Softmax variant ensures a probabilistic distribution over adapter activations, potentially beneficial in scenarios demanding explicit competition among adapters. The ReL.U variant introduces sparsity, which might reduce interference by selectively activating only relevant adapters. Finally. the Tanh activation offers an alternative to the affine variant, while providing nearly equal weights for relevant adapters and allowing deteriorating information to be actively removed using negative coefficients.
Our results, as tabulated in Table indicate that the ability to actively remove interfering information key to the CIL setting, with the affine and Tanh variants achieving the best performance. On the other hand, allowing negative coefficients may result in removing relevant information, which may explain their reduced performance the DIL setting. However, the DG setting clearly establishes that having non-uniform selection of adapters, along with the ability to remove interfering information, holds the key to out-of-distribution generalization. We conclude that while all separation functions perform well, the affine function achieves the best performance overall.

---

Impact of Adapter Count. We conduct detailed amalysis examining the sensitivity of MIRA to the number of task-specific adapters employed (1. 2, 5, and 10 adapters per task). The results, tabulated in Table 8 indicate clear improvement as the number of adapters per task or domain increases, highlighting the importance of capturing diverse task-specifie nuances. However, increasing the adapter count beyond 5 yields marginal returns in performance improvement. Thus, our experiments suggest an optimal trade-off around five adapters per task, balancing efficiency and accuracy effec- tively. However, since efficiency is not the main concern of this work, we use 10 adapters per domain or task in all our experiments, since this gives us the best absolute results.
Table 7: Comparison of different sepa- ration functions used for retrieval from memory.
Table 8: Effects of number of adapters trained for each task/domain.
Sep. Fune
CIL Acc.
DIL Acc.
DG Acc.
Avg.
#Adapters
CIL Acc.
DIL Acc.
DG Acc.
Avg.
Affine
67.29
69.18
61.19
65.89
63.75
69.08
61.21
64.68
Sottmax
66.87
69.21
60.82
65 63
2
66.95
69.04
61.19
65.72
60.90
Tanh
66.73
68.96
60.94
65.54
10
67.29
69.18
61.19
65.89
Performance Overhead of Hopfield Keys. We measure the inference-time latency and memory overhead introduced by trained integrated Hopfield keys (after AlgorithmB] on an adapters-loaded ViT backbone, relative to the same backbone without keys (i.e., using standard LoRA). On DomainNet- DIL with ViT-B/16 (LAION initialization), the average latency is 0.0241s with keys vs. 0.0240s without, i.e., a negligible 0.4% overhead. Using the identity-basee -based query module with 5 adapters per task across 6 tasks and key dimension 768 adds only ~ 276K parameters to an 86M- -aM-maemee model (< 0.4% increase in memory).
5 Conclusion
We presented MIRA, an architecture inspired by biologically plausible AMs, that unifies DG, DIL, and CIL scenarios. It uses an explicit UHN module for storing and retrieving low-rank weight adapters. Empirically, MIRA delivers SoTA performance across all three settings, often surpassing specialized baselines by significant margins, while requiring only minor objective tweaks rather than entire architectural changes. Beyond raw accuracy, our results demonstrate that coupling deep networks with neuro-plausible memory mechanisms yields flexible, reusable circuitry that can continually incorporate new information and adapt swiftly to distributional shifts. We believe MIRA marks step toward closing the gap between neural musk-switching in biology and continual adaptation in artificial systems, and opens rich avenue for exploring memory-centric, learning paradigms at scale, from vision to multimodal generative models.
Acknowledgment and Funding Transparency Statement
Susmit Agrawal supported by the German Research Foundation (DFG): SFB 1233, Robust Vision: Inference Principles and Neural Mechanisms, TP C2, project number: 276693517, and the Reliance Postgraduate Fellowship. Susmit Agrawal thanks the International Max Planck Research School for Intelligent Systems (IMPRS-IS) and the Reliance Foundation for support. Krishn V. Kher thanks the Prime Minister's Research Fellowship (PMRF) for funding and support. Vineeth N. Balasubramanian thanks Microsoft Research India, Bangalore for funding and support. All authors thank Mr. Aveen Dayal for guidance on choosing datasets for running experiments on the Domain Generalization setting. All authors thank the Indian Institute of Technology, Hyderabad for providing the resources and infrastructure required develop this project.
10

---

A
Appendix
Contents
A.1
MIRA: A Multi-Perspective Analysis
A.1.1
Meta-learning (Hypermetworks)
A.1.2
Functional InterpolaiionEExtapllllion
A.1.3
Test Time Adaptation
A.1.4
Biological Perspective
A.2
Additional Experiments
A.3
Experimental Details
A.4
Dataset Details
A.5
Theoretical Underpinnings
A.6 6
Limitations
A.1 MIRA: A Multi-Perspective Analysis
Our method, MIRA is generic framework that can be viewed from four contemporary. broad perspectives in the general context of machine learning, going beyond the perspective introduced in the main paper. We elaborate on those perspectives below.
A.1.1 Meta-learning (Hypernetworks)
Hypernetoorss models that predict weights of other models [30 13 6 approach to meta-learning by eliminating the need for an inner-loop adaptation (as in MAMI. (20). instead learning output task-specific parameters in forward pass. Since MIRA predicts adapters at each consecutive block hierarchically, the associative memories in MIRA's 's architecture can be viewed as a specific type of hyperntwork.. However, hypemnetorr training is often unstable and is unable to support large models such as ViT architectures. We address this issue by providing "supervision" guide the direction of learning of the underlying hypernetwork. Hypernetworks then essentially become models that map a deterministic set of inputs to their outputs deterministically, resembling the behavior of associative memories, and can be implemented as such. We posit this perspective in this work, wherein weight retrieval is achieved by storing the adapter weights over tasks defined over the training set, and retrieving them differentiabll,, instead of attempting both learn and memorize them simultaneously as done in traditional hypernetwor-- Hopfield Networks (69, Predictive Coding Networks [106] 83 and any such AM may in theory be used in this framework. Practical experiments, however, indicated that despite PCNs [106] exhibiting better compression properties than Hopfield Nets, they usually have unsatisfactory retrieval quality when the vectors to be retrieved have very high dimension. This is expected behavior, especially in context storing weight adapters of large foundational models. In addition, their reliance on algorithms incompatible with backpropagation makes it difficult to integrate them into models that need to be trained end-to-end. Hopfield Nets, on the other hand, provide high fidelity in retrieving such nighhdimensional vectors at scale, and are implicitly differentiable.
They offer single-loop
learning. Thus,
A.1.2 Functional Interpolation/Ettraplaaaoo
In this work, we utilize affine combinations of task-specific adapters for retrieval across different domains. However, our associative memory-centric learned retrieval framework is versatile and can seamlessly accommodate richer, non-linear retrieval mechanisms by modifying the underlying simi- larity metric strategy. In principle, one could design much more expressive (non-linear combination schemes to merge knowledge from multiple domains, rather than restricting to linear interpolation [12]. This an interesting direction of future work for this paper. Such non-linear retrieval ap- models use learned gating function to dynamically select only subset of expert parameters for each input (instead of fixed weighted average) [31]. in using learned retrieval functions (c.g., trainable
proaches have been explored in recent literature, such in sparsely-gated mixture-of

---

hashing or routing instead of standard nearest-neighbor search) yields better scaling and can capture latent structure in the memory, outperforming fixed similarity measures [3I]. Crucially, there is both practical and theoretical evidence of directly retrieving such non-linear ensembles of experts/adapters in Hopfield Networks, such as in [76..
Thus, our method can be viewed as retrieving an interpolated task-specific knowledge proxy (adapter) in a space defined by the chosen functional form of combination. With sufficiently expressive interpolation function, this approach can even enable extrapolation to out-of-distribution tasks. In settings like DG, the target domain may lie outside the convex hull of the source domains, wherein the model must generalize beyond any seen domain mixture. A suitably rich, non-linear combination strategy could, principle, facilitate extrapolation of the memorized adapter weights to novel task distributions [. Our approach thus offers a generalizable framework capable of both capturing nuanced relationships between known tasks and extending learned knowledge to new domains beyond the scope of training distributions. We leave the exploration of such function schemes to future work.
A.1.3 Test Time Adaptation
At test time, the optimal combination of adapters for given input may not be obtained using pre-defined weight coefficients. In general, determining the adapter coefficients that best serve new downstream sample can require solving an optimization problem a per-sample basis, as posited in Equation 3. In other words, Equation 3 formalizes the idea that the best adapter composition {ar, (x)) for an input is obtained by minimizing suitable objective for that specific sample at inference time (instead of applying fixed combination rule). This perspective aligns with the paradigm of test-time adaptation in literature. wherein models trained only on source data adapted to target data during inference time [104 41 As an example, Tent (Fully Test-Time Entropy Minimization) performs online model updates during testing by minimizing the entropy of its predictions for each test batch, thereby adjusting normalization parameters increase the model's confidence on the target distribution [88]. One could view our MIRA framework as implementing this idea via memory-based inference mechanism. Since MIRA learns set of key representations (i.e., associative memory slots in a Hopfield network) during training that are used to derive weights on a per-sample basis, it can also use the learned keys at test time per-sample basis effectively, performing adaptation via associative recall. By casting test-time adaptation as an integral part of inference (through solving a Hopfield memory retrieval optimization akin to Equation 3), MIRA can be viewed also as test-time adaptation strategy within a unified, optimized memory- based framework.
A.1.4 Biological Perspective
Notably, MIRA can also be perceived as a biologically plausible framework that solves multiple set- tings such as DG, CIL, and DIL. While Hopfield Nets are well known biologically implementable AM mechanisms, even the affine combinations that we adopt are well-founded in biological mecha- nisms. Specifically,
Machine (TAM) [101 102). model of the Hippocampus, proposes linear combinations of stored memories as implementable (illustrated in Section (A.2). We further observe that the incremental learning settings, when accompanied by DualGPM [60] as the strategy to mitigate catastrophic forgetting, resolve into Generalized Hebbian learning principles [75 in the gradient space of stored memories.
Lemma 2 (DualGPM Mebbbaan Gradient Subspace Equivalence). Let G. = i C Rd be the set of gradient vectors observed while training on task and let the empirical second-me moment matrix be
N.
Σ,
9:9.
N
In the DualGPM algorithm, for a given energy budget
E (0, we define:
Xj
= arg min keld
2 E.
where >1 ... 2 Aa are the eigenvalues of E,-
Let Ut E [Raxk be the orthonormal busis produced by the DualGPM memory update for the energy budget& e. Independently, les W, dd he the weight matrix obtained as a stationary point of the Generalised Hebbian (Oja) update averaged over each gradient in G. (with row sums equal 1).
2

---

Then,
spa span(W.).
Proof. Step 1: Optimality of DualGPM. We first note that the precise objective that DualGPM tries solve is given by:
U, arg min U U=
Ti(I-UUT)
By the Eckart- Loung-Mirske theorem, the optima is attained at those U, whose columns are the k eigenvectors (subject permutation and sign inversion) of Σ, corresponding to the & largest eigenvalues, Å1.... AI.-..,Ar Ax. Hence, U, satisfies the eigenvalue equation
E,U, U,A,
A = diag ..
Step 2: Fixed points of the Hebbian rule. The Generalized Hebbian update rule [? for gradient vector g E G. given by:
AW "(gg" W W diag(WTT 99 W))-
Considering the weight update averaged over all g E Ge and imposing stationary conditions on the same yields:
E, W - W diag(W W) = O.
Tre-multiplying by W.T and noting that each row sum W,T equals one, shows that the diagonal term on the right is precisely the eigenvalue matrix of the projected covariance, so the above is identical to 4 with U, replaced by W..
Consequently, U, and W, have the same k-dimensional eigenbasis, corresponding to the & eigenvec- tors of E, with the k-largest eigenvalues, which proves the assertion.
Thus, Lemma 2 limplies that there exists an orthogonal matrix RExk such that U, W,R.
Implications. This lemma elevates the biological analogy behind DualGPM into provable equiv- alence: the algorithm's 's batch SVD update computes exactly the same principal gradient subspace that online Oja-style Hebbian learner would converge The result has three immediate conse- quences: (i) Theoretical grounding: Any optimal variance or noise filtering guarantees enjoyed by Hebbian PCA now apply to DualGPM's memory, providing principled basis for its strong empirical resistance to catastrophic forgetting; (ii) Algorithmic unification: Projection-basel continual learn- ing methods can be re-interpreted through the lens of gradient-space Hebbian consolidation; and (iii) Neuro- Neuro-inspired design: By demonstrating that protecting past tasks is tantamount to a Hebbian consolidation step, the lemma bridges continual learning research with synaptic consolidation theories in neuroscience, motivating biologically grounded extensions such as local online updates or neural gating via associative memory.
A.2 Additional Experiments
Multiple Initializations and Backbones. For completeness, Table AT reports DIL results for MIRA across architectures and initializations. In addition to the ViT-B/16 backbone used in the main paper, we include ViT-B/32, whose performance is generally lower, reflecting the sensitivity of Hopfield components both initialization and backbone choice. We also report results with the ViT-in21k initialization alongside the LAION initialization used in the main results.
Comparable experiments for the DG setting appear in Table A2 Notably, using ViT-in21k, compare MIRA against bare ViT-B/16 backbone with the same initialization but without adapters, and find that adapters with learned Hopfield keys yield substantial gains even over such strong baselines.
Choice of 9(). In the tables in the main paper, we set g described in the conceptual framework to be an identity function. We tabulate the results below for different choices of g. We find that this function has minimal impact on performance, indicating that the transformations performed within the different layers of ViT are strong and fairly sufficient by themselves constitute an eigenbasis wherein the appropriate keys can be found via gradient descent.

---

Table AI: Comparison with SoTA DIL methods on three standard datasets. Baseline numbers have been taken from prior work. MIRA is evaluated on multiple ViT backbones and initializations for completeness. Best results are highlighted bold, and results within 2% of the best are underlined.
DIL
Avg. Acc
Dataset
Method
Avg. Acc.1
Forgetting
Fine tuming
33.04.10.89
23.23.10.74
31.68
EWC
45
68.62:10.92
25.94=0.98
$1.39
1.2P 37
73.83-10.26
23.43=0.68
68.50
S-Prompts
75.11-22.31
25.66=6.23
Digits
DualPrompt 9
76.42+00.46
26.33=0.62
72.62
CODA-P 8D
77.42:10.71
22.20=00.18
73.70
LAE
22
79.09.1
21.86=0.440
72.43
ICON
A
84.83:10.51
12.67=0.61
78.18
Main (MIRA)
82.46.10.12
8.49:10.43
82.46
VT--B332 (MIR)
83.06::0.12
18.42=0.43
83.06
Fine-tuning
23.52:40026
3.09-
22.53
EWC 45
73.86-10.38
1091:0.12
53.88
LwF 50
74.35=00.52
0.81
54.44
80.72440.39
S-Prompts M
86,50=+0.46
0,92
77.39
CORe50
DualPrompt 96
81.41=:0.22
0.21-
76.69
CODA-P M0
84.36
0.64
81.11
LAE 22
83.09.10.71
0.151
80,10
89.011:0.33
0.100.2211
Main (MIRA)
93.89.10.33
0.00:10.00
93.89
VIT-B/32 (MIRA)
91.28:1:033
0.00::0.00
91.28
Fine tuning
39.52.10.32
28.61:10.44
37.48
BWC 43
41.58:10.26
26.79=0.15
47.31
43.74.10.27
1.2P 97
48.55
19.71 1.29
54.73
S-Prompts
50,80:10063
4.20-+0.53
DomainNet
DualPrompt
51.33.10.10
9.60
56.94
CODA-P 60
49.13:
25.96 1.13
57.17
LAE 22
44.67++0.62
28.99=0.64
ICON 65
54.44:1:0.1
13.32:0.446
59.94
Main (MIRA)
69.18=+0.10
4.07=20.15
69.18
ViT-in2Ik (MIR)
59.44:10.10
12.06=0.15 15
59.44
(MIRA)
59.36.10.10
11.60.10.2
Prefixes as memories. The proposed MIRA framework bears resemblance to the complementary learning system implemented by hippocampal-cortice connections in the brain [77]. This framework, however, models the hippocampus overlays. The analogue such a mechanism in contemporary deep learning architectures is Prompt Tuning [98] in PEFT literature. In particular, the prefix tuning [55] variant of prompt tuning can be directly integrated into the MIRA framework to implement system analogous to a neuroscientific framework, with the pretrained network serving the role of the cortical circuits with powerful generalization capabilities, and the prefixes stored associative memories serving as task-specific representations. We compare the prefix-tuning based approach with original MIRA framework. Our results indicate that this variant maintains comparable performance to storing overlay weights, showing that MIRA can be adapted to different PEFT methods.
memory storage for representations, rather than neural
A.3 Experimental Details
For training task-specific adapters in the Adaptation stage across all datasets and settings, rank LoRA adapters trained for 5 epochs with a learning rate of 1e-3, For CIL and DIL experiments, set the DualGPM threshold to 0.7. The AdamW optimizer is used with weight decay of 1e-3 across all experiments as well. All our experiments are performed on a single RTX A6000 Ada GPU with 48GB VRAM, machine having 96-core Intel Xeon CPU and 128GB RAM.
In the Consolidation stage, all experiments in DIL. and CIL. settings ran for 2 epochs. addition, we initialized the CIL classifiers in the Consolidation stage with the weights learned in the Adaptation

---

Table A2: Comparison with SoTA DG methods on 4 standard DG datasets with different initializations and ViT backbones. Bold = best; underlined = within 2% of best.
Mathod
PACS
VI.CS
OfficeHoome
DomainNet
SWAD
91.30+0..1
79.40+0.4
76.90+0.1
51.70+0.8
74.33
CLIP +7
96. 40.1
81.70.0.1
40.0
57.50
79.85
SMA
02. 40.2
70.
78. 10
55.90
76.95
80
93.70+0.1
82.70+0.1
78. 78.60
63.80
77.68
CoOp 111
96.204o.1
77 77.60
83 23.90
20.
79.88
MIRO
95.60
62.20
62.50
54.00
78.58
SEDGE
9
52.20
20.70
24.70
78. 43
GESTUR 52
96.00+0.0
82 +2.80+0.1
84.20+0.1
58.90
80.48
PEGO
90.30+o.1
20.
21.20
57.30
80.30
Main (MIRA)
97.0110.0
82. 10.10
87 87.3620.3
61.19.o.1
81.92
ViT-in2lk (Base model)
68.89+0.0
73.00+0.5
+1.23+0.3
66.36
VTT-in2Ik (MIRA)
71.28+0.0
18.18
82. 42.52
16.99...
68.71
VIT-R/32 (MIRA)
94.11+0.0
81.6840.5
53.07 79.00
77.11
Table A3: Comparison of different choices of for DIL and DG settings.
g
DIL
DG
Identity
69,18
61.19
Linear
69.22
60.98
3-layer MLP
69.22
61.12
Table A4: Comparison of Prefix Tuning vs LoRA tuning in MIRA.
MIRA Variant
DIL.
DG
Afffaldfault
69.18
61.19
MITR--Prefiees
69.61
60.72
stage for the corresponding label set. Note that this is not effective in the DIL setting, as even though the label sets are the same, the distribution of inputs to the classifier changes, and hence the scope of knowledge transfer the linear classifier head is limited in this setting. In the Consolidation stage of the DG setting, we run PACS, OfficeHome, and VL.SS for 10 epochs, while DomainNet is just run for one epoch. We rescale all images to 256 x 256 during both training and evaluation and take a 224 x 224 crop from this rescaled image (random crop during training, center crop at inference) as input to the model. We apply a random horizontal flip as a training augmentation all cases, and an additional mixup augmentation for the DG setting. In all CIL and DIL settings, we use the AdamW optimizer with a weight decay and learning rate le-3, while in the DG setting, we set the learning rate to 7e-4.
A.4 Dataset Details
DomainNet. DomainNet a large-scale benchmark comprising approximately 600,000 images across 345 categories, distributed over six distinct domains: Real, Clipart, Infograph, Painting, Quickdraw, and Sketch. Each domain introduces a unique visual style, presenting significant domain shifts. In the DIL setup, each domain is treated exposed to data from one domain at a time while maintaining a consistent label space. This setup challenges models to generalize across diverse visual domains without forgetting previously learned knowledge. In the CIL setup, the dataset divided into 5 experiences, each experience containing 69 classes from all domains combined. Unlike the DIL setting, the label space in the CIL setting grows with each experience. In the DG setup, models are trained on 5 domains conjointly and evaluated on the 6th unseen domain.
separate experience, with the model sequentially
DN DN4IL is a curated subset of DomainNet, specifically designed for evaluating domain- incremental learning methods. It retains the six domains from DomainNet but focuses reduced
5

---

set of 100 classes to facilitate controlled experiments on domain shifts. The dataset emphasizes the challenges posed by significant distributional differences between domains, making a suitable benchmark for assessing the robustness of continual learning algorithms.
iDigits. iDigits is domain-incremennte benchmark constructed by combining four digit recognition datasets: MNIST, SVHN, MNIST-M, and SYN. Each dataset represents a distinct domain with varying visual characteristics. In the DIL setting, the model is trained sequentially on each domain to maintain performance across all domains despite the domain shifts. This benchmark is particularly useful for studying the effects of domain shifts in simpler classification tasks. In the CIL setting, all datasets are jointly split into 5 training experiences, each experience containing 2 classes from each of the datasets.
C0Re50. CORe50 a dataset designed for continuous object recognition, consisting of 50 household objects recorded under 11 different environmental conditions. Each condition introduces variations such as background changes, lighting, and occlusions. In the domain-incremennal setup, each environmental condition treated as a separate domain, and the model learns to recognize the same set of objects across these varying conditions. A key difference from other DIL datasets is that CORe50 uses 3 of the 11 domains as the test and incrementally trains on the other 8 domains. This setup evaluates a model's 's ability to generalize object recognition across different real-world scenarios. It can thus also be viewed as a combination of DIL and DG settings, where the test set comprises of unseen domains. A forgetting of < O indicates that the models' performance remains the same or improves on the unseen domains as new domains are incrementally learned. The CIL setting is similar to DomainNet and iDigits the dataset is split into 5 experiences of 10 classes each, encompassing all 11 training domains.
CDDB. CDDB (Continual Deepfake Detection Benchmark) is dataset aimed at evaluating continual learning methods in the context of deepfake detection. It comprises a collection of deepfake videos generated using various known and unknown generative models. In the DIL framework, each generative model represents different domain, and the model is sequentially trained to detect deepfakes from these diverse sources. CDDB challenges models to adapt to new types of deepfakes while retaining the ability detect previously encountered ones. We particularly evaluate on the CDDB-hare subset, comprising five domains: GauGAN, BigGAN, WildDeepfake, WhichFaceReal, and SAN.
ImageNet-R, ImageNet-R is a dataset comprising 30,000 images of 200 ImageNet classes, with images rendered in various styles such art, cartoons, graffiti, embroidery, and video games. This dataset is designed to evaluate the robustness of models to distribution shifts. In the CIL setup, the 200 classes are divided into 5 or 10 tasks, each containing 40 or 20 unique classes. The model is trained sequentially on these tasks to learn new classes while maintaining performance on previously learned ones.
VLCS. VLCS is benchmark dataset for domain generalization, comprising images from four distinct domains: PASCAL VOC2007, LabelMe, Caltech-101, and SUN09, Each domain contains images labeled across five shared object categories: bird, car, chair, dog. and person. The dataset includes total of 7,510 images, with domain-specinn distributions. In the DG setup, models trained on three domains and tested on the remaining one, evaluating their ability to generalize to unseen domains.
PACS. PACS is an image dataset designed for domain generalization, consisting of four domains: Photo, Art Painting, Cartoon, and Sketch. Each domain contains images from seven categories: dog, elephant, giraffe. guitar, horse, house, and person. The dataset comprises a total of 9,991 images, with varying numbers across domains. PACS introduces significant domain shifts due to the diverse visual styles, making it a challenging benchmark for DG methods.
OfficeHome. OfficeHome is a benchmark dataset for domain adaptation and generalization, con- taining images from four domains: Art, Clipart, Product, and Real- World. Each domain includes 65 categories of everyday objects, totaling approximately 15,500 images. The dataset presents substantial domain shifts due to differences in image styles and acquisition methods. In the DG setup, models are trained on three domains and evaluated on the fourth, assessing their ability to generalize to unseen domains.

---

A.5 Theoretical Underpinnings
Lemma stated that the MIRA's AM formulation accommodates the optimal solution for Equation as long the optimal coefficients come from a kernel. It. however, does not prove that MIRA attains those optimal coefficients, specifically via the Consolidation phase of the training. Under certain mild assumptions on the kernel detailed below, we show that our method indeed converges to the true optimal coefficients. Our proof strategy largely follows [4 where, for convenience we also assume that the similarity function is softmax instead of an affine function as we have stated in the main text.
Theorem 1. Let Hy denote the reproducing-kenan Hilbert space induced by the kernel k(., )), and assume an optimal solution to Eqn. 3 {at, (z)),, admits representation in finite eigenbasis of the integral operator associated with k. Further along the lines of Att-SVM in {84), define
T.L
W(x) K.
Σ
(5)
1=1d=1
Assume that the kernel k is such that the following condition is true:
(ai,(2)) arg min [[ s.L. (Iiopt, ==) Iu 1, Vt opt,, i E [n].
(6)
Then the Consolidation stage of MIRA induces the query modules and learnable keys to converge to the optimal coefficients. at, (2)
Proof. The proof follows from 84 by replacing the matrix W by our ensemble of adapters and
inheriting their assumptions is, specifically in Lemmas 1, 2, 4, 12, and Theorem 4.
A.6 Limitations
This work highlights the benefits incorporating neuroscientific insights into deep learning archi- tectures, especially in the context of biologically plausible memory mechanisms. In particular, it proposes potential mechanism in which such task-switching can occur in biological systems with the aid of associative memories. The work constraints to task settings such as CIL, DIL, and DG; extensions to related settings such as Versatile Incremental Learning or Multi-Task Learning, or even other PEFT methods, would be interesting future extensions of our framework. All experiments provided are based on computational models from deep learning research; analogous neuroscience experiments may need to be conducted conclusively declare if memory mechanisms are indeed used in the stated manner in biological systems. Besides, validating this framework on non- ViT architectures such as ResNets is also possible, and may help extend this work more generally to all architectures.

---

References
[1] Emmanuel Abbe, Samy Bengio, Aryo Lotfi, and Kevin Rizk. Generalization on the unseen, logic reasoning and degree curriculum june 2023. In URL httpss//opeereview. net/forrm, 2023.
[2] Susmit Agrawal, Krishn Vishwas Kher, Madhumitha V, and Vineeth N. Balasubramanian. Can memory networks play a role in task task-specific modulation of neural circuits? In New Frontiers in Associative Memories, 2025.
[3] Elahe Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A gen- eral continual learning method based on complementary learning system. arXiv preprint arXiv:2201112604 2022.
[4] Devansh Arpit, Huan Wang, Yingbo Zhou, and Caiming Xiong. Ensemble of averages: Improving model selection and boosting performance in domain generalization. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[5] Guangii Bai. Qilong Zhao, Xiaoyang Jiang, and Liang Zhao. Saliency-Guided Guided Hidden Associative Replay for Continual Learning.
[6] Shawn L. E. Beaulieu, Lapo Frati, Thomas Miconi, Joel Lehman, Kenneth O. Stanley, Jeff Clune, and Nick Cheney. Learning to continually learn. In European Conference Artificial Intelligence, 2020.
[7] Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. BitFit: Simple parameter-officient fine- tuning for transformer-based masked language-moeels In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1-9, Dublin, Ireland, May 2022. Association for Computational Linguisties.
[8] Jan Benner, Martina Wengenroth, Julia Reinhardt, Christoph Stippich, Peter Schneider, and Maria Blatow. Prevalence and function of heschl's gyrus morphotypes in musicians. Brain structure & function, 222()):3587-360 November 2017.
[9] Andrea Bocincova, Timothy J. Buschman, Mark G. Stokes, and Sanjay G. Manohar. Neural signature of flexible coding in prefrontal cortex. Proceedings of the National Academy of Sciences, 119440)=220044001 19, 2022.
[10] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems, 33:15920-1555 15930, 2020.
[11] Junbum Cha, Sanghyuk Chun, Kyungjae Lee, Han-Cheol Cho. Seunghyun Park, Yunsung Lee, and Sungrae Park. Swad: Domain generalization by seeking flat minima. Advances in Neural Information Processing Systems, 34:22405-224111 2021.
[12] Junbum Cha, Kyungjae Lee, Sungrae Park, and Sanghyuk Chun. Domain generalization by mutuallinformation regularization with pre-trained models. In European conference on computer vision, pages
Springer, 2022.
[13] Vinod Kumar Chauhan, Jiandong Zhou, Ping Lu, Soheila Molaei, and David A Clifton. A brief review of hypemetworks in deep learning. Artificial Intelligence Review, 57(9):250, 2024.
[14] Liang Chen, Yong Zhang. Yibing Song. Zhiqiang Shen, and Lingqiao Liu. LFME: A simple framework for learning from multiple experts domain generalization. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[15] Michael Crawshaw. Multi-task learning with deep neural networks: A survey. CoRR,
abs/2009.0977) 09909, 2020.
[16] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer. Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x 16 words: Transformers for image recognition at scale. In International Conference Learning Representations, 2021. 2021.
8

---

[17] Arthur Douillard, Alexandre Ramé, Guillaume Couairon, and Matthieu Cord. Dytox: Trans- formers for continual learning with dynamic token expansion. Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9285-9295, 2022.
[18] John Duncan. An adaptive coding model of neural function in prefrontal cortex. Nature Reviews Neuroscience, 2(1 ((1))820--22) Nov 2001.
[19] MJJ. Buchler E.L. Buehler. X-lora: Mixture of low-rank adapter experts, flexible framework for large language models with applications in protein mechanics and design. 2024.
[20] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference machine learning, pages 1 1126- 1135. PMLR, 2017.
[21] Haotian Fu, Yixiang Sun, Michael Littman, and George Konidaris. Knowledge retention for continual model-based reinforcement learning. In Proceedings of the Forty-second Interna- tional Conference on Machine Learning (ICML), 2025.
[22] Qiankun Gao, Chen Zhao, Yifan Sun, Teng Xi, Gang Zhang, Bernard Ghanem, and Jian Zhang. A unified continual learning framework with general parameter-efficient tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 11483-1149) 2023.
[23] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier transform. In Forty-first International Conference on Machine Learning, ICMI. 2024, Vienna, Austria, July 21-27, 2024, 2024.
[24] Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. Parameter-efficient fine-tuning with discrete fourier transform. Forty-first International Conference on Machine Learning. ICML 2024, Vienna, Austria, July 21-27, 2024, 2024.
[25] Cornelia Geberl, Signe Brinkløv, Lutz Wiegrehe, and Annemarie Surlykke. Fast sensory-moto reactions in echolocating bats to sudden changes during the final buzz and prey intercept. Proceedings of the National Academy of Sciences, 112(13):4122 4127, 2015.
[26] S. N. Gowda et al. A cognitive-inspee insppreeddddd multi module architecture for continual learning. Under review as a conference paper at ICLR 2023, 2023. https::/openzeview.. net/
paf ?id=wP LE LEzBcSC7p)
[27] Shruthi Gowda, Bahram Zonooz, and Elahe Arani. Dual cognitive architecture: Incorporating biases and multi-memory systems for lifelong learning. arXiv preprint 1rXiv:2310.11341, 2023.
[28] Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, Abs/1410.5401, 2014.
[29] Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska- Barwińska, Sergio Gómez Colmenarejo, Edward Grefenstette, Tiago Ramalho, John Agapiou, Adrià Puigdomènech Badia, Karl Moritz Hermann, Yori Zwols, Georg Ostrovski, Adam Cain, Helen King, Christopher Summerfield, Phil Blunsom, Koray Kavukcuoglu, and Demis Hassabis. Hybrid computing using neural network with dynamic external memory. Nature, 538(7626):411-476 Oct 2016.
[30] David Ha, Andrew Dai, and Quoc V Le. Hypernetworks, arXiv preprint 16X9006609 2016.
[31] Xu Owen He. Mixture of a million experts. arXiv preprint arXiv:2407.04113 2024.
[32] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song. Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distrrbbtion generalization. ICCV, 2021.
[33] Benjamin Hoover, Krishna Balasubramanian, Dmitry Krotov, and Parikshit Ram. Dense associative memory with epanechnikov energy. In New Frontiers in Associative Memories, 2025.

---

[34] John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8):2554-255 1982.
[35] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In Kamalika Chaudhuri and Ruslan Salakhutdinov. editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 2790-2799 PMLR, 19-15 Jun 2019.
[36] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models, arXiv preprint ar axXiv:2006.06685, 2021.
[37] Jiajun Hu, Jian Zhang, Lei Qi, Yinghuan Shi, and Yang Gao. Learn to preserve and diver- sify: Parameter-effiiiet group with orthogonal regularization for domain generalization. In European Conference on Computer Vision, pages 198-216 Springer, 2024.
[38] Zhiwu Huang, Yabin Wang, Danda Pani Paudel, Mohamad Shahbazi, Xiaopeng Hong, and Luc Van Gool. inaremnncrellttal learning: Survey and performance evaluation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023.
[39] Zhiwu Huang, Yabin Wang, Danda Pani Paudel, and Luc Van Gool. S-prompts learning with pre-trained transformers: An occam's razor for domain incremental learning. Advances in Neural Information Processing Systems (NeurlPS), 2022.
[40] Nam Hyeon-Woo Woo, Moon Ye-Bin, and Tae-Hyun Oh. Fedpara: Low-rank hadamard product for communication-efffcieent federated learning. arXiv preprint urXiv:2/08.0600 2021.
[41] Yusuke Iwasawa and Yutaka Matsuo. Test-time classifier adjustment module for model- agnostic domain generalization. Advances in Neural Information Processing Systems, 34:2427- 2440, 2021.
[42] Kishaan Jeeveswaran, Elahe Arani, and Bahram Zonooz. Gradual divergence for seamless adaptation: A novel domain incremental learning method. arXiv preprint 11X1 2024.
[43] Nikola Kasabov, Helena Bahrami, Maryam Doborjeh, and Alan Wang. Brain- Inspired Opatio-Temport Associative Memories for Neuroimaging Data Classification: EEG and fMRI. 10(12):1341.
[44] Yoav Kessler. The role of working memory gating in task switching: A procedural version of the reference-back paradigm. Frontiers in Psychology, Volume 2017, 2017.
[45] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Baabsskk-Bawwinska. al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-3222 2017.
[46] James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska. Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521-3222 2017.
[47] Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M Asano. Vera: Vector-based random matrix adaptation. In The Twelfth International Conference on Learning Representations, 2024.
[48] Dmitry Krotov and John J. Hopfield. Dense associative memory for pattern recognition. In D. Lee, M. Sugiyama, U. Luxburg, Guyon, and Garnett, editors, Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016.
10

---

[49] Jogendra Nath Kundu, Rahul Mysore Venkatesh, Naveen Venkat, Ambareesh Revanur, and R Venkatesh Babu. Class-incrementa. domain adaptation. In Computer Vision-ECCy 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XIII 16, pages 69. Springer, 2020)
[50] Chaehyeon Lee, Sungyong Baik, Dongmin Park, Junho Yim, Seunghyun Lee, Jaejun Yoo, and Nojun Kwak. Domain generalization with stochastic weight averaging. In International Conference on Learning Representations (ICLR), 2021.
[51] Seung-Hee Lee and Yang Dan. Neuromodulation of brain states. Neuron, 76(1):209-222, October 2012.
[52] Byounggyu Lew, Donghyun Son, and Buru Chang. Gradient estimation for unseen domain risk minimization with pre-trained models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4436-4440 2023.
[53] Chuqiao Li, Zhiwu Huang, Danda Pani Paudel, Yabin Wang, Mohamad Shahbazi, Xiaopeng Hong, and Lue Van Gool. continual deepfake detection benchmark: Dataset, methods, and essentials. In Winter Conference on Applications of Computer Vision (WACV), 2023.
[54] Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy M. Hospedales. Deeper, broader and artier domain generalization. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 5543-5551, 5551, 2017.
[55] Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli, editors, Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Pupers), pages 4582 Online, August 2021. Association for Computational Linguistics.
[56] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and muchine intelligence, 40(12):2935-2944 2017.
[57] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Trunsactions on Pattern Analysis and Machine Intelligence, 40(12):2935-2947, 2018.
[58] Ziyue Li, Kan Ren, Xinyang Jiang, Bo Li, Haipeng Zhang, and Dongsheng Li. Domain generalization using pretrained models without fine-tuning, arXiv preprint arXiv:2203.04600. 2022.
[59] Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual learning. In 2023 IEEE/CVF Conference on Computer Vision and Puttern Recognition (CVPR), pages 78 7816-7825, 2023.
[60] Yan-Shuo Liang and Wu-Jun Li. Adaptive plasticity improvement for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, pages 78 7816-7825, 2023.
[61] Charles J. Limb and Allen R. Braun. Neural substrates of spontaneous musical performance: An fmri study of jazz improvisation. PLOS ONE, 362)e1679, 2008. ott
[62] E K Miller and Cohen. An integrative theory of prefrontal cortex function. Annu Rev Neurasci, 24: 107-202 2001.
[63] Beren Millidge, Tommaso Salvatori, Yuhang Song, Thomas Lukasiewicz, and Rafal Bogacz. Universal hopfield networks: A general framework for single-shot associative memory models. In International Conference on Machine Learning, pages 15561-15583. PMLR, 2022.
[64] Ha Quang Minh, Partha Niyogi, and Yuan Yao. Mercer's theorem, feature maps, and smoothing. In Proceedings of the 19th Annual Conference on Learning Theory, COLT'06, page 154-168, Berlin, Heidelberg, 2006. Springer-Verlle

---

[65] Min- Yeong Park, Jae-Ho Ho Lee, and Gyeong-Moon Park. Versatile incremental learning: To- wards class and domain- domain-agnostic incremental learning. In European Conference on Computer Vision, pages 271 271-288. Springer, 2024.
[66] Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE International Conference on Computer Vision, pages 1406-1415, 2019.
[67] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-87877 8763. PmLR, 2021.
[68] Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl. Michael Widrich, Thomas Adler, Lukas Gruber, Markus Holzleitner, Milena Pavlović, Geir Kjetil Sandve, et al. Hopfield networks is all you need. arXiv preprint arXiv:2008 02217, 2020.
[69] Hubert Ramsauer, Bernhard Schäfl, Johannes Lehner, Philipp Seidl, Michael Widrich, Lukas Gruber, Markus Holzleitner, Thomas Adler, David Kreil, Michael K Kopp, Günter Klambauer, Johannes Brandstetter, and Sepp Hochreiter. Hopfield networks is all you need. In International Conference on Learning Represenlations, 2021.
[70] Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interference arXiv preprint 18Xiv: 181111800..111.... 2018.
[71] Mattia Rigotti, Omri Barak, Melissa R. Warden, Xiao-Jing Wang, Nathaniel D. Daw, Earl K. Miller, and Stefano Fusi. The importance of mixed selectivity in complex cognitive tasks. Nature, 497(7451):585-559) May 2013.
[72] Grzegorz Rypeść, Sebastian Cygert, Valeriya Khan, Tomasz Trzciński, Bartosz Zieliński, and Barthomiej Twardowski. Divide and not forget: Ensemble of selectively trained experts in continual learning, 2024.
[73] Angeles Salles, Clarice Anna Diebold, and Cynthia F. Moss. Echolocating bats accumulate information from acoustic snapshots to predict auditory object motion. Proceedings of the National Academy of Sciences, 117(46):29229 -92338, 2020.
[74] Tommaso Salvatori, Beren Millidge. Yuhang Song. Rafal Bogcaz, and Thomas Lukasiewiez. Associative memories in the feature space, 09 2023.
[75] Terence D Sanger. Optimal unsupervised learning in single-layer linear feedforward neural network. Neural networks, 2(6):459-47,) 1989.
[76] Saul Santos, Vlad Niculae, Daniel McNamee, and Andre FT Martins. Sparse and structured hopfield networks. arXiv preprint arXiv:2402.13725, 2024.
[77] Anna C Schapiro, Nicholas B Turk-Browne, Matthew M Botvinick, and Kenneth A Norman. Complementary learning systems within the hippocampus: a neural network modelling ap- proach to reconciling episodic memory with statistical learning. Philosophical Transactions of the Royal Society B: Biological Sciences, 372(1711):201606)
2017.
[78] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoe Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv: 170.00370. 2017.
[79] James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion: Continual customization of text-to-imaae diffusion with c-lora. Transactions Machine Learning Research.
[80] James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante- Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. Coda-prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEFF/CVF conference on computer vision and pattern recognition, pages 11909-11919, 2023.
12

---

[81] Sainbayar Sukhbaatar, arthur szlam, Jason Weston, and Rob Fergus. End-to-end memory networks. In C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc., 2015.
[82] Mufeng Tang, Helen Barron, and Rafal Bogacz, Sequential memory with temporal predictive coding. arXiv preprint arXiv:2305. 1982, 2023.
[83] Mufeng Tang, Tommaso Salvatori, Beren Millidge. Yuhang Song, Thomas Lukasiewicz, and Rafal Bogacz. Recurrent predictive coding models for associative memory employing covariance learning. PLoS computational biology, 19(4)=e010719, 2023.
[84] Davoud Ataee Tarzanagh, Yingcong Li, Christos Thrampoulidis, and Samet Oymak. Trans- formers as support vector machines, 2024.
[85] Antonio Torralba and Alexei A. Efros. Unbiased look at dataset bias. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1521-1528, 2011.
[86] Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 2013.
[87] Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5018-5027, 2017.
[88] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006. 10726, 2020.
[89] Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, and Tao Qin. Generalizing to unseen domains: A survey on domain generalization. In Proceedings of the Thirtieth International Joint Conference on Artificial Inielligence (IJCAI), pages 4627-4635. 2021.
[90] Liyuan Wang. Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Patlern Analysis and Machine Intelligence, 46(8):5362- 5383, 2024.
[91] Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained trans- formers: An occam's 's razor for domain incremental learning. Advances in Neural Information Processing Systems, 35:5682-566 5695, 2022.
[92] Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-prompts learning with pre-trained trans- formers: An occam's razor for domain incremental learning. Advances in Neural Information Processing Systems, 35:5682-5695, 2022.
[93] Yufei Wang, Xudong Wang. Arun Mallya, Abhijit Guha Roy, Ser-Nam Lim, and Vikas Verma. Coda-prompt: Continual decomposition of adaptation with prompting for rehearsal- free continual learning. In Advances Neural Information Processing Systems (NeurIPS), 2022.
[94] Yufei Wang, Xudong Wang, Abhijit Guha Roy, Arun Mallya, Ser-Nam Lim, and Vikas Verma, Efficient continual learning with learnable adapters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[95] Zhenyi Wang, Enneng Yang, Li Shen, and Heng Huang. A comprehensive survey of forgetting in deep learning beyond continual learning, 2024.
[96] Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen- Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, et al. Dualprompt: Complementary prompting for rehearsal-free continual learning. In European conference computer vision, pages 631 631-648. Springer, 2022.
[97] Zifeng Wang, Zizhao Zhang, Chen Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEFF/CVE conference on computer vision and pattern recognition, pages 139-149, 2022.
13

---

[98] Zifeng Wang, Zizhao Zhang, Chen- Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 139-149, 2022.
[99] Melissa R. Warden and Earl K. Miller. Task-dependent changes in short-term memory in the prefrontal cortex. Journal of Neuroscience, (5807)):5001-158000 2010.
[100] Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. 2015. Cited by: 299.
[101] James C R Whittington, Timothy H Muller, Shirley Mark, Guifen Chen, Caswell Barry, Neil Burgess, and Timothy E Behrens. The Folman-Eichenbon machine: Unifying space and relational memory through generalization in the hippocampal formation. Cell, 183(5):1249- 1263.e23, November 2020.
[102] James C. R. Whittington, Joseph Warren, and Tim E.J. Behrens. Relating transformers to models and neural representations the hippocampal formation. In International Conference Learning Representations, 2022.
[103] Yujie Wu, Rong Zhao. Jun Zhu, Feng Chen, Mingkun Xu, Guoqi Li, Sen Song, Lei Deng, Guanrui Wang. Hao Zheng, Songchen Ma, Jing Pei, Youhui Zhang, Mingguo Zhao, and Luping Shi. Brain-inspired global-local learning incorporated with neuromorphic computing. 13:65.
[104] Zehao Xiao and Cees GM Snoek. Beyond model adaptation at test time: A survey. arXiv preprint arXiv:2411.0363 2024.
[105] Shih- Ying Yeh, Yu-Guan Hsieh, Zhidong Gao, Bernard B W Yang, Giyeong Oh, and Yanmin Gong. Navigating text-to-image customization: From lyCORIS fine-tuning to model evaluation. The Twelfth International Conference on Learning Representations, 2024.
[106] Jinsoo Yoo and Frank Wood. Bayespen: A continually learnable predictive coding associative memory. Advances in Neural Information Processing Systems, 35:29903-222 2e0ie--9914,, 14, 2022.
[107) Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengeheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. Adaptive budget allocation for pframeter-efficient fine-tuning, In The Eleventh International Conference on Learning Representations, 2023.
[108] Bingchen Zhao, Haoqin Tu, Chen Wei, Jieru Mei, and Cihang Xie. Tuning layernorm in attention: Towards efficient multi-modal LLM finetuning. In The Twelfth International Conference on Learning Representations, 2024.
[109] Kaiyang Zhou, Ziwei Liu, Yu Qiao, Tao Xiang, and Chen Change Loy. Domain Generalization: A Survey. 45(4):4396-441
[110] Kaiyang Zhou, Jingkang Yang, Chen Change Loy. and Ziwei Liu. Learning to prompt for linguae models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 140-149, 2022.
[111] Kaiyang Zhou, Jingkang Yang. Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):237722348, 2022.
[112] Yanqi Zhou, Tao Lei, Hanxiao Liu, Nan Du, Yanping Huang, Vincent Y. Zhao, Andrew Dai, Zhifeng Chen, Quoe Le, and James Laudon. Mixture-of- experts with expert choice routing. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS 222 Red Hook, NY, USA, 2022. Curran Associates Inc.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The sections of the papers address all claims made in the abstract and introduc- tion.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
limitations section has been included in the Appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

Justification:
All proofs have been provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: High-level details have been provided in the Experiments and Results section. Additional details have been included in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer:
[Yes]
Justification:
The code shall be publicly released upon acceptance of the paper.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
public httlc//guiessseeeeee wwwpgdddeeeeeeuuoinnmommm Codeeuimmeesiiooeoo ss onPolicy) for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https?
//nips. w/npps.cppubbll wttppcccppppppctttetcccccccccccp /etsc///uu////eeeeeeoeoo.secsses ies CodeeSumm ss ooooooooliio oooiccyyy icy) for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
The required details have been provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
The presented results include the mean and standard deviations across three runs.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative emror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: All details regarding required compute have been provided in the Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics
Answer: [Yes]
Justification:
The authors have confirmed to the NeurIPS code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: The work does not have any societal impacts, as it proposes general technique in improving ML systems.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
18

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All appropriate sources have been cited.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
assets are released, the license, copyright information, and terms of use the package should be provided. For popular datasets, papers thcode com dataaettettt has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
19

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
20

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, or originality the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs any important, original, or non-standard components.
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: http://eeeii..p..ee http://rrr.....ceoeeeereereeeeee http://cccccccoeeeeeeee/////ee/e 2025/ LLM) for what should should not described.
2025/
21


Reviews:
[{"review_id": "nAKE0bvbS7", "reviewer": "Reviewer_TP33", "summary": "This paper proposes MIRA (Memory-Integrated Reconfigurable Adapters), a unified framework that tackles domain generalization (DG) and continual/incremental learning (CL) within a single architecture, inspired by biological memory mechanisms. Drawing from neuroscience, MIRA incorporates Hopfield-style associative memory (AM) modules into a shared model backbone, allowing per-sample dynamic reconfiguration via learned keys and stored adapter weight updates.\n\nThe key technical innovation lies in embedding Hopfield memory in each layer of a Vision Transformer (ViT), where memory keys are learned post-hoc to align with intermediate activations, enabling robust, context-aware retrieval of task-specific representations. This design allows MIRA to store, retrieve, and interpolate adapter updates across tasks and domains without explicit task identifiers.\n\nEmpirically, MIRA demonstrates strong state-of-the-art (SoTA) performance on standard DG and continual learning benchmarks, outperforming many task-specific baselines by large margins (up to 10%). Ablation studies further highlight the importance of both associative memory and key-learning mechanisms in achieving this performance.", "questions": "1) I noticed that your results outperform many adapter- or prompt-based CL methods (e.g., DualPrompt), but you did not include HIDE-PET [HIDE-PET: continual learning via hierarchical decomposition of parameter-efficient tuning], which recently reported stronger performance than many prior methods.\n\n2) Please discuss whether the associative memory has a limit in terms of how many adapter updates it can meaningfully store and retrieve. Can retrieval quality degrade due to interference or memory saturation?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Quality: The paper presents a well-engineered and carefully evaluated system, integrating associative memory into transformer-based architectures in a novel way. Ablation studies are thoughtfully designed, validating the core components (e.g., associative memory and post-hoc key learning).\n\nClarity: The paper is clearly written and well-motivated, with biological inspiration nicely woven into the design rationale.\n\nWeaknesses: Some insight into computational overhead (both in time and memory) compared to traditional adapters or CL methods would be helpful for practical deployment concerns.\n\nThe method is built on ViT backbones, but there is no exploration of how MIRA behaves across different pretrained models (e.g., different architectures, initialization scales, or training paradigms).\n\nAlthough effective on benchmark CL datasets, it is unclear how well MIRA would scale to hundreds or thousands of tasks/domains—a realistic scenario in continual learning.", "was_revised": true, "final_justification": "I have read rebuttal carefully. As athours' pointed out, HIDE-PET indeed achieves approximately 6% higher accuracy than authors' current implementation. While I acknowledge that MIRA is intentionally designed as a versatile and general-purpose continual learning method, I find that it may lack practical applicability in its current form. Therefore, I am considering updating my score accordingly to 3.", "rebuttal": "We sincerely thank the reviewer for their positive and insightful feedback. We greatly appreciate your recognition of our work’s clear motivation, clarity of writing, and the novel integration of Associative Memories (AMs) into Transformers. We are also thankful for your acknowledgment of our state-of-the-art (SoTA) results across both Domain Generalization (DG) and Continual Learning (CL) settings within unified frameworks, and for recognizing the thoroughness of our ablation studies.\n\nBelow, we address each of your points in detail:\n\n**Computational Overhead**:\nWe kindly refer you to our detailed response titled '**Time, Memory, Scalability**' addressed to Reviewer **SLZL**, which comprehensively covers concerns regarding computational overhead. Briefly summarizing, the inference overhead of our associative memory mechanism is minimal, primarily involving only two additional matrix-vector multiplications per Transformer layer. Similarly, the memory overhead incurred by MIRA is negligible compared to the total model size.\n\n**Different Initializations and Architecture Sizes**:\nWe greatly appreciate your insightful suggestion to test different initializations and architecture sizes. To thoroughly address this, we are currently conducting additional experiments evaluating MIRA’s performance across multiple configurations:\n\nArchitectures: ViT-Large (24 layers), ViT-Huge (32 layers)\n\nInitializations: ViT pre-trained on ImageNet-21k, and self-supervised DINO initialization.\n\n**MIRA Scalability**:\nThank you for highlighting scalability. MIRA leverages Modern Hopfield Networks whose theoretical storage capacity scales exponentially with embedding dimension (768 in our case), thus comfortably accommodating thousands of adapters without retrieval degradation. Given this exponential capacity, MIRA can, in principle, scale effectively to numerous tasks/domains well beyond the scale typically encountered in existing standard continual learning benchmarks.\n\n**Comparison with HIDE-PET**:\nWe sincerely thank you for recommending the HIDE-PET method for comparison. Upon your suggestion, we conducted preliminary experiments with HIDE-PET on the iDigits dataset (5 tasks, LoRA rank-4 adapters, ViT-CLIP initialization). HIDE-PET indeed yielded $6\\\\%$ higher accuracy than our current implementation of MIRA under this particular setup. While we anticipate further improvements through hyperparameter tuning, we emphasize that MIRA is intentionally designed as a versatile and general-purpose method for simultaneously addressing DG, DIL, and CIL, rather than aiming solely to surpass specialized, single-purpose methods.\n\nSpecifically, HIDE-PET is explicitly optimized for incremental learning (CIL/DIL) and does not report performance on Domain Generalization tasks. Therefore, direct comparisons are not strongly warranted given our fundamentally different objectives and broader scope. Our primary contribution remains the unified treatment of memory retrieval and adapter adaptation across diverse settings, elucidating the general principles of memory mechanisms rather than exclusively focusing on maximal performance on isolated benchmarks.\n\n**Memory Capacity**:\nWe reaffirm that Modern Hopfield Networks, as integrated into MIRA, have robust theoretical and empirical backing for their exponential storage capacity relative to pattern dimensionality. Given our chosen adapter representation dimension (768), the number of adapters tested remains safely below the threshold where retrieval interference would significantly degrade performance. Therefore, the memory capacity constraints are comfortably satisfied within our current experimental framework.\n\nWe thank the reviewer again for their thoughtful and constructive feedback, significantly aiding in strengthening our manuscript and clarifying our contributions."}, {"review_id": "6Uf9DgWV3b", "reviewer": "Reviewer_gkzH", "summary": "The proposed MIRA is a biologically inspired unified framework for adapting multiple tasks or domains without catastrophically forgetting. MIRA equips each layer with a Hopfield-style associative memory that stores task specific LoRA style low rank adapter updates indexed by learned keys. At inference time, the model generates query vectors from input activations, which retrieve the most appropriate adapter(s) via similarity-based matching. Depending on the separation function, this retrieval may result in a single adapter or a weighted combination. This dynamic, per-sample retrieval enables effective adaptation to domain generalization, class-incremental learning, and domain-incremental learning using a single architecture.", "questions": "•How critical is the associative memory mechanism to MIRA’s performance? Please provide ablations comparing MIRA with simpler alternatives such as Static adapter selection, attention based adaption mix, LoRA without any memory retrieval to understand the true contribution of the associative memory and validate over conventional baselines.\n\n•What is the computational and memory overhead of MIRA? Can the authors quantify inference-time latency, GPU memory usage, and per-layer retrieval cost? \n\n•Why is post-hoc key learning preferred over joint optimization with adapters? Please clarify whether joint training was attempted, what limitations or performance issues motivated decoupling,  whether hybrid approaches were considered as this directly impacts the interpretability and stability claims made by the paper", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n•The paper is easy to read and algorithms clearly explain the flow of adaptation and consolidation. It also articulates a clear motivation for bridging continual learning and generalization by drawing parallels between neural computation.\n\n•The model addresses DG, DIL, and CIL within a single architecture by integrating LoRA style adapters with Hopfield-style associative memory per layer. It equips each layer of a frozen backbone with a Hopfield-style associative memory to dynamically retrieve task-adaptive parameters (adapters) on a per-sample basis without needing task labels or explicit domain identifiers to enable generalization and mitigate catastrophic forgetting.\n\n•Keys for memory retrieval are not trained jointly with adapters, but learned post-hoc, reducing the interference during adapter learning and separates adaptation from consolidation adding interpretability to the retrieval process\n\n•Adapters are retrieved on a per-input basis, allowing the model to dynamically tailor its behavior to each input, supporting fine-grained generalization.\n\n•It stores compressed adapter weights than the raw data making it efficient in terms of memory storage.\n\n•The architecture is modular making it compatible with existing transformer-based pretrained backbones.\n\nWeaknesses:\n\n•While MIRA introduces a combination of mechanisms, its core components Hopfield-style associative memory, LoRA adapters, and task specific fine tuning are well studied and not individually novel.\n\n•The framework performs key/query similarity operations at every layer and scales with the number of stored adapters. However, the paper lacks analysis of inference latency, runtime performance, or memory usage, making it unclear whether the approach is computationally feasible. The use of per-layer query modules adds to the model's complexity but is under specified and not ablated.\n\n•The experiments do not include statistical significance testing, such as confidence intervals or hypothesis tests, limiting the robustness of reported improvements. Furthermore, there is no strong comparison against simpler retrieval heuristics making it difficult to isolate the contribution of the associative memory mechanism.\n\n•Although the model claims improved interpretability through post-hoc key learning, the resulting adapter combinations, especially when involving multiple overlapping adapters may be difficult to interpret in practice. The lack of visualization or sparsity constraints weakens this claim. Additionally, the architectural modularity is not explored in depth, and their influence on retrieval quality is not clearly studied.\n\n•The framework assumes episodic task access in a sequential setting and is not evaluated in task free or online continual learning scenarios. Its performance also hinges on the assumption that low rank adapters are sufficient to capture all domain-specific variations, which may not hold in cases with more severe distribution shifts.\n\n•While MIRA demonstrates strong short-term retention, it is not evaluated on long sequences of tasks, where memory saturation or key/query overlap could degrade retrieval precision.", "was_revised": true, "final_justification": "Upon careful review, I feel that most of my concerns have been addressed, and I will adjust my score to Borderline Accept.", "rebuttal": "We sincerely thank the reviewer for their detailed and constructive feedback. We greatly appreciate the reviewer’s acknowledgment of the clear readability of our paper, the strong and well-grounded motivation, the broad applicability of the MIRA framework to DG/DIL/CIL settings simultaneously, the ability of MIRA to perform per-sample adapter retrieval, and the efficient storage of adapter weights as opposed to full training samples, significantly improving memory efficiency. However, we respectfully clarify that MIRA does not explicitly aim to address interpretability in adapter retrieval. Our paper does not claim interpretability as an objective or contribution; although interpretability might potentially be introduced by specialized similarity functions in future work, this is beyond the scope of the current paper.\n\nWe now address each of the reviewer’s specific concerns in detail:\n\n**Novelty**:\nWe agree with the reviewer that some individual components of our method have appeared previously in isolation. However, we emphasize that our core innovation lies precisely in integrating these known elements into a cohesive and novel architecture, specifically by embedding associative memory (AM) directly into transformer-based architectures, and crucially doing so during the training process itself. This architectural integration represents a novel and meaningful step forward in this area, which, to our knowledge, has not been explored before.\n\n**Computational Overhead**:\nWe fully acknowledge the importance of this point. We have provided a detailed response to the related concern titled '**Time, Memory, Scalability**' raised by Reviewer **SLZL**. In brief, the computational overhead introduced by MIRA during inference is minimal compared to standard LoRA forward passes, as the associative memory retrieval operation adds only two matrix-vector multiplications per layer, a negligible overhead when compared to the computational demands of the transformer’s attention mechanism. Likewise, the memory overhead, driven mainly by storing keys for adapter indexing, is minimal compared to the size of adapters and pre-trained transformer weights.\n\n**Statistical Significance Testing**:\nWe respectfully highlight that our paper already includes confidence intervals across nearly all primary experimental results (Tables 2–5), directly addressing the concern regarding statistical significance. For additional clarification or any further specific analysis, we kindly refer the reviewer to the comment later in this rebuttal titled, '**Naive Retrieval Heuristics**'.\n\n**Interpretability**:\nWe respectfully reiterate that interpretability in retrieval was neither claimed nor demonstrated in our current work. Nonetheless, we appreciate the reviewer’s insightful suggestion of including visualizations to better illustrate retrieval mechanisms. In response, we are currently conducting an additional experiment: for each domain/task, we quantify how frequently adapters specific to that domain/task yield the highest key-query match when processing input samples belonging to that domain/task. Initial results indicate this metric is highly sensitive to the choice of similarity function. We kindly request the reviewer to clarify their interpretation of the phrase \"architectural modularity,\" to ensure our experimental analyses align precisely with their suggestion.\n\n**Episodic Task Access**:\nWe respectfully clarify that our choice to use LoRA adapters was explicitly intended as a proof-of-concept demonstration, as clearly stated in lines 217-219 of the paper. MIRA, as presented, is a generalizable and flexible paradigm, and is in no way restricted to episodic task access or dependent on the specific choice of adapter architecture. Indeed, more complex adapters beyond LoRA can easily be integrated within the MIRA framework to yield even stronger performance. As described in Algorithm 1 and discussed in Section 3 (final paragraph), MIRA can be effectively combined with any continual learning strategy that addresses catastrophic forgetting over extended task sequences. Evaluations on fully online or task-free continual learning benchmarks would require additional time beyond the discussion period; however, we believe that the core contributions of MIRA regarding memory integration, adaptation, and consolidation have been clearly demonstrated within our chosen experimental contexts.\n\n**Memory Capacity**:\nRegarding memory capacity concerns, we clarify that the theoretical retrieval limit of Universal Hopfield Networks scales exponentially with representation dimensionality (here, 768 dimensions). Consequently, the number of adapters considered in our experiments is significantly below this theoretical bound, ensuring stable and reliable adapter retrieval without degradation.\n\n**Naive Retrieval Heuristics**:\nWe sincerely appreciate the reviewer’s thoughtful suggestion to explore naive retrieval baselines. The associative memory mechanism is critical to MIRA to the extent that we require a memory mechanism that is general enough to support a wide range of key-query matches, as well as have a retrieval strategy that is differentiable. Both of these are satisfied in the Universal Hopfield Network framework, which explains our choice as such. Moreover, given our strong motivation from biological mechanisms, associative memories are widely accepted to be highly active in the brain, and thus our proposal better fits into a biological motivation.\nTo thoroughly address the concern regarding conducting ablation studies under naive retrieval baselines, we are conducting additional ablation experiments as follows:\n\n**Static Adapter Selection**: Using a single trained adapter across all tasks/domains.\n\n**Attention-based Adapter Mixing**: Learning a shared linear layer to dynamically combine adapters without memory retrieval.\n\n**Random Adapter Retrieval**: Training adapters independently per task and randomly selecting adapters at inference.\n\nWe anticipate these experiments will yield deeper insight into the advantages provided by MIRA’s associative memory retrieval and aim to include these results during the discussion phase. Furthermore, we underscore the critical role associative memory plays in our framework, aligned closely with the biological motivation and plausibility underlying our design choices.\n\n**Computational Overhead**:\nWe kindly refer again to our detailed response provided under '**Time, Memory, Scalability**' addressed to Reviewer **SLZL** for comprehensive coverage of this issue.\n\n**Post-Hoc Key Learning**:\nFor detailed discussion regarding post-hoc versus concurrent key learning strategies, please refer to our response titled '**Adaptation vs. Consolidation**' addressed to Reviewer **C7LV**. This response explicitly discusses why we advocate the two-stage training approach, highlighting its practicality, flexibility, and alignment with biological mechanisms of task learning and memory consolidation.\n\nWe thank the reviewer again for their valuable and insightful feedback, helping strengthen our manuscript further."}, {"review_id": "i4gXXrWULU", "reviewer": "Reviewer_SLZL", "summary": "This paper proposes MIRA (Memory-Integrated Reconfigurable Adapters), an architecture inspired by biological associative memories. MIRA stores task-specific low-rank adapter updates in Hopfield-style associative memory modules atop a frozen backbone (ViT) and retrieves an affine combination of stored adapters per input using learned keys. The method unifies domain generalization (DG), class-incremental learning (CIL), and domain-incremental learning (DIL) under a single framework with only minor objective modifications. This frame work is conducted on DG, CIL and DIL task.", "questions": "(See Weaknesses)\n\nDoes retrieving affine combinations of adapters for inputs ever lead to negative transfer or performance degradation when tasks are highly dissimilar? If so, how can such interference be mitigated?\n\nWhat is the architecture and parameter overhead of the query modules (g_l) used to transform layer inputs to queries? How sensitive is performance to their capacity and design choices?", "limitations": "The paper does not provide thorough analysis of inference latency, memory footprint, or retrieval time complexity when storing a large number of adapters, which is critical for deployment in continual learning at scale.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths :\n\nUnlike prior adapter-based PEFT methods (LoRA, DualPrompt) or associative memory works that store data or features, MIRA stores adapters and utilize the key to choose a proper adaptor by the input.\n\nMIRA adapts to diverse learning settings with the adaptor. The paper is supported by exceptionally strong empirical results. MIRA is shown to achieve SOTA performance across numerous datasets in all three settings.\n\n\n Weaknesses: \n\nWhile the method achieves high accuracy, there is no thorough evaluation of inference time, memory overhead, or scalability when storing and retrieving large numbers of adapters across tasks. (The framework's performance improves with more adapters per task, and the main experiments use 10)\n\nThe framework is designed for scenarios where task data is available during a training phase (either all at once for DG or sequentially for CL). As currently formulated, MIRA cannot perform zero-shot adaptation to a completely novel task encountered only at inference time, as it would lack the necessary adapters in its memory.", "was_revised": true, "final_justification": "Thank you for your thorough rebuttal. My concerns have been resolved, and I will maintain my score.", "rebuttal": "We sincerely thank the reviewer for their positive and encouraging feedback. We appreciate your recognition of MIRA's exceptionally strong empirical results across Domain Generalization (DG), Domain-Incremental Learning (DIL), and Class-Incremental Learning (CIL) within a unified framework, and your acknowledgment of our novel approach in utilizing Associative Memories (AMs) distinctively compared to prior works.\n\nBelow, we carefully address each of your insightful points:\n\n**Time, Memory, and Scalability**: We thank the reviewer for highlighting the importance of time complexity, memory overhead, and scalability analyses. We acknowledge this gap and confirm that detailed experiments addressing these concerns are currently underway, with full results anticipated by the discussion phase.\n\nInitial observations suggest that MIRA’s inference time closely matches that of the standard LoRA-based model without associative memory. This is due to our method’s minimal computational overhead, as we only require two additional matrix-vector multiplications per layer per input, a cost substantially overshadowed by the computational demands of the model's attention mechanisms.\n\nRegarding memory overhead, additional storage requirements are minor. Keys for indexing adapters, along with potential parameters of the query module (if parameterized), introduce negligible memory costs compared to the scale of adapters and pretrained models. Specifically, using an identity-based query module as presented in our paper, having 5 adapters per task across 6 tasks with keys dimensionally sized at 768 adds only about 276K parameters to the model's existing 86M parameters, representing less than a 0.4% increase. Additionally, from a theoretical standpoint, Modern Hopfield Networks guarantee stable retrieval for an exponentially large number of patterns relative to representation dimensionality (768 here). Thus, our adapter count remains comfortably within stable retrieval limits. Notably, as detailed in lines 316-319 of our manuscript, empirical benefits plateau beyond 5 adapters per task.\n\n**Zero-shot Performance**: We respectfully clarify our viewpoint on zero-shot performance. Domain Generalization inherently tests a model’s ability to adapt without explicit task-specific information at inference, which aligns with zero-shot adaptation scenarios. In DG, MIRA successfully retrieves ensembles of adapters rather than single adapters, optimizing performance on previously unseen tasks or domains.\n\nEquation (2) of our paper explicitly captures this notion by defining performance metrics directly over unseen test distributions. Our method's efficacy under zero-shot conditions depends significantly on the extrapolation ability of the adapter selection (separation/similarity) functions, extensively discussed in Appendix Section A.1.2. We kindly request the reviewer to refer to this section for an in-depth theoretical explanation.\n\n**Negative Transfer**: Thank you for raising this important point. Negative transfer, where affine combinations of adapters can destructively interfere, indeed poses potential challenges. Our results indicate that although theoretically possible, destructive interference is rare in practice. This is potentially because the consolidation phase learns near-orthogonal keys for such task pairs. To further mitigate this risk, future research could explore designing similarity functions that encourage orthogonality or independence among adapters or their keys, a valuable suggestion we intend to investigate.\n\n**Query Module Ablations**: We acknowledge your suggestion regarding further clarification of query module choices. We have conducted comprehensive ablation studies specifically addressing the impact of query module selection. These experiments and analyses are provided in detail under the subsection titled \"**Choice of $g()$**\" within Appendix Section A.2 of our paper. We respectfully invite the reviewer to revisit this section for the complete ablation study.\n\nWe thank the reviewer again for these constructive suggestions, which have significantly enhanced the robustness and comprehensiveness of our manuscript."}, {"review_id": "v930ZPH5DD", "reviewer": "Reviewer_C7LV", "summary": "The paper proposes a unified framework using Hopfield associative memory (AM) with reconfigurable adapters to address Domain Generalization (DG), Domain-Incremental Learning (DIL), and Class-Incremental Learning (CIL). A fixed ViT backbone is supplemented with task-specific adapters stored as \"values\" in AM. Dynamically learned \"keys\" enable sample-based retrieval of relevant adapters during inference. Training involves adaptation and consolidation stages. Evaluation is performed on both domain generalization and continual learning tasks, and demonstrate the superiority of the proposed method.", "questions": "The two-stage training pipeline is a distinctive feature of the thesis. However, it would be valuable to elucidate whether this approach was essential or if it conferred particular benefits. In my view, it seems more like a limitation. Could the keys and memories be learned concurrently? For instance, Mixture of Experts (MoE) models do not necessitate two-stage learning.", "limitations": "N/A", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n- First integration of AM with parameter-efficient fine-tuning (PEFT) for unified DG/CIL/DIL. Post-hoc key learning dynamically aligns adapter retrieval with task inputs.  \n- Achieves SOTA on multiple benchmarks, outperforming specialized methods in most tasks with significant margin.\n\n**Weaknesses**\n- Lacks ablation studies for Post-hoc key learning vs. other trivial keys such as random keys and more naive retrieval methods (e.g., image similarity)\n- It would be nice to discuss more about adapatation versus consolidation, especially in CL setting. E.g., what will happen if consolidation is applied only in the end like the case in DG\n- [Not confident, remind me if there is some misunderstanding] The thesis should discuss the relationship between the proposed method and existing techniques, such as, at least, **Neural Modulation** and **Mixture of Experts**. There appear to be significant similarities that warrant exploration and comparison.\n[1] Beaulieu, Shawn, et al. \"Learning to continually learn.\" ECAI 2020. IOS Press, 2020. 992-1001.\n[2] Zhou, Yanqi, et al. \"Mixture-of-experts with expert choice routing.\" Advances in Neural Information Processing Systems 35 (2022): 7103-7114.\n- Some minor typos\n\n     Undefined acronyms (e.g., \"MHN\" in Line 147).  \n     Incorrect table references (Table 1 vs. Table 3).  \n     Symbolic errors (e.g., \"[]\" in Line 201).", "was_revised": true, "final_justification": "After carefully reviewing the rebuttals and feedback from other reviewers, I will maintain my current evaluation.\n\nFirst, while the authors assert that revisions have been incorporated in response to my concerns, I regret that I could not locate these additions in the revised manuscript. Clarity on where specific revisions address my points would be helpful.\n\nSecond, regarding the technical contributions: The memory-key-value design and two-stage optimization approach, while technically sound, appear to align closely with established mechanisms in frameworks such as self-attention and Mixture-of-Experts (MoE).  As the other reviewers has pointed out, It needs some more analyses on both training and testing efficiency, and practicality analyses.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. We greatly appreciate the reviewer recognizing our contributions as the first work integrating Associative Memories (AMs) into the learning architecture to unify Domain Generalization (DG), Domain-Incremental Learning (DIL), and Class-Incremental Learning (CIL), enabling dynamic alignment of adapters to task inputs. We also thank you for acknowledging our achievement of state-of-the-art results on standard benchmarks.\n\nBelow we respond point-by-point to your valuable comments:\n\n**Random Keys**:\nThank you for suggesting an experiment with random keys. We agree this would provide insightful comparisons. We are currently conducting an experiment where we train the Adaptation phase normally, but do not train the Consolidation phase to learn the keys. We anticipate this setup will yield lower performance compared to the standard MIRA configuration, as the model will lack the capability to optimally select adapter combinations tailored to specific tasks.\n\nRegarding ablations against naive retrieval methods, we emphasize that our method specifically retrieves adapters based on learned proxy keys, distinctly different from traditional data-sample retrieval methods (e.g., \"image similarity\"-based retrieval). However, in alignment with your request and suggestions from Reviewer **gkzH**, we are also performing additional experiments employing simpler heuristics for adapter retrieval. We kindly direct your attention to our detailed response under '**Naive Retrieval Heuristics**' to Reviewer **gkzH** for further insights.\n\n**Adaptation vs. Consolidation**:\nThank you for highlighting this important distinction; we fully agree it merits clearer articulation in the main paper. The key distinction between **Adaptation** and **Consolidation** in our framework is as follows. **Adaptation** focuses on learning task‐specific parameter adjustments, instantiated here as LoRA‐style adapters, that enable rapid fine‐tuning to each new domain or task. **Consolidation**, by contrast, is concerned with retrieving the appropriate adapter weights at test time without access to any meta‐task labels; in our implementation, this is achieved via learned keys conditioned on the input. Crucially, our DG and CL protocols remain intact regardless of the particular mechanisms we employ for adaptation and consolidation.\n\nAddressing your specific question, if consolidation were only applied after all tasks/domains had been experienced, it would necessitate storing data from all experiences, violating the fundamental continual learning principle. This approach would degenerate into a conventional supervised setting, losing its incremental learning properties and failing to address the continual learning scenarios central to our motivation. Our current two-phase approach inherently respects incremental constraints, avoiding data replay and maintaining practical applicability in genuine continual learning environments.\n\n**Multi-perspective Analysis of MIRA**:\nWe appreciate your suggestion regarding broader contextualization. While MIRA is indeed novel and distinct from existing methods, we acknowledge conceptual overlaps with other areas of machine learning, including Meta-Learning and Mixture-of-Experts (MoE). We have thoroughly explored these connections in Appendix Section A.1:\n\nThe relationship to the meta-learning paradigm (e.g. Neural Modulation) is elaborated in Section A.1.1.\n\nAnalogies with MoE approaches are extensively discussed in Section A.1.2.\n\nWe kindly request the reviewer to refer to these appendix sections, which explicitly delineate our method’s position within the broader ML literature, clarifying how MIRA uniquely contributes beyond established frameworks.\n\n**Two-stage Training Approach**:\nThank you for raising this insightful consideration. Indeed, keys and memories could theoretically be learned concurrently in a single stage. However, we deliberately adopt a two-stage training approach motivated by practical and theoretical reasons:\n\n**Computational feasibility**: Training adapters with keys for large-scale foundation models can be computationally expensive and resource-intensive, limiting scalability and practical applicability. MIRA’s flexibility allows independently trained task adapters (potentially obtained from external sources) to be incorporated seamlessly in the Consolidation phase, significantly reducing computational demands and costs in such settings.\n\n**Biological motivation**: Inspired by biological learning processes, we distinguish short-term, rapid task adaptation (Adaptation) from long-term memory integration and retrieval (Consolidation). This mirrors neurobiological evidence where organisms exhibit distinct phases for learning new information and integrating it with existing knowledge.\n\n**Minor Corrections**:\nWe thank the reviewer for pointing out the minor typographical errors; we have carefully corrected these in the revised manuscript.\n\nWe once again sincerely thank the reviewer for their thoughtful and constructive comments, significantly aiding our refinement and enhancing the clarity and impact of our manuscript."}]

Author rebuttal:

Meta-review: This paper was assigned to four reviewers. Unfortunately, most of the reviewers were either unresponsive or provided limited, unconstructive feedback. Specifically:

Reviewer TP33 did not follow up on the corrections made by the authors. As a result, the rating appears outdated since the authors had already addressed the concerns raised.

Reviewer gkzH, despite multiple reminders from both the AC and the authors, did not actively participate in the discussion. The reviewer provided only a brief justification at the end of the discussion phase, even though the authors had addressed all major issues.

Reviewer SLZL was similarly unresponsive during the discussion phase despite several reminders.

Reviewer C7LV provided a justification that was not entirely fair. The authors had directed the reviewer to supporting material in the appendix, which was ignored. Furthermore, the reviewer’s second comment was vague, claiming the work was “similar to self-attention and Mixture-of-Experts,” which was inaccurate. The authors, however, provided a strong rebuttal.

Given the mixed reviews, the lack of constructive engagement from several reviewers, and after carefully reading the paper myself, I (AC) recommend acceptance as a poster.

Reasons:

1. The biological integration of Hopfield dynamics into current feedforward architectures is novel and compelling.

2. Incorporating these components significantly improves model performance across all three downstream tasks.

3. The authors provided a thorough rebuttal, including new baseline experiments, ablation studies, and model analyses (e.g., inference time and compute efficiency).

Overall, the idea is innovative and encourages the community to think beyond standard feedforward networks and Mixture-of-Experts approaches.
Decision: Accept (poster)