Before Forgetting, There's Learning: Representation Learning Challenges in Online Unsupervised Continual Learning
Authors: Cameron Ethan Taylor
Shreyas Malakarjun Patil
Constantine Dovrolis
Venue: Accepted by TMLR ()

Abstract: This paper addresses the Online Continual Unsupervised Learning (O-UCL) problem, where a learner must adapt to a stream of data arriving sequentially from a shifting distribution without storing past data or relying on labels. This challenge mirrors many real-world machine learning applications, where efficient training and updating of large or on device models is critical. We first explore the unique challenges of O-UCL and identify a secondary failure mode in addition to catastrophic forgetting.  We demonstrate that the presence of transient, small-scale biases in an online data stream can significantly impair learning. Unlike traditional notions of distribution shift that manifest over long timescales, we highlight how biases occurring at the level of individual batches or short segments—while imperceptible in aggregate—can severely hinder a model’s ability to learn, a phenomenon we call ``catastrophic non-learning''. We further showcase how an auxiliary memory can be used to solve both catastrophic forgetting and catastrophic non-learning, but that the criteria for the ideal memory for each are in conflict. In response to these findings, we introduce a dual-memory framework which incorporates specifically designed modules to mitigate both catastrophic non-learning and forgetting. We validate our findings on challenging, realistic data streams derived from ImageNet and Places365, comparing against multiple baselines to highlight the distinct nature of this problem and the need for new approaches in O-UCL.

Published in Transactions on Machine Learning Research (11/2025)
Before Forgetting, There's Learning: Representation Learn- ing Challenges in Online Unsupervised Continual Learning
Cameron Taylor
cameron.tayloo&@gttecc.ccc edu
School of Computer Science, Georgia Institute of Technology
Shreyas Malakarjun Patil
smpatil 7 edu com com
School of Computer Science, Georgia Institute of Technology
Constantine Dovrolis
constantine@ggatcch.dde c.dovrolis@cyi.ac..
Schhol of Computer Science, Georgia Institute of Technology
Computational Science and Technology Reseorch Center (CaSToRC), The Cyprus Institute
Reviewed on OpenReview: https: // openrreveee..... net/ formm? id= YZunnyyy
Abstract
This paper addresses the Online Continual Unsupervised Learning (O-UCL) problem, where learner must adapt to stream of data arriving sequentially from shifting distribution without storing past data or relying on labels. This challenge mirrors many real-world machine learning applications, where efficient. training and updating of large or on device models is critical. We first explore the unique challenges of O-UCL and identify a secondary failure mode in addition to catastrophic forgetting. We demonstrate that the presence of transient, small-scale biases in an online data stream can significantly impair learning. Un- like traditional notions of distribution shift that manifest over long timescales, we highlight how biases occurring at the level of individual batches short segments while impercep- tible in aggregate-can can severely hinder model's ability to learn, a phenomenon we call "catastrophic non-learning". We further showcase how an auxiliary memory can be used to solve both catastrophic forgetting and catastrophic non-lcarning, but that the criteria for the ideal memory for each are in conflict. In response to these findings, we introduce a dual- dual-memory framework which incorporates specifically designed modules to mitigate both catastrophic non-learning and forgetting. We validate findings on challenging, realistic data streams derived from ImageNet and Places365, comparing against multiple baselines to highlight the distinct nature of this problem and the need for new approaches in O-UCT. The code is available on github here
1 Introduction
The Continual Learning problem has been widely studied recent years with many papers approaching the problem from different perspectives Wang et al.) (2024). While continual learning works typically focus on understanding and combating catastrophic forgetting, this not the only challenge that true "continual" learner would face. In many real-world scenarios, where data is collected directly from the environment, stor- ing and labeling this data may be impractical due to costs privacy concerns. Even with the vast amounts of potential pre-training data, many potential application domains will have growing and evolving distri- butions. It would be ideal for these models to grow and expand their understanding as their environments evolve.
Recently, to address some of these challenges the Unsupervised Continual learning (UCL) problem was introduced in Rao et al. (201)). UCL poses the problem of sequentially learning from series of tasks (D1, Dr).. The learner receives one task dataset Dj at time and sample batches by x. such

---

Published in Transactions on Machine Learning Research (11/2025)
that 2, Dj- Given candidltt learning model Fo(:) parameterized by 8, the objective is to learn to extract representations (a:) that are useful for other down-stream tasks such as classification or clustering. Popular approaches for UCL include continually learned generative models Ye & Bors (2020); Achille et al.
(2018); ; Ramapuram et al. (2020); Rao et al. (2019), memory replay [Cha et al. (2021) ; Madaan et al.
2021 ; distillation Fini et al. (2022) : Gomez- Villa et al. (2024); Sadeghi el al. (2024) or both distillation
and replay [Zhang et al. (2024).
Other recent work has focused on the online version of continual learning known as Online Continual Learning (OCL) Lopez-Paz & Ranzato 2017). OCL poses the problem of learning online from stream S(t). Examples
(2,, 3i) arrive one at a time, in some cases including a task identifier 4. Each example is seen only once.
While these methods address the challenges selecting good memories in an online manner [Aljundi et al.
(2019a); Guo et al. (2022) ; Aljundi et al. (2019b): De Lange & Tuytelaars (2021); Chaudhry et al. (2018)
Wei et al. (2023).
continually learning on top of a pretrained model Harun et al. (2023) Hayes et al. (2019)
Hayes & Kanan
(2000) Hayes et al. (2020), little focus is given to the challenges of learning online directly.
While the UCL and OCL problems represent important steps in moving the field of continual learning to- ward more realistic scenarios, we consider even more challenging scenario where the learning is done both without labels and online. We refer to this problem as Online Unsupervised Continual Learning (O-UCL). O-UCL is inspired by many real-world machine learning applications where a critical challenge is the train- ing and updating deployed models in an efficient manner. The problem was first introduced in [Smith et al.) (2019) where was called Unsupervised Progressive Learning (UPL). Several other papers followed which looked at various aspects of the UPL/O-UCL problem such as improving memory building Yu et al. (2023 better augmentations Michel et al. 2002), , sleep as form of offline optimization Taylor et al. (2025
While these papers focus an online and unlabeled version of the CL problem, they still consider learning sequentially from a series of distinct tasks, where the primary challenge is to avoid catastrophic forgetting while learning from limited amount of data, Another closely related work is Hu et al. (2021a),, which inves- tigates pretraining from unsupervised streaming data. While conceptually similar, their approach is oriented toward learning pretrained networks from the stream that are subsequently adapted to downstream tasks. In contrast, our work emphasizes constructing and maintaining useful representations directly throughout the stream itself that are useful as possible given the stream data seen far.
Critically, instead of focusing on the"single-pase setting, argue that learning from online stream is instead challenging because of the inherent lack of control over the ordering and sampling of incoming data, In many real-world scenarios, data does not arrive in perfectly IID manner at small timescales, even if it appears IID when aggregated over longer periods. For instance, a robot navigating an environment may encounter a series of visually similar frames while turning a corner, causing its representation learning to be temporarily biased toward a subset of visual features. Similarly, a recommendation system that updates in real-time may see burst of interactions from niche user group, skewing its learned representations before the broader population is observed. In both cases, the learner is not just at risk of forgetting past knowledge but may fail to develop a meaningful representation in the first place. This perspective shifts the focus from solely mitigating catastrophic forgetting to understanding when and why learning itself can fail due to transient biases in the data stream. Our work highlights this fundamental challenge and demonstrates its impact, providing insights into how online learning dynamics differ from conventional continual learning settings.
1.1 The O-UCL Problem
We consider an online representation learning setting in which an unlabeled data stream arrives sequen- tially. Rather than assuming fully non-stationary process, we model the stream as piecowisc-stationary. meaning that there exist distinct but unknown phases during which the data distribution remains stable before shifting unpredictably
Formally, let S(t) denote the data stream, where data points x, E X arrive sequentially. We assume that time is partitioned into phases Tı, T2,T2,.. ... .,, Th, with unknown boundaries. Within each phase Te, the data follows a stationary distribution (1)),, which is drawn from finite but unknown set of classes N₂- Consecutive phases
2

---

Published in Transactions on Machine Learning Research (11/2025)
Learning (OCL)
Unsupervised Continual Learning (OCL)
Online Unsupervised Continual Learning (OUCL)
task
Took LT!
Stream
Labsted Data
boundarie
Figure 1: Explanation of the key differences between OCL, UCL, and O-UCL (the setting considered in this work)
may introduce new classes, reintroduce previously seen ones, shift their relative frequencies, crcating an evolving but structured learning environment.
The goal is to learn a representation function
fe : Z
parameterized by 0, that captures salient features of the data encountered up to the current phase Tk. Unlike conventional representation learning, where fo is trained on a static dataset, here it must evolve over time, adapting incrementally as new phases arrive.
Specifically, by the end of phase Tg. the model should have learned latent space 2g that enables effective downstream such as clustering or classification over point, i..,
over the union of all classes encountered up that
N₁ U...UNU
At each timestep 4, the model may incorporate the newly arrived data It (or mini-batch thereof) into parameters 0, but it does not control the order or composition of the stream. The learner may also store auxiliary information such as aggregate statistics, embeddings, or even a selection of raw images aid in representation learning. Unlike many continual online learning problems, we do not impose strict cap on the total amount of storage; however, emphasize that memory usage should be efficient, ensuring that the storage requirements remain manageable as the stream progresses. The challenge is not only to determine what to store but also how to leverage this auxiliary memory to maintain robust and generalizable representations while adapting to distribution shifts over time.
At any time 6, the model's performance is assessed using classification accuracy and clustering purity over an evaluation set DR. which contains data sampled from N₃ ....U UN. The challenge is not only avoid catastrophic forgetting of past knowledge but also to ensure that learning remains stable in the presence of small, local biases within individual phases phases-biasco that may not be apparent when viewing the data distribution of the current phase in aggregate.
2 Challenges of Online Unsupervised Continual Learning
The O-UCL problem introduces multiple challenges for learner, but primarily focus two key aspects: (1) learning online from poorly mixed, highly biased data streams, and (2) maintaining memory-cfficient representations while avoiding forgetting, despite the absence of class or phase labels. To systematically investigate these challenges, we conduct a series experiments using a consistent model architecture, training objective, and hyperparameter configuration across all settings.
In each of the following subsoctions, we utilize a neural network F-(-) as our core learning model. Specifically, F.(+) is ResNet-11 18 backbone with projection head trained using the SimCI.R Chen al. (020aa)
3

---

Published in Transactions on Machine Learning Research (11/2025)
Model Performance
Image Pareeelie--n 100 Lows Curem
Places 3-5--6665---000 Loss Curves
Classitcato Accuracy
Imagelett 100
Place365 100
Thaiiing berraiio
Thiiing Deratiom
Figure 2: KNN- of models (left) trained with varying amounts of data for fixed number of iterations. The four models were trained with 10% of the data for 10 epochs, 25% for the data for 4 epochs, 50% of the data for 2 epochs, and 100% of the data for 1 epoch (online). Additionally, loss curves for single seed per model on ImageNet-100 (middle) and Places365-100 (right). The results represent the mean and + one standard deviation over three independently seeded trials for both ImageNet-100 and Places365-100.
self-supervised learning framework. The training objective follows contrastive loss formulation, where positive pairs are constructed using the augmentations from Chen al. (2020a). While adopt SimCLR primary learning paradigm, our framework is agnostic to the specific self-supervised objective, and alternative contrastive
non-contrastive approaches should also be applicable. The model is optimized using SGD with a fixed learning rate of 0.6, a batch size of 512 and temperature of 0.5. All models are trained for the equivalent of 100 epochs in terms of total training iterations.
For each experiment, the total number of classes 100 from either ImageNet (TmageNNet- 100) or Places365 (Places365-100) 100). Each experiment done with a train, validation, and test split of 80%, 10%, and 10% rospectively. All results presented as the mcan over three independently seeded trials, with error surements represented as plus minus one slandard deviation. Further details on the construction of phases and stream sampling blases vary by subsection and are discussed individually in each.
2.1 Online Learning "Single-Pass Pass"
A common assumption in online learning research is that each data point should only be seen once, often referred to as the "single-pase" criterion Lopez-Paz & Ranzato (0117) Hu et al. (2021b) TTylor et al.) (2025): Yu et al. (2023). This restriction significantly reduces the number of training iterations and forces the model to extract meaningful representations in a highly constrained setting. However, it remains unclear whether this assumption fundamental to online learning or whether the assumption that a model cannot revisit any examples "on-demand" sufficient criteria for simulating realisitic learning scenario.
We seek to understand:
Do we need to have the repetition of ezomples in multiple epochs in order learn well?
To investigate this, compare several learners under controlled conditions. One learner is trained "online" and visits a total of N examples exactly once each. This is equivalent to training for a single epoch or equivalently for a "single-pass". We compare this with several learners which observe N unique examples repeated for epochs (i.e., revisiting each example E times). For these comparison learners, we keep the total number of training iterations fixed and vary and N₂ such that E N₂ N. All learners observe the same total number of training batches. If revisiting is necessary, it suggests that online learning requires multiple exposures to the same samples for effective training. Conversely, if performance is equivalent, it challenges the notion that single-pass (defined as only seeing each example exactly once) learning is defining constraint of online learning.
In Figure 2 we showcase the performance of the learners described above. The "online" learners performance, represented by the red bars is compared with other Icarners which train for 2, 4, epochs, while socing Ng N, N 2 and of the examples respectively. The results show that there is no statistically significant difference between any of the learners. Additionally, while the loss curves show that none of the models have
4

---

Published in Transactions on Machine Learning Research (11/2025)
imageNet-
Maces365-100
Classification Accuracy
Accuracy
68
Training Progress
Training Progress
Unblased
50 Classes
Classes
10 Ciaases
5 Classes
1 Class
Figure 3: KNN accuracy comparing several learners trained under varying degrees of biased streams. Results are shown for 100-class subsets of ImageNet and Places363. Each result represents the mean and standard deviation over six trials consisting of dataset samples and two random seeds.
"converged" to a nearly constant loss value, they follow very similar paths further indicating that despite the number of unique data points NE, being very different, the dominant factor in performance is the total number of training iterations.
Based on these results, we argue the criteria that the learner sees each image only once should not be a requirement of an online learning scenario, but instead the requirement should be that it cannot store all the images and cannot control when or if an image is seen again. This allows for the creation of significantly longer streams, created from academic datasets, granting us more flexibility to understand other unique challenges in online learning such as the one presented in the next subsection.
2.2 A New Challenge in Online Learning
An intrinsic difficulty in online learning from data streams is that, even when the long-term distribution of the stream is stationary, the short-term composition of samples may exhibit significant deviations. In practice, data streams are rarely well-mixed; there are often temporal correlations or local biases that cause certain subsets of the data distribution to overrepresented for short periods.
For example, consider robotic agent navigating an environment. If the agent moves through a narrow corridor, its observations may be dominated by limited set of objects textures, biasing its short-term experience. Similarly, in autonomous driving scenario, consccutive frames may contain nearly identical visual features due to temporarily static viewpoint. While these biases may dissipate over longer timescales, they can significantly impact the learning dynamics when training occurs in an online manner. Unlike long- term distribution shifts, which have been extensively studied in the continual learning literature Wang et al.] (2024) these short-term biases introduce a differeent challenge: they may distort the loss landscape for each batch. This because biased batches skew gradient directions away from minimizers of the long-term distribution, effectively reshaping the loss landscape locally. This makes optimization trajectories diverge from globally useful reprosentations, making it difficult for the learner to find a solution that generalizes to the full data distribution.
We explore this by investigating:
How do short-term or local biases in the stream sampling process impact learning?
To quantify the effect of short-term biases in stream samples, we conduct an experiment where we introduce varying degrees of local sampling bias. We consider a stream S(t) consisting of K unique classes and control the degree of bias by restricting how many classes appear within each batch over a short window of time. Specifically, instead of sampling batches uniformly from the full distribution, we constrain each batch to drawn from a subset of classes, where the size of this subset varies across conditions.
We compare six different learners, each trained under a different degree of short-term bias:
5

---

Published in Transactions on Machine Learning Research (11/2025)
Classification Accuracy
Class Balance
Average Memory Duration
Werott Duration
1500
0.40
1000
25%
75%
100%
3000
4000
5000
1000
3000
4000
5000
Leamning Progress (%)
Training Steps
Training Steps
voservoir
-
Quecue
Loss
Upper Bound
Lower Bound
Figure 4: Classification accuracy (left), class balance (center), and average memory duration (right) for models utilizing memory buffers with different selection and removal strategies. The training stream is constructed from 100 classes of ImageNet with a bias of one class per batch. Classification results represent the mean and standard deviation of three independent trials, while class balance and average memory duration are computed from single trial.
1. Unbiased baseline: Each batch is sampled IID from all 100 classes, ensuring well-mixed training data.
2. 50 class bias: Each batch is sampled from a subset of 50 randomly selected classes.
3. 25 class bias: Each batch is sampled from subset of 25 randomly selected classes.
4. 10 class bias: Each batch is sampled from subset of 10 randomly selected classes.
class bias: Each batch is sampled from a subset of 5 classes.
6. 1 class bias: Each batch is sampled from only a single class at a time.
The impact of these biases evaluated on two datasets, ImageNet and Places365.
Figure presents the results of this experiment. As expected, the model trained with unbiased samples (blue curve) achieves the highest performance. However, even mild levels of bias (orange and green curves) result in noticeable degradation in downstream classification performance. With more severe bias (red, purple, and brown curves), performance deteriorates dramatically, confirming that local sampling biases significantly hinder representation Icarning.
Unlike catastrophic forgetting, where knowledge of past distributions is lost, these short-term biases intro- duce a different failure mode: catastrophic non-learning. Since the learner takes gradient steps based on a distribution that is not representative of the long-term data, it struggles to construct a meaningful repre- sentation that generalizes beyond its immediate experience. This highlights the need for approaches that mitigate the effect of short-term biases when learning from non-randomly ordered data streams. It may also be possible that this problem is unique to the SimCLR approach and might be mitigated by another SSL approach such as MoCo Chen et al. (2020b) which utilizes a memory bank. We explore this possibility and demonstrate why this not the case in Appendix
2.3 A Memory Buffer for Biased Streams
As discussed in the previous subsection, the learner has no control over the stream sampling process, meaning that each batch may be poor representation of the overall distribution. This results in biased gradient estimates, which can hinder learning. One way to mitigate this issue is by maintaining small memory buffer to store past examples, enabling the model to construct more representative batches. However, the of a memory buffer may depend on factors such as which examples are stored and for how long.
We seek to understand:
What are the important criteria for designing an effective short-term bias miligating memory buffer?

---

Published in Transactions on Machine Learning Research (11/2025)
To answer this, we consider two criteria for memory selection: 1) Class balance balance-ensutten ennuring that the buffer maintains diverse set of examples across classes. 2) Example duration- duration-controllinnn controlling how long individual examples persist in memory to prevent overfitting. An ideal memory buffer should maximize class balance, minimize example duration, and prioritize high-loss contributing examples to enhance learning efficiency.
We evaluate the impact of different memory selection strategies by comparing three buffer implementations along with lower and upper bound:
Lower Bound: Trains from biased stream without any auxiliary memory buffer.
Reservoir Sampling: Maintains uniform sample of past data, ensuring class balance but reducing throughput and learning value.
FIFO Queue: Stores all incoming examples in a first-in, first-out manner, maximizing throughput but potentially sacrificing class balance and learning value.
Loss Buffer: Selects subset of high-loss examples from cach batch, aiming to balance throughput and class diversity.
Upper Bound: Trains from an unbiased stream without any auxiliary memory buffer.
To quantify the effectiveness of each buffer, we measure classification accuracy, class balance (via KL diver- gence between buffer class distribution and a uniform distribution), and ezample duration (average time an example remains in memory).
As shown in Figure 4 all memory buffers significantly improve performance compared to training without memory. The FIFO and Loss Buffers achieve similar accuracy, whereas the Reservoir Buffer underper- forms, which we attribute to its higher average memory duration. This suggests that while class balance is important, excessive memory persistence can negatively impact learning by reducing exposure to recent data.
Additionally, the Loss Buffer slightly outperforms the FIFO Queue, likely due to its improved class balance and selective retention of challenging examples. These findings indicate that the optimal buffer must balance short-term diversity with retention of informative samples.
2.4 From Local Biases to Catastrophic Forgetting: The Impact of Multi-Phase Streams
The previous section examined the impact of local sampling biases a stream where all classes remained present throughout, training. While such biases hinder representation learning, they do not introduce per- manent shifts in the data distribution. However, in many real-world settings, the set of observed categories changes over time, requiring the learner to adapt to new data while retaining past knowledge.
In this subsection, we transition from a single-phase stream (100 classes throughout) to multi-phase stream (five phases of 20 classes each), which introduces catastrophic forgetting Wang et al. (2024). Unlike catas- trophic non-learning, where biased sampling prevents effective learning, catastrophic forgetting occurs when past representations are overwritten by newer information. While we showed in the previous subsection that buffers with relatively high throughput are the most effective for dealing with the short-term biases, this will likely not be effective for avoiding the long-term shifts which introduce catastrophic forgetting.
We now investigate:
How well does buffer designed for avoiding catastrophic non-learning also mitigate catastrophic forgetting?
To study this, we modify the previous stream by partitioning it into five sequential phases, each containing a disjoint set of 20 classes. Once a phase ends, its classes are never seen again. In this subsection only, we consider a phase-wise evaluation (similar to task-ineremental evaluations in traditional continual learning). Phasc-wise accuracy for phase T, is measured immediately after training on it, denoted as A, while final accuracy on T₁ after completing all K tasks is A,k. Phase-wise forgetting is quantified as the difference between the initial and final accuracy each task, defined as F. Ai, Aik-
7

---

Published in Transactions on Machine Learning Research (11/2025)
Phase-Wise Accuracy
Forgetting
00
Accuracy
60
Forgetting
6-
40
20
Phase
Phase 1
Phase 2 Phases
Phase
Phase 4
Phase C
Phose
Phase 2
Phase
Phases
Figure 5: Phase-wise classification accuracy and forgetting for several memory buffer implementations. The stream consists of 100 classes from ImageNet, split into five phases of 20 classes cach. Performance is mca- sured at the end of each phase. Results represent the mean and standard devialion over three independently seeded trials.
We evaluate two memory strategics along with lower and upper bound:
No-Mem Lower Bound: Learns with no memory or buffer to improve learning or avoid forgetting.
Reservoir Sampling: Maintains a uniformly sampled memory to improve retention.
Loss Buffer: Prioritizes high-throughput learning by frequently updating stored examples.
Unbiased Upper Bound: Learns from an unbiased stream, that still consists of 5 total phases.
Unlike previous experiments measuring overall accuracy, we now track phase-wise wise accurccy and phose-wise forgetting. Accuracy evaluated at the end of each phase, while forgetting is measured as the difference between peak and final accuracy for each phase.
Figure 5 presents the results. The Loss Buffer achieves strong accuracy during training. closely matching the performance of unbiased upper-bound. However, it suffers significantly higher forgetting compared to Reservoir Sampling.
This highlights key trade-off: high- thbthhhouuhhut learning optimizes initial performance but increases forget- ting. The Loss Buffer rapidly adapts to new phases but at the cost of overwriting older representations, whereas Reservoir Sampling, while less efficient in short-lerm learning, better preserves past knowledge. the next subsection, we explore the important design criteria of a memory for avoiding forgetting, that must select examples online and without labels phase boundary information.
2.5 A Memory Buffer for Avoiding Forgetting
In most existing work, a memory buffer would be assumed to have fixed capacity. However, because we consider streams which may have extremely long even infinile length, we argue that more appropriate criteria is to focus growth efficiency with respect to the number of novel concepts objects introduced. We propose that an effective replay memory must achieve slase-ialanc--strringg new information when distribution shifts oceur while remaining efficient, preventing uncontrolled or unnecessary growth.
The key challenge that, without class labels or phase boundaries, the learner must autonomously determine which examples to retain. This further complicated by the evolving latent space Z(t), where feature distances and similarity relationships shift over time.
We explore this by asking:
How can we design memory which maintains balanced and efficient example set under these conditions?
To explore this, we examine similarity- basod memory that selectively expands when novel examples appear and periodically consolidates redundant entries. Specifically, samples compared against stored

---

Published in Transactions on Machine Learning Research (11/2025)
Memory Growth
Memory Allocation
37500
15000
Merry Nowc
13000
Merooy Size
12300
12500
10000
Memorr
10000
7300
7500
5200
5000
2300
0
Prase
Phase Pless
Wrese
These
Rar
Phowe
manes
Phanses
Phanes
Phnnns
Phaness
Phanes
Phanes
Steam Pragression
After Phases
Figure 6: Memory growth and per-phase storage distribution for a Aimilarity-based memory. The stream consists of 100 classes from ImageNet partitioned into ten phases. Even-numbered phases are four times longer, creating significant class and phase imbalance. The memory growth resembles a sort of "step function" where a spike in growth is observed after the introduction of novel data at the beginning of each new phase and then a period of relative stability and minimal growth is reached until a now phase is introduced.
amples using cosine similarity. If an example sufficiently dissimilar (beyond an adaptive novelty threshold), it is added to the memory. To prevent uncontrolled growth, consolidation step periodically prunes highly redundant examples the latent space evolves.
To evaluate this mechanism, we track memory growth in a data stream with no class labels or phase bound- aries, where the distribution shifts progressively. We measure: 1) Memory size over time-does the buffer grow indefinitely or stabilize? 2) Storage distribution across phases-das the buffer allocate memory equitably despite imbalanced exposure?
As shown in Figure 6 our similarity-based based memory successfully expands when novel information appears (annotated as "novelty growth spike") and stabilizes in size over time (annotated as "memory growth sta- bilization").. Despite significant phase imbalances, memory allocation per phase remains close to uniform, indicating that the buffer reacts to distribution shifts rather than raw frequency. Importantly, some phases require more storage due to increased novelty, preventing a strictly uniform allocation. However, it is im- portant to note that these variances in storage allocation are not correlated with the length of the phase.
Moreover, memory growth trends slightly downward as the stream progresses, suggesting that earlier features remain useful, reducing the need for continuous expansion. These results highlight that, even in the a absence of explicit labels or phase information, well-designed similarity-based memory can maintain compact yet adaptive replay buffer. Something which, to the authors knowledge, has not been demonstrated previously.
3 A Memoryy Architecture for Online Unsupervised Continual Learning
The results from Section 2.3 demonstrated that in an online setting, short-term biases in batch sampling can significantly impair lcarning. In Scction 2.4] we showed that mitigating catastrophic forgetting requires a different memory strategy what is useful for online learning. While an ideal memory would balance short- and long-term biases by maintaining diverse set of examples, the two differ fundamentally in their effective throughput. Relying solely short-term memory leads to catastrophic forgetting, older examples are continually overwritten and never permanently stored. Conversely, relying exclusively on long-term memory results in poor overall performance, since the model primarily learns from a relatively static and limited subset of examples. Motivated by these findings, we propose a dual-memory architecture that combines two memory modules to accommodate ouline learning from a non-uniform stream:
A Short-Term Memory (STM) designed to maintain high-throughhut and balanced batch com- position for improving online learning efficiency.
A Long-Term Memory (LTM) designed expand dynamically while maintaining a compact set of diverse and representative examples to prevent. forgetting.
9

---

Published in Transactions on Machine Learning Research (11/2025)
Stream
STM
LTM
Similarity
Top-K
debresh
Sprune
Mowe
Rase
Auster
4,
Ausen
VI
Figure 7: I) Images arrive one at time as a stream. II) Batch is sampled from B,, BSTM, and BLTM and fed into the encoder Fo,k- ITI) The individual losses are collected and those coming from stream examples are passed to the STM. IV) The STM collects the top K examples from Bs, evicting the oldest examples. V) Recently evicted STM examples are compared with existing LTM examples, the examples are dissimilar enough (> detresh), they are added the LTM. VI) Examples are partitioned into partitions based on age and examples, which are outliers within their respective partition (z score 2prume), based on high similarity with their closest neighbor, are pruned.
This framework provides principled way to balance learning efficiency and memory retention in the online unsupervised continual lcarning (O-UCL) setting. We also emphasize that while we provide specific imple- mentations for the STM and LTM, the contributions of this paper focus on the need for a dual-memory system based on the conflicting criteria necessary to design an appropriate STM and LTM.
3.1 Short- Term Memory (STM)
The STM is responsible for mitigating the adverse effects of short-term biases in the data stream. Since in- coming batches may be highly imbalanced, the STM must (1) sample examples that maximize learning utility and (2) ensure that examples are not overrepresented in future batches. As demonstrated in Subsection 2.3 these requirements suggest two key criteria:
High class diversity:
The buffer should contain examples from as many different classes as possible to prevent dominant subset from biasing training.
Low retention duration: Examples should be replaced frequently to maximize exposure to new samples.
STM Sclection Mechanism At cach time stop t, the STM maintains a buffer Bamm fixed size My To populate this buffer, we select the top K examples from the incoming batch X. that have the highest per-example loss, which serves proxy for their learning utility.
Xetm, X
Σ TEX
Endividuall (nddviduu((Fo(z))..
(1)
Once the STM reaches capacity, the oldest examples are evicted in frst-in-first-out (FIFO) manner. These evicted examples are then passed to the LTM for potential long-term storage.
10

---

Published in Transactions on Machine Learning Research (11/2025)
Individual InfoNCE Loss Following the standard SimCLR framework Chen al. (0200a). each imput is augmented into two different views, 2i and Z;j, and passed through an encoder Fg followed by projection head ge, yielding the embeddings:
2g = 4 [o)o(((
go
(2)
The contrastive loss for single example is computed as:
exp(sim(22 2,)/T)
(1)))))(E( - log Ex exp(sim =x)/T)²
(3)
where sim, 25) ||= for example is then defined as the average loss across both views:
2, is the cosine similarity, and 7 is the temperature parameter. The individual loss
Luddividual (Fo(x)) 1 ((n)toCC((F0 + (InoNNC((13)21))
3.2 Long- Term Memory (LTM)
While the STM helps improve online learning, it does not address long-term retention. The LTM must (1) ensure maintenance of representalive set of examples and 2) do so in efficient manner (preventing uncontrolled memory growth). As demonstrated in Subsection 2.5 these requirements suggest two key criteria:
1. Balanced Storage Allocation: The memory should maintain balanced set of examples with respect to all classes or objects seen so far.
Memory Efficiency: The memory should be efficient in its selection examples, expanding only when novel data encountered.
LTM Selection Mechanism To achieve this, we rely on a similarity-based selection strategy that dy- namically expands storage based on novelty. The LTM stores a growing set of embeddings Zmm derived from past data. At each time step, evicted STM examples are compared against existing LTM examples using cosine similarity:
= 1 - (Zettn 7tmm).
(5)
For each example embedding 2,, we compute its nearest neighbor distance in the LTM:
di min Di.j-
(6)
J B
If di,close > dihresh, the example is added to memory. The threshold dthresh is updated dynamically based on the --percentile nearest neighbor distances in the current memory:
duhresh Pp(fdi,close i Bum)).
(7)
To reduce computational overhead, we maintain a frozen copy of the encoder Fa(z), updating it every u steps ensure stability and efficiency. This step critical for achieving the growth pattern we observe Subsection 2.5 and also allows the computational requirements be significantly reduced, by avoiding the need to recalculate all Zim each step.
11

---

Published in Transactions on Machine Learning Research (11/2025)
Memory Consolidation As training progresses, some stored examples may become redundant due to the evolving feature space. To maintain elficiency, we periodically prune redundant examples based on their average pairwise cosine distance:
1
di
Des
Mia
However, since embeddings introduced carlier training tend to have higher pairwise distances than more recent embeddings, pruning based on global statistics can lead to undesirable bias toward removing older examples. To account for this, we partition the memory into P chronological bins based on insertion time, such that each partition contains examples from distinct time period:
B1tm U Bp where Bp fxx
Bitm ETTp
(9)
Here, P represents the time range for partition p, ensuring that each partition contains examples added to memory during specific interval. This helps to reduce any bias towards different phases or classes that may occur based on the duration with which they have been observed.
For each partition, we compute the mean Pp and standard deviation op of the pairwise distances and use this to compute the z-score for each example within its parition. An example is pruned its Z-score falls below predefined threshold žprune:
B, B {xr zi < āprune
(10)
By partitioning based on insertion time and performing per-partition pruning, prevent systematic bias against newer, relatively more similar, memory entries and ensure that only truly redundant examples are removed.
3.3 Batch Sampling Strategy
At each training step, construct batch X, of size B from the stream, STM, and LTM:
B, + Batm + Btm = B.
(11)
We allocate a fixed fraction (B, = 12.5%B) to stream samples to maintain exposure to new data. The remaining batch is split dynamically between STM and LTM, weighted by their relative memory sizes:
Mstm Bas,
Mitm BMs M
B
Bum
Ba = B.-
(12)
M
This ensures that early training relies more on STM (where forgetting is less a concern), while later training shifts towards LTM as memory accumulates.
This framework ensures that online learning benefits from high-throughput STM for fast adaptation while maintaining dynamic LTM for long-term retention.
4 Experimental Results
In this section, we evaluate the proposed dual-memory system in terms of representation quality, memory efficiency, and adaptability to distribution shifts in an online unsupervised setting without explicit labels.
12

---

Published in Transactions on Machine Learning Research (11/2025)
4.1 Experimental Details
Training Details All models use a ResNet-18 backbone. Training follows the SimCLR self-supervised
approach Chen et al. (2020a), including data augmentations and the InfoNCE loss. Optimization is performed using SGD with lcarning rate of 0.6, momentum of 0.9, and weight decay of 1 x 10-". Downstream classification is evaluated using K-nearest neighbors (KNN) with K = 50, while clustering is assessed via K-means, with K = 2 Ny applied to the learned embeddings.
Stream Construction We evaluate models data streams constructed from ImageNet Deng et al. (2009) and Places365 Zhou et al. (0117). cach containing 100 randomly selected classes. To introduce batch-level bias, each batch is sampled from a single class randomly drawn from the current phase. Streams are divided into 10 sequential phases of 10 classes each (except where otherwise noted), with alternating phase durations to induce an imbalance: even-numbered phases span the equivalent of 400 epochs (batch size 512), while odd-numbor numberd phases span 100 epochs. The dataset is split into 80% for training, 10% for supervision (used to train KNN), and 10% for testing. For both datasets resize the images to 128x and apply standard normalization.
Evaluation Metrics Models are evaluated on two downstream tasks: classification and clustering. Clas- sification accuracy is measured after each phase, as well as averaged over all evaluations. Clustering per- formance is quantified using cluster purity. We adopt class-ineremental learning evaluation protocol [Zhou et al. 2004)) where the classification and clustering tasks expand to include all previously seen classes, without phase labels during training evaluation. Since performance degradation may result from either increased class ambiguity or catastrophic forgetting, we compare against baselines (detailed below) that provide reference points for expected performance.
4.2 Baselines
Due to the uniqueness of the proposed approach, existing methods such as STAM Smith et al. (019 PCMC Taylor et al.] (2025). and SCALE Yu et al. 2023) are not directly applicable. STAM was designed for simpler datasets (c.g., MNIST, SVHN) and struggles with complex images Taylor et al. 2025). PCMC
relies on an offline sleep phase, making it unsuitable for our continuous learning setting. SCALE employs memory clustering at each step, resulting in quadratic complexity with respect to memory size, rendering it
infcasible for long streams. Other potential basclines memory approachos in the CL. litcrature such as Yoon
et al. ) Rebulli et al. (211);; Tiwari et al. (022) would also be difficult to adapt to our setting due to their reliance on labeled examples, task boundaries or offline caleulations between tasks.
Instead, we evaluate five baselines to establish performance bounds:
Supervised Offline A ResNet-18 is trained on the full distribution of 100 classes in an offline supervised setting using cross-entropy loss. The model is trained for 100 epochs with SGD (Ir = 0.1, momentum 0.9, weight decay 1 x 10-+), ensuring the same total mumber of training iterations as the streaming setting. This serves as an upper bound, illustrating performance achievable under idcal conditions with labeled data and no sampling biases.
Self-Supervised Offline An encoder is trained on the full dataset in an oflline manner using the SimCLR objective. Training follows the same procedure as the supervised baseline but without class labels and without phasc-level biases. This bascline quantifics the performance gap between our model and an offine--rained self-supervised learner, under equivalent training conditions.
LTM Only An encoder trained under the default streaming setting but using only the long-tern memory module. The LTM is implemented as the similarity memory described in Section 3.2 The LTM is modified to take in all streaming batch examples instcad of those evicted from the STM. This bascline isolates the performance of the model excluding the STM, providing insight into the extent which the STM assists in learning on top of the LTM which mitigates forgetting.
13

---

Published in Transactions on Machine Learning Research (11/2025)
ImageNet-100
Places365-100
Accuracy
Phase
Praseo
Pitsaes
Prasess
Phoses
Phasess
Plasees
Plases
Phases
10 Phases
Phose
Plases
Phasro
Phaaes
Phesees
Phases
Phasee
Plasee
Phases
10 Plasees
Progress
Stream Progress
No-Mem
STM-OniP
LTM-Only
Dual-Mem
Self Supervised
Supervised
Figure Classification and clustering performance per phase comparing our proposed dual-memory ap- proach with several baselines meant serve as upper and lower bounds. The top rows represents classifica- tion accuracy and the bottom row represents cluster purity. Each data point represents the mean over three independently seeded trials with error represented as + one slandard deviation.
STM Only An encoder trained under the default streaming sctting but using only the short-term memory module. The STM is implemented as the loss-buffer described in Section 3.1 This baseline isolates the impact of excluding the LTM, providing insight into the extent of forgetting over time. Performance expected to initially match the full model but degrade as the number of phases increases due to cumulative forgetting.
No Memory An encoder trained without any memory module. This baseline represents the lower bound on performance, highlighting the benefits of incorporating memory-based replay mechanisms in the proposed approach.
4.3 Primary Results
We evaluate the performance of our dual- dual-memory system on phase-imbalanced stream, comparing with several baselines. Figure 8 presents classification and clustering accuracy across phases.
Understanding the Upper Bounds The supervised and self-supervised offline baselines provide ref- erence points for how challenging the classification and clustering tasks become the number of classes increascs, despite no changes in the models themselves. Even in these offline settings, performance declines as more classes are introduced, highlighting the inherent difficulty of class-incremental learning. The per- formance drop from Phase 1 to Phase 10 for the self-supervised model is 24.9%, while for the dual-memory model, it 28.7%, suggesting that a significant portion of the decline due to task complexity rather than forgetting. The additional 3.8% degradation in our model reflects an estimate of the impact of catastrophic forgetting.
Forgetting and Online Learning Challenges Comparing the dual-memory system to the STM-only, LTM-only, and no-memory baselines illustrates the necessity of both short-term and long-term memory, The no-memory model exhibits extremely poor initial performance, which deteriorates further duc to both learning failures and catastrophic forgetting. The STM-only model starts with much higher accuracy, demon- strating its ability to mitigate early-stage learning failures. However, its performance steadily declines and converges near the no-memory bascline, confirming that STM alone is insufficient for long-term retention. Additionally, the LTM model struggles carly due to the biased streams, however as the stream goes on its performance stabilizes. However, it's overall performance is hindered by learning from stagnant. (low-
14

---

Published in Transactions on Machine Learning Research (11/2025)
Classification Accuracy
0.65
20000
0.50
Accuracy
12500
0.55
10000
0.30
7500
0.45
Classes
Progress
5 Tasks
10 Tasks
20 Tasks
Figure 9:
system trained on streams of varying task granularity (5, 10, 20) tasks. Each stream is created with alternating task lengths, but with varying total number of classes per task and total tasks.
Classification accuracy and LTM growth throughout the stream for the
throughput) pool of examples present in only the LTM. In contrast, the dual-memory approach maintains significantly higher accuracy, with only modest decline, demonstrating its ability to prevent both learning failures and forgetting.
Dataset-Specific Trends The observed trends hold across both classification and clustering tasks for ImageNet and Places365. However, performance degradation is more pronounced in Places365 despite its higher initial accuracy. This is likely due to its higher per-class data availability but lower inter-class variance, making early phases easier while increasing ambiguily in later phases.
4.4 Adaptability to Varying Stream Granularity
A key requirement for the LTM is its robustncss to different stream conditions. Figure g examines the impact of phase granularity by varying the nunber of phases while keeping the total number and ordering of classes fixed. This results in streams where the model experiences different. levels of class diversity and varying durations of stable distributions, without any changes to hyper-parameters.
Impact Classification Performance The left subfigure of Figure shows classification accuracy after every 20 newly introduced classes. In the five-phase stream, this corresponds evaluations phase boundaries, whereas in the twenty-phase stream, evaluations occur every four phases after the same number of classes has been observed. Despite initial differences, all configurations exhibit a similar trend, with the 20-phase stream achieving slightly higher early performance, suggesting that finer-grained updates improve initial learning. However, all streams converge to similar accuracy by the end, demonstrating that the dual-memory system remains effective across varying phase granularities.
Impact on Long-Term Memory Growth The right subfigure of Figure illustrates the LTM's growth over time. While the overall trends remain consistent, streams with fewer phases exhibit faster growth rate. This effect arises due to the similarity threshold dehresh being slightly larger in finer-grained streams, leading to more selective memory expansion. We suspect this may be caused by quicker convergence for simpler class sets presented in the smaller phases. Despite these differences, final performance remains unchanged, indicating that. the LTM dynamically adjusts its capacity based stream conditions without degrading accuracy.
Practical Implications These results suggest that the stream's phase gramularity known beforehand, larger value of /3 for lower phase granularity streams may be beneficial. However, the dual-memory approach demonstrates strong adaptability, ensuring reliable performance across a broad range of phase gramularities without requiring hyper-paraneter tuning.
15

---

Published in Transactions on Machine Learning Research (11/2025)
4.5 Impact of Memory Consolidation Sensitivity
Table 1 examines the effect of varying the LTM consolidation sensitivity, controlled by the pruning threshold Sprune. We report final classification accuracy, clustering purity, and final LTM memory size for different values of Aprune on both ImageNet-1000 and Places365-100 streams. The setting Sprune oo represents model with consolidation.
Trade-Off Between Memory Efficiency and Performance Across all settings, reducing zprune sig- nificantly decreases total memory usage, demonstrating the effectiveness of consolidation in preventing un- bounded growth. However, for moderate pruning thresholds (žprune 2.0), performance remains nearly unchanged, indicating that many pruned examples were redundant. When pruning is more aggressive (Sprume = 1.75), slight drop in accuracy and purity is observed, suggesting that excessive consolidation may remove informative examples.
Long-Term Memory Growth Stability Although total memory usage at Tprume = oo is higher than other configurations without consolidation, we observed a more interesting trend in the growth rate of the memory. Specifically, without any pruning the size of the memory continues to grow slowly, despite a period of fixed distribution. This caused by the definition of novelty continuing to shift over time. By adding consolidation, the LTM is able to reach steady stale where the number of additional "novel" examples is equal to the number of pruned examples. This leads to a slowly improving LTM example set, while keeping the total size fixed.
Practical Considerations These results highlight the importance of consolidation in achieving a stable, bounded memory size while maintaining high performance. Moderate pruming (e.g.; Eprune 2.0) effectively balances memory efficiency and representation quality, making it practical choice for long-term online learning scenarios. Additionally, we observe fairly similar performance on both datasets and expect that similar choices would perform well on other datasets.
Table 1: Comparison of classification accuracy and cluster purity for various zprune thresholds in the memory consolidation step. Results represent the mean across three independently seeded trials + one standard deviation.
ImageNNtt-l Net-100
Places365-100
Zprune
A
P
[TTM]
A
LTT
CO
41.0 0.1
30.0 + 0.4
19472 + 813
40.7
27.2 + 0.3
32152 + 1587
2.25
40.5 0.5
30.0 0.5
18377 + 584
4.10
2.1
29576 + 831
2.0
41.3 0.2
15845 + 868
4.2
27.6
27554 + 601
1.75
38.1
+ 0.6
2.1
10451 571
3.69
2.7
9490 1547
5
Discussion
This work highlights the unique challenges of Online Unsupervised Continual Learning (O-UCL). We examine the commonly held "single-pass" criteria which is common among online learning works and argue that it should not be fundamental assumption of ouline learning. We further propose be more relevant to an online Iearner, emphasizing the impact of short-term biases that hinder lcarning and cause what we call calastrophic non-learning. Additionally, we examine the unique challenges which the O-UCL environment presents in terms of building a memory to be used for replay. Arguing that such a memory in the O-UCL setting should not have fixed memory cap, but should instead focus on efficient and balanced growth, despite the lack of labels, phase identities, and even when presented with imbalanced streams.
challenge which may
16

---

Published in Transactions on Machine Learning Research (11/2025)
To address these challenges, we proposed dual-memory framework that integrates short-term memory (STM) for high-throughput learning and long-term memory (ITTM) that dynamically selects and retains novel examples based on evolving latent space similarities.
Our experimental results demonstrate that traditional or single-memory approaches fail to simultancously mitigate both catastrophic non-learning from biased streams and calastrophic forgetting due to long-term shifts. In contrast, the dual--mmoory system significantly improves representation retention and adaptation, maintaining significantly improved classification and clustering performance across challenging ImageNet-1100 and Places365-10 100 streams.
While our work demonstrates promising empirical results, we have not provided a corresponding theoretical analysis or justification of the proposed method. This omission stems from the inherently unconstrained nature of the O-UCL problem, which makes only minimal assumptions regarding key factors such as the independence and identical distribution (IID) of the data, the number of classes, or the total amount of available data. These relaxed conditions render rigorous theoretical treatment particularly challenging. Nevertheless, developing such analyses represents an important and promising direction for future research.
Our proposed approach is also memory and computationally efficient, retaining only a small fraction of the training stream while maintaining competitive performance. For ImageNet-1000 100, the long-term memory (LTM) stored an average of 15,845 + 868 images, corresponding to approximately 14.7% of the total training set, with an overall storage footprint (including short-term memory, STM) of ~ 840 MB. For Places365- 100, the LTM stored 27,554 + 601 images, or roughly 6.8% of the training data, requiring 1.35 GB of storage. The memory systems of memory images to be done in parallel with the forward pass and other model updatos, ensuring the system is able to compute between 3-6 iterations per second on single V100. These results highlight that our approach achieves substantial compression of the training stream while keeping memory usage within practical bounds. Despite the potential for our proposed approach to scale further, this work was done with limited computational resources and scaling experiments to the full ImageNet.k or Places365 datasets would be computationally prohibitive and so left. for future work.
also implemented parallel workers, allowing for the augmentations
These findings underscore the necessity of addressing both catastrophic forgetting and non-lcarning in real- world continual learning. Future work could explore further optimizations in adaptive memory selection, self-supervised objectives tailored for O-UCL, and scalable memory consolidation strategies to enhance online unsupervised representation learning.
References
Alessandro Achille, Tom Eccles, Loic Matthey, Chris Burgess, Nicholas Watters, Alexander Lerchner, and Trina Higgins. Life-long disentangled representation learning with cross-domain latent homologies. Ad- vances in Neural Information Processing Systems, 31, 2018.
Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. Advonces in neural information processing systems, 32, 2019a.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019b.
Hyuntak Cha, Jacho Lee, and Jinwoo Shin. Co21: Contrastive continual learning. In Proceedings of the IEEE/CVP International conference on computer vision, pp. 9516 9525, 2021.
Arslan Chaudhry, Marc' Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-gem. arXiv preprint arXiv:1812.00/.50 2018.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PmLR, 2020a.
17

---

Published in Transactions on Machine Learning Research (11/2025)
Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming Hc. Improved basclines with momentum contrastive learning. ar aixiv preprint orXiv:2033.044 04297, 2020b.
Matthias De Lange and Tinee Tuytelaars. Continual prototype evolution: Learning online from non- stationary data streams. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 8250 8259, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei, Imagenet: A large-scale hierarchical image database. In 2009 IEFE conference computer vision and poltern recognition, pp. Teee, 2009.
Enrico Fini, Victor G Turrisi Da Costa, Xavier Jlameda-Pinedi Elisa Ricci, Karteck Alahari, and Julien Mairal. Self-supervised models are continual learners. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 9621-9630, 2022.
Alex Gomez-Ville Bartlomiej Twardowski, Kai Wang, and Joost Van de Weijer. Plasticity-optimized com- plementary networks for unsupervisod continual learning. In Proceedings of the IEEE/CVF Winter Con- [erence on Applicalions of Compuler Vision, pp. 1690 1700, 2024.
Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximiza- tion. In International conference on machine learning, pp. 8109-8126. PMLR, 2022.
Md Yousuf Harun, Jhair Gallardo, Tyler L Hayes, Ronald Kemker, and Christopher Kanan. Siesta: Efficient online continual learning with sleep. arXiv preprint arXie:2308.1072) 2023.
Tyler L Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear discrimi- nant analysis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp. 220-221, 2020.
Tyler L Hayes, Nathan D Cahill, and Christopher Kanan. Memory efficient experience replay for streaming learning. In 2019 International Conference on Robolics and Aulomalion (ICRA), pp. 9769 9776. IEEE, 2019.
Tyler I. Hayes, Kushal Kafle, Robik Shrestha, Manoj Acharya, and Christopher Kanan. Remind your neural network to prevent catastrophic forgetting. In European conference on computer vision, pp. 466 483. Springer, 2020.
Dapeng Hu, Shipeng Yan, Qizhengqiu Lu, Lanqing Hong, Hailin Hu, Yifan Zhang, Zhenguo Li, Xinchao Wang, and Jiashi Feng. How well does supervied pre-training perform with streaming data? anXiv preprint arXiv:2104. 12081, 2021a.
Huiyi Hu, Ang Li, Daniele Calandriello, and Dilan Gorur. arXiv:2111.01956 2021b.
One pass imagenet.
arxiv preprint
David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Divyam Madaan, Jachong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang. Represcntational continuity for unsupervised continual learning. arXio preprint orXtv:2110.0697 2021.
Nicolas Michel, Romain Negrel, Giovanni Chierchia, and Jean-François Bercher. Domain-aware augmenta- tions for unsupervised online general continual learning. arXiv preprint arXix:2309.0080 06896, 2023.
Jason Ramapuram, Magda Gregorova, and Alexandros Kalousis. Lifelong generative modeling. Neurocom- puting, 404:381 400, 2020.
Dushyant Rao, Francesco Visin, Andrei Rusu, Razvan Pascanu, Yee Whye Teh, and Raia. Hadsell, Continual unsupervised representation learning. Advances in neuraa information processing systems, 32, 2019.
18

---

Published in Transactions on Machine Learning Research (11/2025)
Selvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001 2010,, 2017.
Mohammadreza Sadeghi, Zihan Wang, and Narges Armanfard. Forward-backward knowledge distillation for continual clustering. arXiv preprint 2rx1:22405.1923, 2024.
James Smith, Cameron Taylor, Seth Baer, and Constantine Dovrolis. Unsupervised progressive learning and the stam architecture. orXiv preprint Stt::1904.00222 2019.
Cameron Ethan Taylor, Vassilis Vassiliades, and Constantine Dovrolis. Patch-based contrastive learning and memory consolidation for online unsupervised continual learning. In Vincenzo Lomonaco, Stefano Melacci, Tinne Tuytelaars, Sarath Chandar, and Razvan Pascanu (eds.), Proceedings of The 3rd Conference on Lifelong Learning Agents, volume 274 of Proceedings of Machine Learning Research, pp. 938 938-958. PMLR, 29 Jul 01 Aug 2025.
Rishabh Tiwari, Krishnateja Killamsetty, Rishabh Iyer, and Pradeep Shenoy. Ger: Gradient coreset based replay buffer selection for continual learning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 99-108, 2022.
Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024.
Yujie Wei, Jiaxin Ye, Zhizhong Huang, Junping Zhang, and Hongming Shan. Online prototype learning for online continual learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18764-18774, 2023.
Fei Ye and Adrian G Bors. Learning latent representations across multiple data domains using lifelong vaegan. In Computer AaiionFECCV 2020: 16th European Conference, Glasgow, UK, August. 23-28, 2020, Proceedings, Part XX 16, pp. 777-795, Springer, 2020.
Jachong Yoon, Divyam Madaan, Funho Yang, and Sung Ju Hwang- Online coreset selection for rehearsal- based continual lcarning. arXiv preprint arXiv:2106.cm 01050..6.0..,. 2021.
Xiaofan Yu, Yunhui Guo, Sicun Gao, and Tajana Rosing. Scale: Online self-supervised lifelong learning without prior knowledge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognilion, pp. 2484 2495, 2023.
Yipeng Zhang, Laurent Charlin, Richard Zemel, and Mengye Ren. Integrating present and past in unsuper- vised continual learning. arXiv preprint arXiv:2404.19132 2024.
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. Places: A 10 million image database for scene recognition. IEEE transactions pattern analysis and machine intelligence, 40(6): 1452 1464, 2017.
Da-Wei Zhou, Qi- Wang, Zhi-Hong Qi, Han-Jia Ye, Do-Chuan Zhan, and Ziwei Liu. Class-incremental learning: A survey. IEEE Transactions on Pallern Analysis and Machine Intelligence, 2024.
19

---

Published in Transactions on Machine Learning Research (11/2025)
Table 2: Hyperparameters used in our experiments for ImageNet-100 and Places365-100.
Parameter
ImageNet-100
Places365-100
Description
Training Parameters
Backbone
Nes-81118
ResNet-18
Architecture used for feature extraction. Optimization algorithm used for training.
Optimizer
SGD
SGD
Learning Rate (Ir)
0.6
0.6
Initial learning rate for encoder training. Momentum parameter for SGD.
Momentum
0.9
0.9
Weight Decay
10-6
1 x 10-6
Regularization strength for weight decay. Number of samples per training batch.
Batch Size
512
512
Betream
64
64
STM Parameters
STM Size (Mstm)
2048
2048
Maximum number of examples stored in short-
term memory.
K
10
10
Number examples to select for STM each iteration.
LTM Parameters
B
90
87.5
Sensitivity for calculating adaptive novelty threshold dthresh Number of initial examples stored in LTM. Frequency (in steps) at which LTM updates its stored embeddings.
LTM Init Size (Minit)
512
512
Update Interval (u)
500
500
Pruming Threshold (Zprune)
2.0
2.0
Z-score threshold for memory consolidation pruning.
Partitions (P)
10
Number of partitions for age-based pruning in LTM.
Evaluation Parameters
KNN
50
50
Number of nearest neighbors for KNN.
KMeans
2*N
2
Number of clusters for KMeans where N is classes seen so far.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Table 3: Dual Memory System Augmentation HaperePasameetes
Symbol
ImageNet-100
Places365-100
Color Jitter
0.8
0.8
Brightness
0.8
0.8
Contrast
0.8
0.8
Saturation
0.8
0.8
Hue
0.2
0.8
Horizontal Flip
0.5
0.5
Vertical Flip
0.0
0.0
Random Gray Scale
0.2
0.2
Gaussian Blur
0.5
0.5
Kernel Size
5
12
01
0.1
0.1
02
2.0
2.0
Crop Min
0.08
0.08
Crop Max
1.0
1.0
Classrication Accuracy
Training Loss
STM Cass Balance
65
Class Balorcce
Nccunng
0.4
Rane
Rtane.3
Plased
Plase
Plane
Puose
1000
Stream
Stream
Phese Evoluad line
Max 512
Kap 1024
Ken 2048
Mun 4096
Figure 10: Classífication accuracy (left), training loss (center), and STM class balance (right) for the dual- memory system with varying STM capacities on the ImageNet-100 datastream.
A Additional Dual-Memory System Analysis
A.1 Impact of STM Size
In this subsection, we examine the impact of the short-term memory (STM) capacity on the performance and learning dynamics of the dual-memory system. Figure 10 presents the classification accuracy (left), training loss (center), and STM class balance (right) for four different values of MSTM, which denotes the size of the STM.
Analyzing the classification accuracy, we observe relatively minor differences across the different STM ca- pacities, highlighting the efficiency of the STM in facilitating learning even with minimal memory usage. However, a slight decline in performance is noticeable when the STM size is reduced to 512, suggesting a lower bound beyond which performance begins degrade. Additionally, the training loss exhibits clear decrcasing trend as the STM size increases. This trend expected, as larger STM provides more diverse and representative set of samples, leading more accurate estimation of the true gradient for the current phase's classes.
Examining the STM class balance in the right subfigure, we observe that as the STM size decreases, class bal- ance improves. This indicates that smaller STM capacities roduce greater bias in the batch samples drawn from memory, thereby making the learning task more challenging. These results underscore the importance of maintaining an appropriately sized STM ensure representative set of stored examples. However, they also suggest that effective learning can be achieved with relatively modest STM sizes, demonstrating the efficiency of the meml-y approach in managing memory constraints.
21

---

Published in Transactions on Machine Learning Research (11/2025)
Classification Accuracy
LTM Size
LTM Class Balance
0.7
0.8
Balance
17t Stas
05
10000
0.4
0.2
0.
0.0
Prone:
meso
Prose
Privee
Plase
Prosee
Prise
Phose
Progress
Progress
Phase Evalustionn:
Bota 87.5
Bcta
Sets 92.5
Figure 11: Classification accuracy (left), LTM size (middle), and LTM class balance (right) for the dual- memory system with varying novelty sensitivity values 3 on the ImageNet-100 datastream.
A.2 Impact of |
In this subsection, we analyze the impact of the long-term memory (LTM) novelty threshold sensitivity parameter, 8. Figure presents the classification accuracy (left), LTM size (center), and LTM class balance (right) for the dual-memory system across four different values of [3.
Examining the classification accuracy, we observe minimal differences among the three lower values of 8 with only 3 = 92.5 exhibiting a noticeably lower performance. The middle plot, which illustrates the growth of the LTM, reveals a clear relationship between increasing values of 8 and the overall size of the LTM over time. Notably, the step-function. like growth pattern remains consistent across all values of B, indicating that the model's desired bohavior is robust to variations this paramcter.
Finally, considering the class balance (right), we observe that the models achieving the highest classification accuracy corresponding the lowest values of B tend to exhibit the lowest class balance. This suggests that when the novelty threshold is overly restrictive, the LTM struggles to construct a representative set of exemplars, ultimately degrading overall classification performance.
A.3 Dual-Memory System Memory Usage
The total memory consumption of the dual-memory system for storing images is given by the sum of the memory allocated to the short-term memory (STM) and the final size of the long-term memory (LTM) at the end of the stream. In the case ImageNet-100, with a parameter setting of B = 90 and žprune = 2.0, the LTM stored an average of 15845 868 images. The ImageNet-100 datasct comprises 100 classos, cach containing 1350 images. The training subset consists of 80% of these images per class, resulting in total of 1080 images per class and an overall training set size of 108,000 images. Consequently, the dual-memory system retained approximately 14.7% of the total training images.
Since these images are resized to 128 x 128 pixels and stored in uint8 format, the storage requirement per image is approximately 48 KB. Given that the LTM stored 15,845 images and the STM maintained 2,048 images, the total storage requirement approximately 840 MB.
Similarly, for Places365-100, the LTM stored an average of 27554 + 601 images. The number of images per class in Places365 is not entirely balanced, but the training subset consists of approximately 404,000 images. As result, the dual-memory system retained around 6.8% of the total training data throughout the stream. The storage requirement for these images amounts to approximately 1.35 GB.
A.4 Novelty Detection Dynamics
Figure 12 presents the evolution of the novelty threshold and the number of detected novelties throughout the stream. Observing the novelty threshold over time, we note a steady increase during the first three phases. This trend arises because the model begins from a randomly initialized state and must progressively structure the latent space to better scparate the incoming data. Upon reaching the fourth phase, distinct pattern emerges: the novelty threshold momentarily drops at the introduction of each new phase before recovering to an approximately stable level.
22

---

Published in Transactions on Machine Learning Research (11/2025)
ImageNet-100
Novelty Threshold
Novelties Detected
0.3
Count
10
0.2
Natelly Shrinkage
0.1
Phose
Phade 2
4 Phase 5
Phose Phase
Phose 8 Phase
Phose 10
Phase
2 Phase
Phase 5
Phose Phass
Phase 8 Phoke
Phase 10
Stream Progress
Stream Progress
Seed
Seed 2
Seed
Places365-100
Novelty Threshold
Novelties Detected
Novely
0.25
Siluus
0.20
Novolty Shrnncage
Count
0.15
0.10
0.05
Phose 1
Phase? Phate 3
Phase Phase
Phase Phase
Phase Phase
Phase
Phose 1
Phase Phate 3
Phase Phace
Phace 6 Phase
Phase B Phate
Phase
Progress
Progress
Seed
Seed 2
Seed
Figure 12: Novelty threshold dehresh (left) and novelties detected per step (right) for Image.Net-100 (top) and Places365-100 (bottom). Results are shown for three seeds the phase-imbalanced streams from the primary results in the main paper. The annotations highlight the important trends in the novelty threshold (novelty shrinkage) and the count of noveltities detected per step (novelty spikes) which result from changes in the underlying data distribution (new phases).
Examining the plot of detected novelties, observe pronounced spike at the onset of each new phase. This phenomenon occurs because the introduction of a new phase leads to a rapid expansion of the long-term memory (LTM) as novel examples are incorporated (novelty spikes). However, since this data represents a previously unseen distribution that the encoder has not yet adequately learned, the newly added repre- sentations are initially clustered more tightly in the latent space. This transient effect causes temporary reduction in the novelty threshold dthresh, as the average pairwise distances among data points decrease (novelty shrinkage). As the stream progresses and the model refines its representation of the new phase, the spread of the data increases. In the earlier phases, even the initial data undergoes significant restructuring as the model adapts, diminishing the relative impact of new phase introductions-sinn since, at that stage, all incoming data is effectively novel.
B LTM Adaptation with Repeating Classes
In this section, examine the long- long-term memory (LTM) system's ability to not only adapt to novel data introduced in new phase but also to selectively disregard repeated data that has already been encountered. This characteristic is critical for memory efficiency, as the LTM's capacity is elfectively unbounded. In scenarios where the phases of the data stream cycle through previously observed classes or data, an ideal model should be capable of recognizing and ignoring such redundant information the context of LTM storage, while still leveraging it improve classification performance through continued training on the streaming data.
23

---

Published in Transactions on Machine Learning Research (11/2025)
ImageNet-100
LTM Growth
LTM Class Balance
8000
Class Balance
0.8
LT14 Size
8000
0.6
4000
2000
0.2
0.0
Phone 1
Phone
Phose 3
Phone
Prsse (mspost
Phose
Phose 2
Phose
Phose 4
Proae (Poppest
Stream Progress
Stream Progress
Seed
Seed 2
Seed
Figure 13: LTM growth (left) and LTM class balance (right) for a data stream with repeated phase. The stream consists of four initial phases, each introducing 10 new classes, followed by fifth phase that reintroduces the samne classes from Phase 1, but in a shuffled order.
Figure 13 presents the LTM growth (left) and LTM class balance (right) for a data stream containing a repeated phase. The first four phases each consist of 10 classes, following the same setup as our primary experiments. However, instead of introducing unique set of classes in the fifth phase, reintroduce the same classes from Phase 1, albeit in a different order. Ideally, the LTM should recognize that this phase does not introduce novel information and, therefore, should exhibit minimal growth, with only minor changes in class balance.
The results indicate that while there a modest increase in the LTM size at the onset of the repeated phase, the growth is substantially lower than what would be expected the phase contained truly novel classes. Furthermore, the class balance remains largely unchanged, suggesting that the new additions primarily serve to correct class-wise imbalances in the LTM that arose during the first phase. This effect may stem from adjustments in the latent space as the encoder continued to refine its representations over the course of training. Since the LTM's selection of stored examples during the first phase was based on an undertrained encoder, some those early storage decisions may have been suboptimal. Encountering the same classes again with a more refined encoder likely enabled improved decisions regarding which examples should be retained.
Additionally, due the consolidation step, some of the images originally stored from Phase 1 were pruned from the LTM, meaning that the net increase in stored images from Phase is slightly lower than the raw growth observed in the figure. Specifically, at the end of the first phase, the LTM contained 2,873 images. By the end of the stream, the total number of images stored for Phase had increased to 3,328, representing a total increase of approximately 15%.
These findings demonstrate that the proposed LTM mechanism is capable of appropriately adapting shifts the underlying data distribution, effectively distinguishing between truly novel data and previously encountered information. This ability ensures both efficient. memory usage and robust long-term adaptation in streaming environments.
C Phase-Wise Classification Performance
While the evaluations in this work primarily focus a maassinnrementll setting, where single output space expands new classes are introduced, also analyze the model's phase-wise performance. This evaluation measures the classification accuracy of cach class independently and then averages the accuracy across all classes introduced within the same phase. Since the classification task becomes increasingly challenging as the stream progresses due to the growing number classes we assess performance based on the final evaluation at the end of the stream. Given the highly imbalanced nature of the phase durations in the stream, expected that the model's ability to discriminate between classes introduced in shorter phases may be affected. To quantify this effect, we compare the phase-wise performance of the dual-memory model
24

---

Published in Transactions on Machine Learning Research (11/2025)
Imageliet 100 Phose Wise Performance
Imageliet Phase Performence
Pader
Made
Plae
Pade
Prises
Paaee
Dad
DoAMetey
Spevbal
Figure 14: Phase-wiss wise classification performance for ImageNet-100 (left) and Places365-100 (right). Phase- wise performance is measured the average individual class performance at the final evaluation, averaged over the phase they were introduced. Phases marked with are to indicate their duration is 25% of the other phases length. The results represent the mean over three trials with the error representing = one standard deviation.
against the two upper-bound basclines considered in Figure 8 We omit the lower-bound basclines, as both exhibit significantly reduced performance, making trend analysis less reliable.
Figure 14 presents the phase-wise classification performance for ImageNet-100 (left) and Places365-100
(right). Phases denoted with an asterisk correspond to significanly shorter durations. Despite the im- balanced length of the training phases, the test data used for the evaluation balanced, showing that the model able to reach similar performance on the classes which are observed for significantly fewer training steps. If the model's performance were biased toward longer phases, we would expect to observe systematic reduction in accuracy for these asterisk-marked phases. The results indicate that while some variance exists across phases, the relative performance trends closely follow those of the upper bounds. This is particularly notable, as the upper-bound models were trained offline without exposure to phase imbalances. Thus, their performance trends more accurately reflect differences in the intrinsic difficulty of classifying the classes within each phase, rather than artifacts introduced by phase imbalances. These trends remain consistent across both the ImageNet-100 and Places365-100 data streams.
Overall, these findings demonstrate that the dual-memory system effectively mitigates the challenges as- sociated with highly imbalanced class and phase distributions. The model's performance aligns with the inherent difficulty of the classification task rather than being disproportionattey influenced by variations in data availability across phases.
D Further Online vs Offline Experiments
In this section, we extend the experiments presented in Section 2.1 which analyzed the importance of repeated exposure to training examples. Specifically, we examine the impact of both the number of training iterations and the total amount of unique training data on model performance.
Figure 15 presents a grid of experimental results where each row corresponds to a different number of training iterations, and cach column represents different percentage of the overall datasct used for training. The results in the first row for both ImageNet-1000 and Places365-100 are identical to those in Figure 2 From progressively increase the number of training iterations
these initial results, the figure. Unlike the prior experiment, where the "online" model observed each training example exactly once, we now explore models trained with varying total amounts of data and varying numbers of training iterations.
move downward in
Let N denote the total number of unique examples available in the dataset, and let T represent the total number of training iterations. The top-right cell in Figure 15 corresponds an "online" setting where 100% of the dataset (N) is observed once over 200 iterations (i.e., a single pass over the data). Conversely, the
25

---

Published in Transactions on Machine Learning Research (11/2025)
ImageNet
Places365
292-
14.3
14.2
14.8
14.0
MBC
23.9
23.3
23.7
23.9
Approxiraaee Iterations
493
16.7
17.3
17.8
17.4
1660
28.0
28.1
28-1
28.1
25
1000
21.0
21.6
21.7
21.5
33.9
33.9
34.1
34.1
2090
31.0
31.3
31.6
20005
43.0
25
49.3
43.9
44.0
51.6
53.3
15
/
Data Percentage
Data Percentage
Figure 15: Classification accuracy for varying total number of training iterations and N (total number of unique training examples). Results in the first row represent the equivalent of 10, 5, 2, epochs and N. TO 3 N. N. from left to right, ensuring an equivalent number of training iterations. Each subsequent row from top T: I
to bottom represents a multiple of the previous row in terms of iterations, which also increases the effective epochs (or repeats of individual images).
bottom-right cell corresponds to training regime where the model undergoes 20,000 iterations, equivalent to approximately 100 epochs over the dataset.
Examining the results within each column, we observe that performance consistently improves as the number of training iterations T increases, which aligus with expectations. However, for the first four rows, the total amount of data used (i.e., the percentage of N) has little impact on final accuracy. It is only in the last two rows where the model undergoes a significantly larger number of iterations that increasing the dataset size results in mcasurable performance gain. Since the number of iterations per row is fixed, models trained smaller dataset size necessarily experience a higher number of repetitions per example. These results suggest that strong performance is primarily driven by the total number of training iterations T and the dataset size N, rather than the frequency of repeated exposure to individual examples.
To further validate this conclusion, we compare the performance trends between ImageNet- and Places365-100. Despite similar upper bounds on classification accuracy, overall performance on Places365- 100 is significantly higher. This discrepancy can be attributed the larger dataset size of Places365-100, which allows for more unique training examples per iteration. Even when each example is seen only once, the larger N provides sufficient training signal to match or exceed the performance of a model trained "offline" on smaller or equivalent dataset size with an identical total number of iterations T. Consequently, we conclude that an "online" learner with access to a sulficiently large data stream can achieve comparable or superior performance to a model trained in a traditional "offline" setting, provided that N and T are sufficiently large.
E Comparison with MoCoV2 on Biased Streams
It is possible that the extreme reduction in performance induced by the bias streams in Section 2.1 are unique, or particularly substanital due to our use of the SimCLR Chen et al. (2020a) approach for self supervised training. One possible alternative which may have intrinsic benefits MoCo Chen al. 2020b
which alrcady utilizes a memory bank which may function somewhat like the STM in mitigating the impact of short-term bias and catastrophic non-learning. To examine this, we perform an experiment comparing a model trained with the SimCLR. Chen et al. (2020a) approach to model trained with the MoCoV2 Chen let al. (2020b) approach. For the SimCLR model we use the same hyper-parameters as described in Table 2 and for MoCoV2 we use the hyper-parameters for ImageNet described in the paper Chen et al. 2020b).
Figure[[6sshow The results in Figure [6]show that the model trained with MoCoV2 actually has a slightly worse performance than SLR. We suspect that this because while MoCo does have a memory bank of past representattins it does not store raw examples. Because the model struggling learn at all from the highly biased samples, the stored representations are not useful and actually function as form of noise, pushing the already highly
26

---

Published in Transactions on Machine Learning Research (11/2025)
ImageNet-100
Places365-18s 100
Classiicttion Accuracy
25%
30%
75%
100%
25%
50%
75%
100%
Training Progress
Training Progress
Sin
McCoV2
Figure 16: Classification accuracy of model trained with SimCLR and MoCoV2 methods on biased streams for 100 epochs. Each result represents the mean and standard deviation over six trials consisting of 3 dataset samples and two random seeds.
biased positive batch away from large set of negatives which not all that meaningful to begin with. We see that MoCo closes the performance gap slightly later in training the memory bank is filled with improved samples, but we can clearly see that it does not provide any performance benefit when compared with the SimCLR approach.
F Full Algorithm Psuedocode
27

---

Published in Transactions on Machine Learning Research (11/2025)
Algorithm Python-Like psuedocode of the dual-memorr system algorithm.
Total batch size
atatm capacity
new examples STM per
novelty threshold senstivity
M_init initial size LTM
nu encoder/d_ threehh update frequency 2_prune nemory consolidation sensitivity
# number of LTM partitions during consolidation
# encoder The backbone with associated loss (e.g. simclr/zeenet18)
10
stm init_ mnit_sta(M_stm, K)
11
1tm init_ltar (encoder... (el__dd deepcopy (), beta, m_init, mu, 2_prune, P)
for X_stream in (e)errat(())
# Calculate batch sizes per source
и size ltm.size() () B_m =B- sistream.size() B_stm, B_ltm (ste. M) B_m, (1tm. size M)
Create batch and perform training step 1_stm, X_ltm stm sanple 1_stream, 1_stm, 1_1tm encoder. twttttrraaannnnnnnnottttt tranattttttttppoottttttteee stream, X_stm, x_1tm))
sample (B_lt)) 1tm)
21
22
23
Update the STN and LTM stm_evicted stm stm_update 1_stream, x_stream) ltm_ Stm_update (1tm, stm_evicted)
24
26
28
if (t % mu) == 0:
27
imm.dncceer encoder deepcopy () 1tm tan_consolddaee (11m))
28
def stm (ta_update 1_stream, K_strean);
evicted_exampe examples stn (stu # pop examples from the stm queue nev_examples X_stream [argsort [_storam (1_strean) [-stm. k:]] # top examples stm enqueue (nev_exxa) exe_eexm
38
37
def sim_update (1tm, stm stm_ evicted)
38
encoder (stm (stm_evictttteeee evicted)
examples
39
(2_sta ltm 2tm. pairuise distances between stm_evicted LTX close_d min(D, axis-1) distance to nearest neighbor per example
40
41
42
add_ inds where (close_d 1tn. Ltm.d_tt thresh)
43
ltm. X. add eeicteeviitedeeeeee [add_inds]) 1t (2_ stm (ndd)
44
45
46
return 1tm
47
def 1tm (mmcconsolidate(1aa)))
D (1em.2 itm. calculate pairvise distances for each 1tn embedding close_d (D, axis=1) # closest embedding for each example
1tm. 1thre_thresh (ercentile 1tm. beta) update d_thresh
partitions partition(1ta (1tm, ltm.P) split the embeddings into partitions
55
58
for partition partitions:
57
Calculate per partition distances and similarity score D.P =1 (partition. a partition. scores 2_score ltm.2_prune.
58
59
60
01
Keep only the dissimilar examples inds shere(score) partition. x partition. {[kee_indds partition. Z partition. Perrition.2(kopp... [[keepiind]
62
2_prune)
63
64
return ltn
28


Reviews:
[{"review_id": "v3hpTUV4x4", "reviewer": "Reviewer_kkFm", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- Novelty and relevance: The dual-memory approach addresses a realistic and challenging online learning scenario not fully addressed in prior work.\n\n- Comprehensive experimental evaluation: Evaluations include classification, clustering, phase-wise analysis, LTM growth, memory consolidation, and robustness to stream granularity.\n\n- Insightful analysis of memory mechanisms: The paper clearly separates STM and LTM roles and quantifies their impact on catastrophic non-learning and forgetting.\n\n- Scalability and practicality: Memory efficiency and adaptive consolidation are well-motivated, with storage requirements explicitly quantified.\n\n- Reproducibility: Hyperparameters, evaluation protocols, and detailed metrics are provided for both ImageNet-100 and Places365-100.\n\nWeaknesses:\n\n- Limited theoretical analysis: While the empirical results are compelling, the paper lacks formal guarantees or theoretical justification for memory selection and novelty threshold adaptation.\n\n- Generality of LTM thresholds: The novelty threshold (β) and pruning z-score (zprune) are tuned per dataset. Providing guidance on automatic adaptation or transferability to new datasets would strengthen the work.\n\n- Comparison to existing continual learning methods: Although baseline selection is justified, including direct comparisons to other unsupervised or self-supervised continual learning approaches (e.g., STAM, SCALE) would better contextualize the contributions.\n\n- Clarity in figures: Some plots, particularly those showing LTM growth and novelty threshold dynamics, could benefit from additional annotations or explanations for readers unfamiliar with online unsupervised learning.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission addresses the challenges of Online Unsupervised Continual Learning (O-UCL), where models learn from non-stationary data streams without labels or phase information. The paper identifies two key problems in this setting:\n\n1. Non-learning due to short-term biases in streaming batches, which prevent effective representation learning.\n\n2. Catastrophic forgetting caused by long-term shifts in the data distribution, where earlier learned representations are overwritten by new information.\n\nTo tackle these challenges, the authors propose a dual-memory architecture comprising:\n\n- Short-Term Memory (STM): Maintains a high-throughput, diverse buffer of recently observed examples to improve online learning efficiency and mitigate short-term biases.\n\n- Long-Term Memory (LTM): Dynamically stores a compact set of representative examples, selectively expanding when novel data appears and pruning redundant entries to prevent uncontrolled growth.\n\nThe paper also introduces a batch sampling strategy that balances contributions from the stream, STM, and LTM, ensuring rapid adaptation while preserving long-term knowledge.\n\nKey experimental contributions include:\n\n- Comprehensive evaluation on complex datasets (ImageNet-100 and Places365-100) under phase-imbalanced streams, showing that the dual-memory system significantly outperforms single-memory or no-memory baselines.\n\n- Analysis of memory efficiency and adaptability, demonstrating that the LTM can grow efficiently, maintain class balance, and appropriately handle repeating or redundant data.\n\n- Investigation of hyperparameter sensitivity (e.g., LTM novelty threshold and pruning) and stream granularity, showing robust performance across various online conditions.\n\n- Empirical evidence that online learners with sufficient data and iterations can achieve comparable or superior performance to offline learners, even without repeated exposure to individual examples.\n\nOverall, this work provides a novel framework for efficient, adaptive online learning that mitigates both catastrophic non-learning and forgetting, and introduces a memory design that could serve as a foundation for future research in online unsupervised representation learning.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Clarify novelty detection and threshold dynamics: Explain how the LTM novelty threshold adapts over time and its influence on memory growth and class balance.\n\n- Phase-wise evaluation details: Provide more information on how per-class accuracy is aggregated into phase-level performance, especially for imbalanced phase durations.\n\n- Memory consolidation rationale: Justify the choice of pruning partitions and the zprune threshold, and show how they affect LTM efficiency and representation quality.\n\n- Ablation studies: Include results demonstrating the impact of varying STM and LTM sizes on forgetting, classification, and clustering performance.\n\n- Scalability and generalization: Discuss computational and memory costs, and the approach’s applicability to larger or more imbalanced datasets beyond ImageNet-100 and Places365-100.", "broader_impact_concerns": "- Data bias propagation: The method relies on large image datasets (ImageNet, Places365) that may contain inherent biases. Without careful consideration, the learned representations could perpetuate social or cultural biases when applied in real-world systems.\n\n- Privacy considerations: Although the model stores subsets of training images in memory, repeated storage of sensitive or personally identifiable content could raise privacy concerns if applied outside benchmark datasets.\n\n- Environmental impact: The continual learning setup involves frequent updates and maintenance of both STM and LTM, which could increase computational and energy costs, especially for large-scale applications.\n\n- Misuse potential: Models capable of online learning from streaming data could be deployed in surveillance or other sensitive applications without explicit consent, raising ethical concerns.\n\n- Limited demographic coverage: The datasets used are predominantly curated images and may not reflect global diversity, which could limit fairness and generalization in practical deployment.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "4hCpCAfRT1", "reviewer": "Reviewer_KaMd", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n- The problem setting of online continual unsupervised representation learning is significant and practical, as the capability of a pre-trained model is expected to scale with more unlabeled data. In particular, recent large language models (LLMs) typically have a cutoff date for their learned knowledge. Although a separate continual pre-training stage is widely used after the initial pre-training stage in industry, enabling LLMs to learn directly from fully streaming data would further greatly enhance their self-evolving capability.\n\n- Regarding the method, the dynamically expandable long-term memory is interesting and is demonstrated to effectively address the O-CUL problem. In particular, the selection and updating mechanism is reasonable and avoids dependence on any extra hyperparameters.\n\n- The implementation details and empirical analysis are comprehensive, which enhances the understanding of the proposed framework.\n\n**Weaknesses**\n\n- Lacks discussion or comparison with relevant prior work [A] on representation learning from unlabeled streaming data.\n\n- The claimed new challenge of catastrophic non-learning due to short-term biases is not fully convincing. According to the authors, typical local biases occur when the data distribution is overrepresented for short periods. This explanation is plausible when the representation learning method is SimCLR, which relies on contrastive learning with all examples in the same batch. However, the claim becomes less convincing if the method is not SimCLR but, for example, MoCo [B] with a memory bank or other approaches that do not require contrastive learning.\n\n- The claim that the proposed Short-Term Memory (STM) and Long-Term Memory (LTM) are based on conflicting criteria is not easy to understand. It appears that both STM and LTM require high class diversity. This raises the question of why STM is necessary if LTM already exists. In particular, is STM still necessary when the representation learning method is MoCo [B] instead of SimCLR? I suggest providing complete ablation results in Figure 8 by including the results for an “LTM-only” configuration.\n\n- All experiments are conducted using SimCLR with ResNet-18 on ImageNet-100 or Places365-100, which is a rather narrow experimental setting. If the submission aims to substantiate all its claims with empirical results and analysis, it is necessary to extend the experiments by incorporating more diverse representation learning methods and larger datasets, such as the full ImageNet.\n\n- Several claims are difficult to understand due to missing empirical or theoretical support.\n(i) In Section 2.1, the statement “we argue that the criteria for an online…” overlooks that the one-pass setting studied in Hu et al. (2021) is practical for scenarios where data can only be seen once.\n(ii) In Section 2.2, the claim “they may distort the loss landscape…” lacks clarification on how the loss landscape would be affected.\n(iii) In Section 2.3, the phrase “prioritize high-value examples” is unclear—what exactly constitutes a high-value example?\n\n**References**\n\n- [A] How Well Does Self-Supervised Pre-Training Perform with Streaming Data? ICLR 2022\n\n- [B] Momentum Contrast for Unsupervised Visual Representation Learning. CVPR 2020", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission investigates the problem of online continual unsupervised learning (O-CUL). The authors identify a new challenge in O-CUL, termed catastrophic non-learning, which arises from short-term biases that hinder online learning. To address both catastrophic non-learning and forgetting, the authors propose a dual-memory framework comprising a short-term memory (STM) and a long-term memory (LTM). Empirical evaluations using SimCLR on ImageNet-100 and Places365-100 demonstrate the effectiveness of the proposed O-CUL framework.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I suggest that the authors include discussions of relevant work, conduct full ablation experiments, add all relevant baseline methods, and provide complete clarifications or support for their claims—ensuring the claims are convincing with either comprehensive empirical or theoretical evidence. Detailed questions are provided in the weaknesses section.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "o3XdMANIIr", "reviewer": "Reviewer_wqoj", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n\n1.  The paper is well-structured.\n2.  The targeted O-UCL setting is highly challenging and closely reflects real-world application environments.\n3.  The paper thoroughly investigates and questions the \"single-pass\" assumption in online learning through detailed experiments. It also identifies the phenomenon of \"catastrophic non-learning\" in data streams with temporal correlations and local biases.\n4.  The paper designs a dual-memory buffer system with STM and LTM that effectively stores the most valuable samples efficiently and outperforms the single-memory baseline on ImageNet and Place365.\n\n**Weaknesses**\n\n1.  While the exploration of the \"single-pass\" assumption in online learning are insteresting, there are some concerns regarding the specific experimental setup in Figure 2. It is unclear whether the models trained on different data percentages (e.g., 50% data for 2 epochs) have fully converged, as convergence speeds vary with the dataset size. Comparing the results of these training settings might be unfair, given that online learning is restricted to a single epoch. It is recommended to provide training loss curves for Figure 2 to further clarify the impact of convergence on the results. Additionally, to validate the limitations of the single-pass setting for online learning, could explore performing multiple gradient descent steps on each mini-batch of samples to achieve convergence at each mini-batch step? This might lead to severe overfitting, which would further illustrate the constraints of the single-pass assumption.\n2.  In Figure 3, it is observed that the performance of the most severe bias (1 class, green curve) is slightly better than that of the mild bias (10 classes, blue curve). This anomaly needs clarification. It is also suggested to add results for a \"50 classes\" condition to bridge the gap between the unbiased and biased scenarios in this experimental result.\n3.  The proposed method primarily focuses on memory buffer design, an area already extensively studied in online learning[1,2,3]. However, the experimental section only uses simple baselines and does not consider comparisons with other existing online learning methods that feature sophisticated memory buffers.\n4.  The algorithmic pseudocode for the proposed method is not provided.\n\n**Reference**\n\n[1] Rebuffi, S., Kolesnikov, A., Sperl, G., & Lampert, C.H. (2016). iCaRL: Incremental Classifier and Representation Learning. 2017 CVPR.\n\n[2] Yoon, J., Madaan, D., Yang, E., & Hwang, S.J. (2021). Online Coreset Selection for Rehearsal-based Continual Learning. ArXiv.\n\n[3] Tiwari, R., Killamsetty, K., Iyer, R.K., & Shenoy, P. (2021). GCR: Gradient Coreset based Replay Buffer Selection for Continual Learning. 2022 CVPR.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the Online Unsupervised Continual Learning (O-UCL) problem. The paper identify 'catastrophic non-learning' as a new failure mode, where biases in the online data stream significantly impair learning. The paper demonstrates that an auxiliary memory can mitigate both catastrophic forgetting and catastrophic non-learning, but the ideal memory criteria for each are in conflict. In response, they propose a dual-memory framework consisting of a Short-Term Memory (STM) to handle short-term biases and a Long-Term Memory (LTM) to prevent forgetting. The STM prioritizes high class diversity and low retention duration by selecting high-loss examples , while the LTM dynamically expands based on novelty and consolidates redundant samples using a similarity-based approach. Experimental results on ImageNet and Places365 datasets validate the effectiveness of the dual-memory system in maintaining representation quality and adapting to distribution shifts, outperforming single-memory baselines.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1.  Provide the training loss plots for the experiments in Figure 2 to illustrate the impact of model convergence on the results for both online and offline learning scenarios.\n2.  Conduct an experiment to verify the model's learning performance when multiple gradient descent steps are performed on each mini-batch to achieve convergence, and analyze the potential overfitting issues that may arise in such a scenario.\n3.  Clarify Figure 3 and include experimental results for a '50 classes' setting to provide a more complete understanding between unbiased and biased conditions.\n4.  Include more sophisticated memory buffer baselines from prior work in online learning.\n5.  Include the algorithmic pseudocode for the proposed dual-memory framework (the pipeline of STM and LTM) to enhance clarity and reproducibility.\n6.  Typo in Figure 1: the abbreviation 'OCL' in the middle image should be changed to 'UCL'.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)