Label Delay in Online Continual Learning
Authors: Botos Csaba
Wenxuan Zhang
Matthias Müller
Ser-Nam Lim
Philip Torr
Adel Bibi
Venue: NeurIPS 2024 poster (2024)
Keywords: Online continual learning
delayed feedback
computational constraint
efficient learning
Primary area: online_learning

Abstract: Online continual learning, the process of training models on streaming data, has gained increasing attention in recent years. However, a critical aspect often overlooked is the label delay, where new data may not be labeled due to slow and costly annotation processes. We introduce a new continual learning framework with explicit modeling of the label delay between data and label streams over time steps. In each step, the framework reveals both unlabeled data from the current time step t and labels delayed with d steps, from the time step t−d. In our extensive experiments amounting to 1060 GPU days, we show that merely augmenting the computational resources is insufficient to tackle this challenge. Our findings underline a notable performance decline when solely relying on labeled data when the label delay becomes significant. More surprisingly, when using state-of-the-art SSL and TTA techniques to utilize the newer, unlabeled data, they fail to surpass the performance of a naïve method that simply trains on the delayed supervised stream. To this end, we introduce a simple, efficient baseline that rehearses from the labeled memory samples that are most similar to the new unlabeled samples. This method bridges the accuracy gap caused by label delay without significantly increasing computational complexity. We show experimentally that our method is the least affected by the label delay factor and in some cases successfully recovers the accuracy of the non-delayed counterpart. We conduct various ablations and sensitivity experiments, demonstrating the effectiveness of our approach.

Label Delay in Online Continual Learning
Botos C.
Wenxuan Zhang" Mohamed Elhoseiny?,
Matthias Müller
Ser-Nam Lim",
Philip H.S. Torr',
Adel Bibi'
University of Oxford
King Abdullah University of Science and Technology
Intel Labs
University of Central Florida
Abstract
A critical yet often overlooked aspect in online continual learning is the label delay, where data may not be labeled due to slow and costly annotation processes. We introduce a new continual learning framework with explicit modeling of the label delay between data and label streams over time steps. In each step, the framework reveals both unlabeled data from the current time step t and labels delayed with d steps, from the time step d. In our extensive experiments amounting to 25000 GPU hours, we show that merely increasing the computational resources is insufficient tackle this challenge. Our findings highlight significant performance declines when solely relying on labeled data when the label delay becomes significant. More surprisingly. state-of-the-art Self-Supervised Learning and Test-Time Adaptation techniques that utilize the newer, unlabeled data, fail surpass the performance of a naïve method that simply trains on the delayed supervised stream. To this end, we propose simple, robust method, called Importance Weighted Memory Sampling that can effectively bridge the accuracy gap caused by label delay by prioritising memory samples that resemble the most to the newest unlabeled samples. We show experimentally that our method is the least affected by the label delay factor, and successfully recovers the accuracy of the non-delayed counterpart. The implementation for reproducing our experiments can be found at https: : htt///gtttb/.////////////e com/botcs/label-dees exp.
Introduction
Machine learning models have become the de facto standard for a wide range of applications, including social media [1], finance [2], and healthcare [3]. However, these models usually struggle when the distribution from which the data is sampled is constantly changing over time, which is common in real-world scenarios. This challenge continues be active area of research known as Continual Learning (CL). However, most prior art in CL examines this problem with presumption of the immediate availability of labels once the data is collected. This assumption rarely holds in real-world scenarios.
Consider the task of monitoring recovery trends in patients after surgeries. Doctors gather health data from numerous post-operative patients regularly. However, this data does not immediately indicate broader recovery trends or potential common complications. To make informed determinations, sev- eral weeks of extensive checks and tests across multiple patients are needed. Only after these checks completed, the gathered data be labeled as indicating broader "recovery" or "complication" trends. However, by the time the data is gathered, assessed, labeled, and a model is trained, new
Equul Contribution
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Evaluare fo
Input data
1
Data stream Sx
Send for
t-4 4
t
3
2
labeling
Time step
{gf
Labels
Annotator Sd
Figure I: Illustration of label delay. This figure shows a typical Continual Learning (CL) setup with label delay due to annotation. At every time step t, the data stream Sx reveals a batch of unlabeled data {x') J. on which the model fo is evaluated (highlighted with green borders). The data is then sent to the annotator Sy who takes d time steps to provide the corresponding labels. Consequently, at time step t the batch labels {)' corresponding to the input data from time steps before becomes available. The CL model can be trained using the delayed labeled data (shown in color) and the newest unlabeled data (shown in grayscale). In this example, the stream reveals three samples at each time step and the annotation delay is d = 2.
patient data might follow trends that do not exist in the training data yet. This leads to repeating cycle: collecting data from various patients, assessing the trends, labeling the data, training the model, and then deploying it new patients. The longer this cycle takes, the more likely is going affect the model's reliability, a challenge we refer to as label delay.
In this paper, we propose a CL setting that explicitly accounts for the delay between the arrival of new data and the corresponding labels, illustrated by Figure 1. In our proposed setting, the model is trained continually over discrete time steps with label delay of d steps. At each step, two batches of data are revealed to the model: unlabeled new samples from the current time step f, and the labels of the samples revealed at the step d. First, we show the naïve approach where the model is only trained with the labeled data while ignoring all unlabeled data. While this forms a strong baseline, its performance suffers significantly from increasing the delay d. We find that simply increasing the number of parameter updates per time step does not resolve the problem. Hence, we examine a number of popular approaches which incorporate the unlabeled data to improve this naïve baseline. We investigate semi-supervised learning, self-supervised learning and test-time adaptation approaches which are motivated for slightly different but largely similar settings. Surprisingly, methods considered, none could outperform the naive baseline given the same computational budget. Motivated by our extensive empirical analysis of prior efficient method that outperforms every other approach across large-scale datasets; in some scenarios it even closes the accuracy gap caused by the label delay. Our contributions are threefold:
of 12 different
this new setting, we propose simple and
We propose a new formal Continual Learning setting that factors label delay between the arrival of new data and the corresponding labels due to the latency of the annotation process.
We conduct extensive experiments (~ 25, 000 GPU hours) on various Online Continual Learning
datasets, such as CLOO [4], CGLM [5], FMoW [6] and Yearbook [7]. Following recent prior art on Budgeted Continual Learning [8, 9]. we compare the best performing Self-Supervised Learning [10]. Semi- Suni--suereeeed Learning [1 I] and Test Time Adaptation [12] methods and find that none of them outperforms the naïve baseline that simply ignores the label delay and trains model on the delayed labeled stream.
We propose Importance Weighted Memory Sampling to rehearse past labeled data most similar to the most recent unlabeled data, bridging the gap in performance. IWMS outperforms the naïve method significantly and improves over Semi-Supervised. Self-Supervised Learning and Test-Time Adaptation methods across diverse delay and computational budget scenarios with negligible increase in computational complexity. We further present an in-depth analysis of the proposed method.

---

Algorithm Single OCL time step with Label Delay
The Stream Sx reveals batch of images {2z 2=m] De:
2. The model Sa, makes predictions { for the new revealed hatch (xD)
3. The Annotator S3 reveals labels {u!
Fi=is
The model fa, is evaluated by comparing the predictions (0))==) i- and true labels {o where the true labels are only testing;
5. The model fa, is updated to fec+1 using laheled data -²=1 {x7 oo N== and unlaholed data Uf= aia=-tt under computational budget C.
2 Related Work
Label Delay in Online Learning. While the problem of delayed feedback has been studied in the online learning literature [13, 14], the scope is limited problems of spam detection and other synthetically generated, low-complexity data [15, 11] and often views input images as "side in6o"[16]. Additionally, methods and error bounds proposed in[17, 18, 19, 20] are more focused on expert selection rather than representation learning, most of which cannot generalize to unstructured, large-scale image classification datasets.
Continual Learning. Early work on continual learning primarily revolved around task-based continual learning [21, 22], while recent work focuses on the task-free continual learning setting[23, 24, 4]. This scenario poses challenge for models to adapt as explicit task boundaries are absent, and data distributions evolve over time. GDumb[25] and BudgeiCL[8] demonstrate that minimalistic methods can outperform most offline and online continual learning approaches. RealtimeOCL [9] shows that Experience Replay [26] is the most effective method, outperforming more popular continual learning methods, such as ACE [21], LwF [27], RWalk [28], PoLRS [4], MIR [29] and GSS [30], when methods are normalized by their computational complexities.
Semi-Supervised Learning. While the labels arrive delayed, our setting allows the models to use new unlabeled data immediately. Possible directions to leverage the most recent unlabeled data entail
(or often referred to as their broader category, Semi-Supervised Learning) Superviuee Learning (S4L) methods [31]. Pseudo-labeling techniques predict the labels of the samples before their true labels become available to estimate the current state of the joint distribution of input and output pairs. This in turn allows the model fit its parameters on the estimated data distribution. On the other hand, S4L integrates self-supervieed learning, such as predicting the rotation of an image or the relative location of image patches, with the -upervisdd learning framework. We replace the early self self-supervised tasks of S4L [31] with more recent objectives from Balestriero al. [10].
methods [11] and Self-Supervised
Test-Time Adaptation. Besides semi-supervised learning, TTA methods are also designed to adapt models with unlabeled data, sampled from similar distribution the evaluation samples. Entropy regularization methods like SHOT [32] and TENT [33] update the feature extractor or learnable parameters of the batch-normalization layers [34] to minimize the entropy of the predictions.
SAR [35] incorporates an active sampling scheme to filter samples with noisy gradients. More recent works consider Test Time Adaptation in an online setting [36] or Continual Learning setting [37]. In our experiments, we fine-tune the model with ER [26] across time steps and adapt copy of the model with TTA to the most recent input samples at each time step.
3 Problem Formulation
We follow the conventional online continual learning problem definition proposed by Cai et al. [4]. In such a setting, we seek to learn a model fe : X + y on a stream where for each time step t {1, 2, ... the stream S reveals data from time-varying distribution D, sequentially in batches of size n. At every time step, fo is required to predict the labels of the coming batch {af) first. Followed by this, the corresponding labels {u(} are immediately revealed by the stream. Finally, the model is updated using the most recent training data ((x), H
This setting, however, assumes that the annotation process is instantaneous, i.e., the time it takes to provide the ground truth for the input samples is negligible. practice, this assumption rarely holds. It is often the case that the rate at which data is revealed from the stream S is faster than the rate at
3

---

Algorithm 2 Importance Weighted Memory Sampling
1. At time step for each unsupervised batch of size n, (=() the model fe computes predictions (77 For every predieted lahel 8/, select labeled samples from the memory huffer {(ag) .3 ) where yj = ;7:
Compute pairwise cosine feature similarities K cos (h(a)), h(x))) between each unlabeled sample =f and
sclected memory samples =/":
Select the most relevant supervised samples (2g ,86 by sampling & E (1 MM} from multinomial distribution
with parameters K
Update the model fo using the selected supervised samples, aiming to match the distribution of the unlabeled data.
which labels for the unlabeled data can be collected, as opposed to it being instantaneously revealed. To account for this delay in accumulating the labels, we propose pepeea setting that accommodates this lag in label availability while still allowing for the model to be updated with the most recent unlabeled data. We modify the previous setting in which labels of the data revealed time step will only be revealed after time steps in the future.
At every time step E, the Annotator St reveals the labels for the samples from d time steps before, i..., {(x)- y. }== while the data stream Sx reveals data from the the current time step, i.e.. {-t]== Recent prior art [25, 8. 9] introduces more reasonable and realistic comparisons between continual learning methods by imposing a computational complexity constraint on the methods. Similarly to [25, 8, 9]. in our experiments the models are given a fixed computational budget C to update the model parameters from 0g to 4c+1 for every time step To that end, our new proposed setting can be formalized per time step f, alternatively to the classical OCL setting, as described in Algorithm 1.
Note that this means at each time step 4, the stream reveals a batch of non-corresponding images {= and labels {yî Tyf-dT -d-n "}== as illustrated in Figure 1. With the label delay of d time steps, the images themselves revealed from time step d to time step & can be used for training, despite that labels are not available.
A naive way to solve this problem is to discard the unlabeled images and only train labeled data ( of > However, it worth noting that the model is still evaluated the most recent samples from Sx. Thus, training on the labeled training data leads to the model at least being steps delayed. Since in our setting the distribution from which the training and evaluation samples are drawn from is not stationary, this discrepancy severely hinders the performance, as discussed in detail in Section 5.
Furthermore, we shall show in Section 6 that the existing paradigms, such as Test-Time Adaptation and Semi-Supervised Learning, struggle to effectively utilise newer, unlabeled data to bridge the aforementioned discrepancy. Our observations indicate that the primary failure is from the excessive computational demands of processing unlabeled data. To that end, we propose Importance Weighted Memory Sampling that prioritises performing gradient steps on labeled samples that resemble the most recent unlabeled samples.
4 IWMS: Importance Weighted Memory Sampling
To mitigate the challenges posed by label delay in online continual learning, we introduce novel method named Importance Weighted Memory Sampling (IWMS). Recognizing the limitation of traditional approaches that either discard unlabeled data or utilize it in computationally expensive ways, IWMS aims to bridge the gap between the current distribution of unlabeled data and the historical distribution of labeled data. Instead of directly adapting the model to fit the newest distribution with unlabeled data, which is inefficient due to the lack of corresponding labels, IWMS cleverly adjusts the sampling process from memory buffer. This method ensures that the distribution of selected samples closely matches the distribution of the most recent unlabeled batch. This nuanced selection strategy allows the continual learning model to effectively adapt to the most recent data trends, despite the delay in label availability, by leveraging the rich information embedded in the memory buffer.
As discussed in Section 5, using the most recent labeled samples for training leads to over-fitiing the model to outdated distribution. Thus, replace the newest supervised data by batch which

---

CLOC
CGLM
FMoW
Yearbook
#21.1
100
Orline Accurary
15
16.4
10-
01
11
100%
200l
3008
07
216
1k
12
28
3k
1010
2000
Time step
Timee step
Time step
Time step
Figure 2: Effects of Varying Label Delay. The performance of a Naive Online Continual Learner model gradually degrades with increasing values of delay d.
we sample from the memory buffer, such that the distribution of the selected samples matches the newest unlabeled data distribution. The sampling process is detailed in Algorithm 2. It consists of two stages: first, at each time step t, for every unsupervised sample x in the batch of size n, we compute the prediction 7. and select every labeled sample from the memory buffer (a "/ such that the true label of the selected samples matches the predicted label v V/- In the second stage, compute the pairwise cosine feature similarities K,, between the unlabeled sample of and the selected memory samples ar34 by Kij cos (h(=;)), (xx))) (). where h represents the learned feature extractor part of fa, directly hefore the final classification layer. Finally, we select the most relevant supervised samples (ar v/ by sampling k (1... M] from multinomial distribution with parameters k. Thus, we rehearse samples from the memory which (1) share the same true labels the predicted labels of the unlabeled samples, (2) have high feature similarity with the unlabeled samples.
To avoid re-computing the feature representation h(z*)) for each sample in the memory buffer after every parameter update, we store the corresponding features the input data computed for the predictions during the evaluation (Step 4 in Algorithm 1). This technique greatly reduces the computational cost our method, but comes at the price of using outdated features. Such trade-off is studied in detail by contemporary Sepf-Super Literature [38, 39, 40] observing significant impact on performance. We ablate the alternative option of selecting samples based only on their similarity in the Supplementary Material A.1I.
5 The Cost of Ignoring Label Delay
To better understand how label delay influences the performance of model, we begin with the Naïve approach, i.e., ignoring the most recent data points until their label becomes available and exclusively training on outdated labeled samples. More specifically, we are interested in measuring the performance degradation under various label delay d and computational budget C scenarios. To this end, we conduct experiments over 4 datasets, in 4 computational budget and label delay settings. We analyse the results under normalised computational budget (Section 5.2) and demonstrate that the accuracy drop can be only partially recovered by increasing the computational budget (Section A.5).
5.1 Experimental Setup
Datasets. We conduct our experiments four large-scale online continual learning datasets, Continual Localization (CLOC) [4], Continual Google Landmarks (CGLM) [5], Functional Map of the World (FMoW) [6], and Yearbook [7]. The last two adapted from the Wild-Time challenge [41]. More statistics of the benchmarks are in Supplementary. We follow the same training and validation set split of CLOC in [4] and CGLM in [5] and the official released splits for FMoW [6] and
Yearbook [7].
Architecture and Optimization. Similarly to prior work [9, 8), we use ResNet18 [42] for backbone architecture. Furthermore, in our experiments, the stream reveals mini-batch, with the size of 128 for CLOC, FMoW, Ycarbook and 64 for CGLM. We use SGD with the learning rate of
5

---

0.005, momentum of 0.9, and weight decay of 10 5 We apply random cropping and resizing to the images, such that the resulting imput has resolution of 224 x 224.
Baseline Method In our experiments, we refer to the Naive method as the one naively training one labeled data. We apply the state of the art continual learning mechanism under computational constrains [9], Experience Replay (ER) [26], to eliminate the need to compare with other continual learning methods The memory buffer size is consistently 2 samples throughout our experiments unless stated otherwise. The First-In-First-Out mechanism [26, 4] to update the buffer. The discussion of the effectiveness of the memory buffer can be found in Section A.18. We report the Online Accuracy [4] at each time step in Step 4 of Algorithm under label delay d. In our quantitative comparative analysis. for simplicity, we use the final Online Accuracy scores, denoted by Acca. Refer to Section A.17 for the discussion of more metrics.
Computational Budget and Label Delay. Normalising the computational budget is necessary for fair comparison across CL methods, thus, we define C = as the number of FL.OPs required to make one backward pass with a ResNet18 [42], similarly BudgetCL[8] and Realime0CL[[]], We discuss its relation to the wall-clock training time in Section A.16. When performing experiments with a larger computational budget, we take integer multiplies of C to apply C parameter update steps per stream time steps. The proposed label delay factor d represents the amount of time steps the labels are delayed with. Note that, for 1, d O, our experimental setting is identical to prior art[4, 9].
5.2 Observations
In Figure 2, we analyze how varying the label delay d E {0, 10, 50, 100) impacts the performance of Naïve on four different datasets, CLOC [4], CGLM [8], FMoW [6] and Yearbook [7]. The label delay impacts the online accuracy differently across all scenarios, thus, below provide our observations case-by-case.
On CLOC, the non-delayed (d = 0) Naive achieves Acco = 20.2%, whereas the heavily delayed counterpart (d 100) suffers significantly from the label delay, achieving only Acc100 11.7% Interestingly, label delay influences the accuracy in monotonous, but non-linear fashion, as half of the accuracy drop is caused by a very small amount of delay: Acc1o Acco = -.4%% In contrast, the accuracy degradation slows down for larger delays,i.e., the accuracy gap between two larger delay scenarios (d = 50 + 100) is rather marginal Acc Accso = 1.1%. We provide further evidence the monotonous and smooth properties of the impact of label delay with smaller increments of d in the Supplementary Material A.3.
For CGLM the accuracy gap landscape looks different: the majority of the accuracy decrease occurs by the smallest delay d 0 + 10, resulting in Accio Acco -7.9% drop. Subsequent increases (d = 10 - 50 and d = 50 + 100) impact the performance to significantly smaller extent: Accsu Acc1p
1% and Acc100 Accso 0.5%.
In the case of FMoW, where the distribution shift is less imminent (i.e., the data distribution varies less over time), the difference between the delayed and the non-delayed counterparts should be small. This is the case for the satellite image data in the FMoW dataset, where the accuracy drops are -2.8% 2%. 1.9% for d = 0 + 10 + 50 + 100, respectively.
The Yearbook's 's binary classification experiments highlight an important characteristic: if there is a significant event that massively changes the data distribution, such as the change of men's appearance in the 70's [7] the non-delayed Naive (d = 0) suffers a small drop in Online Accuracy (at the middle of the time horizon t 130), but quickly recovers as
data starts to appear. In contrast, under small and moderate delay (d = 10, 50), the decline is more emphasised and the recovery is delayed (at I = 120, 180, respectively). Alongside with more detailed investigation, we provide visual examples of the dataset to support our claims in the Supplementary Material A.4.
5.3 Section Conclusion
Over- or Under-fitting, While in the experiments we report results under a single computational budget C per dataset, it is reasonable to suspect that the results might look different under smaller or larger budget. To this end, we ablate the effect of C over various delay scenarios, on multiple datasets in the Supplementary Material A.5.

---

CLOC (C=2)
CGLM (C=8)
FMoW (C=16)
Yearbook (C=16)
Online Aocatucy id 100) Online Acturac (d 50) Onitze Accutacy 10)
22.14
07 0k
10-
10.56
17:398
1
141.0
24.1
16330
10.00
13.4A
40
60
01.50
08.26
08.1
63 0000
100
92.00
92.24
32.1
10
20
60-
40
100k
200k
500k
3k
100
20-
20.
21.14
80
61.788
100
16
14.99
12:66
15.38
60
07.50
00
77.2e
10-
0.5 4
19:54
12266
1::.3
12.74
55.5A
20.10
(0)
77.0AA
70.54
5
01
01-
01
100k
2000
100k
0
21
5k
71
1k
24
3k
100
2001
20.3
80
100
4077.0
30.00
13:1.
200
05:00
15
11:70
63.9
66688
10-
11.00
10
12.74
12.1
10
63:5
05.34
8.6
0.0
20-
63.04
10
300t 200k 300k
21
3k
0
100
2001
Time step
Time Time-step
Time stop
Time stop
Naäve wishout delay
Naive
IWMS
SCL
Paeudo-Label
TTA
Figure 3: Comparison of various unsupervised methods. The accuracy gap caused by the label delay between the Naïve without delay and its delayed counterpart Naïve. Our proposed method, IWMS, consistently outperforms all categories under all delay settings on three out of four datasets.
Common patterns. We argue that the consistent, monotonic accuracy degradation, present in all of our experiments, is due the non-stane -on-ttationary property the data distribution that creates distribution shift. Our hypothesis is supported by the findings of Yao et al. [41]. A complementary argument is presented by Hammoud across the labels, i.e., images of the same categories arrive in bursts, allowing an online learning model to easily over-fit the label distribution even without using the input images.
[43], stating that the underlying datasets have high temporal correlations
Motivation for Delay Specifie Solutions. As our experiments suggest so far, label delay is indeed an extremely elusive problem, not only because inevitably results in an accuracy drop, but because the severity of the drop itself is hard to estimate a-priori. We showed that the accuracy gap always increases monotonically with increasing delay, nevertheless the increase of the gap can be gradual or sudden depending on the dataset and the computation budget. This motivates our efforts of designing special techniques address the challenges label delay. In the next of experiments, we augment the Naïve training by utilizing the input images before their corresponding labels become available.
Utilising Data Prior to Label Arrival
In our proposed label delay experimental setting, we showed the larger the delay the more challenging it is for Naïve, method that relies only older labeled data, to effectively classify new samples. This is due to larger gap in distribution between the samples used for training and for evaluation. This begs the question of whether the new unlabeled data can be used for training to improve over Naïve, as it is much more similar to the data that the model evaluated on.
We propose four different paradigms for utilizing the unlabeled data, namely. Importance Weighted Memory Sampling (IWMS), Semi-Supervised Learning via Pseudo-Labeling (PL), Self-Supervised Semi-Supervised Learning (S4L) and Test-Time Adaptation (TTA). We integrate several methods of each family into our setting and evaluate them under various delays and computational budgets. In particular, adapt each paradigm individually by augmenting the parameter update (Step 5 of

---

Algorithm 1) of Naive, described in detail in the following subsections. Furthermore, to quantify
the how much of the accuracy gap (Ga = A Naive Acc is recovered, we use the formula R2 =
A Ace namely the improvement of the method divided by the extent of the accuracy gap [Gal
for a given delay factor d.
6.1 Experiment Setup
Importance Weighted Memory Sampling (IWMS). The only additional cost of IWMS compared to Naïve is the cost of evaluating the similarity scores, which is still less than 1% of the inference cost for 100K samples, and can be evaluated in parallel, therefore we consider it negligible. Since method simply replaces the newest supervised samples with the most similar samples from the replay buffer, we do not require any additional backward passes to compute the auxiliary objective. Therefore, the computational budget of our method is identical the Naïve bascline, i.e.. CIMwS 1.
Self-Supervised Seni-Super Learning. For integrating S4L methods, we adopt the most effec- tive approach through iterative optimization of both supervised and unsupervised losses. We report the best results across the three main families of contrastive losses, i.e., Deep Metric Learning Family (MoCo [38], SimCLR [44] NNCLR [45]), Self-Distillation (B [46] and SimSIAM [47], and DINO [40]), and Canonical Correlation Analysis (VICReg [48], Barlow [49]. SWAV [50]. and W-MSE [51]).
For fair comparison, we normalise the computational complexity of the compared methods. According to [8, 9]. Naive augmented with Self-Supervised Learning at each time step takes two backward passes, since they augment each input images to two views, thus Cs4L. = 2. We provide further explanation of our S4L adaptation in the Supplementary Material A.2.
Pseudo-Labeling. To make use of the newer unlabeled samples, we adopt the most common Semi-Supervised Learning technique [11]: Pseudo-Labeling (PL). To predict the labels of the samples before their true label becomes available we use a surrogate model go- After assigning the predicted labels (if) } to each input data {xf} at time step 1. for i = 1.n, the main model fo is updated over the union of old, labeled memory samples and new pseudo-labeled samples {(aT, of )== {(a], Ø using standard Cross Entropy loss. Once fo is updated, we update the parameters of the surrogate model following the momentum update policy [11] with hyper- parameter X, such that Onew = AQoaa + (1 X)
For simplicity, we ignore the computational cost of the surrogate model gs inferring the pseudo-labels ÿ. Nevertheless, the main model fe is trained double the amount of samples Naïve, n labeled and n pseudo-labeled, therefore we define CpL. = 2.
Test-Time Adaptation As done for other paradigms, have extensively evaluated all reasonable candidates to adapt traditional TTA methods to our setting. We find performing the unsupervised TTA step the most effective when only single update is taken (in Step 5 of Algorithm 1), exactly before the evaluation step (Step 2 of Algorithm 1) of the next step. Therefore, for all the parameter updates apart from the last one perform identical steps to Naive Furthermore, we found TTA updates severly impact the continual learning process of the Naive when the parameters are iteratively optimised across the two objectives. Thus, before each TTA step, clone the model parameters 0 to a surrogate model gæ, by performing the TTA step (with hyper-parameter) using the newest batch of unlabeled data Ø next time step.
eVoLTIA {2t} and perform the evaluation (Step 2 of Algorithm 1) of the
To represent the state of the art in TTA, we adapt and compare the following methods: TENT [33], EATA [52], SAR [35], and CoTTA [37], in Figure 3. Furthermore, for the result of our hyper- parameter tuning is provided in the Supplementary Material A.7.
For fair comparison, we train and evaluate all TTA methods under normalised computational budgets (a detailed breakdown of the considerations can be found in the Supplementary Material A.15) More specifically, under fixed computational budget C, at every time step, we perform C supervised steps fe identically to Naïve followed by single step of TTA.
8

---

6.2 Observations
Figure illlustrates our most important results of our work. It shows to what extent we can recover the accuracy gap caused by the label delay between the Naive without delay and its delayed coun- terpart Naive. We evaluate our proposed method, /WMS, and compare it against the three adopted paradigms, S4L, PL and TTA. We report the best performing method of each paradigm with hyper- parameters tuned on the first 10% of each label delay scenario (further detailed the Supplementary Material A.6 and 12).
IWMS. On the largest dataset, containing 39M samples, CLOC [4], the accuracy drop of Naive is Ga = -4.5%, -7.5% -8.6% for d = 10, 50, 100, respectively. Our proposed method, IWMS, achieves Accd- 17.3%, 14.2%, 13.1% final Online Accuracy, which translates Ra- 33%, 19%, 16% recovery for d = 10, 50, 100, respectively. While there is slow decline over in- creasing delays, the improvement over Naive consistent. On CGLM [8], the accuracy drop is G2 = -7.8% 8..8, 99.3% for the three increasing delays, respectively. IWMS exhibits outstand- ing results, Acca = 24.1%, 23.5%, 22.9% meaning that the accuracy gap is fully recovered by the method for d = 10. More specifically, the recovery is R, = 100%. 93%, 87% for d = 10, 50, 100. The results on FM [6] are even more surprising, as IWMS not only recovers the accuracy gap but ousperforms the non-delayed Naive counterpart in the d = 10 scenario. More specifically, the accuracy drops for the increasing delays are G2 2.5%, 3. 3.%%4.4% and Ra = 140%, 67%, 45%. We hypothesise this is due to the fact that under large C, repeated parameter updates with sub- optimal sampling strategies lead to over-fitting to the outdated state of the data distribution, as explained in detail in Section 7. On Yearbook [7], IWMS performs on-par with Naïve in every sce- nario. The accuracy gaps are G. = - 5%.- 20.5%, -44 whereas the recover scores are marginal: Ra 1%, 0%, 0%. We argue this is due two factors: the brevity of the dataset in comparison the other datasets and the difficulty of the task without prior knowledge on appearance and fashion trends.
Semi-Supervised Methods. S4L and PL performs very similarly to each other under all studied scenarios: the largest difference in their performance 0.7% on Yearbook, under d = 50 label delay. Therefore, we report their performance together, picking the better performing variant for numerical comparisons. Notice that in every scenario the delayed Naive bascline performance is not be achieved, which is due to the computational budget constraint. More specifically, since CSsL- = 2 x CNavve. optimising the standard classification objective over the older, supervised samples for twice the number of parameter updates is more beneficial across all scenarios than optimising the Pseudo- Laddo-g classification objective or the Contrastive loss over the newer unlabeled images. In the Supplementary Material 13, we provide further evidence and explanation of this claim. On CLOC, S4L slightly outperforms PL by +0.1% for all label scenarios, however R, = -27%, -2% 77% CGLM, S4I. outperforms PL by +0.6%, for all
for d 10, 50, 100, respectively. Similarly, label scenarios and achieves a negative recovery score R. = - 29%, - 27%, 23%. On FMo and Yearbook, the differences between the accuracy of Naïve, S4L and PL. are negligible as the largest improvement over Naive is +2.3% on Yearbook under the large label delay scenario d = 100.
TTA. In Figure 3, we find that TTA consistently under-performs every method, including the delayed Naïve, under every delay scenario on the CLOC, CGLM and FMoW datasets Nevertheless, on Yearbook TTA successfully outperforms IWMS, S4L, PL and Naive by up +1.7% in the moderate label delay scenario d = 50. Over the four dataset, the exact extent of the recovery of the accuracy gap R for d 10, 50, 100, respectively, is as follows:
CLOC R. 87%, -44%, -36%, on CGLM Ra= = - 77%,- 67%- 62%, on FMoW Ra = - 480%, 227%, - 159% and on Yearbook R. 4%, 11%, 11%. The disproportionateey severe negative result on FMoW is due the otherwise small accuracy gap G. = - 2.5%, -5 5.2%, - 7.4%. More importantly, we hypothesize that TTA fails to outperform Naive because the common assumptions, upon which TTA methods were designed, are broken.
7 Analysis of Importance Weighted Memory Sampling
We first perform an ablation study of our IWMS to show the effectiveness of the importance sampling. Then, we show our performances under different computational budgets and buffer sizes. We provide further in-depth analysis of the information retention abilities of the considered methods in the Supplementary Material A. 13 and the effect of memory size in A. 14.

---

Sampling Strategy (d=10)
Sampling Strategy (d=100)
Online Accurccy
20
Online Accurcy
20
18.4
17.3
15
10
10
WR I02
-
TIF
0
0
2k
41
6k
2k
4k
6k
Time step
Time step
Figure 4: Effect of sampling strategies We report the Online Accuracy under the least (d = 10) and the most challenging (d 100) label delay scenarios on CGLM [5].
Analysis on Memory Sampling Strategies. Note that while method, IWMS is prioritised sampling approach, it has some similarities to Naive, except for the sampling strategy. While the Naive method uses the most recent labeled data and randomly sampled mini-batch from the memory buffer for each parameter update, our method provides a third option for constructing the training mini-batch, which picks the labeled memory sample that is most similar to the unlabeled data. When comparing sampling strategies, we refer to the newest batch of data as (N), the random batch of data (R) and the importance weighted memory samples as (W).
In Figure 4, we first show that in both delay scenarios (d 10 and 100) replacing the newest batch (N) with (W) results in almost doubling the performance: +8.5% and +9.1% improvement over Naïve, respectively. Interestingly enough, when we replace the (N) with uniformly sampled random buffer data (R) we report significant increase in performance. We attribute this phenomenon to the detrimental effects of label delay: even though Naïve uses the most recent supervised samples for training, the increasing discrepancy caused by the delay d 10 and d 100 forces the model to over-fi on the outdated distribution.
Conclusion and Future Work
We motivate modeling real-world scenarios by introducing the label delay problem. We show how severely and unpredictably it hinders the performance of approaches which naively ignore the delay. To address the newfound challenges, Labeling, S4L and TTA) and propose our own technique (IWMS). We provide extensive empirical evidence over four large-scale datasets posing various levels of distribution shifts, under multiple label delay scenarios and, most importantly, under normalised computational budget. IWMS simply stores and and reuses the embeddings of every observed sample during memory rehearsal where the most relevant labeled samples to the new unlabeled data are rehearsed. Due to its simplicity, the robustness against changes in the data distribution can implemented very efficiently.
adopt the three most promising paradigms (Pseuodo-
Acknowledgement
Botos Csaba partially funded by Intel and partially by Meta AI Research. This work is supported by a UKRI grant Turing AI Fellowship (EP/W002981/) (01009981/)) and EPSRC/MURI grant: EP/N019474/1. Adel Bibi acknowledges the funding from the KAUST Office of Sponsored Research (OSR-CRG2021- 4648) and the support from Google Cloud through the Google Gemma 2 Academic Program GCP Credit Award. The authors thank Razvan Pascanu and João Henriques for their insightful feedback. We also thank the Royal Academy of Engineering.
10

---

References
[1] Jamal Abdul Nasir, Osama Subhani Khan, and Iraklis Varlamis. Fake news detection: A hybrid cnn-rnn based deep learning approach. International Journal of Information Management Data Insights, 1()):100007, 2021.
[2] Luyang Chen, Markus Pelger, and Jason Zhu. Deep learning in asset pricing. Management Science, 70(2):114-750, 2024.
[3] George Gkotsis, Anika Oellrich, Sumithra Velupillai, Maria Liakata, Tim JP Hubbard, Richard JB Dobson, and Rina Dutta. Characterisation of mental health conditions social media using informed deep learning. Scientific reports, 7(1):4514, 2017.
[4] Zhipeng Cai, Ozan Sener, and Vladlen Koltun. Online continual learning with natural distri- bution shifts: An empirical study with visual data. In International Conference on Computer Vision (ICCV), 2021.
[5] Ameya Prabhu, Zhipeng Cai, Puncet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online continual learning without the storage constraint. arXiv preprint arXiv:2305.09255 2023.
[6] Gordon Christie, Neil Fendley, James Wilson, and Ryan Mukherjee. Functional map of the world. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pages 6172-6180, 2018.
[7] Shiry Ginosar, Kate Rakelly, Sarah Sachs, Brian Yin, and Alexei A Efros. A century of portraits: visual historical record of american high school yearbooks. In Proceedings of the IEEE International Conference on Computer Vision Workshops, pages 1-7, 2015.
[8] Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K Dokania, Philip HS Torr, Ser-Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[9] Yasir Ghunaim, Adel Bibi, Kumail Alhamoud, Motasem Alfarra, Hasan Abed Al Kader Ham- moud, Ameya Prabhu, Philip HS Torr, and Bernard Ghanem. Real-time evaluation in online continual learning: A new hope. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[10] Randall Balestriero, Mark Ibrahim, Vlad Sobal, Ari Morcos, Shashank Shekhar, Tom Goldstein, Florian Bordes, Adrien Bardes, Gregoire Mialon, Yuandong Tian, et al. A cookbook of self-supervised learning. arXiv preprint XrXiv22304.12210 2023.
[11] Heitor Murilo Gomes, Maciej Grzenda, Rodrigo Mello, Jesse Read, Minh Huong Le Nguyen, and Albert Bifet. A survey on semi-supervised learning for delayed partially labelled data streams. ACM Computing Surveys, 2022.
[12] Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey test-time adaptation under distribution shifts. arXiv preprint 2tt.:2333.1536611 2023.
[13] Marcelo J Weinberger and Erik Ordentlich. On delayed prediction of individual sequences. IEEE Transactions on Information Theory, 2002.
[14] Chris Mesterharm. On-line learning with delayed label feedback. In International Conference on Algorithmic Learning Theory, 2005.
[15] Hanqing Hu and Mehmed Kantardzic. Sliding reservoir approach for delayed labeling in streaming data classification. In 2017 Proceedings of the 50th Hawaii International Conference
on System Sciences, 2017.
[16] Pooria Joulani, Andras Gyorgy, and Csaba Szepesvári. Online learning under delayed feedback. In International Conference on Machine Learning, pages 1453-1461. PMLR, 2013.
[17] Ludmila I Kuncheva and J Salvador Sánchez. Nearest neighbour classifiers for streaming data with delayed labelling. In 2008 Eighth IEEE International Conference on Data Mining. IEEF, 2008.
11

---

[18] Kent Quanrud and Daniel Khashabi. Online learning with adversarial delays. Advances in neural information processing systems, 2015.
[19] Joshua Plasse and Niall Adams. Handling delayed labels in temporally evolving data streams. In 2016 IEEE International Conference on Big Data (Big Data), 2016.
[20] Haoran Gao and Zhijun Ding. A novel machine learning method for delayed labels. In 2022 IEEE International Conference on Networking, Sensing and Control (ICNSC), 2022.
[21] Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations (ICLR), 2021.
[22] Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[23] Rahaf Aljundi, Klaas Kelchtermans, and Tinne Tuytelaars. Task-free continual learning. In Conference on Computer Vision and Pattern Recognition (CVPR), 2019.
[24] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances Neural Information Processing Systems (NeurIPS), 2019.
[25] Ameya Prabhu, Philip HS Torr, and Puneet K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European Conference on Computer Vision (EECV), 2020.
[26] Arslan Chaudhry, Mareus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Dokania, Philip HS Torr, and Mare' Aurelio Ranzato. Continual learning with tiny episodic memories. arXiv preprint, 2019.
[27] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935-299 2017.
[28] Arslan Chaudhry, Puneet Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European conference on computer vision (ECCV), pages 532-547, 2018.
[29] Rahaf Aljundi, Eugene Belilovsky. Tinne Tuytelaars, Laurent Charlin, Massimo Cac- cia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal inter- fered retrieval. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché- Buc, E. Fox, and R. Garnett, editors, Advances in Neural Information Processing Systems 32, pages 11849-11800 11860. Curran Associates, Inc., 2019. URL. http://paperss hwtp::///pprrre.i.......p nips. cc/paper/ 9357 onl ine continual wew.--ng-rnnniiitteeee www.wiitttaaaaaaa.a.a.aaaaaaaaaa www.iiieeeeeeeeereeeeeeeee Dtt.rrtg......aaaaaaaaiiieeirrrr
-df..
[30] Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances neural information processing systems, 32, 2019.
[31] Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, and Lucas Beyer. S41: Self-supervised semi- supervised learning. In Proceedings of the IEEE/CVF international conference computer vision, pages 1476-1485, 2019.
[32] Jian Liang, Dapeng Hu, and Jiashi Feng. Do we really need to access the source data? source hypothesis transfer for unsupervised domain adaptation. In International Conference on Machine Learning (ICML), 2020.
[33] Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. In International Conference on Learning Representations (ICLR), 2021.
[34] Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint 17Xiv:1502.03167, 2015.
12

---

[35] Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations (ICIR), 2023.
[36] Motasem Alfarra, Hani Itani, Alejandro Pardo, Shyma Alhuwaider, Merey Ramazanova, Juan C Pérez, Zhipeng Cai, Matthias Müller, and Bernard Ghanem. Revisiting test time adaptation under online evaluation. arXiv preprint arXiv:2304.047999 2023.
[37] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai. Continual test-time domain adaptation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
[38] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Conference on Computer Vision and Puttern Recognition (CVPR), 2020.
[39] X. Chen, S. Xie, and K. He. An empirical study of training self-supervised vision transformers. In International Conference on Computer Vision (ICCV), 2021.
[40] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Eemerging properties in self-supervised vision transformers. In International Conference on Computer Vision (ICCV), 2021.
[41] Huaxiu Yao, Caroline Choi, Bochuan Cao. Yoonho Lee, Pang Wei W Koh, and Chelsea Finn. Wild-time: benchmark of in-the-wild distribution shift over time. Advances in Neural Information Processing Systems, 35: 10399-1332,, 2022.
[42] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
[43] Hasan Abed Al Kader Hammoud, Ameya Prabhu, Ser-Nam Lim, Philip S. Torr, Adel Bibi, and Bernard Ghanem. Rapid adaptation in online continual learning: Are we evaluating it right? In International Conference on Computer Vision (ICCV), 2023.
[44] Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning (ICML). 2020.
[45] Debidatta Dwibedi, Yusuf Aytar, Jonathan Tompson, Pierre Sermanet, and Andrew Zisser- man. With a little help from my friends: Nearest-neighbor contrastive learning of visual representations. In International Conference on Computer Vision (ICCV), 2021.
[46] Jean-Bastin Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. Bootstrap your own latent a new approach to selff-suprvisee learning. In Advances in Neural Information Processing Systems (NeurIPS), 2020.
[47] Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
[48] Adrien Bardes, Jean Ponce, and Yann LeCun. VICReg: ization for self-supervised learning. In International Conference on Learning Representations (ICLR), 2022.
invariance-covarti regular-
[49] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self- supervised learning via redundancy reduction. In International Conference on Machine Learning (ICML), 2021.
[50] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignment. Advances in Neural Information Processing Systems (NeurIPS), 2020.
[51] Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for self-supervised representation learning. In International Conference on Machine Learning
(ICML), 2021.
13

---

[52] Shuaicheng Niu. Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Confer- ence on Machine Learning (ICML), 2022.
[53] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition, arXiv: 1556, 2014.
[54] Zhiyuan Chen and Bing Liu. Lifelong machine learning, volume 1. Springer, 2018.
[55] Enrico Fini, Stéphane Lathuiliere, Enver Sangineto, Moin Nabi, and Elisa Ricci. Online continual learning under extreme memory constraints. In Computer Vision-ECCV 2020: 16th Europeun Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part XXVIII 16, pages 720-735. Springer, 2020.
[56] David Lopez-Paz and Marc' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
[57] Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, and Jonghyun Choi. Online continual learning contaminated data stream with blurry task boundaries. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9275-9284, 2022.
[58] Genevieve E Flaspohler, Francesco Orabona, Judah Cohen, Soukayna Mouatadid, Miruna Oprescu, Paulo Orenstein, and Lester Mackey. Online learning with optimism and delay. In International Conference on Machine Learning. PMLR, 2021.
[59] Vinicius MA Souza, Diego F Silva, Gustavo EAPA Batista, and João Gama. Classification of evolving data streams with infinitely delayed labels. In 2015 IEEE 14th International Conference Machine Learning and Applications (ICMLA), 2015.
[60] Enrico Fini, Victor G Turrisi da Costa, Xavier and Julien Mairal. Self-supervised models are continual learners. In Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Elisa Ricci, Karteek Alahari,
[61] Xiaofan Yu, Yunhui Guo, Sicun Gao, and Tajana Rosing, Scale: Online self-supervised lifelong learning without prior knowledge. In Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
[62] Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28-51, 2022.
[63] Maorong Wang, Nicolas Michel, Ling Xiao, and Toshihiko Yamasaki. Improving plasticity in online continual learning via collaborative learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23460 2366022369, 2024.
[64] Yiduo Guo, Bing Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International conference on machine learning, pages 8109-8126. PMLR, 2022.
[65] Yujie Wei, Jiaxin Ye, Zhizhong Huang. Junping Zhang, and Hongming Shan. Online prototype learning for online continual learning. In Proceedings the IEEE/CVF International Conference on Computer Vision, pages 18764-18774, 2023.
[66] Yanan Gu, Xu Yang, Kun Wei, and Cheng Deng. Not just selection, but exploration: Online class-incremental continual learning via dual view consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7442-7451, 2022.
[67] Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in neural information processing systems,
2020.
[68] Hyunseo Koh, Minhyuk Seo, Jihwan Bang. Hwanjun Song, Deokki Hong, Seulki Park, Jung- Woo Ha, and Jonghyun Choi. Online boundary-free continual learning by scheduled data prior. In The Eleventh International Conference Learning Representations, 2023.
14

---

A Supplementary Material
A.1 Dataset Statistics
We conduct our experiments on four large-scale online continual learning datasets, Continual Lo- calization (CLOC) [4], Continual Google Landmarks (CGLM) [5], Functional Map of the World (FMoW) [6], and Yearbook [7]. The last two are adapted from the Wild-Time challenge [41]. More statistics of the benchmarks are in Supplementary.
The first, Continual Localization (CLOOC) [4] which contains 39M images from 712 geolocation ranging from 2007 to 2014. The second is Continual Google Landmarks (CGLM) [5] which contains 430K images over 10788 classes. Followed by that, we report our experiments on Functional Map of the World (FMoW) [6] adapted from the Wild-Time challenge [41]. The dataset contains 14,696 satellite images, from 2002 2017, with the task of predicting the land type. Last, we show our results on the Yearbook dataset [7] containing 33,431 frontal-facing photos from American high- school yearbooks. The photos were taken in the time- oiee-period between 1930-2013 and represent changes in fashion, gender and ethmicity over the years. The task is a binary classification problem: predicting the gender of the student based on the photo.
A.2 Implementation Details of S4L
For integrating S4L methods, we adopt the most effective approach through iterative optimization of both supervised and unsupervised losses. This process involves optimising the standard Cross Entropy loss on labeled data (similar to Naive) and minimising contrastive loss on unlabeled data, utilising balanced approach until exhausting the computational budget. We conducted an exhaustive search over the possible multi-objeccive optimisation variants (such as iterative and joint optimisation) and determined the best result is achieved when the contrastive loss is minimised separately for the first half of the parameter update steps, followed by minimising the supervised loss for the second half of the update steps. We report the best results across the three main families of contrastive losses, i.e.. Deep Metric Learning Family (MoCo [38], SimCLR [4d] NNCLR [45]), Self-Distillation BarlowTwin [49], SWAV [50], and W-MSE [51]).
(BYOL [46] and SimSIAM [47], and DINO [40]), and Canonical Correlation Analysis (VICReg [48],
For fair comparison, normalise the computational complexity [8, 9] of the compared methods. We find that while SSL methods may take multiple forward passes, potentially with varying input sizes, the backward pass is consistently done only once among the variants, therefore, the number of backward passes to measure the computational complexity of the resulting methods. According to this computational complexity constraint, Naïve augmented with SSL at each time step takes two backward passes, one for computing the gradients of the Cross Entropy over the labeled samples and for the Contrastive Loss over the unlabeled samples, thus Csa1. 2.
choose
A.3 Monotonous Online Accuracy Degradation
We argue the persistent drop in the Online Accuracy is due the stattttttonayy property of the data distribution that creates a distribution shift. Our hypothesis is supported by the experimental results, illustrated in Figure the Online Ace gradually decreases as the function of label delay d, at any given time step f. Furthermore, in Figure 6, we summarize the final Online Accuracy scores, i.e., the Online Accuracy value at the final time step of each run
Our claims are reinforced by the findings of Yao et al. [41]. A complementary argument is presented by Hammoud et al. [43], stating that the underlying datasets have high temporal correlations across the labels, i.e., images of the same categories arrive in bursts, allowing an online learning model to easily over-fit fit the label distribution even without using the input images.
A.4 Qualitative Analysis of Label Delay
case study of the distribution shift in the Yearbook experiments. While Online Accuracy is a well established performance metric for Online Continual Learning [4. 8, 5, 43], it can conceal some of the most important characteristics of the underlying dataset. To highlight a direct connection between the distribution shift and its immediate impact on the model performance, we illustrate the
15

---

C=1
8
CLOC0 Omlan Accurayy
7.5
20
6.01
10
16-
10
2.5-
5-
0.00
10%
2006
30%
10%
200%
30%
10%
2008
30
5k
10k
18k
CGLM OOllmo Ansitary
15-
6-
10-
5-
5-
1k
2x
11:
2k
2k
2k
Time
Time
Time
Time
d=0
d=3
d=10
d=000
d=50
d=
d-90
d=1
dm3
d=10
d=00
d=30
d=100
Figure 5: Monotonous degradation of Online Accuracy with regards to label delay d, over multiple datasets, CLOC [4] and CGLM [8], under various computational budgets, C 2, 4, 8. The accuracy gradually drops at every time step t as the function of the label delay d. However the extent of the degradation is non-linear: The initial smallest increases in label delay have severe impact performance. In contrast, the rate of degradation slows down even for an order of magnitude larger increments when the labels are already delayed. See Figure for the summary of the final values.
the
CLOC delay profile
CGLM delay profile
25
C=10
C=12
C=3
Final Online Accuracy
C=5
Final Online Accuracy
C=8
C=2
20-
C=1
C=6
C=1
C-2
15
Ca4
15
C-1
10-
25
50
75
100
25
50
75
100
Delay
Delay
Figure 6: Delay Profile. Each trajectory shows the Final Online Accuracy, i.e., the Online Accuracy evaluated at the last time step of each run, at a fixed computational budget C. On both datasets the most severe accuracy degradation occurs the first quarter (d -> 25). In contrast, on CGLM [8], the degradation is not significant in lower compute regimes C 4.
Top-1 Accuracy of the current batch at each time step in Figure 7. The experimental settings are identical to the main experiments on Naïve, detailed Section 5.2.
In this experiment, describe several observations: first, the models perform at per-chance level accuracy until the first batch of labeled data arrives. Notice that the per-chance level not 50% because the dataset is biased (contains improves over time, the random classifier's accuracy gets closer to 50%.
male than female portraits). However the ratio
Before the distribution shift. In the smallest delay scenario (yellow curve), the delay is identical to a lag of three years between making the predictions and receiving the labels. Under such delay, the model quickly reaches close-to-optimal accuracy under just few time steps and performs identically
16

---

Fraction of hair type males
Yearbook
Long Hair
Tep- Accuracy of Cumee Botch
100
Afro
Fraction of population
60
No delay yeurs
0.10
40
17 yoars
0.05
34 years
1838
1000
1930
1580
1800
2000
20.0
2000
0.00
1960
2020
Year
Years
Figure 7: (Left) Top-1 Accuracy of Naive the current batch (of time step of Yearbook. (Right) Report from Ginosar et al. [7] on "the fraction of male students with an afro or long hair." The drop in Top-1 Accuracy over time strongly correlates with the change in appearance of one the classes in the Yearbook [7] dataset. The larger the delay, the longer it takes to recover the close-to-perfee accuracy.
Example of Class F
Example of Class M
Year
1969
1972
1975
1978
1981
1984
Figure 8: Examples from the Yearbook dataset [7] during the time where the visual appearance of men (bottom row) changes drastically resulting in an accuracy drop of an online classifier, regardless of the label delay.
to the non-delayed counterpart (blue curve). In the moderate delay scenario (green curve), the model stays "idle" for longer time (equivalent of 17 years) because of the delay of the first labeled batch. Nevertheless, the delayed model reaches similarly good performance after a time steps. Interestingly, the severely delayed model (red curve), exhibits steep increase in performance, at 1972, exactly 34 years after observing the first sample (t = 1938).
During the distribution shift. The steep increase in the most severely delayed scenario (red curve) coincidentally overlaps with a major distribution shift in the appearance of one This shift simultaneously impacts the performance of all four models, however the rate at which their performance recovers differs, due to the label delay. While in general it is an immensely difficult problem to detect and trace the changes of the data distribution, due to hidden latent variables (such as socio economic factors, genetic diversity of the population, cultural and political trends), Ginosar et al. [7] identified and tracked many of such variables. One of these factors, namely the "fraction of male students with an afro or long hair", is highly correlated (in the temporal dimension) with the accuracy drop in our experiments, illustrated in the right-hane hand side of Figure 7.
two classes.
The reason behind the accuracy drop. the qualitative experiments
the section titled "What time specific patlerns is the classifier using for dating?". Ginosar et al. [7] reports that convolutional neural networks, such as VGG [53], learn to extract features from the hairstyles of the subjects. Although the task is slightly different, classification of the year of the photograph, we hypothesise
17

---

d
d 10
d 50
100
CLOC- Onlan Aceuacy
6
0.0
30
30
25:24
15:4
625.2
14.8
11:10
09.6
09.54
140
11.00
00
10
13.74
10
10:5
10
13.5
10.7
00:34
10.0
00:0
100k
20ll
300%
100%
2002
3001
100k
2008
500%
100%
200k
300K
OGLM Oaline Accuacy
22200
20
201
20
4 1
14.78
14.3
12.44
11.38
1:
400.7
10-
1
10
10.0
100
077
11 mm
1550
044
133,6
2k
2k
36
24
3k
1k
1k
2k
Time
Time step
Time
Time
C=6
Figure 9: Diminishing returns of increasing the computational budget C over four label delay regimes = 0, 10, 50, 100, on two datasets. While in many real-world scenarios simply increasing the budget C improve the overall performance, when the labels are delayed the improvements may become marginal. Interestingly, this phenomena is emphasized on the CLOC [4] dataset, as the trajectories collapse single curve the delay increases 0 100. In contrast, on CGLM [8] the relative improvements, i.e., the vertical distances between the lines, may shrink going from 10, but stay consistent for d 10 100. The final scores are summarized by Figure 10.
that one of the most discriminative features learned by the model are related to the hairstyles, as it is the most influential variable in terms of the accuracy of four independently trained models.
After the distribution shift. The recovery of the accuracy can be characterised by two factors: 1) the severity of the level degradation and 2) the duration of the recovery. Both factors show strong dependency on the underlying label delay factor: the larger the delay the larger the degradation and the longer the recovery length. Notice how elosely the slightly delayed, yellow curve (d = 3 years) follows the non-delayed, blue curve in terms of duration, while the extent of the accuracy drop is larger for the delayed counterpart. On the other hand, the moderately and severely delayed models (green and red curves, respectively) apparently reach a lower-bound in performance degradation, where larger delay does not further reduce the accuracy. Nevertheless, the recovery of the severely delayed model is slower and occurs later than the moderately delayed model.
A.5
The impact of label delay on the scaling property of the computational budget
The exploration of the impact of label delay computational efficiency and accuracy across different settings reveals important insights into the performance and scalability of Naïve, an Experinece Replay model [26], which simply waits for every sample to receive its corresponding label before using it as a training data. In this section, through extensive quantitative comparison under different label delay d and computational budget C. regimes, we offer comprehensive overview of how these key factors interact to influence model performance on two large-scale datasets: CLOC [4] and
CGLM [8].
Diminishing Returns. Figure highlights the phenomenon of diminishing returns on investment in the computational budget C across four different label delay regimes (d 0, 10, 50, 100). Notably, while augmenting typically yields performance improvements, these gains become increasingly marginal in the presence of delayed labels. The impact of label delay is markedly pronounced in CLOC dataset, where the performance trajectories converge into a singular trend as the delay escalates from d = 0 to d = 100. Conversely, the CGLM dataset exhibits contraction in the relative improvements (vertical distances between performance trajectories) as delay transitions from d to 10, yet these differences remain relatively stable for delays extending from d 10 to d 100.
18

---

CLOC C scaling profile
CGLM- C scaling profile
25
Final Online Accurwcy
Final Online Accuracy
20
20
15
15
10
10
10
2
6
10 12
Computational Budget C
Computational Budget
d
5
100
Figure 10: Compute Scaling Profile. Each trajectory shows the Final Online Accuracy, i.c., the Online Accuracy evaluated at the last time step of each run, at fixed computational budget C. We show sub-lineaa improvement w.r.. subsequent increases in C, even in the non-delayed (d = 0) scenario. Moreover, the influence of label delay datasets: while on CLOC [4] large delays (d = 100) prevent the model from benefiting from more parameter updates, CGLM [8] label delay (for d > 1) only seems offset the Final Online Accuracy, but does not impact rate of improvement.
the scaling property varies between the two
Compute Scaling Profile. In Figure 10, the concept of Compute Scaling Profile is introduced, displaying the Final Online Accuracy- - the accuracy measured at the last time step of each run for various levels of computational budget C. This figure elucidates the sub-linear scaling performance improvements with respect to incremental increases in C, trend observable even without label delays (d = 0). The effects of label delay diverge between the datasets; CLOC sees a significant impediment to performance gains from additional parameter updates at high delays (d = 100), while in CGLM, the delay primarily shifts the Final Online Accuracy without diminishing the rate of improvement.
Gradual Monotonous Degradation. Figure presents nuanced view of how Online Accuracy monotonically degrades with increasing label delay (d) across different computational budgets (C = 1, 1, 4, 8). This degradation is not linear: initial increments in label delay incur a steep decline performance, whereas the rate of decline moderates for larger increments of delay, showcasing nonlinear impact model accuracy over time.
Delay Profile. Finally, Figure 6 encapsulates the Delay Profile, depicting the Final Online Accuracy at various computational budgets (C). Both datasets exhibit the most substantial accuracy reductions in the initial quarter of delay increments (d 0 25). Interestingly, the CGLM dataset demonstrates a negligible degradation in lower computational regimes (C < 4), indicating a potential resilience or adaptive capability under specific conditions.
While increased computational budget generally improves the performance, the presence of label de- lays introduces complex dynamic that significantly hinder these benefits. The distinct behaviors observed across the CLOC and CGLM datasets further suggest that the dataset characteristics play a pivotal role in the decision making whether investment in additional compute is warranted not. We suggest that such decision should be made on a case by case basis, rather than extrapolating from publicly available benchmarks.
A.6 Breakdown of SSL methods
In Figure 11 we show the performance of the best performing SSL based methods after hyper- parameter tuning. We observe that the performance of the SSL methods highly dependent on the dataset and the delay setting. However, we apart from MoCo v3 [39], the methods perform similarly to Naïve on CLOC. On the other hand on CGLM they have insignificant differences in performance, but consistently underperform Naïve.
19

---

CLOC (d=10, C=2)
CLOC (d=50, (=2)
CLOO (d=100, C-2)
CGLM (d=10, C=8)
OGLM (d=50, (=
CGLM (d=100), C=
Ohinw Acnmayy
7.5
5.0
2.5
10.4 2001 201
1000. 2004, SRRK
1006, 21R1 2014
21
41
6k
82
28 4
18
$6
26 41
Ck
8k
Time
Time stop
Timc stap
1000 50cp
Time stpp
Time step
Naive
Figure 11: Comparison of the best performing SSL based methods after hyper-parameter tuning
CLOC (d-10, C-2)
CLOC (d=50, C=2)
CLOC (d=100, C-2)
CGLM (d=10, C=8)
OGLM (d=50, C-8)
CGLM (d-100, C=8)
16
1003
Onlan Auairayy
Jo
01
2.5
06
10.0 2002 2006
L1ME 37.6
1006 200K 2006
25 45 0E
25 42
12
45 Ck
8k
Time
Time
Time
Timo
Tim:
Time
Figure 12: Comparison of the best performing TTA based methods after hhper- tuning
CLOC 10, C=2)
CLOC 10, C=10)
CLOC (d C=20)
10.0
10.0
Online Accunaty
Onilne Acemey
7.5
7.5
5.0
Online
BVOL
Stuckerr
25
Berie Nase
2.5
McDa Va
Naii Cwll
MLCC V3
-
M4du VS
Baiko Twke
0.0
Vickia
0.0
Nain C-10
10x
2ik
3UK
10k
20k
30k
10k
3ik
Time step
Time step
Thme step
Figure 13: Detailed breakdown of various Self-Supervised Learning methods from each family. Results are shown across varying number of parameter updates C 2, 10, 20 under the d = 10 scenario.
A.7 Breakdown TTA methods
In Figure 12 show the performance of the best performing TTA based methods after hyper- parameter tuning. We observe that the performance of the TTA methods are consistently worse than Naïve both CLOC and CGLM, under all delay settings. We observe that in the most severe delay scenario (d = 100) the performance of EAT [52] and SAR [35] is comparable to Naïve on CLOC, while CoTTA [37] avoids the catastrophic performance drop.
A.8 Comparison of S4L to Naïve when using the same amount of supervised data
While in our main experiments S4L fails to outperform Naive (in Section 6). we show that is mostly due to the computational constraint of our experiments. In order to test our hypothesis, series of experiments on the S4L variants, illustrated Figure 13, In this experiment, instead of limiting the Computational Budget C, directly restrict the number of parameter updates to test optimising the joint objective of Naïve and the given Self-Supervised Learning method improves the performance of the model at all. Our results indicate positive improvement Naïve for MoCo-V V3 [39], SimSiam [47] and B BOL [46] consistently across multiple settings with increasing number of parameter updates.
First, on the left hand side of Figure 13, both the Naïve and the S4I. variants take only a single parameter update per time step (thus C for all, except Naïve, where C 1). On the first 10%
20

---

of the CLOC dataset [4], this results in modest, nevertheless clear improvement over Naïve, up to +0.7%. Followed by that, in the middle, every model takes five parameter updates per time step. Notice that Naïve has stricter computational budget, C = 5, to match the rest of the experiments. Consistently with our findings Section A.5, Naive only benefited marginally from the increase in compute, due to diminishing returns, 7.0% + 7.5%. On the contrary, the previously highlighted S4L variants show a larger improvement over the increase in number of updates, c.g., 7.7% + 9.6%. Consequently, this increases the gap between the Naive and the S4L methods. Finally, on the right hand side of the figure, we show when the models are updated ten times in each time step, the improvement plateaues for both the Naïve and the S4L variants.
Conclusion of this set of experiments two-fold: when granted equal amount of parameter updates, S4L methods outperform Naïve across different settings. However, computing the parameter gradients W.r.t. the joint objective of S4I. costs approximately twice the amount that of the Naîve: Csa1. = 2 x CNaive. Due to the well-known property of Self-SSupe Learning methods, sample inefficiency, our main experiments show that "spending" the compute on more frequent Naïve updates is more beneficial than optimising the joint S4L objective, even when the training data is heavily delayed.
A.9 Examples of the Importance Weighted Memory Sampling on CLOC
Correctly labeled memory recalls. In the subfigure's caption "Newest" refers to the newest unsupervised image observed by the model and "iwm" refers to the sample drawn from the memory by our proposed sampling method. The numbers refer the corresponding true label IDs.
Figure 14:
Incorrectly labeled memory recalls. In the subfigure's caption "Newest" refers to the newest unsupervised image observed by the model and "iwm" refers to the sample drawn from the memory by our proposed sampling method. The numbers refer to the corresponding true label IDs.
Figure 15:
On CLOO, we report similar scores to Naïve due to high noise the data. To provide evidence for our claims we visualize the supervised data sampled from the memory buffer by our Importance
21

---

Weighted Memory Sampling method. In Figure 14, we show that our method is capable of guessing the correct location of the unsupervised sample (the left hand side of the image pairs) and recalling a relevant sample from memory. In contrast, the incorrect memory recalls hurt the performance even though the content of the samples might match. We illustrate such cases in Figure 15, where obvious that in some cases the underlying image content has information related to the location where the picture was taken at. such scenarios, the only way a classifier can correctly predict the labels is by exploiting label correlations, e.g., classifying all close-up images of flowers to belong to the same geo-location, even though the flowers are not unique to the location itself. Or consider the pictures taken at social gatherings (second row, second column from the right), where delayed classifier without being exposed to that specific series of images has no reason to correctly predict the location ID. Our claims reinforced by the findings of [43].
A.10 Visual Explanation of our Experimental Framework
We provide visual guides for explaining our experimental framework. In Figure 16, we emphasize the main difference between our setting and the general setting of partially labeled data-streams: while prior art does not differentiate between old and new unsupervised data, our work focuses specifically on the scenario when all unsupervised data is newer then the supervised data. In Figure 17, show the two types of data that our models work with: outdated supervised data, and newer, unsupervised data. The task is to find way to utilize the newer unsupervised data augment the Naive approach, that simply just waits for the labels to become available to update its parameters. The most challenging component in our experiments is the computational budget factor that allows only certain amount of forward and wackward passes through the backbone.
Data collection
Annotation rate
1:l
collection rate
Anmotation rate
d=4
Figure 16: Our experimental setup (top): After fixed amount of time steps all labels become available. This allows us to focus on utilizing future unsupervised samples effectively. Partial labeling setup (bottom): in the generic setting, when the data collection rate is higher than the annotation rate, some samples might never receive labels.
22

---

Augmented
Naïve
Pact
Ties
Infr
Suit
Pact
128
133
134
Supervised data
Unsupervised data
Figure 17: Experimental setup: in our experiments we show how increased label delay affects the Naïve approach that simply just waits for the labels to arrive. To counter the performance degradation we evaluate three paradigms (Self-Supervised Learning, Test-Time Adaptation, Importance Weighted Memory Sampling) that can augment the Naïve method by utilizing the newer, unsupervised data.
23

---

A.11 Two-stage vs single-shot sample selection
In Section 4, we outlined our proposed wo-stage sample selection method, IWMS. In this experiment we show empirical evidence and analysis on why predicting the class-labels first then doing similarity matching leads better results than simply using similarity score over all the memory samples. In Figure 18, we illustrate the evolution of the similarity scores of the two matching policies. On the left, the matching is done purely based on the similarity scores, whereas on the right only those samples were compared against those memory samples whose labels match the predicted labels. In the middle plot, we show that by implementing the two-stage selection, we increase the effectivity of the similarity matching by large margin, +7.8%.
without
Strlacto Hatriea Chadiatte
100
930
500
1331
1530
10:0
Gulter
2:30
2100
5400
2000
2030
30
35
Stviltuity Seire
Starlaiity State
Figure 18: The evolution of Similarity Scores between the unsupervised and memory samples over time. On each histogram, we plot the distribution of the cosine similarity scores between the feature representations of the yet be labeled samples and the samples in the memory that already received their labels. On the top row we show the initial distributions and going from top down, the evolution of the two distribution is illustrated over the time steps.
24

---

A.12 Extended Literature Review on Online Learning
Online Learning vs Online Continual Learning: Online Learning and Online Continual Learning. while both involve learning from data arriving sequentially, differ fundamentally in scope. Online Leamnng typically deals with single-task streams, often assumed be from an i..dd. distribution, as outlined in section 2.3 of [54] and the introduction of [55]. In contrast, Online Continual Learning (OCL.) is more concerned with non-stationary streams that undergo frequent changes in distribution, where mitigating forgetting is one of several challenges [55, 56, 57].
Non-Li.d. distribution of unsupervised data: While our work focuses on evolving distributions, work such as Weinberger et al. [13] and Flaspohler et al. [58] only considers label delay while the distribution a time-invariant, consequently completely omitting the problem of distribution shift. Majority of the prior online learning work [41, 14, 11, 20, 19, 15, 59, 13, 17, 18, 58] ignores the difference between past and future unsupervised data. In our proposal, all unsupervised data is newer than the last supervised data. We illustrate the difference between the two different types of unsupervised data in Figure 16.
Considering catastrophic forgetting: Continual Learning, both online and offline, is concerned about performing well previously observed data, often referred to backward transfer of the learned representations [41, 14, 11, 20, 19, 15, 59, 13, 17, 18, 58]. This is different from Online learning where the problem of forgetting not considered. Even in more recent Online Continual Learning work, backward transfer have been given slightly lower priority [43, 9, 4] where the authors have reported them only in the appendix.
Furthermore, most of the prior art does not differentiate between the past and future unlabeled data. In our proposal, all unlabeled data is newer than the last labeled data due to delayed amnotation, as illustrated in Figure 16. RealtimeOCI. also considers delay, however, their delay arises from model complexity; in their fast-stream scenario, the stream releases input-label pairs quicker than models update, causing models to be trained on an older batch of samples. In essence, labels are still instantly available in RealtimeOCL, while our work examines delay attributed the non-instantaneous arrival labels. RapidOCL [43] highlighted the exploitation of label-correlation in online continual learning, with focus on measuring online accuracy through future samples. In contrast, our framework allows the models to leverage the more recent, unlabeled data for adaptation. While a growing line of work adapts S4L continual learning make use of unlabeled data in continual learning settings, such as CaSSLe [60] in task- aagnottc settings and SCALE [61] in task-free settings, most previous work did not perform comprehensive examination of PL and S4L under strict computational budget.
25

---

CLOC (C=2)
OGLM (C=8)
FMoW (C=16)
Yearbook (C=16)
100
30
50
57.5
Baloual Trasdes
03.46
56.0
10
90.50
6.7
52.5
63
6.0
50.0
83
112
230k
100k
20%
10k
800
0
20%
20k
6ilk
8k
10%
55..0
Badounl (4
57.0
07776
07.50
6.9
55.0-
90
0.35
40-
34
52.5
688
30-
31.59
6.2
50.0
2000
400%
20
200
40k
000
800
2.02
20k
610k
83
51c
106:
100r
Badounl Thasia 10))
600
600.2A
57.0
07.44a
177
50
6.9
50-
36.76
55.0
15.44
10
6.8
40-
0.7
32.5
53.0
90
6.4
200%
100k
206
805
20k Sumples
20k
Gilk
5k
10%:
Sumples in the Post
Sumple in the Past
the
Poel
# Sample in the Posl
*- Naive w/o delay
Naive
S1L
Pesudo-Label
IWMS
Figure 19: Backward transfer. Measuring forgetting the withheld validation set.
A.13 Analysis on forgetting over past samples
In Figure 19, report the backward transferability of the learned representation. This is done held-out, ordered validation set where the timestamp is used for ordering. On CLOC, all methods perform similarly due poor data quality as reported in the Supplementary Material A.9. On CGLM, our method not only surpasses the performance of others, but achieves ~ 2x the accuracy of the S4L, PL., Naïve and non-delayed Naïve baseline CGLMM. This means that the representation learned by our sampling technique is far more robust and generalises better not only to future but past examples well. On FMoW, the best result is achieved by the Semi-Supervised methods, nevertheless our method outperforms the non-delayed Naïve in all scenarios. Finally, on Yearbook we see that under low label delay 10) all results are clustered around 97%, however IWMS and Naive performs best under larger delays (d = 50, 10).
26

---

Memory Size (d=10)
Memory Size (d=100)
Online Accuracy
Online Accurary
15
45
10
50k
50k
21k
10k
102
2k
4k
6k
2x
4k
6k
Time step
Time step
Figure 20: Effect of memory sizes (right). We report the Online Accuracy under the least (top:
d 10) and the most challenging (bottom: 100) label delay scenarios CGLM [5].
A.14 Analysis on the Memory Size.
We study the influence of buffer size on our proposed IWMS. In particular, we show the performance of our algorithm under the buffer size from 10K 80K Figure 20. Even though IWMS relies on the images sampled from the buffer to represent the new coming distribution, its performances remain robust under different buffer sizes: the largest performance gap between memory sizes of 10K and 80K is a marginal 2.5%.
A.15 TTA Normalized Computational Budget Considerations
We report several TTA methods, such CoTTA [37] and SAR [35], which abuse the absence of formal computational constraints in traditional Test-Time Adaptation settings by computing the entropy of the predictions of the input data up 32x different augmentations. Methods, such as EATA [52] further complicate the complexity normalisation problem by using multiple smaller-sized crops of the input image. To simplify our comparisons, ignore the of model inference, thus CTTA- 1.
A.16 Training Time
Our computation budget C is based on the number of forward-backward passes, which is generally good proxy for time and it has been widely used CL literature [25, 8, 9, 43]. The actual wall-clock training time can be influenced by various factors, c.g., code optimization, hardware, data I/O speed, and implementation.
Here we add the training time for our method, ER, contrastive learning baasid method, pseudo-labeling based method, and TTA methods with the same number of forward hawaardbackwward passes in Table Most of the experiments are using single 110 GPU with 12 CPU. The training time is measured in hours. This table shows that the training time could entirely different due to various other factors.
Table 1: Training times (in hours) for various methods across different datasets. I The CPU allocation
was 6.
CLOC
CGLM
FMoW
Yearbook
Naive ReSSL
52
3.6
2.3
0.20
67
3.0
4.0
0.25
CoTTA
39
5.0
2.5
0.20
Pseudo Labeling
111
4.6
2.5
0.15
IWMs
3.6
3.5
0.20
A.17 Alternative Conventional Metrics
In the Continual Learning literature pletora of metries have been proposed, of which the most popular are Average Accuracy [62], Task based Forward and Backward Transfer [62, 25], Model
27

---

Plasticity [63] which entails Learning Accuracy and Learning Forgetting. While most of these metrics aim capture and aggregate semantically similar aspects of the observed learning dynamics, they have a slight variation in the definition.
In Mai et al. [62], under the "Evaluation Metrics", Average Accuracy is defined as follows:
AverageAccuracy(Al
(I)
j=1
which is closely related to Online Accuracy, but we appreciate the fact that in such a scenario, performance on past iterations is re-evaluated in every step. In our setting, evaluating and reporting this metric would be infeasible, as the number of steps on the CLOC, CGLM, and FMoW datasets is 3-to-4 orders of magnitude larger than the examples provided in the survey (the maximum number of steps in [62] is 20; for comparison, in our experiments the maximum number of "tasks" " is 296, 199).
With a slight variation, Guo et al. [64] details the evaluation metric in section 6.1 as follows:
We first learn from the data stream of all tasks for each dataset, and then test the final model using the test data of all tasks. We report the average accuracy of all tasks from 15 random runs for each dataset.
We argue that this metric is taking an excessively strong measure to remove noise from the metric. In our experiments, we experienced that rerunning the same training with different seed results in negligible (less than 0.) differences in the results. Running the experiments 15 times to evaluate the metric of [64] is infeasible for
This similarly holds for [65] as well, since they report the Average Accuracy and Average Forgetting across 15 runs.
Furthermore, although the main manuscript of [66] does not provide the detail about re- running the Average Accuracy, the corresponding code is set by default to 15 re-runs with different seeds. (follow this URL for reference: https: /ttps://github Coom/anannG/DVC/DDLL 6f
In [67], the number of random initializations is dropped 10, otherwise they report the Average Accuracy as well.
Koh et al. [68] introduces their own metric: Knowledge Loss/Gain Ratio, claiming that the metrics used by [64, 65, 66) are relying on the notion of task boundaries therefore they define new objective that is "appropriate for periodic data distribution". In our paper we cannot make such assumptions about periodicity.
The accuracy metric proposed by Wang et al. [63], Learning Accuracy (LA) using Model Plasticity formally defined for the j-th task as:
4 a
where af is the accuracy evaluated on the test set of task j after training the network from task to task; j. We would like to argue that this metric is similar to the Online Accuracy metric apart from the notion that here the test samples are drawn from a different distribution, whereas the Online Accuracy evaluated on the j-th batch of data before is used for training. If we assume that both the test and the training batch is drawn from the same distribution at time-step j, the two metrics are arguably the same. (Please note that the training is only done on the batch after the evaluation in the case of Online Accuracy.)
To ensure that all relevant metrics can be computed for future reference, we the Naïve and IWMS experiments on the two datasets where IWMS was performing the best and the worst to provide full comparison against the baseline.
We simplified the table representation by splitting the validation data into 100 equal sized ranges along the time axis, such that the ranges would correspond to the training data range:
28

---

Table Accuracy matrix for Naïve method on Yearbook dataset.
Accuracy
te0-+12
tE12-25
tc25-+37
1c37-450
tes0-462
1C62-475
te75-+87
tes7-+100
-1-
0,99
0.98
0.94
0.76
0,56
0.70
0.89
0.88
tr12-25
0.98
0.99
0.97
0.79
0.58
0.72
0.88
0.86
fr25-37
0.99
0.99
0.96
0.75
0.55
0.59
0.77
0.86
fr37-50
0.99
1.00
0.99
0.83
0.65
0.74
0.87
0.90
8750-66
0,94
0.96
0.96
0.88
0.88
0.94
0.96
0.94
0,97
0,99
0.99
0.96
0.93
0,93
0.97
0,97
L175-87
0.99
0.99
0.99
0.96
0.93
0.93
0.96
0.96
U187-100
0.99
1.00
1.00
0.95
0.93
0.96
0.98
0.97
Table 3: Accuracy matrix for IWMS method on Yearbook dataset.
Accuracy
teu-~12
tē12-22
te25-137
te37-+50
te5o-62
te82-75
ters-187
tes7-+100
12
0.98
0.99
0.92
0.74
0.53
0.71
0.86
0.86
I112-25
0.99
0.99
0.78
0.57
0.69
0.86
0.86
tr25-a7
0.99
0.99
0.96
0.73
0.54
0.61
0.78
0.86
t737-00
0,99
1.00
0.99
0.82
0.64
0.74
0.87
0,90
tr50-62
0.97
0.98
0.98
0.91
0.89
0.94
0.96
0.94
tr82-75
0.99
0.99
0.99
0.96
0.94
0.95
0.98
0.97
fr75-87
0.99
1.00
0.99
0.96
0.94
0.95
0.97
0.96
1187-100
0.99
1.00
1.00)
0.95
0.92
0.95
0.98
0.97
A.18 Online Learning without Memory Rehearsal
The main issue of online learning framework is the lack of information retention mechanisms in traditional methods, which crucial for addressing the complexities of real world continual learning tasks, such as training a feature extractor that both learns new concepts faster (forward transfer) without losing the capability to perform well already seen problems (backward transfer).
To highlight that without rehearsing on memory samples the methods suffer significant performance degradation, we implemented the OL algorithm that is mentioned in Section A.12 in the special case in which all the labels (or feedback) arrives in order with fixed constant delay. We ran new experiments (with identical experimental environment described in the main experimental section, Section 6) on the two largest datasets, CLOC and CGLM, with computational budget respectively, for d 10, 50 and C = 2, respectively. The results show extreme underperformance:
29

---

Table 4: Accuracy matrix for Naïve method on CGLM dataset.
Accuracy
tcp-116
te16-+33
1033->50
te50-+66
1656-183
tég3-+100
tru-116
0.10
0.27
0.09
0.07
0.06
0.05
tr'1-333
0.14
0.29
0.24
0.11
0.11
0.08
tr'33+50 50
0.21
0.38
0.37
0.26
0.17
0.13
tr'00666
0.23
0.39
0.39
0.35
0.25
0.15
Ir66 -83
0.25
0.40
0.41
0.38
0.34
0.22
1r'83 1/883000
0.15
0.26
0.26
0.24
0.24
0.19
Table 5: Accuracy matrix for IWMS method on CGLM dataset.
Accuracy
tep->16
t616-433
t033-150
te30-666
tC56-883
tég3-+100
Iro-116
0.15
0.39
0.13
0.11
0.09
0.08
tr16-+33
0.24
0.51
0.45
0.20
0.17
0.15
tr33
0.31
0.55
0.61
0.41
0.25
0.20
tr50-666
0.35
0.57
0.63
0.59
0.38
0.24
tr 5786-+83
0.38
0.59
0.64
0.62
0.60
0.32
tr'3 1'83-1100
0.40
0.60
0.65
0.64
0.64
0.53
Table 6: Online Accuracy of Online-Leeaning (no memory rehearsal) on CLOC
Table 7: Online Accuracy of Online-Learning (no
memory rehearsal) on CGLM
Time Steps
delay=10
delay=50
Time Steps
delay=10
delay=50
delay=100
5000
0.195
0.163
100
0.000
0.000
0.000
15000
2.142
1.354
800
0.463
0.389
0.263
25000
2.960
1.793
1500
0.476
0.319
0.379
40000
3.467
2.157
2200
0.531
0.242
0.257
50000
4.202
2.451
2900
0.465
0.196
0.218
60000
4.838
2,699
3600
0.459
0.172
0.179
75000
5.238
2.898
4300
0.390
0.188
0.187
85000
S.632
3.076
5100
0.419
0.178
0.158
95000
5.849
3.287
5900
0.456
0,253
0.169
105000
6.265
3.727
6600
0.504
0.313
0.175
30

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
The claims in the abstact and introduction are the least speculative, they are stating observations and reflecting on our experiences of the interactions with peers from academia and industry.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long it is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Our work is a wide-range study over many different paradigms and regardless our best efforts to provide extensive and exhaustive experimental comparison over different constraints (4 datasets, 3 computational budget scenarios, label delay settings), such work will be always limited by available resources. Our work aims to showcase the most relevant experiments with the most realistic experimental setups.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faireess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
31

---

3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
Every experimental setup is provided in full-detail.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
32

---

5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [No]
Justification: Our submission reports experiments conducted solely publicly available
datasets. We publish the code if the manuscript is accepted at NeurIPS 2024.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" an acceptable answer. Papers cannot be rejected simply for including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: / //niss. cccccc//p/ www.c/pppcccaaeeecc.eeeeeeeeetle wwwpiii/guus///eeee wttgggee//eeeeeuiieeeeeeeeeeeeee /wt////odoiiiiiiiooonoooooo ssioolllici for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental Setting/Details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
See above.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We do not report error bars and other information about statistical significance in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
33

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (c.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Our main contribution is actually considering the computational resources for fair evaluation Online Continual Learning paradigms that utilize unsupervised data. While we do not report exact inference and parameter update time in miliseconds (due to relying on excessively heterogeneous computational resources), we normalize all our experiments w.r.t. the computational complexity. Guidelines:
Justification:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https::/ http:://eeuipp..eee httpipp...cppublpe cc/public/Btttc wwwbliie/thisstuudelliiiee weliiceeineeeeeeee ines?
Answer: [Yes]
Justification: Yes, the research conducted paper conforms, in every respect, with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
Answer: [Yes]
34

---

Justification: Our work addresses the importance of considering delayed feedback and computational constraints online continual learning research, which has significant impli- cations for real-world machine learning applications. On the positive side, by incorporating these factors into the research, we enable the development more efficient and practical learning algorithms that can operate effectively in resource-constrained environments. This can lead to improved performance and reduced computational costs, making machine learn- ing more accessible and applicable to wider range of societal problems. However, also acknowledge the potential negative impacts, such as the risk misuse or unintended consequences if these algorithms are deployed without proper safeguards considerations for faimess and transparency. By highlighting these issues, we encourage the community to develop responsible and ethical approaches online continual learning that mitigate potential risks while harnessing the benefits for society.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance), faimess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: In our paper, no such risk is posed.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
35

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: See above.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcom thoode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re- re-packaged, both the original license and the license of the derived asset (if has changed) should be provided. If this information is available online, the authors are encouraged to reach out the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No such assets provided with the submission.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We did not conduct crowdsourcing experiment or research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, other labor should be paid at least the minimum wage in the country of the data collector.
Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
36

---

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: See above.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
37


Reviews:
[{"review_id": "2VoxUqiynI", "reviewer": "Reviewer_bdVn", "summary": "The paper addresses the problem of label delay in online continual learning. The proposed framework explicitly models this delay, revealing unlabeled data from the current time step and labels delayed by a specific number of steps. Extensive experiments demonstrate that increasing computational resources alone is insufficient to overcome performance declines caused by significant label delays. The authors introduce Importance Weighted Memory Sampling, a robust method that prioritizes memory samples similar to the newest unlabeled data, improve the performance under label delays scenario.", "questions": "See Weakness above.", "limitations": "See Weakness above.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "-   The paper is well-written and easy to follow.\n\n-   The proposed method is comprehensive and reasonable. In Section 4, the IWMS addresses the critical issue of outdated feature updates.\n\n-   The experiments conducted in the paper are thorough and convincing, covering a variety of datasets and including ablation studies, sensitivity analyses, and more.", "weaknesses": "1. The discussion of related work on the label delay problem is not comprehensive. The authors should consider additional relevant studies in the field of online learning with label delay, such as:\n\n    - Heliou et al. \"Gradient-free Online Learning in Continuous Games with Delayed Rewards.\" ICML 2020.\n    - Wan et al. \"Online Strongly Convex Optimization with Unknown Delays.\" Machine Learning 2022.\n\n    Additionally, recent works on \"asynchronous labels\" (which is a general version of label delay, in which both the feature and label can delay) should be better considered:\n\n    - Zheng et al. \"Asynchronous Stochastic Gradient Descent with Delay Compensation.\" ICML 2017.\n    - Qian et al. \"Learning with Asynchronous Labels.\" TKDD 2024.\n\n2. The paper considers only the simple case of a fixed delay time step $d$, which may not be practical for real-world scenarios. It would be more beneficial if the authors considered a more general case where the delay time step is variable.\n\n3. Although previous research on online learning with delayed feedback focuses on the online learning scenario rather than continual learning, it would be more comprehensive if the authors compared their method with these online learning approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewers' positive comments on our method, experiments, and paper presentation. We address the above concerns here.\n\n**Q1: The discussion of related work on the label delay problem is not comprehensive.**\n\nWe thank the reviewers for pointing out the missing references. We will include the suggested references in the final version of the paper, covering both the label delay and asynchronous labels scenarios. \n\n**Q2: The paper considers only the simple case of a fixed delay time step d.**\n\nWe acknowledge that this is an excellent suggestion and plan to add further ablations on varying delay time steps during the rebuttal period. However, we would like to point out to the reviewer the numerous experiments we have conducted where we vary $d$  across all experiments  (as in Figure 2,3,4), which already demonstrate the superiority of IWMS over other methods. We do not generally expect the performance ranking to differ significantly with variable delay times.\n\n**Q3: Comparison to the online learning approaches**\n\nWe have a thorough discussion on this point in the supplementary A12. Furthermore, We appreciate the reviewer’s suggestion to compare our work with existing research on online learning methods that handle delayed labels. We would like to point the reviewer to the supplementary material A12 (page 23, L756), which explains the primary reasons for the incompatibility of traditional online learning approaches with modern OCL settings. Briefly, the main issue is the general lack of information retention mechanisms in traditional methods, which are crucial for addressing the complexities of real world continual learning tasks, such as training a feature extractor that both learns new concepts faster (forward transfer) without losing the capability to perform well on already seen problems (backward transfer). To highlight that without rehearsing on memory samples the methods suffer significant performance degradation, we implemented the OL algorithm that is analogous to the mentioned papers in the special case in which all the labels (or feedback) arrives in order with a fixed constant $d$ delay.\nWe ran new experiments (with identical experimental environment described in the main experimental section, Section 6) on the two largest datasets, CLOC (39M) and CGLM (580K), with computational budget $\\mathcal{C}=2,8$ respectively, for $d=10,50$ and $d=10, 50, 100$ respectively. The results show extreme underperformance:\n\n[Online Accuracy of Online-Learning (no memory rehearsal) on CLOC]\n\n| Time Steps | delay=10 | delay=50 |\n| --- | --- | --- |\n| 5000 | 0.195 | 0.163 |\n| 15000 | 2.142 | 1.354 |\n| 25000 | 2.960 | 1.793 |\n| 40000 | 3.467 | 2.157 |\n| 50000 | 4.202 | 2.451 |\n| 60000 | 4.838 | 2.699 |\n| 75000 | 5.238 | 2.898 |\n| 85000 | 5.632 | 3.076 |\n| 95000 | 5.849 | 3.287 |\n| 105000 | 6.265 | 3.727 |\n\n[Online Accuracy of Online-Learning (no memory rehearsal) on CGLM]\n\n| Time Steps | delay=10 | delay=50 | delay=100 |\n| --- | --- | --- | --- |\n| 100 | 0.000 | 0.000 | 0.000 |\n| 800 | 0.463 | 0.389 | 0.263 |\n| 1500 | 0.476 | 0.319 | 0.379 |\n| 2200 | 0.531 | 0.242 | 0.257 |\n| 2900 | 0.465 | 0.196 | 0.218 |\n| 3600 | 0.459 | 0.172 | 0.179 |\n| 4300 | 0.390 | 0.188 | 0.187 |\n| 5100 | 0.419 | 0.178 | 0.158 |\n| 5900 | 0.456 | 0.253 | 0.169 |\n| 6600 | 0.504 | 0.313 | 0.175 |\n\nThe results clearly indicate the necessity of memory rehearsal:\nmodels on CLOC saturate at <6.5% for delay=10 and <4% for delay=50. In the case of CGLM dataset the performance collapses in all three delay scenarios <1%.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yWricOIaQq", "reviewer": "Reviewer_dj2M", "summary": "This manuscript delineates a novel method, termed as IWMS, which is devised to address the problem of label delay in online continual learning, where new data may not be labeled due to slow and costly annotation processes. The IWMS exhibits an innovative usage of fine-grained Gaussian Mixture prototypes,  along with mutual information optimizing that endow the proposed method with competitive performance for unsupervised class incremental discovery.", "questions": "See weaknesses above.", "limitations": "Addressed.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "S1. The paper's significance is underscored by its empirical evaluation. The evaluation is comprehensive, with comparisons to different baselines and ablation study providing a compelling demonstration of the strength of this work.\n\nS2. The proposed IWMS method is simple and easy to implement. The proposed memory sampling has the potential to inspire future research in this domain.", "weaknesses": "W1. Although this paper presents a simple method, further technical insights regarding the implementation and each module within the IWMS method would be beneficial. Further efforts regarding technical innovation and methodological novelty would also be beneficial.\n\nW2. The manuscript could delve deeper into the memory buffer (e.g., how to construct and update it, and the additional complexity by it), a factor which could be pivotal for fully understanding the proposed model.\n\nW3. It would be better to analyze and discuss the difference between online learning, continual learning, and online continual learning, and further clarify the topic of this work.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer's recognition for our extensive experiments and the simplicity of our proposed method and the value suggestions. The following are our responses to the reviewer's concerns:\n\n**Q1: Need further explanation regarding the implementation and each module within the IWMS method and technical innovation.**\n\nOur implementation of IWMs is closely related to the experimental setup in Section 5. It is built upon the best performing method, ER, and we further improve it by sampling the buffer samples that are more similar to the unlabeled data. All the implementation details and \"technical innovation\" are detailed in full in Section 1, 3 and 4, in Figure 1 and in Algorithm 1 and 2. Line 149 - provides \"technical insights regarding the implementation and each module within the IWMS method\". Line 154 and 163 - \"delves deeper into the memory buffer\". Section A.12, Line 756 - Analyses the difference between online learning continual learning and online continual learning.\n\nAs we point it out in the main paper in Section 4, L137, the novelty lies in how to learn the unlabeled distribution.  Our experiments in Section 6 demonstrate that existing literature, which focus on operating unlabeled features to learn the most recent distribution, are often neither effective nor efficient. Our approach avoids unnecessary computation on the most recent unlabeled distribution, even if it closely matches the evaluation distribution. Instead, we sample previously labeled data similar to the current unlabeled data. This allows us to construct a pseudo-distribution close to the most recent distribution and conduct purely supervised learning on the pseudo-distribution. \n\n**Q2: Need further explanation regarding the memory buffer.**\n\nThis is all detailed in Section 4, Line 154, but we will reiterate it here for completion.\nAt time step $t$, we keep a memory buffer to store both the raw data and the features of the labeled data from time step $t'$($t'$< $t$) computed in $t'$. We sample the data according to the similarity between the features from the current unlabeled data $x^t$ and the features in the buffer. We then take the buffer data whose features are most similar to x^t to construct a new labeled batch. This batch is our buffer batch. \n\n**Q3: Need to analyze and discuss the difference between online learning, continual learning, and online continual learning, and further clarify the topic of this work.**\n\nWe already had a Section 2 and A.12  in the paper discussing this in full depth. In fact, the title of the section is \"Online Learning vs Online Continual Learning\" and \"Considering catastrophic forgetting\".\n\nWe provide further explanation below:\nOnline Learning and Online Continual Learning both involve learning from data arriving sequentially, but Online Learning typically deals with single-task streams, often assumed to be from an i.i.d. distribution. In contrast, Online Continual Learning is more concerned with non-stationary streams that undergo frequent changes in distribution. Continual Learning, on the other hand, is a broader concept that encompasses any type of learning process where the data is revealed sequentially and the distribution of the data may change over time. Recent discussion in continual learning has focused on the scenario where the computation is fixed for fair comparison.\n\nOur work mainly focuses on the online continual learning scenario, where the distribution of the data changes over time and the data is revealed batch-wise. As motivated in line 26, we further consider the label delay problem, where the labels of the data are not available immediately, and most recent data distribution is unlabeled. To this end, we propose solutions from various literatures and propose our simple yet effective method, IWMs, to address the label delay problem in online continual learning.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RAAiAN0ZWQ", "reviewer": "Reviewer_HwKE", "summary": "This paper introduces the problem of label delay in Online Continual Learning (OCL), where a model is trained continually on an unlabelled stream where labels are revealed with a fixed delay. The problem becomes learning from semi-supervised data with the objective of quickly adapting to the unlabeled distribution of most recent data. After analyzing the impact of label delay, the authors introduce their method: Importance Weighted Memory Sampling. The main idea is to favor replaying past samples which share similar features representation and labels with current unlabeled samples. Eventually, the authors compare various flagship strategies from different connected domains (Self-Supervsied Semi-Supervised Learning, Test-Time Adaptation and pseudo-labeling) and show superior performances of their approach compared through experiments.", "questions": "See weaknesses.", "limitations": "I believe the authors have not addressed the limitations of their work. Please see weaknesses.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The presentation is very clear and the paper is well-written.\n- The motivations of introducing such a realistic problem are clearly defined and I believe the problem of label delay to be interesting to the community.\n- Thorough experiments have been conducted, with a specific care given to computation budget, which is often overlooked in OCL\n- The idea behind the presented approach is interesting, simple and effective\n- The figures are informative and clear", "weaknesses": "**Major Weaknesses**\n\n1. My main concern regarding this paper is the evaluation metric. I understand that Online Accuracy is the usual metric in Online Learning. However, in Continual Learning, the usual metric of interest is the Average Accuracy. The problem with Online Accuracy is that only the performances on the last task are considered, meaning that you could have terrible performances on previous tasks. In fact, given that the proposed approach favors samples which are the most similar to the current distribution, I would expect the model to improve on the current task at the cost of worse performances on previous tasks (since the corresponding samples are less likely to be replayed). I have seen the Figure 19 of the appendix regarding backward transfer, but I am unsure that I understand these values correctly. Backward transfer can often be negative, but I might be unfamiliar with those specific datasets. I would like the authors to either:\n- Clarify why is Online Accuracy the only metric considered. Is it not important to maintain performances on previous tasks too? Is the lastly observed distribution the only one that is important?\n- Show the average accuracy across all tasks in the main draft to ensure that the gain in performances in current task does not come at the cost of lower performances on previous tasks.\n\n2. I wonder why the authors did not compare to existing Continual Learning methods in supervised and semi-supervised cases. Such methods have been cited in appendix (CaSSLE and SCALE). Other methods such as [1,2,3] could also be considered.\n\n3. Why should you have a limited memory size? It seems to me that all data are stored anyway for labeling.\n\n4. The authors did not discuss the potential limitations of their approach.\n\n5. The code is not accessible yet.\n\n**Minor Weaknesses**\n\n6. To my understanding, a similar computational budget does not imply a similar training time. For example, one pass training with Cross-entropy is much faster than one pass with contrastive losses, as they require computing the Gram matrix. it could be enlightening to include training time of various compared methods.\n\n7. This setup is a domain incremental learning setup, I believe this should be stated in the paper.\n\n8. I don't think the memory buffer $M$ is defined in the text.\n\nIf the authors can address my main concerns I would happily raise my score.\n\n**Typos**\n\nl191: One -> on ?\n\nl193 Is the memory really $2^{19}$?\n\n[1] Michel, Nicolas, et al. \"Contrastive learning for online semi-supervised general continual learning.\" 2022 IEEE International Conference on Image Processing (ICIP). IEEE, 2022.\n\n[2] Wang, Liyuan, et al. \"Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021.\n\n[3] He, Jiangpeng, and Fengqing Zhu. \"Unsupervised continual learning via pseudo labels.\" International Workshop on Continual Semi-Supervised Learning. Cham: Springer International Publishing, 2021.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their recognition of our presentation, the problem we study and the solution we proposed, and appreciate the reviewer for the valuable suggestions. Here are our responses to the reviewer's concerns:\n## Q1: Evaluation metric\nUsing the Online Accuracy metric, in online continual learning, appears to be the common practice in most works, see [Online Continual Learning with Natural Distribution Shifts by Cai et al.] [Real-time evaluation in online continual learning: A new hope by Ghuneim et al] [Computationally budgeted continual learning: What does matter? by Prabhu et al] [Rapid Adaptation in Online Continual Learning by Hammoud et al.] . While evaluation on past tasks has been reported, they were often reported in supplementary as the main focus was the most recent task with OCL. The reason is among the motivation prior works provide, [Cai et al.], is that the top priority for a time-ordered stream is the ability to predict current task and future task (note that evaluation is done before training at each time step). Performance on past tasks is not relevant in case those tasks (which happens to be from the past) might never be presented again (think of images of old laptops or images of devices (for classification) that are no longer popular these days nor will they be in the future.). Thus the majority of works considering OCL, they investigate it on streams that are time ordered (unlike classical offline CL), where performance on next batch accuracy (online accuracy) is the top priority while still reporting the past task accuracies for completion is supplementary.\n\nTo reiterate on our results, the reviewer can find the performance on past tasks metric described and evaluated on all settings of the main experiment in Section A.13, Figure 19. The backward transfer in Figure 19 is computed by the average accuracy of the last model on previous data following the evaluation protocol in [Online Continual Learning with Natural Distribution Shifts by Cai et al.]\n\n## Q2: Comparison to existing Continual Learning methods\nAs mentioned in line xx, our work is built on top of previous works of CL with fixed budget [Real-time evaluation in online continual learning: A new hope by Ghuneim et al] [Computationally budgeted continual learning: What does matter? by Prabhu et al] and delayed evaluation [Rapid Adaptation in Online Continual Learning by Hammoud et al.]. As consistently shown in these works, other existing semi-supervised CL methods like CaSSLE and most existing supervised CL methods like regularization based methods are not as effective as ER [On Tiny Episodic Memories in Continual Learning by Chaudhry et al]. [1] is a contrastive learning based method and [3] is a pseudo-labeling based method. We have tried our best to tune these two types of methods and include more advanced variants of them in section 6, although the modified version might be a little bit different from the two suggested ones. \nIn conclusion, despite best efforts, since the computational complexity is constrained these methods failr terribly. This is already well documented in several other prior art that conducts these experiments under fixed computation [Rapid Adaptation in Online Continual Learning by Hammoud et al.] [GDumb: A Simple Approach that Questions Our\nProgress in Continual Learning by Prabhu et al.] where more sophisticated methods generally struggle.\n\nFurthermore, prior to this discussion, we tried using the publicly available implementation of [2] and failed to reproduce their results.\n\n## Q3: Limited memory size\nWe agree with the reviewer that we should not limit the buffer size since all data are stored anyway, and this is what we set for the experiments for small datasets, such as FMoW, Wild-Time and CGLM in our experiments section. The current buffer size set is more due to the experiment time and I/O time. We have tried our best to set the buffer size to be $2^{19}$ (line 193), which will be enough to store all the data in a small dataset like Yearbook, FMoW, and CGLM, but not enough for CLOC(39M, Supplementary A1). We further have buffer size analysis in supplementary A14, showing that our methods is robust to different buffer sizes and the performance is not relied on large buffer size.\n\n## Q4: Potential limitations and code accessibility\nWe agree with the reviewer that we should discuss the potential limitations of our approach. We will add this in the final version of the paper. We will also make our code accessible after the paper is accepted.\n\n## Q5: Training time\nWe agree with the reviewer that the training time can be influenced by various factors. Our current computation budget is based on the number of forward-backward passes, which is generally a good proxy for time and it has been widely used in CL literature[8,9].\nWhile the training time metric could be sensitive to multiple other factors, e.g. code optimization, hardware, data I/O speed, and implementation. \n\nHere we add the training time for our method, ER, constrastive learning-based method, pseudo-labeling based method, and TTA methods  with the same number of forward-backward pass in Table 1. \nMost of the experiments are using a single A100 GPU with 12 CPU. The training time is measured in hours. This table shows that the training time is could be entirely different due to various other factors. \n\n\n|  | CLOC | CGLM | FMoW | Yearbook |\n|---|:---:|:---:|:---:|:---:|\n| Naive | 52 | 3.6 | 2.3 | 0.2 |\n| ReSSL | 67.3 | 3 | 4 | 0.25 |\n| CoTTA | 39 | 5 | 2.5 | 0.2 |\n| Pseudo Labeling | 111.3$^1$ | 4.6 | 2.5 | 0.15 |\n| IWMs | 61 | 3.6 | 3.5 | 0.2 |\n$^1$ the CPU allocation was 6", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "z19YkSME4m", "reviewer": "Reviewer_UAN4", "summary": "This paper outlines a new continual learning framework to handle label delays in data streams, a situation where labels lag behind data collection. Extensive testing revealed that neither increased computational resources nor advanced techniques like Self-Supervised Learning significantly overcome the challenges posed by label delays. To address this, the authors introduce Importance Weighted Memory Sampling, a method that effectively uses memory samples resembling new unlabeled data to bridge accuracy gaps caused by delays, achieving performance on par with non-delayed scenarios.", "questions": "See the weakness.", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. I fully agree with the idea that the assumption of instantaneous annotation in Online Continual Learning (OCL) rarely holds in real-world applications. This makes the following discussions particularly interesting and relevant.\n2. The proposed method is simple and easy to understand.", "weaknesses": "1. While the discussion on the instantaneous nature of the annotation process is intriguing, I remain curious about the practicality of the proposed Online Continual Learning (OCL) model that incorporates delayed labels. In real-world scenarios, the challenge of matching input data at time step $t$ with labels at time step $t+d$ perfectly may prove more difficult than acquiring a data stream with instantaneous annotations.\n2. The proposed setting is not clearly formulized. If the annotator can provide both the image and the corresponding label which makes the memory constraints and proposed label delay quite meaningless, simply train on data stream given by the annotator will be a better choice.\n3. Unclear notations. The notation and evaluation criteria in the paper are somewhat unclear. Unless I am mistaken, it seems the model is assessed based on its performance on unlabeled training data. This approach may overlook inherent problems such as forgetting or performance degradation in Online Continual Learning (OCL), since ideally, the model should be evaluated across all previously encountered data distributions. A clearer definition of the OCL model mentioned in the paper is necessary. Additionally, there is notable existing research on online learning methods that handle delayed labels. A comparison with these methods would enrich the discussion and provide a more comprehensive evaluation of the proposed model's effectiveness in the context of OCL.\n4. More comparisons with existing OCL methods should be included.\n5. The paper does not provide a convincing comparison of its contribution with respect to existing work like[A,B].\n\n\n[A]: Mesterharm, Chris. \"On-line learning with delayed label feedback.\" International Conference on Algorithmic Learning Theory. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. \n\n[B]: Gomes, Heitor Murilo, et al. \"A survey on semi-supervised learning for delayed partially labelled data streams.\" ACM Computing Surveys 55.4 (2022): 1-42.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their efforts reviewing our paper and that the reviewer finds our method to be simple and that they agree on the setup of delayed labels.\n\n**Q1:  the challenge of matching input data at time step 𝑡 with labels at time step 𝑡+𝑑 perfectly may prove more difficult than acquiring a data stream with instantaneous annotations.**\n\nThroughout our work we argue that modeling the delay is essential in OCL settings, and we found that a straightforward experimental model for demonstrating how such delays impact the model performance is the ($x_t, y_{t-d}$) formulation. \n\nThroughout our work, we argue that modeling delay is crucial in OCL settings. We found that a straightforward experimental model, using the (x_t, y_{t-d}) formulation, effectively demonstrates how delays impact model performance. Decoupling the input and label streams could be an interesting generalization, where data and labels arrive asynchronously. We excluded this to maintain focus on our main message. Including this discussion in future work would enhance the manuscript. However, we highlight the numerous experiments varying delay times (as in Figures 2, 3, 4), demonstrating IWMS's superiority over other methods. We do not expect performance rankings to differ significantly with variable delays.\n\n**Q2: The proposed setting is not clearly formalized**\n\nWe will review our manuscript to ensure the notations and explanations are as clear as possible and will add clarifications where necessary to prevent misunderstandings. Feedback from other reviewers (bdVn, dj2M, and HwKE) indicated that our presentation was clear, but we understand that improvements can always be made. Please let us know any specific comments the reviewer has in mind.\n\nThe Annotator does **not** provide the samples, **only** the labels. The scenario, to which the reviewer refers to, in which we simply wait for the annotator to reveal the labels and ignore the delays, is described as the Naïve case as this is the simplest and in practice the most popular approach.\n\n**Q3: part 1: Unclear notations. The notation and evaluation criteria in the paper are somewhat unclear.**\nWe will review our manuscript to ensure the notations and explanations are as clear as possible and will add clarifications where necessary to prevent misunderstandings. Please let us know any specific comments you have in mind.\n\nWe appreciate the reviewer's feedback on the evaluation criteria and the suggestion to include more comparisons with existing Online Continual Learning (OCL) methods. However, we would like to clarify both the original OCL formulation ([Cai] in Figure 3c and our manuscript does indeed address the evaluation of forgetting and performance degradation. Specifically, these aspects are thoroughly considered and illustrated in Figure 4 of the main paper, which evaluates the model's performance across all previously encountered data distributions. We apologize if this was not immediately clear and will ensure that the manuscript highlights these evaluations more prominently.\n\nOCL papers often focus less on forgetting and more on online accuracy, i.e., next batch accuracy, because the data stream is chronologically ordered. In this setup, older data (e.g., from 2010) always precedes newer data (e.g., from 2024) and might never reappear. For example, images of Nokia phones are far less common now than those of iPhones. Thus, the goal is to predict future data accurately. Despite this, we still report forgetting and past data accuracy, detailed in the supplementary material.\n\n\n**Q3: part 2: A comparison with these online learning with label delay**\n\nWe would like to point the reviewer to the supplementary material, which explains the primary reasons for the incompatibility of traditional online learning approaches with modern OCL settings. Briefly, the main issue is the general lack of information retention mechanisms in traditional methods, which are crucial for addressing the complexities of continual learning tasks.\nTraditional online learning methods often assume immediate access to labels and do not account for the challenges of learning from data streams with delays. In contrast, modern OCL approaches, including our proposed framework, focus on retaining and utilizing past information effectively to mitigate forgetting and adapt to new data distributions.\nWe have highlighted these distinctions in our paper and supplementary material to clarify why direct comparisons with traditional methods may not fully capture the unique challenges and contributions of our work. However, we will ensure that these discussions are more clearly articulated in the manuscript.\n\n**Q4: More comparisons with existing OCL**\nCould the reviewer point out which exact method should be included in our comparisons? Please note we used the best performing (under computationally budgeted) OCL as shown recently in [Ghunaim et al.].\n\n\n**Q5: No convincing comparison to [A, B]**\n\nWe appreciate the reviewer's suggestion to improve our comparison with prior work. Figures 16 and 17 in the Supplementary Material visually explain the core difference with the framework considered by [B]. In our framework, all labels become available after a fixed duration, making it fully supervised. Gomez et al. do not consider unsupervised and semi-supervised methods for cases where newer data distribution states are accessed using unsupervised techniques, which is our core contribution. We'll emphasize this distinction in the main paper.\n\nRegarding [A], we address the incompatibility issues of such online learning work from 2005 in the related work section, L66. The methods proposed by [A] describe how to wait for labels that may come in a different order than the data points, involving sorting labeled instances or updating on the most recent labels. Our new experiments show that simply rehearsing on the newest supervised data, as [A] suggests, leads to performance collapse (i.e., per chance accuracy).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces the concept of label delay (well studied in online learning) into the online continual learning setting. The authors first show that a naive method (ignoring the unlabelled data) degrades rapidly as the delay increases (as expected). They then show that existing methods that make use of unsupervised data, such as test-time adaptation (TTA) don’t solve the problem. They then introduce importance weighted memory sampling (IWMS) as a simple method that is claimed to outperform other methods.

The reviewers cited various strengths and weaknesses leading to scores (post author responses) of 6, 5, 6, and 5:
Strengths:
- [bdVn, HwKE] well written / easy to follow
- [bdVn, dj2m, UAN4] IWMS seems like a reasonable solution
- [bdVn, dj2m, HwKE] comprehensive experiments
Weaknesses:
- [bdVn] missing related work
- [bdVn, UAN4] fixed delay assumption may be unreasonable
- [bdVn, HwKE, UAN4] missing comparisons - e.g. to label delay in online learning or other semi supervised or online continual learning methods
- [HwKE] choice of online accuracy as a metric
- [HwKE] discussion of limitations
- [HwKE] availability of code

There was active discussion, and many of these points were resolved satisfactorily. One sticking point was around the choice of metric, but on its own I don’t feel this is reason to reject the paper. Regarding the code request, the authors did put (time limited) code up. I had a look through myself, but found it hard to evaluate without executing as there was a lack of documentation. Some readme files were later added, but I didn’t get a chance to read these. At least the “code will be made available” box can be checked.
The authors also point out that the review from dj2m appears to be LLM generated, and in fact mentions methods that don’t appear in the paper. This review was also the least thorough of the 4, so I would give it less weight in the overall evaluation anyway.

Overall the work appears to be a useful contribution to the field, and is of a high enough standard for the conference. As such I am recommending that the paper be accepted.
Decision: Accept (poster)