Online Continual Learning via Logit Adjusted Softmax
Authors: Zhehao Huang
Tao Li
Chenhe Yuan
Yingwen Wu
Xiaolin Huang
Venue: Accepted by TMLR ()

Abstract: Online continual learning is a challenging problem where models must learn from a non-stationary data stream while avoiding catastrophic forgetting. Inter-class imbalance during training has been identified as a major cause of forgetting, leading to model prediction bias towards recently learned classes. In this paper, we theoretically analyze that inter-class imbalance is entirely attributed to imbalanced class-priors, and the function learned from intra-class intrinsic distributions is the optimal classifier that minimizes the class-balanced error. To that end, we present that a simple adjustment of model logits during training can effectively resist prior class bias and pursue the corresponding optimum. Our proposed method, Logit Adjusted Softmax, can mitigate the impact of inter-class imbalance not only in class-incremental but also in realistic scenarios that sum up class and domain incremental learning, with little additional computational cost. We evaluate our approach on various benchmarks and demonstrate significant performance improvements compared to prior arts. For example, our approach improves the best baseline by 4.6% on CIFAR10.

Published in Transactions on Machine Learning Research (05/2024)
Online Continual Learning via Logit Adjusted Softmax
Zhehao Huang
kinght l.Bsgjuu.ed..co
Institute of Image Processing and Pattern Recognition Shanghas Jiao Tong University
Tao Li
W.tao@sjju.edu.c edu.co
Instibute of Image Processing and Pattern Recognstion Shanghai Jiao Tong University
Chenhe Yuan
vermunftdisjtu.ddu..rr
Deportment of Automation Shonghai Jiao Tong University
Yingwen Wu
Institute of Image Processing and Pattern Recograition Shanghai Jiao Tong University
Xiaolin Huang
viaolinhuungg@@juu..uu.. edu.cr
Institute of Image Processing and Pattern Recognétion Shanghat Jino Tong University
Reviewed OpenReview: http:s//pppneeeeereeeee//////sts
Abstract
Online continual learning is a challenging problem where models must learn from a non- stationary data stream while avoiding catastrophic forgetting. Inter-class imbalance during training has been identified as major cause forgetting, leading model prediction bias towards recently learned classes. In this paper, we theoretically analyze that inter-class imbalance is entirely attributed to imbalanced class-priors, and the function learned from intra-class intrinsic distributions is the optimal classifier that minimizes the class-balanced error. To that end, we present that simple adjustment of model logits during training can effectively resist prior class bias and pursuc the corresponding optimum. Our proposed method, Logit Adjusted Softmax, can mitigate the impact of inter-class imbalance not only class-incremental but also in realistic scenarios that sum up class and domain incremental learning, with little additional computational cost. We evaluate our approach on various benchmarks and demonstrate significant performance improvements compared to prior arts. For example, our approach improves the best. baseline by 4.6% CIFAR10. Codes are available at https://github. com cot.e/ubbooo////ggg/.a.aa wowm//iigt//oonineeeeeeeeee online CL logit adjusted softmax.
1 Introduction
Continual learning (CL) has emerged to equip deep learning models with the ability to handle multiple tasks an unbounded data stream. This paper focuses on the online dass-incrementa.. (class-IL. (Lange et all,
2018
CL problem (Zhou el. all, 2028), which holds high relevance to real-world applications (Wang el all, In online CL, also known as task-free CL, data obtained from an unknown non-stationary stream for single-pass training. Class-IL learning, in contrast task-IL learning, continuously introduces new classes
(0ZZ)
the model as the data, stream distribution changes, without task-identifiers assist classification (xan
Ven all 2022)

---

Published in Transactions on Machine Learning Research (05/2024)
Catastrophic forgetting (McCloskey & Cohen, 1989 Ratclifi, [нАп) is major obstacle to deploying deep learning models in CL. Recent research attributes catastrophic forgetting to recency bins (Chrysakis & Moens, 2123), which causes deep neural networks to classify samples into currently learned classes. In fact, this bias in CL is similar to the dominance of head classes in long-tailed distribution learning (Menon ct all 2021).. The vanilla model trained on a long-tailed distribution suffers from inter-class imbalance and tends to infer samples into classes that possess a majority of samples. Previous works (Ahn ef all, 2020) have also observed that one of the primary causes of catastrophic forgetting is inter-class imbalance throughout training. Growing attention to recency bias and inter-class imbalance has given rise to methods (Kohh et all 2022) to alleviate the negative impact of imbalance, among which recently rehearsal-based methods have been highly successful but still with limitations. Replay buffers will become ineffective for long sequential data streams or tasks with numerous categories. Some methods (Guo-eef ell 2022, Prabhu elaal, 2020) train only on replayed samples to achieve balanced learning but sacrifice most valuable training data and risk overfitting on the buffer. Methods (Caccia et all, 2022) that separate gradient updates for old and new classes effectively prevent the impact of imbalance between them but fail to construct clear classification boundaries between old and new classes.
Upon decomposing sample probability in non-stationary data streams through conditional probability (sample probability = class-conditional x class-prior), revealing that recency bias caused by inter-class im- balance is entirely attributable to imbalanced class-priors. The underlying class-conditional invariant in online class-IL data streams motivates us to learn a function from intrinsic intra-class distributions instead of traditional sample distributions. We propose Logit Adjusted Softmax (LAS) to resist the impact of class- priors and grasp class-condiiionals by simply adjusting the model logits output via input label frequencies in training. Our method is grounded by the optimal classifier that minimizes the class-balanced error in online class-IL setup. Moreover, in the challenging online CL scenarios ((u all 2121) that sum up class- and domain-IL, we show that preserving knowledge in the class- conditiona function can better adapt the learner to changing domains. LAS provides the following three practical benefits in comparison to other previous online CL methods: (1) It can eliminate the prediction bias caused by the imbalance between old and new classes, as well as the inherent inter-class imbalance of the data stream. (2) Tt orthogonal to the mot improving replay strategies and plug-in to most of the rehearsal- based methods, (3) It improves performance with nearly additional computational overhead.
We evaluate LAS on extensive benchmarks over various datasets and multiple setups. Our LAS lifts the plainest Experience Replay (ER) (Chaudhry et al. 201) to state-of-the-ai performance without model expansion or computationally intensive technique, c.g., improving the accuracy of the best bascline by 4.6% on CIFARIO (Krizhevsky, 2009) in the online class-II. setup. Furthermore, we notice that inter-class imbalance dominates forgetting in long sequential data streams, which is rarely evaluated and always underestimated in previous work, so we evaluate on the challenging ImageNet (Deng et all PEER) and iNaturalist (Horm eT al 20m,) , where our proposed method consistently outperforms previous approaches. In addition to the class-TL setup, LAS also succeeds in the blurry setup and and scenarios that sum up class- and domain-II.
We discover the class-conditional invariant and the optimality error for the conditont function in online class-IL. (2) We propose eliminating caas--rioos and learning conditioods separately under the scenarios that sum up class- and domain-IL. (3) We introduce to adjust model logit outputs in training with a batch- wise sliding- winddwwwwwdoww estimator for time-varying class-priors to pursue the class-conditional function.
Key contributions of this paper include: (1)
that minimizes the
2 Problem Setup
2010) with clear task-boundaries, we consider more realistic environment where task-identifiers and baskdboundasies are absent at any time, and the total number of labels is unknown. Specifically, let X' be the instance set and y be the corresponding label set. In online CL, | = x. At time i E T = {1, 2, }. given unknown non-stationary data stream D, over X x Y, the learner samples data batch B, = {a,, v4 |B.l ~ y|r;). We refer to B, as the incoming batch. If a pair of instance and label is not stored in the memory, it will be inaccessible in subsequent training unless resampled.
Beyond the task-IL setting (L & Hotem
2

---

Published in Transactions on Machine Learning Research (05/2024)
Commonly, a constrained memory M (I Σ M) is utilized to enhance online CL: if the buffer is not emply at time i, Retrieval program ensembles several instances and other specific information I to form a buffer batch Br = Retrieval(Be.. M.) = faz, 1 P(x, M[M))- The buffer Updales with incoming batches, M1+1 + (r, Typically, ER (Chaudhry et al, 2 stores instances and labels 4, = ye, retriovals by random replaying, and updates via reservoir sampling (Vitter 1965). Rehearsal helps to alleviate inter-class imbalance when the number of classes is limited, but can not fundamentally eliminale its impact. The minimum class-prior in memory is bounded by the inverse proportion to the number of observed classes, mingey, M.) < 1/D3/ + 0 (t - oo). When the number of seen classes surges, rehearsal will no longer be able to support balanced inter-class learning.
The learner is a neural network parameterized by o = embeddings with dimension D. Following the feature extractor, a single-head linear classifier produces logits, ( = T fo(-): : X' + RDii (for short Py(+) = wjJ fo(.), where w E ERD P R represents the weights corresponding to target classes. The dimension of weights in the classifier can grow as more classes have been observed. The learner trains through surrogate loss averaged on all input instances, La 3/ R - R
[ w]. Function So : X - RRD extracts feature
(V, is the set of all observed labels), lypically the sofumax cross-entropy loss:
CCE (()) -log Eyex, ( log/1
E (ev((t)) (1)))) V'Av
(1)
3 Statistical View for Time-varying Distribution Learning
The standard CL methods learn from the sample probability P(z,yle) of the target distribution Pr (for example D, in practice). The model is encouraged to pursue a posterior probability function PPyy pt) and to minimize the misclassification error Ep, [...... a arg max (x)]] From Bayesian and conditional probability rulc, we notice P(ylæ,, pt) x P(z, plp)) = P(xly, (Pylpe),, revealing that the sample probability P(r, ylpt) of a time-varying distribution controlled by the class-conditional P(rlyy pt) and the class-prior P(ylp)). In unknown non- non-stationary data streams, inter-class imbalance entirely attributes to time- varying class-priors and is independent of dlass-condiionaals Therefore, such factorization of probability motivates us to learn blass-balanced classifier by exclusively pursuing a class-conditional function & P(zly, Pt), which is agnostic to arbitrarily imbalanced class-priors. The class-conditional function has been widely studied statistical learning on stable distributions (Long et all, 2ULZ). Similar motivation regarding the decomposition of the sample probability has also boen discussed in previous work (Nan De Ven et all, 202T).
We further extend the analysis of the cans-conniiional function to non-stationary stream distribution learning and discover the dlas-balaacee optimality of the class-conditional function when learning stream distribu- tions without domain drift, i.e., with fixed class-conditionals, as demonstrated in the following Theorem B
Theorem 3.1. For the time-vorying distribution Pt, given that its class- conditionals keep the same through- out time, i.e., VI, (P(xly, pt) = P( ((sly, Po), the consscconall function salisfies the optimal classifier &y that. minimizes the balaaccde error,
1 E arg CBE( V.), 4: -
arg max D7 (x) = arg max F(*ly, Pt). vEl
(2)
DEDii
(BE( = Dil
Ep, Enlnon (V # arg max + yy (x)]].
(3)
yey.
vE
In other words, the optimal
estimate is the class under which the sample is most likely to Error (Menon_et all, 20)) the current label set Vr, extended
appear. CBE( vi) is the from the misclassification error for class-balanced evaluation, formally in Equation R. Noting that the class- conditional function corresponds to the Bayes optimal classifier when class-priors are uniform. However, in scenarios where the class-priors are inherently imbalanced, the Bayes optimal classifier fails to minimize the claancbal error (Menon et all, 2013). Bias towards the most recently occurring classes does not aid in reducing the balaneed error, but approximation towards real underlying cass-condiiionals helps balanced classification because the classebalanced error averaged from the per-class error rate. Therefore,
3

---

Published in Transactions on Machine Learning Research (05/2024)
to address the impact of inter-closs imbalance and leverage knowledge from intra-class intrinsic distributions, we propose eliminating class-priors and constructing class-conditional function in online CL. The proof of Theorem I is in Appendix Al Following, we discuss two distinct CL scenarios on the critical condition of class-conditionals.
Discussion on online class-IL with time-invariant class-conditionals.. Prior works (Chrysakis & Moens, M have typically assumed no occurrence of domain drift during the learning process in online IL. Although domain drift should be taken into account in realistic scenarios, nearly time-invariant dlasscoonditonaaae are gemuinely feasible in practical situations. For instance, acting as lifelong species observer in the wild, the agent can find that the target class-conditonaals conform to their natural distri- butions, determined by their semantic information and occurrence frequencies. Without intentional human interference, the concept of natural semantics will remain almost unchanged over a prolonged time, i.e., Vt, P(aly. Pr) P( Po). In the experiments, we mainly adhere to the conventional class-TT. configuration of no consideration of domain drift and focus on addressing the issues of inter-class imbalance and forgetting induced by recency bias
Discussion on online CL scenarios with time varying class-conditionals. In the challenging online CL scenarios where P(ylpt) and P(zly, Pt) fluctuate as the data stream flows, both inter-class imbalance and intra-class domain drift are crucial considerations. While this set sinting has been studied in offline incremental setups (Kie et all 2022), there has been no research on this topic under online conditions, to the best of our knowledge. We now present our contribution to bridging this gap. In this online CL setup, we eliminate class-priors and focus on the caas-conndiionall function, which should not favor any specific domain but should blend all observed domains uniformly for optimal decision-making,
arg max D7 (2) arg max- vEDA
Σ P( Pc).
(4)
Since previous distributions are unavailable in CT., determining the optimal uniform domain distribution is intractable. Nevertheless, the disparity between the optimal classifier that minimizes the balanca error and the Icarned class-conditional function could be measured by the similarity between their underlying intra-class distributions. We combine with knowledge distillation Lechniques
Hoiemi, 20T1:; TTo el all,
2020: Kang et all, 2022: Dong et all, 2023) to narrow that disparity in probability space. Results in S6x show that preserving the knowledge in the class-balanced class-conditional function after eliminating class-priors can better adapt to domain drift than the standard posterior function. Therefore, our proposal is scalable to online settings that sum up class- and domain-IT.. Furthermore, it paves the way for developing further efficient solutions for this online scenario by minimizing the intra-class probabilities gap from the optimal class-conditional function, which we intend to explore in future research.
4 Method
4.1
Logit Adjustment Technique
Now our objective becomes excluding class- paisoss and establishing an estimator for current class-con classtoondisonnass, i.c., t, : X + RR.,, exp(#,,.) x P( Pt). However,
notoriously difficult to model the class-conditionals explicitly (Wan De Ven et all, 2021: Zajac et all 2023). To detour this problem, we draw on the Logit Ad- justment technique proposed by (Menon el. all 2I2T). Suppose the optimum scorer obtained by minimizing misclassification error the target distribution Pa at time t is sf X + R exp(s?,) Pyylz, pt). Recall- ing Pyylæ, Pt) & P(zly, Pt) P(llpt), we can derive the relationship between the class-conditional estimator o and the optimum scorer 8t as follows:
arg max e ( (w) arg max (ctt.(t). P(ulp))) arg max (or,(x) In >EDD
(5)
vEDA
sEDA
Equation E induces a straightforward method to approximate class-conditonnals and to achieve a class- balanced classifier: adjusting the model logits output according class-priors P(ylp:) and directly optimizing the softmax cross-entropy loss.
4

---

Published in Transactions on Machine Learning Research (05/2024)
10000
Mothed
sample
10103
non-stationary distibbtion
incoming batch
model
ER
((our)
logits
uptiale
Prodicttoo
memorr
buffer batch
estimator
2000
1000
log't adjustment
Task1 Task2 Task3
Toskd
Task5 Task6
Tosk7
Thok8
Task9 Tosk10
Figure 1: Left is the diagram of Experience Replay (ER) with our proposed Logit Adjusted Softmax and a batch-wise sliding-window estimator (ER-LAS). LAS helps miligate the inter-class imbalance problem by adding label frequencies to predicted logits. The model in ER-LAS still trained via the softmax cross- entropy loss. And right model prediction test samples by Fino-Tune, ER, and ER-LAS on C-CIFAR100 (10 tasks). The gray dashed line indicates the ground truth lask-wise distribution (Ik for each). We count. according to the tasks to which the predicted classes belong,
4.2 Logit Adjusted Softmax Cross-entropy Loss
We now show how to incorporate Logit Adjustment technique into the softmax cross-entropy loss for the aim of addressing the inter-class imbalance issues in online CL. The modified Logit Adjusted Softmax cross-entropy loss defined as follows:
2 (T)
(I,s(y, f(x)) = -log " (ortt)trloggn log +
Σ V y'
Tyyi Tuat
e0),(1))
(6)
EVEr,
where r the temperature scalar, and Tg.t t is the class prior P(y(S() time t. practice, S represents the data point collection from which the model samples input batch each time. Due to the uncertainty of St, it is impossible to pinpoint class priors at each moment. To overcome this barrier, the following $a.3 will provide a simple yet effective method for estimating class- class-priors in the flowing input stream. Applied to rehearsal-basee methods, LLAS will act both on incomiing and buffer batches to fully exploit input data. The right-hand side of Equation B illustrates its distinction to the cross-entropy loss in Equation D, enforcing a large relative margin between the major class and the minor class, i.e., (Tmajor,t/ "minor,))" 2019 Tan cf all, 2020; Menon ct all, 2021)..
(Cao et all
4.3 Estimator for Time-varying Class-priors
In stationary distribution, the Logit Adjustment technique (Collell ct all, 2016) can determine class-priors based on large amount of training data. But when facing an unknown time-varying input data stream, it is required to continuously estimate class-priors Ty. for LLAS at each time t. Therefore, we propose an intuitive batch-wise estimator with sliding window, where the occurrence frequency of a label input batches covered by the sliding time window approzimates the corresponding class-prior that label. Given the length > 0 of the time frame, yy, calculated follows:
Elarit E(s (o weBs 1 y)
Ny.
Ei [B$]
(7)
where (-) is the indicator function of label y and B is the input batch sampled from the data point collection S.. For rehearsal-based methods, the input batch often consists of the incoming and buffer batch, i.c., B = B, U B.M. The length 1 of the time window concerns sensitivity-stabiiity tradc-off (Nagengast et all 201T) with respect to the estimation of class priors, which we further study in the sensitivity analysis of $
Discussion. Logit Adjusted Softmax cross-entropy loss and the batch-wise estimator with sliding window together constitute our proposed LAS approach. Our method orthogonal to previous methods of various replay strategies and knowledge distillation techniques. Fxact joint label distribution of the non-stationary
5

---

Published in Transactions on Machine Learning Research (05/2024)
data stream and the memory retrieval program is unnecessary to approach, allowing to effortlessly incorporate LAS into existing methods and correct their model prediction bias caused by inter-class imbalance at nearly no cost of additional computational overhead. The experiment in Figure (right) verifies the effect of LAS on correcting the prediction bias, which follows the same setting as in SR Fine-Tune, which trains without any precautions against calastrophic lingetting and inter-class imbalance, calegorizes all test samples into the most recently studied task classes. ER (Chaudhry et all, 20R) includes a constrained memory to store previously observed data but still assigns about 38% (instead of the expected 10%) test samples to the most recently learned classes. By contrast, bias, achieves balanced class-posteriors similar to ground truth distribution, and significantly improves ER performance evaluated in the following $A The algorithm of LAS is in Appendix [B.
ER-LAS shown in Figure IXleft) eliminates the recency
Implementation in online GCL. We combine LAS with knowledge distillation in online GCL to preserve a class-balancee class-conditional function over averaged domain distributions. We directly calculate the distillation loss between the outputs of old and current models without logit adjustment. Noting that dist illation necessitates well-defined (ask-boundaries to preserve the previous model for distillation. This requirement presents formidable obstacle in online CL settings, where such boundaries are absent. To investigate the efficacy of our proposed method under the online GCL sctting, we allow to acquire task- boundaries in relative experiments. The algorithm of LAS with knowledge distillation for online GCL is A Algorithm 5 in Appendix B.
5 Related Work
We next provide some intuition on the effectiveness of our proposed approach by comparing LAS to prior work from the perspective of traditional and continual imbalanced distribution learning. We also highlight the computational efficiency in online conditions.
Methods for mitigating inter-class imbalance in stable distributions. Logit Adjustment (Menon all 202n) technique appears similar to Loss weighting et 20)) methods, yet the two differ sig- nificantly in addressing inter-class imbalance. While Loss weighting methods can balance the representation learning on minority class samples by weighting after the loss between logits and ground truth, it cannot rectify prior class bias and therefore cannot address recency bias, In contrast, Logit Adjustment technique directly balances the class-priors on logits, eradicating the impact of prior class imbalance on model classi- fication and resolving recency bias. In addition to Loss weighting methods, there are also other methods such as Weight normalization (Kang all, 2T2()), Resampling (Kubāä & Matwin, mнио), and Post-hoc correction (o all, 20TT). Different from these methods and the original Logit Adjustment technique, our adapted LAS possesses firm statistical grounding for non-stationary distributions. We compare with these inter-class imbalance mitigation methods in Appendix F4
Methods for mitigating inter-class imbalance in non-stationary distributions. The fundamental ER (Chaudhry et all 201 and recently proposed ER-ACE (Caccia at 2022) represent two extreme cases of our approach. ER corresponds to the case where O, and LLAS degenerates into the conventional cross-entropy loss function Lee in Equation M. losing the ability to alleviate inter-class imbalance. ER- ACE employs asymmetric losses for incoming and buffer batches, considering only the classes present in the current batch for incoming, ie., T + oo, and all previously seen classes for replaying, i.e., 7 = O, to mitigate representation shift. However, completely separating the gradients of current and past classes blocks the construction of inter-class decision boundaries. Our method lies between ER and ER-ACE, not only pursuing clas-condditonaal function but also encouraging large relative margins between old and new classes in online class-IL, i.e., always (Trew,t/TooM..)" > 1 in Equation B derived from their imbalance. We also notice highly related Logit Rectify methods (Zhon et al 2122) designed for offline task-IL, which we compare in Appendix E.
Computational efficiency. Online CL cannot ignore real-time requirements because memory and training time is usually limited in practical scenarios. Compared to traditional Softmax, Logit Adjusted Softmax slightly increases the computational cost of O(D)). Our suggested estimator raises the calculation time by O( + BBM + (il)) and the memory cost by O(Di)). In contrast to the time and storage overhead of the model and the memory, such an increase is negligible and lower than in previous works. Our experiments
6

---

Published in Transactions on Machine Learning Research (05/2024)
primarily compare methods with computational costs similar to approach. Noting that CL methods based on contrastive learning (Guo el all, 2022; 202)) may consume substantially more computational resources than our algorithm. We present a performance comparison with these methods in Appendix F
6 Experiment
In this section, we conduct comprehensive experiments to demonstrate the effectiveness of our proposed LAS. First, we investigate the performance of LAS in the online class-II. sccnario with class-disjoint tasks and in the online blurry CL scenario without clear task-boundan boundaries. Then, we evaluate LASs gains on rehearsal- based methods in the online class-TT. setup and gains on knowledge distillation approaches the online CL setup that sums up class- and domain-IL. Finally, we study the extreme variants of our method, the necessity of the suggested batch-wise estimator with sliding window, and the hyperparameter scnsitivity of our LAS.
Benchmark setups. We use 5 image classification datasets combined with 3 kinds of CL setups to form 8 benchmarks. Among datasets, CIFAR10 (Krizhevsky, 2008) has classes. CIFAR (Krizhevsky, 200)) has 100 classes, and they can also be categorized into 20 superclasses with 5 domains. TinyImageNeet (Le & Yang 20) has 200 classes. ImageNet ILSVRC 2012 (Deng et all 2009) has 1,000 classes, evaluating method performance
the long sequence data stream. iNaturalist 2017 (Horn all, 2077) has 5,089 classes. The distribution of images per category in iNaturalist follows the observation frequency of the species in the wild, so the data stream possesses inherent inter-class imbalance. As to CL setups, online class-IL (C) (Aljundi et all, 201 splits a dataset into multiple tasks with uniform disjoint classes, e.g., C-CIFAR10 (5 tasks) is split into 5 disjoint tasks with 2 classes each, except for C-iNaturalist (26 tasks) that is organized into 26 disjoint tasks according to the initial letter of cach class. Online blurry CL (B) (Koh all 2022) has both class-IL. IL distributions and blurry task boundaries. It divides the classes into Naturry % disjoint part and (100 - Niturry %) blurry part. The disjoint part classes only appear in fixed tasks, while the blurry part classes occur throughout the data stream but with inherent inter-class imbalance represented by blurry level Malurry. We split CIFAR100 and TinylmageNet into 10 blurry tasks according to (Koh et all, CL (S) (Xie_ et all 2022) covers class- and domain-IL setup, where incoming data contains images from new classes and new domains. We only apply this online CL setup on CIFAR100. The learner needs to predict superclass labels. Each superclass has 5 subclasses representing different domains within the same class. See Appendix Q for more details about benchmark sctups.
2022) with disjoint ratio Nuturry = 50 and blurry level Mulurey = 10. Online Sum-Class-Domee
Training Protocol. For all experiments, unless otherwise specified, following (Buzzega el al, 2020). We use the full ResNet18 as the feature extractor. For small-scale datasets, we start training from scratch. We pro-train models on 100 randomly selected classes from C-ImagoNet and then perform online learning on the remaining 900 classes (Gallasd all, 2021).. As for C-iNaturalist, we pre-train models on the entire ImageNet dataset. A single-head classifier is applied to classify all seen labels. We use SGD optimizer without momentum and weight decay. The learning rate is set to 0.03 and kept constant. Incoming and buffer batch sizes are both 32. On C-ImageNet and C-iNaturalist, horizontal-flip, and normalization. Some literat ure
set both batch sizes to 128. We apply standard
data augmentation, including candom-resized-com et all 2022) assumees that data arrive one-by-one in online CL, in which case we can accumulate samples as a batch to help model optimization convergence. We discuss the performance under varying batch sizes and per-sample updating in Appendix |. For online CL, only one epoch is used to run all methods for each task, and gradient descent is performed only once per incoming batch. By default, we set T == 1.0 and / for LAS. We report means and standard deviations of all results across 10 independent runs.
Evaluation Protocol. A commonly used metric is the final average accuracy Ar. Another common metric is the final average forgetting (Chaudhry et al] 2020) Fr. For blurry sctup, we follow (Koh all, 2022) to add the Area Under the Curve of Accuracy AsuC to evaluate the model performance throughout training. The detailed computation of each metric is given in Appendix DI
Basclines. We consider 7 rehcarsal-based methods for online CL to compare: ER. (Chaudhry et all, 20)
uses reservoir update and random replay. DER++ (Buzzega et all, 2021) replays samples with previous
logits for distillation loss. MRO (Chrysakis & Moens. 2028] only trains from memory. SS-IL (Ahn et all,
7

---

Published in Transactions on Machine Learning Research (05/2024)
Table 1: Final average accuracy Ar (higher better) on C-CTFARIO (5 tasks), C-CTFAR100 (10 tasks), and C-TinylmageNet (10 tasks). M is memory size.
Dataset
C-CIFARIO
C-CIFAR100
C-TinylmageNet
Method
M 0.5k
M 1k
M 2k
M 0.5k
M 1k
M 2
M 0.5k
M
M 2k
ER
1..9
4
50.3
12.920.3
16.52o.4
19.8
8.8
11.0=a.2
14.320.3
DER++
49.4
49.7.3.0
48.920.9
8.920.4
13.1
12.310.4
5.940.2
8.0
9.520.3
MRO
43.4
19.3
55.920.6
11.520.1
18.320.2
23.1
5.940.1
9.220.1
13.420.2
SS-IL
47.7+0.7
52.6+0.6
51.7+0.4
19.2+n.2
2L.5+0.2
24.2+0..2
13.1+0.2
14.9-a.1
17.1+0.9
CLIB
48,4+0.9
54.8+1.0
55.9+1.0
15.9+0.2
20.7+0.2
25.3+0.3
8,3+o.1
12.1+0.2
15.9+0.2
ER-ACE
44.411.0
48.1
51.221.2
18.620.4
22.510.
25.010.a
11.410.
14.820.2
16.420.4
ER-OBC
45.1
16.420.6
46.020.4
15.620.2
17.910.2
22.110.3
9.110.1
13.2g0.1
16.420.1
ER-CBA
45.0+1.6
54.2+1.1
56.3+0.9
20.1+0.6
23.0+0.a
26.0+0.6
12.3+0.7
13.6=0.s
17.0=0.6
ER-LAS
51.7
55.3+1.a
60.5
20.1+0.2
25.7+0.
27.0+0.a
13.7=0.a
15.5+0.2
18.7+0.2
Table Final average accuracy AT (higher is better) and final average forgetting Fy (lower is better) on C-ImageNet (90 tasks) and C-iNaluralist (26 tasks). We show the results of top-3 methods. Memory sizes
Table 3: AUC of Accuracy AAUC and final average accuracy Ar (both higher is better) B-CIFARI00 (10 tasks) and B- T-TinlmmaeeNet (10 tasks). We show the results of top-3 methods. Memory sizes are M
are M = 20k.
2k.
Dataset
C-lmageNet
C-iNaturalist
Dataset
B-CIFAR100
B- B-TinyImageNet
Method
Ar t
Fr
Ar f Fr
Method
AT 1 / AAUC
Ar AAUC
ER
31.8 / 38.610.2
4.7 / 18.0
ER
19.621.8/b e/161
16.220.2/12.422
ER-ACE
33.420.2 11.320.1
5.7
1.1
ER-ACE
18.3+1.0 115.2=0..
16.4-o. 12..2oo
MRO
35.8
10.2
5.0
0.4
CLIB
21.9t0.a/18.0
15.9 o...
ER-LAS
3
9.0
8.1
2.8=0.a
ER-LAS
24.9 24.9tus/20.3.1m 20.3
19.4_o./15.1.0
2020) separates the loss for present and absent classes. CLIB (Koh et all 2022) updates by sample-wise
importance and only trains on replayed samples. ER-ACE (Caccia all
212 employs the asymmetric
loss to reduce reprosentation shift. ER-OBC (Chrysakis & Moens 2023) additionally updates the classifier by balanced buffer batches. ER-CBA (Wang el all 2023) introduces continual bias adapter inserted after the classifier and conducts dual optimization on input and buffer batches. In addition, we enhance 3 methods of replay strategy: MIR (Aljundi et al] 201 retrieves the memory samples most interfered with by the model updating. ASER,, (Shim et all 2020) calculates Shapley values of samples to update and retrieve. OCS (Yoon et all, 2022) selects coreset with high affinity to replay. Also, knowledge distillation Josses in 3 approaches are augmented by LAS: LwF & Hoiem 20TF) distills on logits of previous classes. LUCIR (Hon et all, 2013 distills on normalized features. GeoDL (Simon et at 202T) also distills in the feature space but measures by the geodesic path.
6.1 Results on Online Class-IL Scenarios
Accuracy results. Table D and Table & show the final average accuracy for C-CIFARIO, C-CIFAR100, C- TinylmageNet, C-ImageNet,, and C-iNaturalist with various memory sizes. ER-LAS consistently outperforms all compared baselines, achieving 60.5% (+4.6%), 27.0% (+1.7%), and 18.7% (+1.6%) on C-CIFAR10, C- CIFAR100, C-TinylmageNet respectively compared to the best baselines. Compared to only considering replayed samples in MRO and CLIB or scparating the gradients between old and new classes in SS-IL and ER-ACE, LAS optimizes for a class-balanced function for incoming and buller batches and enforces large relative margins between imbalanced classes, resulting in better performance. Considering that the challenging C-ImageNet and C-iNaturalist. benchmarks possess substantially longer sequences of data stream than the above three benchmarks, where the recency bias problem caused by inter-class imbalance becomes severely critical, we also apply LAS to boost the performance of E.R. We present the results of the top-3
8

---

Published in Transactions on Machine Learning Research (05/2024)
Table 4: Final average accuracy Ar (higher is bet- ter) by replay stratogy methods w/o and w/ LAS on C-CIFAR100 (10 lasks). Gains are shown in paren- theses. M is memory size.
Table 5: Final average accuracy Ar (higher is bet- ter) by knowledge distillation approaches w/o and w/ LAS S-CIFAR100 (20 tasks). Gains are shown in parentheses. M is memory size.
Dataset
C-CIFAR100
Dataset
S-CIFAR100
Method
M 0.1k
M 0.5k
Method
M 0.11
M 0.5k
ER
6.5
12.940.3
ER
20.4
27.3
ER-LAS
10.7 (4.2 f)
20. 10.2 (7.2
ER-LAS
24.1 20.2 (3.71)
31.5 (4.2 t)
MIR
6.6
12.0
LwF
23.9
30.1 2o.3
MIR-LAS
11.8
(5.2 f)
20.1 (9.1 t)
LwF-LAS
26.020.2 (2.1 | f)
22.4 (2.3 f)
ASER
7.810.2
13.8
LUCIR
20.1
29. 29.4.o.3
ASER,,LAS
9.510.4
(1.7
18.0 (1.2 f)
LUCIR-LAS
25.010.2 (1.9 f)
32.6 (3.2 t)
OCS
9.4+o.1
16.2 +0.2
GeoDL
20.6+0.2
30.1 -0.2
OCS-LAS
12.7 (3.3 f)
21.010.3 (1.8 f)
GeoDL-LAS
25.2+0.2 (4.6
32.8=a.2 (2.7
baselines (MRO, ER-ACE, ER) C-ImageNet and C-iNaturalist. ER-LAS obtain 39.3% (+3.5% C-ImageNet and 8.1% (+2.4%) on C-iNaturalist compared to the best baselines. To ensure fair comparison on C-iNaturalist, we present both the final average accuracy directly evaluated on the test data like other methods do, and the results of the final average classbbalanced accuracy aligning with our Theorem f in Appendix F.7. Our extensive evaluations demonstrate the superior performance of our LAS by effectively alleviating inter-class imbalance in the online class-IL sctup with ncarly no additional computation cost (Table M). ER-LAS is only slightly slower than ER, contributing its real-world online applications.
Forgetting rate. We compare the final average forgetting of ER-LAS with top-3 performed baselines (MRO, ER-ACE, ER) on C-ImageNet and C-iNaturalist. As shown in Table 2, ER-LAS achieves the least forgetting rale on C-ImageNet and only forgets more than MRO and ER-ACE on C-iNaturalist. However, the lowest forgetting rate (e.g., 0.4% of MRO) does not necessarily guarantee the highest accuracy (8.1% of ER-LAS) because of the stability-plasticity dilernma (Kim & Han, 212X). the following sensitivity analysis of bel we show that although lower forgetting rate can be obtained by deliberately tuning hyperparameters in our L.AS, better stability--paastcity trade-off worth noting that in long sequence benchmarks, compared to ER without considering inter-class imbalance, methods trying to address recency bias not only remarkably reduce forgetting rates but also bring about improvements in accuracy, underscoring the importance of inter-class imbalance as a top priority in lifelong class-IL. We provide prediction results on C-ImageNet to further support our officacy of climinating recency bias in Appendix I.S. We also evaluate the final average forgetting on C-CIFARIO, C-CIFAR100, and C-TinylmageNet in Appendix ET
be achieved by the optimal hyperparameters. Tt is
6.2 Results on Online Blurry CL Scenarios
We compare ER-LAS with the best 3 baselines B-CIFARI00 and B-TinylmageNet Table B shows that ER-LAS can outperform all basclincs on Ar and AAUC- For example, ER-LAS improves the best bascline by 3.0% Ar and 2.5% AAUC on B-Tinylmageeet. In fact, our method is particularly suitable for the online blurry CL setup because LAS alleviates the detrimental effects of inter-class imbalance both inherently in the data stream and between new and old classes. The results of ER-LAS further confirm that such an advantage can help obtain high accuracy throughout learning under the realistic online blurry CL setup with challenging inter-class imbalance problems.
9

---

Published in Transactions on Machine Learning Research (05/2024)
Ag
Table 6: Ablation study about two extreme situa-
30 20 20 1: 10 5 3 I
3 1
tions of 7 and about randomly assigned (Random)
Window length
Window length
or macro statistical (Macro) caass-priors on C-
5
CIFAR100 (10 tasks). M 2k.
15
8025 20 15 10
Method
==0
T=
00
Random
Macro
LAS
Art
19.4-o4
22.7+oo
20.6*02
22.1+a.a
27.0=o.a
Fr 4
29.1-o.4
2.7-0.4
23.5+0.2
14.2+a.a
10.7+0.4
0.3
0.5
0.3
10.0
0.1
0.3
003
08
00
in
11
13
30
100
Temperature scalar
Temperature scalar
Table 7: Training time compared with top-3 fast methods C-CIFAR100 (10 tasks) by one Nvidia
Figure 2: Final average accuracy (darker is better, left) and final average forgetting (lighter is better, right) of various hyperparameter combinations on CIFAR100 (10 tasks). M 2k.
Geforce GTX 2080 Ti. M = 2k:
Method
ER
ER-ACE
MRO
ER-LAS
Training Time (8) 77.4 (x0.4)
84.7 (x1.22 80.0 (x1.20)
82.6 (x1.00)
6.3
Gains on Enhanced Methods
Rehearsal-based methods on online class-IL scenarios. We verify the performance boost of LAS by plugging it into ER, MIR, ASERg, and OCS. These three baselines train via softmax cross-entropy loss with different replay strategies, which harmonize with our approach. Table EI shows that LAS can significantly improve ER and its variants (+1.7%++9.19)) in the online class-IL setup. Although these methods with various memory management strategies benefit from our LAS, the gains depend on the estimation of class- priors from retrieval, as a relatively smaller boost is observed on ASER, which has sophisticated strategy to manage memory.
Knowledge distillation methods on online ClmmClass-Domain CL scemarios. To further investigate LASs offectiveness in alleviating inter-class imbalance, we combined with knowledge distillation approaches in the difficult and realistic online CL setup that sums up class- and domain-IL. Table E summarizes the results. LCE represents the basic CE loss used in ER. Knowledge distillation losses obtain higher accuracy by adapting intra-class domain drift. Augmented by our LAS, consistent gains (+2.1% eliminating inaslimmblaneed prior bias. The results demonstrate the validity of our proposal to separately handle class-condition ionals and class-priors in non-stationary stream learning. Tt. also showcases the perfor- mance improvement of eliminating imbalanced prass-priors by our method in this CL setup. Noting that we allowed the knowledge distillation methods to preserve old models boundaries, which intractable in real-world online CL. In future studies, we will explore the efficient and task-free method for handling intra-class domain drift to further refine the solution to online Sum-Class. Clan-CCas-DDoain CL.
are observed by
6.4 Ablation Studies
Extreme variants of LAS. We investigate the performance of two variants of our method by pushing 7 towards two extremes. When T = 0, LAS degenerates into the traditional softmax cross-entropy loss in ER. In t = 00, we set (Tw,e/Ty,)** = 0 in Equation H when /y Ty, < 1, otherwise we keep this coefficient and set 7 = 1 to ensure runnable. It achieves a similar effect as separating the gradient of new and old categories in ER-ACE, reducing representation shift. As shown in Table G, the performance of = is similar to ER as expected. = oo benefits from a remarkably low forgetting ratc. However, our proposed LAS with == achieves the highest accuracy, indicating that enforcing relative margin between classes based on the imbalanced class- class-priors priors obtain better stabtlity-plastiiity trade-off.
Necessity of batch-wisc estimator with sliding window. We empirically validate the necessity of our designed estimator. We randomly assign each prior of seen classes by a uniform distribution U(0, 1] and normalize them to 1, as Random. We also explicitly calculate the joint label distribution of the current data stream and the memory replay, as Macro, which is intractable in practice. Results Table & demonstrato that Randon degrades to performance similar ER, and Macro also inferior our proposed estimator. We conjecture that the online CT. model concerns more about the distribution within current. short-term
10

---

Published in Transactions on Machine Learning Research (05/2024)
imput batches than the macro distribution of sequential data stream and memory. Therefore our batch-wise estimator can better exploit the Logit Adjustment technique to improve performance.
Hyperparameter sensitivity analysis. We conduct the sensitivity analysis of the hyperparameters T and in our method in Figure D. ER-LAS is robust to a wide range of In practice, if the distribution fluctuations in the stream can be discerned, we recommend setting short I for streams that change rapidly and vice versa. As to temperature scalar 7, it has distinct impacts on accuracy and forgetting rate. Although a larger T can enable models to forget remarkably less, the best accuracy result is achieved around 1.0. Therefore the cuability-llasticiyy trade-off for target applications can be achieved by tuning and 1 together.
7
Conclusion
We discover the class-condiiional invariant and prove the optimality of the class-condite conditional function that minimizes the class-balanced crror in online class-IL. As a corollary of our thcoretical analysis, we introduce Logit Adjusted Softmax with a batch-wise sliding-window estimator to purse the class-conditional function. Extended to online GCL, knowledge of the learned class-conditional function should be preserved for adap- tation to domain drift. Under conditions without model expansion or computationally intensive techniques, extensive experiments demonstrate that LAS can achieve statc-of-the-art performance on various benchmarks with minimal additional computational overhead, confirming the effectiveness and elficiency of our met to mitigate inter-class imbalance. It is effortless to implement LAS and plug it into rehearsal-basee methods to correct their recency bias and boost their accuracy. Rehearsal-free approaches with LAS for online CL could be a subject of further study. Furthermore, we will continue to investigate efficient approaches to handling online domain drift, contributing to practical online GCL. applications in the real world.
8 Broader Impacts and Limitations
Broader Impacts. The implication of our research on the community of CL learning is two-fold. Firstly, our proposed LAS method is simple yet highly effective in climinating the recency bias caused by inter- class inbalance. LAS can be easily implemented and enhance the performance of other online CL methods. Moreover, LAS readily adapts to real-world uses, such as robot environment adaptation, object detection in autonomous driving, and rcal-time recommendation systems, improving augmented applications accuracy. Secondly, we propose partitioning non-stationary data stream learning into the class-conditional and class- prior functions and empirically demonstrate effectiveness. Our approach lays the framework for future research on online general CL and can provoke more task-free methods to address domain drift. Overall, our work is unlikely to have a negative impact on socicty.
Limitations. As the focus of our proposed method in this paper is primarily on addressing the impact of inter-class imbalance and forgetting induced by recency bias on CL performance. We eliminate the imbalanced class-priors to improve performance. However, can not provide any benefits when faced with domain-II. scenarios with only intra-class domain drift and no inter-class imbalance. Our next rescarch focuses on developing task-free domain-IL methods to address intra-class domain drift efficiently. We hope to integrate method with task-free domain-IL methods to form comprehensive solution for online general CL. Another limitation of our approach is that it is not effectively applicable to rehearsal-free online CL. Such a drawback does not stem from the constraints of our theoretical framework but rather from the difficulty in balancing inter-class margins when using the logits adjustment technique to pursue the class- conditional function. For detailed discussions, please refer to the experimental results in Appendix In the future, we will stick rescarching rehcarsal-free online CL.
Acknowledgments
The authors would like to thank the anonymous reviewers for their insightful comments.
The research leading to these results has received funding from National Key Research Development Project (BBYFFF114420)) National Natural Science Foundation of China (62376155). Shanghai Municipal Science and Technology Research Program (22511105600 and Major Project (2021SHZD7X0102),
11

---

Published in Transactions on Machine Learning Research (05/2024)
References
Hongjoon Ahn, Jihwan Kwak, Su Fang Lim, Hyeonsu Bang, Hyojun Kim, and Taesup Moon. Ss-il: Separated softmax for incremental learning. 2021 IEEE/CVE International Conference on Computer Vision (ICCV), pp. 824 833, 2020.
Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, and Tinne Tuytelaars. Online continual learning with maximally interfered retrieval. ArXiv, abs/1198.04472, 2019.
Eden Belouadah and Adrian Daniel Popeseu. I12m: Class incremental learning with dual memory. 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 583- 592, 2019.
Pietro Buzzega, Matteo Boschini, Angelo Porrello, Davide Abati, and Simone Calderara. Dark experience for general continual learning: a strong, simple baseline. Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2022 (NewrIPS), 2020.
Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. The Tenth International Conference on Learning Representations (ICLR), 2022.
Kaidi Cao, Colin Wei, Adrien Gaidon, Nikos Arechiga, and Tengyu Ma. Learning imbalanced datasets with labelldlistrbbution-awwae margin loss. Advances in Newral Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019 (NIPS), 32, 2019.
Francisco Manuel Castro, Manuel J. Marin-Jiménez, Nicolás Guil Mata, Cordelia Schmid, and Alahari Karteck. End-to-end incremental learning. ArXiv, abs/1807.095338 2018.
Arslan Chaudhry, Mareus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, Puneet Kumar Doka- nia, Philip H. S. Torr, and Marc' Aurelio Ranzato. Continual learning with tiny episodic memories. ArXiv, 111119922.14866 2019.
Arslan Chaudhry, Nacemullah Khan, Puneet Kumar Dokania, and Philip H. S. Torr. Continual learning in low-rank orthogonal subspaces. Advances in Newral Information Processing Systems 33: Annual Confer- encE on Neural Information Processing Systems 2020 (NewrIPS), 2020.
Aristotelis Chrysakis and Maric-Francine Moens. Online bias correction for task-free continual learning.
In The Eleventh International Conference on Learning Representations (ICLR), 2023. URL https:// http:/aw.eee//foorr//d/d/aaaae
Guillem Collell, Drazen Prelec, and Kaustubh R. Patil. Reviving threshold-moving: a simple plug-in bagging ensemble for binary and multiclass imbalanced data. ArXiu, abs/ 111666666666 2016.
Yin Cui, Menglin Jia, Tsung- Yi Lin, Yang Song, and Serge J. Belongie. Class- balanced loss based on effective number of samples. 2019 IEFE Conference on Computer Vision and Pattern Recognition (CVPR), pp- 9260 9269, 2019.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, K. Li, and Li Fei-Fei. Imagenet: A large-scale hicrarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 248- 255, 2009.
Jiahua Dong, Wenqi Liang, Yang Cong, and Gan Sun. Heterogeneous forgetting compensation for class- incremental learning. In Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 11742 11751, 2023.
Jhair Gallardo, Tyler L. Hayes, and Christopher Kanan. Self-supervised training enhances online contin- ual learning. In British Machine Vision Conference, 2021. URL http::://pp..emmaaticcccooo....
Corpus 1D:23235258
Yiduo Guo, B. Liu, and Dongyan Zhao. Online continual learning through mutual information maximization. In International Conference on Machine Learning (ICML.), 2022.
12

---

Published in Transactions on Machine Learning Research (05/2024)
Yiduo Guo, Bing Liu, and Dongyan Zhao. Dealing with cross-task class discrimination in online continual learning. 2023 IEEE/CVI Conference on Computer Vision and Pattern Recogration (CVPR), 2023.
Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alexander Shepard, Hartwig Adam, Pietro Perona, and Serge J. Belongie. The inaturalist species classification and detection dataset. 2018
IEEE Conference Computer Vision and Pattern Recognition (CVPR), pp. 8769 8778, 2017.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incre- mentally via rebalancing. 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 831-839, 2019.
Bingyi Kang, Saining Xie, Marcus Rohrbach, Zhicheng Yan, Albert Gordo, Jiashi Feng, and Yannis Kalan- tidis. Decoupling representation and classifier for long-tailed recognition. 8th International Conference on Learning Representations (ICLR), 2020.
Minsoo Kang, Jaeyoo Park, and Bohyung Han. Class class-incremental learning by knowledge distillation with adaptive fcature consolidation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16071 16080, 2022.
Dongwan Kim and Bohyung Han. On the cability-plasticitn dilemma of class-incremental learning. ArXiv, (bss/234400166 2023.
James Kirkpatrick, Razvan Pascanu, Neil C. Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinske Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114:3521 3526, 2016.
Hyunseo Koh, Dahyun Kim, Jung-Woc Woo Ha, and Jonghyun Choi. Online continual learning on class incre- mental blurry lask configuration with anytime inference. The Tenth International Conference on Learning Representations (ICLR), 2022.
Alex Krishevsky. Learning multiple layers of features from tiny images. In ArXiV, 2009.
Miroslav Kubát and Stan Matwin. Addressing the curse of imbalanced training sets: One-sided selection. In Proceedings of the Fourteenth International Conference Mochine Learning (ICML), 1997.
Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Ales dis, Gregory G.Slabaugh, andTTinnel wytellurss ferfyinggorggettingnneee hiw.ddassiiiiaaiintssss.... IEEE Transactions on Pottern Analysis and Mochine Intelligence, 44 3366- 3385, 2019.
Acontinullearringaureey
Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge. In ArXiV, 2015.
Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40:2935 2947, 2016.
Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael 1. Jordan. Conditional adversarial domain adaptation. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018 (NIPS), 2017.
Zheda Mai, Ruiwen Li, Hyunwoo J. Kim, and Scott Sanner. Supervised contrastive replay: Revisiting the nearest class mean classifier in online class-incremental continual learning, 2021 IEEE Conference Computer Vision and Pattern Recognition Workshops (CVPR Workshops), pp. 3584-3594, 2021.
Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of Learning and Motivation, 24:109-165, 1989.
Aditya Krishna Menon, Harikrishna Narasimhan, Shivani Agarwal, and Sanjay Chawla. On the statisti- cal consistency of algorithms for binary classification under class imbalance. In Proceedings of the 30th International Conference on Machine Learning (ICML.,, 2013.
13

---

Published in Transactions on Machine Learning Research (05/2024)
Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, and Sanjiv Kumar. Long-tail learning via logit adjustment. 9th International Conference on Learning Represeniations (ICLR), 2021.
Arne J. Nagengast, Daniel A. Braun, and Daniel M. Wolpert. Risk-sensitivity and the mean-variance trade- off: decision making in sensorimotor control. Proceedings of the Royal Society B: Biological Sciences, 278: 2325 2332, 2011.
Ameya Prabhu, Philip H. S. Torr, and Puneet Kumar Dokania. Gdumb: A simple approach that questions our progress in continual learning. In 16th European Conference Computer Vision (ECCV), 2020.
Roger Rateliff. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97 2:285 308, 1990.
Dongsub Shim, Zheda Mai, Jihwan Jeong, Scott Samner, Hyunwoo J. Kim, and Jongseong Jang. Online inssmmnnreeettall continual learning with adversarial shapley value. In Thirty- Fifth AAAI Conference on Artificial Intelligence (AAAT), 2020.
Christian Simon, Piotr Koniusz, and Mehrtash Harandi. On learning the geodesic path for incremental lcarning. 2021 IEEE/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1591- 1600, 2021.
Jingru Tan, Changbao Wang, Buyu Li, Quanquan Li, Wanli Ouyang, Changqing Yin, and Junjic Yan. Equalization loss for long-tailed object recognition. In 2020 TEEE Conference on Computer Vision and Pottern Recognition (CVPR), pp. 11662 11671, 2020.
Xiaoyu Tao, Xinyuan Chang, Xiaopeng Hong, Xing Wei, and Yihong Gong. Topology-preserving class- incremental learning. In 16th European Conference on Computer Vision (ECCV), 2020.
Gido M Van De Ven, Zhe Li, and Andreas S Tolias. Class-incremental learning with generative classifiers. In 2021 IEEE Conference Computer Vision and Pattern Recognition Workshops (CVPR Workshops), pp. 3611-3620, 2021.
Gido M van de Ven, Tinne Tuytelaars, and Andreas S Tolias. Three types of incremental learning. Nature Machine Intelligence, 4(12):1185-1197, 2022.
Jeffrey Scott Vitter. Random sampling with a reservoir. ACM Trans. Math. Softw., 11:37 57, 1985.
Quanziang Wang, Renzhen Wang, Yichen Wu, Xixi Jia, and Deyu Meng. Cba: Improving online continual learning via continual bias adaptor. In 2023 IEEE/CVF International Conference on Compuler Vision (ICCV), pp.
2023.
Zifeng Wang, Zheng Zhan, Yilan Gong, Geng Yuan, Wei Niu, Tong Jian, Bin Ren, Stratii Toannidis, Yanzhi Wang, and Jennifer G. Dy. Sparcl: Sparse continual learning on the edge. ArXiv,
2022.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Raymond Fu.
Large scale incremental learning. 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 374-382, 2019.
Jiangwei Xie, Shipeng Yan, and Xuming He. General incremental learning with domain-aware categorical representations. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2022.
Mengya Xu, Mobarakol Islam, Chwee Ming Lim, and Hongliang Ren. Class-incremental domain adaptation with smoothing and calibration for surgical report generation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, 2021.
Jachong Yoon, Divyam Madaan, Eunho Yang. and Sung Ju Hwang. Online coresct selection for rehcarsal- based continual learning. In International Conference on Learning Representations (ICLR), 2022. URL https://openrevieu http://ewerreaeeeeeeeeee/eraaaaa htt.:/oorum//d///9905500...
14

---

Published in Transactions on Machine Learning Research (05/2024)
Michal Zajac, Tinne Tuytelaars, and Gido M van de Ven. Prediction error-based classification for class- incremental learning. Arxiv, 0os//2005.18800 2023.
Chen Zeno, Itay Golan, Elad Hoffer, and Daniel Soudry. Task agnostic continual learning using online variational bayes. ArXin, abs/180s 0ts//183311023,, 2018.
Da-Wei Zhou, Qiwen Wang, Zhiyuan Qi, Han-Jia Ye, De chuan Zhan, and Ziwei Liu. Deep class-incremental learning: A survey, ArXiv, abs/2302.03648, 2023.
15

---

Published in Transactions on Machine Learning Research (05/2024)
A Proof
A.1 Proof in Online Class-incremental Scenarios
To tackle the issue of inter-class imbalance, extensive research (Menon all, 2013 Collell et all, 2016 Menon
ala 20ZII has been conducted on the optimal classifier that minimizes the class-bal class-balanced error for stable distributions. Actually, previous arts have proposed the following Theorem about this optimal classifier:
Theorem A.1. For time- cnne-invoriant distributions, the optimal estimate that minimizes the class-balanced error is the class under which the sample probability is most likely:
arg min CBE(D,3),
arg max (x))) = arg max P(arl)) vEly
(8)
VEDI
(2U3); Collell et all (2016) states that this optimal classifier is independent of arbitrary imbalanced label distributions P(().. The class-conditional function in stable distributions naturally minimizes the class-balance error. From Theorem A and given the condition of fixed class-conditinnals i.c., Vt, P(rly, Pc) P(zly, Po), we can derive the proof Theorem B as follows:
Theorem Menon et al
arg max D (x) = arg max yEDi
P(xly.p))
= arg max P(xly.
(9)
DED? t
yE|
A.2 Proof in Online General Continual Learning Scenarios
Without any prior information about the distribution of the test data, we assume that its distribution should conform to uniformly joint distribution of all observed class distributions. Therefore, the final intra-class distribution is E._,PE Px p.). Therefore, the result of Equation + is from definition.
Let Paly and Orly be the underlying distributions the optimal classifier that minimizes the class-balanced error and the learned class-conditional ional function represents, respectively. The class-balanced error gap between the optimal classifier (e.(), x F(zly, p) = Pxly and the learned class-conditional function exp( (x)) & P(lly. q) = Gnly can be formalized follows:
CBE( (3)) CBE(
Bp, Eeiyo (arg max Paly * arg max dolyll
(10)
D
y'E.. V'E),
c,(4",4)
DE
de de(Palp:9rrs)
Equation III describes the disparity -)(: from the optimal solution by similarity measure de(:.-) in the probability space. Aligning two dlass-conditioaal requires techniques for domain generalization and concept shift. In the future, we will explore efficient class-conditional alignment techniques in the context of online CL.
B
Algorithm
We give the algorithm of Experience Replay in Algorithm M The algorithm our proposed Logit Adjusted Softmax enhanced Experience Replay in Algorithm Ø is mainly based on Algorithm We also apply our method to online GCL by combining with knowlodge distillation, as shown in Algorithm g
C Benchmark Details
C.1 Dataset Details
We list the image size, the total number of training samples, the total number of test samples, and the total number of classes for the datasets (CIFAR10 Krizhevsky (2000). CTFAR100 Krizhevsky (200g).
16

---

Published in Transactions on Machine Learning Research (05/2024)
Algorithm Experience Replay (ER) Chaudhry et al (401
Input: Data stream {Di]
Initialize: Learner ( model parameter Q memory buffer Mi +- 0, label set 1/ 0.
for t= 1 to I do
Sample incoming batch B, from D,
) < V/-1 U Ust (
B.M t Retricval(BA B,,Ma
2 + (cconcal Br, Bi +)
|B E 1B11+1MM (or(U)
SGD(
M4+1 t (M,,
end for
Algorithm 2 Experience Replay with Logit Adjusted Softmax (ER-LAS)
Input: Data stream {Dr) temperature scalar т, sliding window estimator length / Initialize: Learner ((.), model paramcter θ memory buffer Mi t {}, label set 3) + 0-
for t= to T do Sample incoming batch B, from D, ( 1B:l)
Di + B.M + ReRtiievallB,, (B,, for y in y/ do
Ty, t compute class-priors from Equation 7 end for (cconcati Br, ), O) SGD |B.++BBMT D1 LLAS(U;,2))
Me11 <- Updatel BI,Mi) end for
TinylmageNet Le & Yang (2 ImageNet Deng et al. (2008) and Naturalist Horn et al (2017) in
Table In the former four Alass-balanced datasets, each category contains an equivalent number of training and test samples. However, within iNaturalist, an inherent imbalance exists between classes, posing greater challenge. We download the dataset of iNaturalist from http:::/giitubb.com/iisspjppp
/natt _
C.2 Continual Learning Setup Details
In online class-IL Aljundi el al] (2011)), classes of CIFARIO, CIFAR100, TinyImage and ImageNet are evenly split from the total into each task. And the classes in different tasks are disjoint. For example, C-CIFAR10 (5 tasks) is split into 5 disjoint tasks with 2 classes cach. As result, the numbers of training samples, testing samples, and classes are the same in each task, except iNaturalist. We divide the iNaturalist into 26 disjoint tasks according to the initial letter of the category. The numbers of classes in each task are shown in Figure E It shows that the number of classes varies significantly among each task. Noting that the classes within each task are also imbalanced. The comprehensive inter-class imbalance issues C-iNaturalist (26 tasks) pose great challenges online CL methods.
In online blurry CL Koh all (2022), the classes are divided into Naturry % disjoint part and (100 Noturry%) %) blurry part. The classes that belong to the disjoint part will only appear in fixed tasks, while all other classes in the blurry part will occur throughout the data stream. In each task, (#train-(T- (T- instances will be sampled from the training data of head blurry classes and Maturr instances will be sampled from the training data of remaining blurry classes, which forms the apparently class-inbalanced blurry part samples. The classes in blurry part play the role of head classes in turn across different tasks. During inference, the model will predict on test samples from all currently observed classes. We split CIFAR100 and TinyImageNet into 10 blurry tasks according to [Koh et.al (2022) with fixed disjoint ratio Nuturry 50 and blurry level Maturry 10. Next we take B-CIFAR100 (10 tasks) example.
17

---

Published in Transactions on Machine Learning Research (05/2024)
Algorithm Knowledge distillation with Logit Adjusted Softmax (KD-LAS)
Input: Data stream {Dr temperature scalar T, sliding window estimator length | Initialize: Learner ( model parameter Q memory buffer Mi <- O, label set Vi +- 0.
for t= 1 to I do
Sample incoming batch B, from D,
) < V/-1 Ust ( B. + Reeriicaal(B, for in V/2 do
yy, + compute class-priors from Equation end for
I P(concal B1, B, θ) gold -- #(concat BB, B, Gald)
(CLAS(U) 21)+ 2f 0) SGD( | 11B.++BB,11
Mi+1 < -Update(B,,MM,
ends a task. then
Save the old model Gold + O
end if end for
Table 8: Dataset information for CIFAR10, CIFAR100, TinylmageNet, ImageNet, and iNaturalist.
Dataset
Image Size
# Train
# Test
# Class
CIFAR10 Krizhevsky
(009
3 32 x 32
50,000
10,000
10
CIFAR100 Krizhevsky
2004
3 x 32 x 32
50,000
10,000
100
TinyImageNet Le & Yang (2
3 x 64 x 64
100,000
10,000
200
ImageNet Deng et all (2011)
3 x 224
x 224
1,281,167
50,000
1000
iNaturalist Horn all (20177
3 x 299
299
579,184
95,986
5089
In B-CIFARIO0 (10 tasks, Nuturry = 50, Maturry = 10, #train = 500 per class, #class 100), the disjoint part contains 50 classes, and each task possesses 5 disjoint classes training data. On the other hand, the blurry part comprises the other 50 classes, and each lask has head classes. The head classes contain 500 9 10 410 training samples, whereas the remaining 45 blurry classes only have 10 training samples each for the current task. Thercfore the model in this sctup will continuously observes disjoint new classes stream flows and imbalanced classes overlap across all tasks, encountering severe problem of inter-class imbalance.
In online Sum-Class-Domain CL Xie el all (2022), incoming data contains images from new classes and new domains. We only apply online Sum-Clas Class-Domain CL on CIFAR100. Similar to the online class- II. setup, we partition the CIFAR100 dataset into tasks, each with 5 subelasses. However, the model is required to predict superclasses, with each subclass representing distinct domain within them. Each domain of superclass has the same number of training samples. As depicted in Figure EL different superclasses appear in various tasks. Also, varying number of superclasses occur in each tasks. And the distribution within each superclass changes across different domains. Therefore, S-CIFAR100 (20 tasks) possesses both inter-class imbalance and ra-class domain drift, i.e, changing class-priors and class-conditionals Next, we discuss scenarios where mixed class and domain distributions jointly change, presenting a greater challenge than the sum of individual online class- and domain-II. learning problems. We refer to this case as "Mix- Class-Domaa This increased difficulty arises due the possible coupling between domain shift and class distribution shift in Mix-Class-DDae Domain CT.
18

---

Published in Transactions on Machine Learning Research (05/2024)
700
600
Number of classes per task
500
400
300
200
100
A B
C
D E GH J WKLLNNO R S NUVWXYY
Task
Figure 3: The number of classes per task in divided iNaturalist. Each one of these 26 tasks contains categorics with the same corresponding initial letter.
Class-ILe IL: The joint distribution of sample x and label can be written as p(a, y. p(aly. f),p(y. t), where p(aly. t) is constant throughout time t. p(y. t) changes with data flow and brings about inter- class imbalance.
Domain-IL: A common assumption in domain-IL that each class has the latent domain indicator 2. The joint distribution of sample x, label y, and domain indicator a can be decomposed into p(x, y/ z,t) p(z|2, y, p(=yy, p(y,t), where only p(zly,t) changes over time t, while y, and p(y,() keep constant.
Sum-Class Domaain: This is special case of the Mix-Class- Dommmln CL problems, where domain shift. and class distribution shift are separable. In the decomposition of the joint distribution, p(x, y, t) p(z|zz y, f), p(= (=lw, (), p(y, (), only p(æ|2, () remains invariant and independent of t, while p(zly, t) and p(y,t) t) vary over time, giving rise to intra-class imbalance and inter-class imbalance issues, respectively.
Mix-Classs Domas-- We consider the inherent coupling between domain indicators & and class labels y, rather than their mutual independence which allows for decomposition. In this case, the joint distribution can only be decomposed as pfr,y, 2, = p(r)2, y, 4)p(z, y, f). The fact that domain indicators and class labels are inseparable forms more challenging problem than the separable Sum-Class Dlass-Dommin CL, which may lead to more forgetting.
Unfortunately, there is also a lack of standard benchmarks for the indecomposable case. As one of the most challenging and realistic scenarios for practical applications, it deserves further in-depth investigation in future research.
D Metrics Details
Assume test samples of task j is S3 {Z,,, )n) The number of test samples in each class y for task is Ný. The model trained on task is 4, The seen class set at task i is V. The accuracy aig task j after training on task is formalized as follows:
1 NO
Σn
arg max vy (Ix) 3n
(11)
Y'E
5
19

---

Published in Transactions on Machine Learning Research (05/2024)
Task
Task2
Task3
Task4
2
2
Subclaas
1
10
20
10
20
20
0
10
20
Task5
Task6
Task
Task8
2
# Subelass
1-
0
20
10
Task9
Task 10
Task
Task 12
2
# Subelass
0
10
20
10
20
10
20
10
20
Task
Task 14
Task
Task16
2
Suholass
0
10
20
10
20
0
10
20
0
10
20
Task
Task18
Task
Task20
2
Subelass
1
1
10
20
10
20
10
20
0
10
20
Figure 4: An illustration of the occurrence of subclasses within each superclass for every task in S-CIFAR100 (20 tasks). The y-axis represents the number of occurrences of subclasses. The z-axis represents the 20 superclasses. Worth noting that each subclass is distinct domain.
For long-tailed data streams with inherent inter-class imbalance, also consider more appropriate met- ric, namely class- balanced accuracy o instead of standard accuracy as for evaluation. Class-balanced accuracy excludes prior class imbalances and prevents the overestimation of trivial solutions with high prob- abilities for major classes.
1 VEJ;
Σ
1 arg maxa bay (Ix) =v) yeE)
(12)
D
No
[(2, (ven.,))((sa. (m.m. 3.,
The corresponding final average accuracy AT and final average class-balanced accuracy Arhl can be calculated as follows:
Ar
T
aT
(13)
j=l
T
ccbl
-
Σ
TT
cbl
(14)
J=1
In datasets such as CIFAR-10 and CIFAR-100, where class-priors are uniform, the final average accuracy is equal to the final average class-balanced accuracy, consistent with analysis.
20

---

Published in Transactions on Machine Learning Research (05/2024)
The final average forgetting Fr can be computed Chaudhry et al. (20220) as follows:
T Σ ==1
FT
max -11.
(ai.) - aT,)).
(15)
We follow Koh ct al (2022) to add the Area Under the Curve of Accuracy AAUC in the online blurry CL setup. AAUc is the average accuracy to { of samples}. We simplify the calculation of AaUC by replacing (# of samples} with (# of steps}. Then this metric can be calculated as follows:
K E f(k An) An, N
1
AAUC
(16)
where represents the total number training steps, An denotes that we sample the accuracy f(+) of the model every steps, and K is the total number of sample intervals. We set An 5 in the experiments.
E
Implementation Details
E.1 Baseline Implementation
We follows list the hyperparameter configurations for the baseline methods mentioned in this paper, along with their sources of code implementation.
For ER (Chaudhry et al.
2014 we set the learning rate as 0.03. The code source is https://githubb.cm
Satageelabmammot..
For DER++ (Buzzega al. 2020,, set the learning rate 0.03. is set 0.1, and B is set to 0.5. The
code source is https://gitue com wta.m/aaaaallaaaammoottooo mammooth
For MRO ( Chrsskii & Moens 2023) we set the learning rate as 0.03. The code source is https://githut
com www.aaaaaamaaamaaaaa.a
For SS-IL (Am et al, 2020), we set the learning rate as 0.03. We update the teacher model every 100 steps. The code source https://githi www. cotp:///oocoooo0000001-----1----
For CLIB (Koo et all, 2022), set the learning rate 0.03. The period between samplo-wise importance updates is set to 3. The code source is httpp::/ggtthu....
For ER-ACE Caccia et al 2222 we set the learning rate as 0.03. The code source is https://giiie
Com/ connpcluucss4/AM1
For ER-OBC (Chrysakis & Moens 2023) we set the learning rate as 0.03 for both training and bias correction.
The code source https://githun com/ chrysakis/0BC,
For ER-CBA (Wang all 2023), we set the learning rate as 0.001 for C-CIFAR10, and 0.01 for C-CIFAR100 and C-TinyImageNet. The code source is htttps://githhi hwtp://///aae///---oiieeee----
For MIR (Aljundi et al. 2014 we set the learning rate as 0.03. The number of subsampling in replay is 160.
The code source is https: github. wwt.coooooop//oc cot./oppimmms//aaaimaaaalllllala
Interfered Interferee__etrieeval
For ASER,, (Shim all, 2020), we set the learning rate as 0.03. The number of nearest neighbors perform ASER is 5. We use mean values of adversarial Shapley values and cooperative Shapley values. The maximum number of samples per class for random sampling is 6.0 times of incoming batch size. The code source is
For OCS (Yoon et all, 2022), we set the learning rate as 0.03. The hyperparameter 7 that controls the degree of model plasticity and stability is set to 1000.0. The code source is http::://openreevee.eeee/e/eeeee
For LwF (L Hoiem 2016)) we set the learning rate as 0.03. The penalty weight a 0.5 and the temperature scalar is to 2.0. The code source is https://github. com http///ammallaammmmtttt...
21

---

Published in Transactions on Machine Learning Research (05/2024)
For LURIC (Hon et all 201M), set the learning rate as 0.03. Abase is set to 5.0 for all the experiments.
The code source is https://
http::////tttbb //gggtttuu..... com/hshustc/CVPRR9 Incremental Learning.
For GeoDL (Simon al 1)21), we set the learning rate as 0.03. The adaptive weight 33 is set to 5.0. The
code source is Sttps://githut com/ cwwccco/c////eeeeeee bbrrysts/geodesiccomm. continual. learning.
We also list the hyperparameter configuration for the bascline methods used in this appendix with their sources of code implementation.
For SCR (Mai et all 2021), set the learning rate 0.03 and the temperature
0.07. The code
source is https: thub. http:///atttooaaa/ooaeeeeee--- mot how.aa-eeaooiiiimmmaaaaaaaannnn
For OCM (Guo el all 2122), we use Adam optimizer and set the learning rate as 0.001. The code source is
https://githin com/ gydpku/UCK
For BiC cfal, 2014. we sct the learning rate as 0.03. We split 10% of the training data into validation set for training the bias injector with 50 epochs. The softmax temperature 7 is 2.0. Distillation loss also applied after bias correction. The code source is https://github. com/ sairin122//B1/
For (CEstastro et aL, 2018, we set the learning rate as 0.03. In the process of balanced fine-tuning we set the learning rate as 0.003 and train 30 epochs. The code source is sstps://githhe com/ hanm/PtrrikkZH] 1-o-EEnd-nncrmeene Learning.
For [Beonaaan Popescu, 2 we set the learning rate as 0.03. We calculate the mean and variance of each batch online to re-scale the outputs. The code source is sttp:://ggttubb..mm///eelllllll
class--ncrementll--eeyre
For LUCIR (Hon et all 201H) we set the learning rate as 0.03. Abase is set to 5.0, K is set to 2, and m is set to 0.5 for all the experiments. The code source is ottp://///////ggg/o/o//////////t Incremental Learning.
E.2 Ablation Implementation
In the ablation study of h we employ two extreme variants of LAS, along with two estimators. Random estimator randomly assigns class-priors. Macro uses statistical information to assign class-priors. Now, we elaborate on how these four methods are implemented. Recalling our proposed Logit Adjusted Softmax cross-entropy loss in Equation
Fy (1) Ty.t
(y, = - llog
log[1 +
Σ V'
Tyya Tyat
(x)
(17)
Even t (o)toroo
T is simple special case that sets the temperature scalar to
needs modification because directly setting the hyperparameter to large value to pursue would cause troubles when Ty/t//y, > 1, as it would lead to infinity coefficient and result in gradient explosion, obstructing the gradient descent optimization algorithm. Therefore, shown in Equation IS, we set the coefficient to only when T/ < 1, while retaining - = 1 for all other situations to enable successful model training. The significantly low forgetting rate and competitive accuracy observed in the experimental results suggest that this approach closely approximates T as expected.
(Tv.a/Ty,))
-
=
f0.
(Tgle//T..)) <
(18)
otherwise
Random samples each prior of seen classes from a uniform distribution U(0, 1]. Then they are normalized to 1.
Macro computes the joint label distribution by taking into account the occurrence frequencies of each class in the current data. stream, well as the label probabilities in the memory buffer, to serve the current.
22

---

Published in Transactions on Machine Learning Research (05/2024)
class-priors. It is worth noting that since the distribution of the data stream is unknown during training, Macro cannot be directly obtained and serves only reference for comparing and validating the necessity of batch-wise estimators. For instance, in C-CIFAR (5 tasks), when it comes to the 2nd task, 2 classes in the data stream are the same quantity, and the classes in the memory buffer also contain a similar number of samples from the previous task. The incoming and buffer batch sizes are also the same. At this point, the 4 classes probabilities that may appear in the input batch are all equally likely, i.c., 1/4. When comes to the 5th task, the data stream still consists of 2 classes with the same label probabilities, but the memory buffer now stores 8 classes that have appeared before. Therefore, it can be calculated that the class- blass-prioss of the 2 classes in the data stream are 1/4, while the class-priors of the 8 classes in the memory buffer are 1 1/16. M Macro represents a statistical oracle, but experiments show that its performance is inferior to batch-wise est estimators, indicating that in online CL, the model may pay more atlention to the label distributions within each batch rather than the label distributions across the sequential tasks.
F
More Experimental Results
F.1 Results on C-MNIST
We provide the results for MNIST of the online class-IL setting in Table g. ER-LAS still achieves competitive performance, highlighting the effectiveness of our method in addressing simple online CL problems. However, we observe that on small datasets like MNIST, the performance improvement achieved by our method is limited. This limitation arises because the tasks in MNIST are relatively easy, and the forgetting causod by inter-class imbalance not prominent. Therefore, in our experiments, we have primarily focused on more challenging scenarios with considerable classes large-scale datasets like ImageNet. Considering that online CL. aims to handle potentially infinite data streams, we believe that scalability to large-scale datasets is crucial in validating the effectiveness of online CL algorithms.
Table 9: Final average accuracy AT (higher better) on C-MNIST (5 tasks). M is memory size.
Method M 0.5k
1k
ER
87.020.2
88.01o.2
90.7
+ER++
92.3+o.1
93.9+0.0
94.2+0..
MRO
87.4+0.2
88.9+o.1
92.6+0.1
SS-IL
88.7+0.s
90.3+0.2
91.7+0.1
CLIB
88.420.3
90.610.0
91.920.0
ER-ACE
90.420.0
92.41o.1
93.810.2
ER-OBC
90.0
89.7
91.6+0.4
ER- CBA
90.1
90.24o.1
91.3+0.1
ER-LAS
91.7.0.1
92.8+o.1
94.0+o.1
F.2 Results on MNIST without Rehearsal
Despite the current mainstream online continual learning methods incorporating memory replay of samples and achieving satisfactory performance, a range of studies (Zajac et all, 2123 & Hotem, 20G Kirkpatrick all 2011 Zeno et all, 2018) have also explored effective continual learning approaches without rehcarsal. Our theoretical foundation of the class-conditional function paves the way for our rehearsal-free applications. In our proposed LAS loss function, the use of replayed samples is not necessary, allowing for direct application in online continual learning scenarios without replay. We conducted experiments on the C-MNIST dataset with no memory. The results in Table C show that LAS outperforms previously proposed methods. PEC achieves superior performance to LAS, benefitting from the expansion of new models for each class. Although our theorems hold without the need for rehearsal, the implementation of the logit adjustment technique requires support from replay data. As indicated on the right-hand side of Equation (G) the original paper, LAS adjusts inter-class classification margins by leveraging the imbalanced class-priors between major and minor classes, achieving balanced learning. When the number of minor classes decreases to zero, LAS
23

---

Published in Transactions on Machine Learning Research (05/2024)
fails to adjust the inter-class classification margins and degrades into learning based only on the currently encountered classes.
Table 10: Final average accuracy Ar (higher is better) on C-MNIST (5 lasks) without rehearsal.
Method
Ar t
LwF
19.8+0.0
EWC
19.820.0
Labels trick
45.723.5
PEC
92.3+0.1
LAS
48.4+1.2
F.3 Results when Batch Sizes are varied
While typically samples arrive one by one in the online learning data stream, advanced algorithms CaarrtthssCCaaaann all 2022; Chrysakis & Moens 2023) are commonly designed to update the model by accumulating a certain number of incoming samples as batch. This is because per-batch updating is generally more advantageous for model optimization convergence and well-defined classification boundaries than updating each individual sample. However, in some situations with constrained computational resources, only very small batch sizes are available or the batch sizes vary. Based on this concern, we consider two sctups related to changing the batch size: one is various batch sizes for the entire online training process, and the other is varying the batch size during training. We conduct experiments on online C-CIFAR10. We begin with brief introductions to these two setups.
1. Evaluating the batch size change for the entire online training process examines the macro robustness of our method to the hyperparameter of batch size. In the manuscript, set both incoming and buffer batch sizes to 32. We now experiment with corresponding batch sizes of 4, 16, 64. Smaller batch sizes bring more gradient updates for the model, but each contains less information for forming inter-class margins. Larger batch sizes may lead to overfitting on the memory buffer, thereby reducing performance.
2. Varying the batch size throughout the entire ouline training process examines the micro robustness of the batch size. This is a practical scenario where the frequency of incoming data may vary different stages, requiring time-varying batch sizes. In this experiment, we only vary incoming batch sizes while keeping buffer batch sizes at 32. We consider 3 cases of changing incoming batch sizes:
Inereasing incoming batch sizes during training, specifically for C-CIFAR10: 2, 4, 8, 16, 32, as Increase. The inter-class imbalance issue intensifies.
Decrcasing incoming batch sizes during training, specifically for C-CIFARI0: 32, 16, 8, 4, 2, as Decrease. The inter-class imbalance issue is alleviated.
Randomly sampling incoming batch sizes from a uniform distribution U[2 32] each stage, as Random. This is a fusion of the previous two cases, where the impact of inter-class imbalance varies during training.
The results in Table DD and Table I show that ER-LAS consistently achieves the best accuracy across various and varying batch sizes, highlighting the robustness of method to batch size variations. In theory, changing batch sizes or the variation of batch sizes during training poses threat to principle of mitigating inter-class imbalance through the climination of class-priors. It only affects our estimation of Lime-varying class-priors. However, the ablation study of estimators in Sixll indicales that online CL models may pay more attention to the current input class distributions. Therefore, our designed batch-wise estimator can timely provide effective approximation at various batch sizes. The potential issues may lie the cases where batch sizes bocome extremely small, such as 1. Following, we discuss this problem in detail and provide recommendations for improvement.
24

---

Published in Transactions on Machine Learning Research (05/2024)
Table 11: Comparison of final average accuracy on online C-CIFARIO with various batch sizes. In the manuscript, set both incoming and buffer batch sizes to 32. We experiment with corresponding batch sizes of 4, 16, and 64. Experimental settings are the samne in Table W. Memory sizes are M = 1k.
Batch size
4
16
32
64
ER
54.0 = 1.8
52.4 = 1.8
45.4 1.8
45.4 + 1.9
ER-ACE
55.1 + 1.9
56.7 = 2.1
48.1+10
46.2 + 1.9
ER-OBC
48.6 1.8
54.7=
46.4
39.0 + 1.8
ER-LAS
5.2
1.3
55.3 1.6
53.1 1.2
Table 12: Comparison of final average accuracy online C-CIFAR10 with varying batch sizes during training. We only vary incoming batch sizes while keeping buller batch sizes 32: Increase incoming batch sizes during training, i... 2, 4, 8, 16, 32. Decrease incoming batch sizes during training, i.c., 32, 16, 8, 4, 2. Randomly sampling incoming batch sizes from a uniform distribution U(2 32] at each stage. Experimental settings are the same as in Table M Memory sizes are M 1k.
Incoming batch size
Increase
Docrease
Random
ER ER-ACH
52.7 + 1.8
65.2 + 1.9
55.1 + 1.8
50.8 + 1.2
62.5 + 1.1
55.1 + 1.9
ER-OBC
53.1 + 1.4
65.3 1.1
51.4 = 1.7
ER-LAS
59.8 1.1
65.7 0.7
59.3 1.9
In fact, training single incoming sample goes against the theory of traditional stochastic gradient descent, which may harm model convergence and hinder the establishment of classification boundaries. Therefore, we maintain the incoming batch size of 1 and consider concatenating various numbers of buffer batch sizes to ensure valid training and practical performance. The experiments are conducted on online C-CIFARIO in order to explore the impact of changed buffer batch sizes on a single incoming batch size.
Table 13: Comparison of final average accuracy on online C-CIFAR10 with fixed incoming batch sizes of and various buffer batch sizes. Experimental scttings are the same as in Table Memory sizes are M 1k.
Buffer batch size
16
64
ER
39.3
2.0
62.4 + 2.0
63.7 1.8
60.6 + 1.9
ER-ACE
27.9 2.1
57.7
58.1
54.5 1.7
ER-OBC
33.2 1.9
64.3
65.3
60.9 1.7
ER-LAS
36.9 1.5
66.4
6.5
62.2 1.3
The results in Table show that when both incoming and buffer batch sizes are 1, ER-LAS performs slightly than the F.R baseline. Nevertheless, simply increasing buffer batch sizes enable ER-LAA to achieve the highest accuracy, This is because the case of extremely small batch sizes of 1 affects our estimation of time-varying class-priors and hinders the construction of classification margins, where slightly increasing buffer batch sizes can serve as an effective approach to refresh our method. Noting that excessive buffer batch sizes can lead to overfitting on the memory buffer and harm performance, as shown in the rightmost column Table
F.4 Comparison to Inter-class Imbalance Mitigation Methods
We mentioned the differences between LAS and other class imbalance miligation methods from an analytical perspective in $E. We worry that since these methods have not been deliberately designed and applied to online CL. in previous works, our direct application may lack credibility and endorsement. As result, we do not compare with these methods in experiments. However, we have indeed conducted experiments with them in the preliminary exploration phase of method. Here, briefly describe our applications and provide
25

---

Published in Transactions on Machine Learning Research (05/2024)
experimental comparisons and analysis. We compare four class imbalance mitigation methods originally for stable distributions. We refer to enietal (2019) and apply the Plass-Balanced loss, which re-weights the loss terms of each class based on the input class distribution, as ER-CBL of Loss weighting. We refer to Kang et all (2021) and normalize the weights of classifiers with || ll, as ER-WN of Weight normalization.
We perform upsampling on the bulfer samples and downsampling by randomly ignoring some incoming samples to maintain consistent input class distributions, as ER-Up and ER-Down of Resampling (Kubhá & Malwin, 1997.
Table 14: Comparison of final average accuracy by ER, ER-LAS, and imbalance miligation methods. Ex- perimental settings are the same as in Table M. Memory sizes are M = 1k.
Dataset
C-CIFAR10
C-CIFAR100
C-TinylmageN
ER
45.4 + 1.8
16.5 0.4
11.0 + 0.2
ER-CBL
48.1 + 1.6
18.8
11.1 + 0.2
ER- WN
46.1 1.5
16.6
11.0 + 0.1
ER-Up
53.6 1.6
0.5
15.0 + 0.1
ER-Down
1.6
1.39
1.2
ER-LAS
55.3 + 1.6
25.7
15.5 + 0.2
The results in Table Iai show that our ER-LAS outperforms all other compared methods for mitigating class inbalance. Next, we will analyze the shortcomings of these methods. ER-CBL re-weights the loss after computing the logits and ground truth, which helps in learning better features of minority classes but fails to climinate the influence of class-priors to achieve balanced posterior outputs. ER-WN ensures that the model output is not affected by class weight bias. However, we find that CL models are still affected by feature drift (Caccia et all, 2022), leading to recency bias. Therefore, these two methods cannot truly solve the inter-class imbalance problem in online CL. ER-Up the closest method to our ER-LAS, but the inter-class imbalance problem intensifies, it results in significant computational burden, whereas our method costs almost no additional computational resources. ER-Down, although maintaining inter-class balance during training, discards a majority of valuable incoming training samples. Furthermore, unlike these four mothods, our proposed LAS is supported by a statistical ground of underlying class-conditionas..
F.5 Results on Offline task-IL Scenarios
In offline lask-IL. settings, learners have access to a whole dataset for each lask and can undergo multiple epochs of training. Previous arts that are highly related to our work have proposed some Logit Rectify methods to alleviate the issue of inter-class imbalance in offline CL and improve learning performance. BiC (WW all, 20g9) adds bias correction layer to the model and stores a portion of input data as the held-out validation set to calibrate this layer and lessen the model's task-recency bias. F2F (Castro ef all, 20E) fine-tunes the model with a balanced dataset after each task. IL2M (Belouadah & Popescu, 211 rescales the model output with historical statistics. LURIC (Hon all, 20M combines cosine normalization, less-forget constraint, and inter-class separation to mitigate the adverse effects of class imbalance. We also compare successful offline CL methods (ER (Chaudhry et all, 20g)) DER++ (Buzzega ct a.l, 2020), and ER-ACE (Caccia el. all, 2)22)). Our experimental settings follow (Buzzega all 2020). LUCIR fails to work on C-TinylmageNet due to too low memory size.
The results in Table [B demonstrate that ER-LAS still achieves competitive performance in offline CL. When combined with knowledge distillation method LwF Hoiem, 2016), LwF-LAS outperform the other compared methods. These finds indicate that our approach can also effectively mitigate inter-class imbalance and improve performance than previously proposed Logit Rectify methods in offline task-IL setups. Considering the severe impact of recency bias in online CL, our main focus is how eliminate the adverse effects caused by inter-class imbalance in online settings, and we design widely applicable LAS algorithm.
26

---

Published in Transactions on Machine Learning Research (05/2024)
Table 15: Final average accuracy Ar (higher better) on C-CIFARIO (5 tasks), C-CTFAR100 (10 tasks),
and C-TinIlmageNet (10 tasks) in the offline condition. M 100. The epoch set 50.
Dataset
C-CIFARIO C-CIFARI00 C-TinylmageNet
BIC
23 a
15.3=0.1
10.1
E2E
51 61.620.3
16.7.0.1
9.0
IL2M
42.1
11.02a.2
8.4
LUCIR
28.9
15.7+0.7
10.2
ER
39.4
11.520.1
8. 8.1,0.0
DER+
55.311.2
14.8
9.420.3
ER-ACE
55.921.0
17.720.7
8.720.2
ER-LAS
53.9+1.0
16.4+0.2
10.3+0.1
LWF-LAS
57.5
22.6+0.1
12. 12.4
70
Final Average Accuracy Ar
65
60
55
50
SCR
45
OCM
ER-LAS
40
1000
2000
3000
4000
5000
6000
7000
Training Time (s)
Figure 5: Comparison with online CL methods based contrastive learning C-CIFAR10 (5 tasks). Memory size M 1k. The --axis represents training time, and the y-axis represents the final average accuracy Ar (higher is better). We evaluate the accuracy and the time efficiency of SCR, OCM, and our ER-LAS at batch sizes of 8, 16, 32, and 64. Noting that the time consumption increases as the batch size decreases.
F.6 Comparison to CL Methods Based on Contrastive Learning
We compare our method with the online CL methods (SCR Maletall (202T) and OCM Guo all (21M2) based on contrastive learning. SCR utilizes the NCM classifier and is trained via supervised contrastive learning. OCM employs contrastive learning to maximize mutual information. These methods based on contrastive lcarning typically require more computational resources, and their performance is influenced by the number of negative samples, but they often achieve better performance. As shown in Figure E, we evaluate the training time and the final average accuracy of our ER-LAS and the contrastive learning- based online CL methods under different batch sizes. SCR and OCM require and 30x more computational time than our method, respectively. Although they achieve higher accuracy than our method, LAS exhibits superior overall computational efficiency.
27

---

Published in Transactions on Machine Learning Research (05/2024)
Table 16: Final average forgetting Fr (lower better) on C-CIFAR10 (5 tasks), C-CTFAR100 (10 tasks), and C-TinImmagNNet (10 tasks). M the memory buffer size.
Dataset
C-CIFARI0
C-CIFARI00
C-Tinylmageleet
Method
0.5k
M 1k
M 0.5k
M 1k
M 0.5k
M 1
M
ER
4
36 1
24.5
31.1
23.211.0
2
38.510.
33.4
27.820.3
+
29.3+1.2
31.6+2.9
32.4+2.3
34.5+a.5
36.4+0.6
38.6+a.2
37.2+0.3
37.2+0.2
MRO
26.1
21.021.2
8.920.8
13.5
9.320.3
6.310.2
11.110.1
10.9
8.420.2
SS-IL
22.0+o.s
20.0+0.9
16.5+0.5
11.8+o.3
10.0+a.2
8.1+0.3
14.5+a.2
12.2+0.2
10,0+0.8
CLIB
30.411.
17.7
16.1
25.940.3
14.920.5
7.610.3
30.110.3
20.420.5
10.820.3
ER-ACE
11.0=1.
16.1
10.1
9.3
7.9
5.6=0.7
13.8+0.a
9.9
7.510.4
ER-OBC
37.3
19.5.0.a
30.5
24.0
22.4+0.4
18.7
36.2+0.2
29.420.2
21.9=0.2
ER-LAS
28.521.3
18.521.4
7.1
22.1
11.510.6
9.31o
26.910.2
19.920.4
10.3
F.7 Forgetting Rate
We compare the forgetting rate of each method C-CIFARIO, C-CTFAR100, and C-TinylmageNet in Table I. In most settings, ER-ACE achieved the lowest forgetting rate, except for when compared to our proposed ER-LAS on C-CIFAR10 with M = 2k, and to MRO on C-TinylmageNet with M 0.5k. Noting that the lowest forgetting rate does not necessarily correspond to the highest accuracy. Moreover, remarkable reductions in the forgetting rate can be achieved by deliberately adjusting the hyperparamoters of our method, but al the cost of accuracy. Currently, our method achieves the optimal stability-plasticne trade-off.
F.8 Prediction Results C- C-ImageNet
ER
Average Accuracy of classes in tasks
ER-LAS
60-
40
20
20
30
50
80
Tasks
Figure 6: Prediction results by ER and ER-LAS on C-ImageNet (90 tasks). We calculate the average
accuracy classes within each task to demonstrate the recency bias.
We present the prediction results of ER and our proposed ER-LAS on C-ImageNet after training, as shown in Figure R. Recalling that ER assigns 38% of the lest. samples to the most recently learned classes in C- CIFAR100 (Figure in (m). ER also outperforms ER-LAS on the last task, but is inferior to ER-LAS on all other tasks. This due to the larger task sequence and the more total number of classes C-ImageNet than in C-CIFAR100, resulting in a much more severe recency bias for the ER method. However, our ER-LAS successfully eliminates the recency bias expected, and result, achieves remarkably lower forgetting
28

---

Published in Transactions on Machine Learning Research (05/2024)
rate and the highest accuracy in the experiments of $6. These results validate that inter-class imbalance is more severe in long sequential tasks and demonstrate that our method can adapt to learning from such highly imbalanced data streams by pursuing the class-conditional function.
F.9 Class- balanced Accuracy on C-iNaturalist
Table 17: Final average accuracy Ar and final average class-balanced accuracy ATd (both higher better) on C-iNaturalist (26 tasks). We show the results of top-3 methods. Memory sizes are M = 20k.
Dataset
iNaturalist
Method
Ar f Agbl
ER
4.
6.25
ER-ACE
5.68+0.01
6.32+0.01
MRO
4.96+a.0
4.47
ER-LAS
8.11+0.01
8.62+0.0r
As aim to pursue the optimal classifier that minimizes the class-balanced error on imbalanced data streams, we also evaluate the class-balanced accuracy of our method and baselines C-iNaturalist. As shown in Table IV ER-LAST achieves the best performance in both accuracy and class-balanced accuracy, validating the effectiveness of our optimization towards this optimal estimator.
29


Reviews:
[{"review_id": "RTUSs5GsT6", "reviewer": "Reviewer_P17Q", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) Paper clearly describes the problem setup and contributions. \n\n2) Paper motivates the proposed method with theory and supports it with experiments. \n\n3) Authors clearly give training times for the proposed method and some of the baselines in the appendix. \n\n\nWeaknesses: \n\n1) Lack of large dataset results: One of the baselines SSIL has results on larger datasets like  ImageNet and Google Landmark Dataset v2. ImageNet dataset consists of 1,000 classes, which has nearly 1,300 images per class. Google Landmark Dataset v2 consists of 203,094 classes, and each class has between 1 and 10, 247 images. Will the results generalize the large number of classes where class imbalance could really have large effect. \n\n2) Related work and comparison with some recent papers seems to be missing. Quick search leads to the following papers [1-4]. Can authors justify why some of these recent papers should not be included? Note that this is not an exhaustive list and there could be more published papers.\n\n3) The paper uses different terms such as online class-incremental, online class-CL, online general CL, etc. without explaining their differences or relations. \n\n4) The paper does not provide any discussion or insights on the results, the limitations, the future work, or the broader impact of the proposed method.\n\n5) Is there any obvious or trivial baseline that one could try to address class imbalance? How does the proposed Logit Adjusted Softmax compare with other methods that use different loss functions, such as focal loss or class-balanced loss, to address inter-class imbalance?\n\n6) How does the batch-wise estimator with a sliding window handle the cases where some classes are missing in the input batches?\n\n\n\n[1] Kang, Minsoo, Jaeyoo Park, and Bohyung Han. \"Class-incremental learning by knowledge distillation with adaptive feature consolidation.\" In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16071-16080. 2022.\n\n[2] Dong, Jiahua, Wenqi Liang, Yang Cong, and Gan Sun. \"Heterogeneous forgetting compensation for class-incremental learning.\" In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 11742-11751. 2023.\n\n[3] Wang, Quanziang, Renzhen Wang, Yichen Wu, Xixi Jia, and Deyu Meng. \"CBA: Improving Online Continual Learning via Continual Bias Adaptor.\" In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 19082-19092. 2023.\n\n[4] Guo, Yiduo, Bing Liu, and Dongyan Zhao. \"Dealing with Cross-Task Class Discrimination in Online Continual Learning.\" In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11878-11887. 2023.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper addresses the problem of online continual learning, where models must learn from a non-stationary data stream while avoiding catastrophic forgetting and inter-class imbalance. The authors propose Logit Adjusted Softmax (LAS), a method that adjusts the model logits output according to class-priors estimated by a batch-wise sliding-window estimator. It can also be combined with existing rehearsal-based methods (and others) to further improve performance. The paper demonstrates improvements of LAS over prior arts through various benchmarks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1) Can at least one of the large scale dataset be included? \n2) Better comparison with state-of-the-art methods \n3) Better discussion on results and limitations.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "QyFectoAzO", "reviewer": "Reviewer_gzvs", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The extensiveness of the experiments is the main strength of this paper. It not only compares against quite a few baselines but also tests some important ablations and parameter sweeps. In my opinion, this is very sufficient and extensive.\n\nThe idea is relatively simple, and the theory is also quite intuitive and straightforward to see, but I like the discussion on the assumption of a fixed class-conditional distribution. I'm not sure why the distillation part is necessary, but it helps strengthen the effectiveness of the method. \n\nOne weakness is on writing which I point out below: \n\n1. Equation (6) and the notion of a margin: I don't quite see why this is a margin. Please point readers to a relevant reference, or explain the authors' intuition more clearly. \n2. The benchmark setup: please describe clearly what was done to the C-iNaturalist (26 tasks). What are the classes for each of the 26 tasks? Which tasks are \"small-scale\"?\n3. In Eqn 11 and 12, please indicate that the samples are drawn from the j'th task, otherwise the index j never appears on the right hand sides.\n4. Please explain clearly how the distillation is performed, what are the losses, and whether a separate NN is trained for each task?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses continual learning where the set of labels changes across tasks but the class-conditional data distribution is invariant through time. The author proposes to optimise a neural network class predictor where the outputs (logits) are combined with a class prior to estimated from a short horizon of the past. The authors showed extensive experiments showing that this simple idea improves existing methods. In addition, in cases where the task boundary is known, the authors propose to use knowledge distillation to save previously trained models.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I only ask the authors to address the three points above.", "broader_impact_concerns": "No discussed. The authors should try to explain why this is unrelated.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "jN4H6Rb02T", "reviewer": "Reviewer_x8rw", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n- The proposed method improves the performance of standard ER on online class-incremental learning problems without substantially increasing the computational cost\n- The proposed method is claimed to be orthogonal to, and can be combined with, replay and knowledge distillation\n- The proposed method is experimentally evaluated on several relatively large-scale benchmarks for online continual learning\n\nImportant issues:\n- The paper claims at several places to be the first to realize that the class-conditional function $p(x|y)$ remains constant in class-incremental learning (and that therefore modeling $p(x|y)$ rather than $p(y|x)$ might be a fruitful approach for class-incremental learning). This claim is not true and should be rectified, as a similar point was made and demonstrated by Van de Ven et al. (2021, CVPR-W; https://arxiv.org/abs/2104.10093). The implementation of the current paper differs from the implementation of this other paper, but the underlying motivation is similar, and this should be acknowledged and discussed.\n- There are several important assumptions that underlie the method of this paper that do not seem to be discussed. Firstly, it is assumed that the “overall” or “ground-truth” class prior $p(y)$ is uniform over all classes. This assumption should be stated and discussed. In some cases (e.g., the experiments that are presented) this might be a reasonable assumption, but what if this assumption does not hold? Secondly, it seems to me that it is assumed that the softmax-classifier that is being learned online is, at any point, “fitted to and converged” to the data distribution that is captured by the sliding time window that is used to estimate the class-prior. I think it is important to make this assumption explicit and to discuss its consequences. For example, I think this highlights the importance of choosing a correct sliding time window. A choice of “1”, as seems to be the default in this paper, does not actually seem reasonable to me. I think it is likely that this choice of “1” only works thanks to the combination of LAS with ER. If this is the case (see also another comment below), this should be clearly discussed.\n- The main “selling point” of this paper is probably the strong performance of ER-LAS relative to the competing methods that are compared against. However, it seems that the provided empirical comparisons paint a too optimistic picture. For example, compared to another recent paper (Zając et al., 2023; https://arxiv.org/abs/2305.18806) that uses a similar set-up (e.g., online, ResNet18), the reported results no longer seem so unequivocally strong. On CIFAR-10 and CIFAR-100, this other paper outperforms ER-LAS (when using a memory buffer of 1k or less) even though the approach of this other paper does not use any replay (and additionally, the problem in this other paper is made strictly more difficult by only having a single class per task). I think this should be discussed in the current paper to put the reported results into perspective, and some of the claims about “best” or “state-of-the-art” performance should be changed.\n- I would like to encourage the authors to additionally include results on a simple dataset (e.g., MNIST). I think it is important to include such an experiment as a sanity check to show that the proposed strategy is able to successfully solve a simple problem.\n- I would also like to encourage the authors to include some results of LAS without using replay (i.e., show results for “finetune+LAS”, it might be OK to just do this on a dataset like MNIST). Based on the theory it seems that LAS should work in this case as well. If in practice LAS fails in this case, I think this is important to show and to discuss why this might be the case.\n- It is claimed in several places in the paper that “general CL” is the “most challenging” continual learning scenario. Could the authors clarify what this means? My impression is that in this paper the term “general CL” is used to refer to any experiment that features a combination of domain- and class-incremental learning. Is the claim that combining these two types of incremental learning results in a problem that is more difficult than the sum of the individual problems? If so, I think the authors should explain why. If not, I think the claims about “general CL” should be rectified.\n- Finally, this paper makes several questionable decisions in assigning credit to previous findings. For example, Goodfellow et al (2014) is cited for “catastrophic forgetting”, but it seems more appropriate to cite McCloskey & Cohen (1989) and Ratcliff (1990) for this. The authors also do not cite the original paper, or preprint, that describes the distinction between task-, domain- and class-incremental learning (van de Ven et al. 2022, Nature Machine Intelligence). For the statement that some methods train only on replayed samples to achieve balanced learning, it seems to me that Prabhu et al. (2020, ECCV) should be cited. I encourage the authors to critically go through their paper for other potential misattributions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper addresses the online class-incremental learning problem. It starts by observing that in this setting predictions of softmax-based discriminative classifiers tend to be biased towards the most recently seen classes (inter-class imbalance). Using Bayes’ theorem, it is then argued that rather than directly learning the conditional distribution $p(y|x)$, in class-incremental learning settings it is advantageous to instead learn class-conditional function $p(x|y)$, as this one remains constant over time. The paper states that directly modeling the class-conditional function $p(x|y)$ is notoriously difficult, and instead proposes to approximately do so using a previously developed Logit Adjustment technique. To adapt this technique to the non-stationary setting, the paper introduces a sliding time-window based estimator for the time-varying class-prior $p(y)$. The paper calls this approach Logit Adjusted Softmax (LAS). With experiments on CIFAR and ImageNet, the paper then shows that when LAS is combined with experience replay (ER), it outperforms several other recently proposed extensions of ER for class-incremental learning.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would encourage the authors to make changes to the paper in order to address the “important issues” listed above.", "broader_impact_concerns": "No concerns in this regard.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: During private discussion, the reviewers all recommended that this paper should be accepted. For example, one reviewer commented, "Initially I had a fair number of concerns, but the authors did a good job addressing and/or clarifying these in their rebuttal and subsequent discussion phase. I don’t have any major concerns left that would make me doubt the validity of the paper’s main claims." All reviewers explicitly stated that the authors sufficiently addressed their concerns in the revision. They were also appreciative of the authors including the suggested limitations in their paper.

The authors are advised to consider all advice from the reviewers in preparing the final version of their paper.
Decision: Accept (poster)