Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning
Authors: Nisha L. Raichur
Lucas Heublein
Tobias Feigl
Alexander Rügamer
Christopher Mutschler
Felix Ott
Venue: Accepted by TMLR ()

Abstract: The primary objective of methods in continual learning is to learn tasks in a sequential manner over time (sometimes from a stream of data), while mitigating the detrimental phenomenon of catastrophic forgetting. This paper proposes a method to learn an effective representation between previous and newly encountered class prototypes. We propose a prototypical network with a Bayesian learning-driven contrastive loss (BLCL), tailored specifically for class-incremental learning scenarios. We introduce a contrastive loss that incorporates novel classes into the latent representation by reducing intra-class and increasing inter-class distance. Our approach dynamically adapts the balance between the cross-entropy and contrastive loss functions with a Bayesian learning technique. Experimental results conducted on the CIFAR-10, CIFAR-100, and ImageNet100 datasets for image classification and images of a GNSS-based dataset for interference classification validate the efficacy of our method, showcasing its superiority over existing state-of-the-art approaches.

Published in Transactions on Machine Learning Research (03/2025)
Bayesian Learning-driven Prototypical Contrastive Loss for Class-Incremental Learning
Nisha L. Raichur
misha hasphh/...kaaaao....mmuuaiitaaaa
Prounhofer Institute for Integrated Circuits IIS, Nürmberg, Germany
Lucas Heublein
Praunhofer Institute for Integrated Circuits IIS, Nürnberg, Germany
Tobias Feigl
Abbias.feigl@iiss..mmmmoooo
Prounhofer Institute for Integrated Circuits IIS, Nürnberg, Germony
Alexander Rügamer
Fraunhofer Institute for Integrated Circuits IIS, Nürrberg, Germany
Christopher Mutschler
Prounhofer Institute for Integrated Circuits IIS, Nürnberg, Germony
Felix Ott
felix. ott feweuiissfrounoffee
Prounhofer Institute for Integrated Circuits IIS, Nürnberg, Germany
Reviewed on OpenReview: htths///ppaareeeeee////momooo-a V9M
Abstract
The primary objective of methods in continual learning is to learn tasks in a sequential manner over time (sometimes from a stream of data), while mitigating the detrimental phenomenon of catastrophic forgetting. This paper proposes a method learn an effec- tive representation between previous and newly encountered class prototypes. We propose protolypical network with Bayesian learning-driven contrastive loss (BLCL), tailored specifically for class-increments learning scenarios. We introduce contrastive loss that incorporates novel classes into the latent representation by reducing intra-class and increas- ing inter-class distance. Our approach dynamically adapts the balance between the cross- entropy and contrastive loss functions with a Bayesian learning technique. Experimental results conducted on the CIFAR-10, CLFAR-100, and ImageNet100 datasets for image clas- sification and images of a GNSS-based dataset for interference classification validate the efficacy of our method, showcasing its superiority over existing stale-of-the-arl approaches. Git: https:// https///gillabcccasp..o hwwpspp.frraaooo.eeeeeeeeeeeeeee gnss/ gnss class incremental learning
1 Introduction
Numerous practical vision applications require the learning of new visual capabilities while maintaining high performance on existing ones. Examples include construction safety, employing reinforcement learning
methodologies (Kirkpatrick et al., 2017), or adapting novel types of interference (as image classification
task) in global navigation satellite system (GNSS) operations (Ott et al. 2024 Brieger et al. 2022 Raichur
ct al. 2022 van der Merwe et al.) 2023 Heublein et al.) 2024b] Gaikwad et al.) 2024 Manjunath et al.
2025 h. Continual Learning describes the setting of sequentially learning novel tasks while avoiding calas-
trophic forgetting, ie., forgetting how to perform well on previously seen tasks (Kirkpatrick et al.) 2017). which commonly occurs when models are trained successively on multiple tasks without revisiting carlier tasks. Recent methodologies try circumvent catastrophic forgetting, for instance by utilization of fea- ture extraction and fine-tuning adaptation techniques (Li & Hoiem 2018), by leveraging off- off-policy learning

---

Published in Transactions on Machine Learning Research (03/2025)
Task
Task
Task 3
Task 4
Task
CIL Model
CIL Model
CIL Model
CIL Model
CIL Model
generolized
blooilos
wwrngiig
weight
@eteralized
weight Averaging
generolized blocker
weight averaging
generalaadd
bicoits
evrrggngg
weight
generalizeed
blocks
spectialined blocks
specialiaed
specialized blacks
spectaliaed
sperialiaed
blocks
blocks
blocks
(tsinulble]
(isinable)
Grainablel
Imsinuble
Insinuble
Bayesian Learning driven Weighting Entropy Contrastive Learning Functions
feature represen- tation.
test
classes to
classes 0
classes to 6
classes to
classes to
Figure 1: Overview of our BLCL approach, which employs Bayesian weighting to balance cross-entropy and contrastive learning loss functions, facilitating efficient representations during updating specialized blocks.
and behavioral cloning from replay to enhance stability (Rolnick et al. 2019), or by selectively decelerating learning on weights important for specific tasks (Kirkpatrick et al. 2017
Previous work (Zhou et al. 2023b) investigated the effect of model size of the specialized components, i.e., shallow and deep layers, and concluded that deeper layers obtain diverse feature representations and is crucial for learning novel tasks. Specifically, the learned data representations between precoding and novel tasks hold considerable importance (Rebuffi al., [017]. This serves as a motivation for our focus on refining the architecture of specialized blocks and enhancing dynamic representations for continual learning. Our proposed method, BLCL, enhances task adaptability by dynamically expanding specialized blocks. These blocks, optimized through targeted hyperparameter scarch, focus primarily on task-specific lcarning.
A related yet distinct task is contrastive learning, which pairs positive and negative instances with corre- sponding anchors to enrich training (Chen et al. 2020 Ott et al. 2023). While continual learning with contrastive techniques is prevalent in semantic tasks (Ke et al.J 2021). contrastive learning's application in continual learning for image classification tasks is limited (Cha et al. 2021). This limitation arises from the potential decrease in diversity of negative samples, affecting the task-specific loss, and the likelihood of the regularizer may hinder learning of new distinctive representations (Cha & Moon 2023). Recognizing the importance of weighting both the classification and contrastive loss functions to achieve a balance be- tween previous and new tasks, our approach utilizes a Bayesian learning-driven strategy (Kendall & Gal, 2017 Kendall ot al. 2018). A notable challenge ariscs from the inconsistency of negative samples between classification and representation learning lasks (Liu el al. 2024).. Bayesian framework facilitates the infer- ence of probabilities for contrastive model samples, eliminating the necessity for computationally intensive hyperparameter tuning scarches (Hasanzadeh et al. 2022. Figure F shows the main idca of our approach, exemplary for GNSS interference classification. For each task, two additional classes are trained and the model weights are averaged with the weights from the previous task to circumvent catastrophic forgetting by adapting only the specialized blocks. We combine the cross-entropy (CE) with a contrastive learning (CL) loss and dynamically weigh them with Bayesian learning. This leads to an optimal feature representation with improved clusters.
The main contribution of this work is to propose a continual learning approach capable of adapting novel classes. Our contributions are as follows: (1) We introduce BLCL, continual learning method initially based on MEMO (Zhou et al. 2023b), which is founded this setting, we introduce dynamic specialized blocks by adding additional blocks as shown in Figure 2 (2) We combine the CE with CL loss, incorporating class embedding distances in a prototypical manner (Snell et al.) 2017).
generalized and specialized blocks. In addition to
(3) We utilize a Bayesian learning- drrien (Kendall et al., 2018) dynamic weighting mechanism for both loss functions, and (4) we conduct extensive ovaluations on CIFAR-10, CIFAR-100 (Krizhevsky & Hinton 2009), ImageNet100, and GNSS (Ott et al., (2024) dataset to demonstrate that BLCL yields an enhanced feature representation, resulting in improved classification accuracy for image-related tasks.
2

---

Published in Transactions on Machine Learning Research (03/2025)
2
Related Work
2.1
Continual/Caas-Incceeennaa Learning
Class-incremental learning (CIL) aims to continuously develop a combined classifier for all encountered classes. The primary challenge in CIL lies in catastrophic forgetting, where direct optimization of the network with new classes leads to the erasure of knowledge pertaining to former tasks, thereby resulting in irreversible performance degradation. The objective is to effectively mitigate catastrophic forgetting (Zhou et al. 2023a Conventional ML models address CIT, through dual Lasks by updating only the model with a single new stage [Lewandowsky & Li, 1995; Kuzborskij et al. 2013 Da et al.) 2014). Feature extraction methods (Donahua Ct al. 2014) maintain the set of shared network paramcters (6,) and task-specific parameters for previously learned tasks (0o) unchanged. Here, only the outputs of one more layers serve as features for the new task, employing randomly initialized task-specific parameters (0,). In finetune (Girshick et al. 2014) methods, O, remains fixed, while O, and 6, are optimized for the new task. Kirkpatrick et al. (2017) | proposed elastic weight consolidation (EWC), which selectively decelerates learning on weights crucial for previous tasks. The method learning without forgetting (TwF)), introduced by Li& & Hoiem| (2018), employs solely new task data to train the network while retaining the original capabilities. Rolnick et al. (2019) proposed CLEAR (continual learning with experience and replay) for multi-task reinforcementt learning, leveraging off-policy learning and behavioral cloning from replay to enhance stability, well as on-policy learning to preserve plasticity. Rebuffi et al. (0017) introduced iCaRL, which learns such that only the training data for small number of classes needs to be present concurrently, allowing for progressive addition of new classes. iCaRL combines a classification loss with a distillation loss to replicate the scores stored in the previous step. Replay, introduced by Ratcliffi (1997). investigates manipulations of the network within a multilayer model and various variants thereof. Dynamically expandable representation (DER) Yaan et al. 2021) considers limited memory and aim to achieve better stability-plastieity trade-off utilizing a dynamically expandable representation. Inspired by gradient boosting, FOSTER (Wang et al., 2022) dynamically expands new modules to fit the residual between the target and output of the original model. Yu et al. (2020) compensated the semantic drift of features for embedding networks without exemplars. PASS (Zhu et al. 2021 and Zhu et al. (2022) mainly focus on previous class prototypes and updating the backbone model.
Conventional CIL methods typically prioritize representative exemplars from previous classes to mitigate forgetting (Luo et al.] 2023). However, recent investigations indicate that preserving models from history can significantly enhance performance. Certain applications necessitate memory-efficient architectures. However, state-of-tho-ars CIL methods are not compared with regard to their memory budget. The memory-effeiien expandable model (MEMO) (Zhou et al., 2023b) addresses this gap by considering accuracy and memory size, analyzing the impact of different network layers. Zhou et al. (2023b) discovered that shallow and deep layers exhibit distinct characteristics in the context of CIL. MEMO extends specialized layers based on shared generalized representations, efficiently extracting diverse representations at modest cost while maintaining representative exemplars. We adopt MEMO as our methodological baseline. However, we observe that results can be further enhanced by incorporating dynamic specialized blocks (the last layers in Figure for which the weights are averaged with the previous task). In our experiments, we compare the performance of Finetune, FWC, LwF, iCarl, Replay, MEMO, DER, and FOSTER. For comprehensive overview of
methodologies, refer to Zhou et al. (202aa): Lange et al. (2022); |3 Masana et al. (023]] | Mai et al. 2022
2.2 Bayesian Contrastive Learning
Contrastive learning establishes pairs of positive and negative instances along corresponding anchors to en- hance training (Chen et al.) 2020). This approach has found extensive application across various domains including multi-modal learning ( (Wei et al. 2016 Rasiwasia et al. 2010 Huang et al. 2020 domain adap-
tation Thota & Lcontidis 2021), and semantic learning tasks (Ke ct al.
2021 Sarafianos et al. 2019). The
contrastive learning paradigun has been exlended to triplet learning (Ou et al. 2023 Schroff et al. 2015).
which utilizes a positive sample, and quadruplet learning (Ott et al. 2024 Chen et al. 2017), which employs a similar sample. The main focus within this field resolves around the optimal selection of pairs to augment training. However, the primary focus of our study resides in determining the optimal weighting between classification and contrastive loss functions.
3

---

Published in Transactions on Machine Learning Research (03/2025)
Cha et al. (2021) demonstrated that representations learned contrastively exhibit greater robustness against calastrophic forgetting compared to those trained using the CE loss. Their approach integrates asym- metric variant of supervised contrastive loss, mitigating model overfitting to limited number of previous task samples by employing a dynamic architecture. However, contrastive learning requires a large number of diverse negative samples to be effective. On smaller datasels (as the CTFAR-10 and GNSS datasets), this lack of diversity can degrade performance (Cha et al., 2021). Similarly, Sy-CON (Cha & Moon 2023) also integrates symmetric contrastive loss. Hasanzadeh al. (2022) utilized distributional representations to provide uncertainty estimates in downstream graph analytic tasks. Lin et al. (2023) employ Monte Carlo dropout on skeleton data for action recognition, generating pairwise samples for model robustness. Liu et al.
(2024) introduced a Bayesian contrastive loss to mitigate false negatives and mine hard negative samples, aligning the semantics of negative samples across tasks. Instead, our approach leverages Bayesian learning as proposed by Kendall et al. (2018) for weighting the CE and CL functions.
3 Methodology
we mathematically formulate the problem of CIL. Subsequently, in Section 3.2 we provide an overview of our proposed method. We introduce a contrastive loss formulation and Bayesian learning-driven weighting strategy Section 3.3 We describe data augmentation techniques in the Appendix A.I
This section provides a detailed description of our methodology. In Section 3.1
3.1 Problem Formulation
We consider T soquentially presented training tasks denoted as ,D.,...,,TTT where cach D ((%, w represents the 1-th incremental step containing ni training instances of images. X signi- fies a sample belonging to class yi E Y,, with Y, representing the label space of task During the training of task t, only data from D' is available. The objective is to continuously develop classification model en- compassing all classes encountered thus far. Subsequent to each task, the model is evaluated for all observed classes J/2 Yi UY, (Zhou et al. 2023a Rebulfi et al. 2017). Typically, classes do not overlap ween different tasks. However, in certain scenarios, previously encountered classes may reappear in subsequent tasks, constituting what is termed as blurry CIL (Xie et al.] 2022 Koh et al. 2022 Bang et al. 2021 2022) For the purpose of this paper, it is noted that the classes are overlapping as we have a. fixed number of representative instances from the previous task classes, so-called ezemplars.
3.2 Method Overview
Our methodology is outlined in Figure 2 designated as BLCL. Our BLCL method is built upon the MEMO architecture (Zhou el al. 2023b), which serves as the baseline. Our key contribution lies in the integration of a Bayesian framework to dynamically balance loss functions. Tnitially, we trained 110 vision encoder models from Hugging Face (Wightman 2019 on the GNSS dataset, identifying the ResNet model as the optimal backbone (sec Appendix A.2 The ResNet model, proposed by He et al. (2016). specifically ResNet18 ResNet32, comprises both generalized and specialized components. The generalized component extracts features from the image dataset, featuring 13 convolutional layers, and is succeeded by one or more (up to four) additional layers. The specialized component consists of blocks whose weights are averaged with the weights from the previous task. These specialized layers contain different numbers of blocks I, adapted to the dataset at hand, are denoted as BI.CT,, where each block comprises of two convolutional layers and two batch normalizations. The output dimension of the convolutional layers is set to 512. We employ average pooling and fully connected (FC) layer, and train utilizing the CE loss Ler. The training process involves freezing the specialized component from previous tasks while updating the generalized and specialized component with classes pertinent to the specific tasks. Morcover, we average the weights of the specialized component. from previous (asks. To maintain adaplability to varying complexities of new classes across tasks, we dynamically adjust each task's layer structure, denoted as BLCL(s) (as depicted in the top right of Figure 2 For example, the notation BLCC..(I.1222) represents a network architecture consisting of eight specialized blocks, where the final layers are arranged in the sequence 1, 1, 2, for each task. This adaptation involves the removal of either one convolutional layer an entire block from the specialized
4

---

Published in Transactions on Machine Learning Research (03/2025)
gener generalized aerialleed (bacckbont
imput
block
BLCh
BLChy
BLCLe
BLCLq
BLCLu4
conv: 128
conv2D: (32x644). (3x33), (2x2)
conv: 128
cOnv: 64
conv: 128
Batch
conv: 64
conv: 256, \2
conv2D: (32x64))
ReaNer1 NesNet322 GNSSS
CIFAR 10
conv:
conv: 256
(3x33), (2x2)
conv: 128, 2
bosi:: 256
conv: 256
BatchNorm:
bayer
AvgPool
FC
weight averuging
Lootal
1
Ace
(W) (c)(W)
20j
log log
Average Pooling 2D:
per for
kernel size-
Task
Task
Task
Task
Task
stride-8, padding-0
CIFAR-10
0. 2,
4.
6, 8, 9
AvgPool
AvgPool
AvgPool
CIFAR-100
10 tasks, with 10 classes per task
FC
RC
PC
ImagoXet100
10 tasks, with 10 classcs per task
Task 1
Task
Task 3
GNSS
0, 1,
3,4
6, 6
7,
9, 10
Figure 2: Overview of our BLCL method. After the generalized component of RosNet18 ResNet32 He
et al. 2016), follow up to eight blocks (1 8) for the CIFAR (Krizhevsky & Hinton 2009) datasets and two blocks (1= 2) for the GNSS (Ott et al., 2024) dataset in their respective specialized blocks. After each task the weights are averaged with the weights of the previous tasks. We propose dynamic architecture with a
different number blocks and convolutional layers, indicated by BLCLQ After the FC layer, follows a CE loss in combination with a Bayesian learning-driven weighting of the contrastive loss (CT.).
model's layer configuration, catering to the evolving requirements of each task. Furthermore, to facilitate a fair comparison with other state-of-the-art methods and to compensate for the additional blocks, we maintain a constant number of exemplars, fixed at 2,000 samples, throughout the training process for all datasets. See the Appendix A.3 for the dynamic adjustment of the layer structure.
3.3 Loss Functions
The lower-level reprosentation of the CIL model is contimuuusy adapted with the addition of cach new class, thereby polentially undergoing significant changes. The primary objective is to incrementally enhance class prototypes, thereby reducing intra-class distances while simultaneously increasing inter-class distances. To facilitate this objective, we introduce a contrastive loss term, denoted as LCL. Achieving an optimal balance between the CE and CL loss functions is crucial. Hence, propose automated weighting mechanism utilizing Bayesian learning techniques.
Contrastive Loss. Pairwise learning characterized by the utilization of pairs featuring distinct labels, serving enrich the training process by enforcing a margin between pairs of images belonging to different identities. Consequently, the feature embedding for specific label lives on a manifold while ensuring sufficient discriminability, i.e., distance, from other identities (Ott et al. 2023 Do et all 2019). CL, the other hand, is a methodology aimed at training models to learn representations by minimizing the distance between positive pairs and maximizing the distance between negative pairs within latent space. We define feature embeddings f(X) E Raxh to map the image input into a feature space Rexh where I represents the output the last convolutional layer with a size of 512 from the specialized model, and x denotes the dimensionality of the layer output. Consequently, we define an anchor sample Xς corresponding to a specific label, posilève sample X from the same label, and a negalive sample X drawn from different. label. For all training samples ((X)) E o objective is to fulfill the following inequality:
d(s(x)) + < (((()))
(1)
5

---

Published in Transactions on Machine Learning Research (03/2025)
where d is distance function, specifically the cosine distance CD(X1, X2) = || x computed between non-zero vectors X1,Xy In of size n, denotes margin differentiating positive and negative pairs, and & is the set of all possible pairs within the training datasct, where N is the number of pairs. For each batch of size bs, we select all conceivable pairs and increase the batch size be(bs+1)
Algorithm 1 BLCL: Algorithm for Incremennlllllll Learning (Python & PyTorch like code)
Input: T task sequence, current task, M model, 7 similarity threshold,
2: E = number of exemplars, N number of epochs, = learning rate
3: for each t in 7 do
4: dataset = load_ task (aakk_dat(()) (t)
5: class similarity = class, similarity(,, srevioss_takee
Based on cosine similarity
if class siisssarmlrrtyy > then
reduce laaeeellaassssMM
Reduce layers in the last specialized block
elsc
retain_ (ayaiin)
Keep all layers from the last specialized block
10:
end if
11:
optimizer = wattiialize_optimmizerteee 01, 02 a)
>. Adam optimizer
12:
{1- 1..,,...,.l 1}) enable enable_trainingg srrraaninngffoggggerralizall and specialized(M, () for epoch 1 to N do for (X, Y) in dataset do
13:
14:
15:
16:
softmax, emb = model (X)
Embeddings of laster layer
17:
Lce = cross_ -ertoss_etrrry___eeee _oss (softmax, Y)
18:
pos pair, neg pair- = form_ Laamai_aars____ (emb) & Select positive and negative pairs for each batch CCL = contrastive _llssss______ [ems( pos pos_pair, neg_pair) loss, 01, d2 Bayesian_ weigtneeegghiigngnee (Cee, La., 01, 02) optimizer.zerg zero grad() (oss.backwarl( ) optimizer.stepp(
19:
20:
b Update deviation parameters
21:
22:
23:
24:
end for end for
25:
26:
M2 = poooragesppeiilizzedd w M,-1)
27: = update_exemplare (t,, E)
b Ensures that the total exemplars are equal to E
28: end for
29: Return: trained M
Bayesian Learning. Typically, naïve methods employ weighted combination of both loss functions to compute the total loss, denoted as Liotal = wi LcE + wyLeL. However, the model performance is extremely sensitive to the selection of weights w; (Groenendijk et al. 2021). Balancing CE and CI. losses is essential because they capture complementary aspects of lcarning: CE ensures class-specific discrimination, while CL enhances feature representations. Optimizing only the CL may lead to poor decision boundaries and degraded classification performance. Dynamic weighting allows the model adapt throughout training, emphasizing representation learning initially and refining decision boundaries later for better generalization. In our approach, we adopt strategy wherein concurrently optimize both objectives using homoscedastic task uncertainty, as defined by Kendall & Gal 2017 Kendall el al. (2018). This homoscedastic uncertainly pertains to the task-dependent aleatoric uncertainty, which remains invariant across input data but varies across distinct tasks (Klaß et al. 2022). Let W (X) denote the output of a noural network with weights W on input X. In scenarios involving multiple model outputs, we factorize over the outputs
p(y / (x) = p(y W (x)(()))))))))))) p (X)),
(2)
with K model outputs Vi.....YK .K Kendall al. 2018 In a classification task, we sample from a probability vector from scaled softmax function's 's output p(y (X). softmax ( (X)). The log likelihood is

---

Published in Transactions on Machine Learning Research (03/2025)
then defined as
logp(y = c((((()))))))
0² w (X) log Σ exp ( fe (x)),
(x)),
(3)
02
where W (X) is the e-th element of (W (X). In a regression lask, we maximize the log likelihood of the model as
log logp(y (y w (X), & 3/2 lly w (x)1 log a
20
for Gaussian likelihood (X)) = =((P (X), o²), where a the model's observation noise parameter (Kendall et al.j 2018). In our case, our model output is composed of two vectors, a discrete output yı for the CE loss and continuous output y2 for the CL loss, which leads to the total minimization objective:
L(W, 02, = logplyi c2 w (x)) = -softmax (y1 W (X)) .)) + log. N (X), og))
= -logg(yy; = </ (x), + 20
y2 - W (x) 2 log 02
(5)
1
CCE X) + CoL (W, X) log σ1 + log 02.
20
203
The final objective is to minimize Equation 5 with respect to and 02, thereby learning the relative weights of the losses (i(W) and C2( in an adaptive manner (Kendall & Gal 2017: Kendall et al. 2018). The hyperparameters 01 and 02 are trainable parameters, initially set to one, indicating equal weighting for both. The Appendix AA.4 provides an illustrative demonstration of the weighting mechanism applicd to both loss functions for all tasks on the CIFAR-10, CIFAR-100, ImageNet100, and GNSS datasets. As the noise parameters 01 and d2 increase where σ1 corresponds to the variable y1 and 02 to Y2 the weight assigned to LCE and LCL, respectively, decreases. Such an adjustment serves as regularizer for the noise term. This trend is observable at the early period of cach task, as new classes introduce more noise, and hence, the weighting of both loss functions decreases. In general, both loss functions have converged after 200 epochs.
dynamically balances the contributions of the CE loss (LCE) and the CL loss (Lez) using Bayesian uncertainty estimation. It introduces two deviation parameters, 01 and Ø2, which represent. the uncertainty associated with each loss term. The function computes weighting factorr as 1/(2ff) and 1/(2o3), ensuring that losses with higher uncertainty contribute less to the total loss. Additionally, a regularization term, log(a,) + log(o2), is included to prevent the deviation parameters from collapsing to zero. The final weighted loss is then returned along with the updated values of 0₁ and Ø2, which are adjusted using gradient-based optimization. This adaptive weighting mechanism ensures that the model prioritizes more reliable loss terms during training, improving robustness in an incremental learning setting.
The function Bayesian_weii
The function update_ extrpear_epa is responsible for maintaining and updating the exemplar memory used in CIL. Given the current task index t and the existing exemplar set E 2, 000, the function determines the number of exemplars per class E E where k is the nunber of classes at the current stage. It then selects subset of training samples for each class using predefined strategy called Herding. The goal is to retain the most representative exemplars while adhering to memory constraints.
4 Experiments
For fair comparability, we conduct experiments on four distinct image datasets: the CIFAR-10, CIFAR- 100, and ImageNet100 datasets and a applications pertinent to continual learning, specifically within the realm of image classification tasks.
dataset. Consequently, we encounter a diverse array of
4.1 Image Classification: CIFAR-10, CIFAR-100, & ImageNet100
The CIFAR-10 KKrizhessyy & Hinton 2009) dataset consists of 60,000 colour images of size 32 x 32. The goal to classify 10 distinct classes (i.c., airplanc, automobile, bird, cat, deer, dog, etc.), with 6,000 images per class. The dataset is split into 50,000 training and 10,000 test images. We train four tasks: task consists of the classes 0, 1, 2, and 3, task 2 consists of the classes 4 and 5, task 3 consists of the classes
7

---

Published in Transactions on Machine Learning Research (03/2025)
(a) 0.
(b) 1.
(c) 2.
(d) 3.
(e) 4.
(f) 5.
(8) 6.
(h) 7.
(1) 8.
(i) 9.
(k) 10.
Figure 3: Exemplary spectrogram samples without interference (classes 0 to 2) and with interference (classes
3 to 10) between intensity [0, -90] with logarithmic scale.
and 7, and task 4 consists of the classes 8 and 9. The CIFAR-100 Krizhovsky & Hinton, 2009) dataset has 100 classes containing 600 images each, and hence, we train 10 classes per task with 10 tasks in total. ImageNet100 a subset of the larger ImageNet dataset, containing 100 carefully selected classes with 1,300 images per class, maintaining balanced distribution. It is commonly used for benchmarking due to its reduced computational requirements while preserving the diversity of ImageNet (Deng et al. 2009).
4.2 Interference Classification: GNSS Snapshots
The GNSS-based (Ot et al. 2024) dataset contains short, wideband snapshots in both E1 and E6 GNSS bands. The datasct was captured at a bridge over a highway. The sctup records 20 ms raw IQ (in-phase and quadraturo-phase phase) snapshots triggered from the energy with a sample rate of 62.5 MHz, an analog bandwidth of 50 MHz and 8 bit bit-width. Further technical details can be found in (Brieger et al. 2022 Heublein et al. 2024a 2025 Figure 3 shows exemplary snapshots of the spectrogram. Manual labeling of these snapshots has resulted in 11 classes: Classes () to 2 represent samples with no interferences, distinguished by variations in background intensity, while classes 3 to 10 contain different interferences such as pulsed, noise, tone, and multitone. For instance, Figure 3k showcases a. snapshot containing a potential chirp jammer type. The dataset's imbalance of 197,574 samples for non-interference classes and between to 79 samples per interference class, emphasizing the under-repressntation of positive class labels. The goal to adapt to new interference types with continual learning. The challenge lies in adapting positive class labels with only limited number of samples available. We partition the dataset into 64% training set, 16% validation set, and a 20% lest set split (balanced over the classes). We train five tasks: lask consists of the classes 0, 1, and 2, task 2 consists of the classes 3 and 4, task 3 consists of the classes 5 and 6, task 4 consists of the classes 7 and 8, and task consists of the classes and 10.
5 Evaluation
Table 1: Evaluation results for C-FAR-100.
Final
Initially, present the results obtained through our BLCL. approach, in comparison with state-of- the-art methods, on both the CIFAR, the Ima- geNet, and the GNSS datasets. Subsequently, we conduct an analysis involving confusion matrices, model embeddings, cluster analysis, and paramo- ter counts. Throughout, we provide the accuracy achieved after each task, the average accuracy over all tasks (Acc.), well as the F1-score and F2-score. For our hardware and training setup, see the Appendix A.5
Method
Acc.
F1
F2
DB
CH
Finetune FWC KKrrkaariic et
26.41
0.01
0.03
4.34
2017
35.81
0.20
0.20
3.11
77
LwF Li & Hoiem (2018)
43.62
0.15
0.17
2.99
72
iCarl Rebulli et al 201
54.83
0.35
0.33
2.67
63
Replay Batchif
1907
52.90
0.34
0.31
2.77
68
MEMO Zhou et al. DER. YYan el. al. 2021
2023b
w/o
68.86
0.57
0.57
3.55
55
67.17
0.58
0.58
3.37
61
FOSTER (Wang et al.
2022
63.51
0.50
0.50
3.52
74
BLCCL CF
70.14
0.57
0.57
3.58
57
BLCLg/+]- CL
5.24
0.01
0.02
5.32
76
BLCLg]a) w/
70.43
0.58
2.59
102
0.50
0.007
0.007
0.019
1.15
Evaluation Results. The evaluation results for the CIFAR datasets are summarized in Table 1 and Ta- ble 2 while those for the GNSS dataset are presented in Table 3 On the CIFAR-10 dataset, MEMO and DER achieve the highest classification results for each task, with the highest average accuracy of 84.78%. respectively 86.77% iCarl also demonstrates strong performance with an average accuracy of 84.47%. How- ever, MEMO with weight averaging slightly decreases results to 84.78%, leading us to train BLC.. without
8

---

Published in Transactions on Machine Learning Research (03/2025)
Table 2: Evalustion results for baseline methods and our Bayesian learning-driven (B) contrastive loss functions for the CIFAR-10 Krizhevsky Hinton, 2009) dataset. We indicate the weighting with (w/) and without (w/o) averaging. Underlined is the best baseline method and bold is the best method in total. We present the final Rovidi- (DB) (4) and Kalinikii-Hrbbas (CH) (t) scores.
Loss/
Task 1
Task 2
Task 3
Task 4
Final
Method
Weighting
Acc.
Acc.
Acc.
Acc.
Acc.
F1
F2
DB
CH
Pinetune
CE
95.20
32.08
24.80
19.80
43.22
0.067
0.110
7.29
1,716
EWC KKrkkpatrick al
2017
95.20
54.30
47.39
36.10
58.24
0.303
0.324
2.12
1,365
LwF
Li
Hotem 2018
95.48
63.55
62.80
56.89
69.68
0.556
0.557
2.14
ICarl
Rebulli et al. 2017
95.35
81.85
80.12
79.57
84.47
0.792
0.791
1.28
1,907
Replay [Ratclil]
1997
95.35
78.95
76.69
76.72
81.93
0.766
0.764
1.18
2,327
MEMO Zhouu el al.
2023b
w/o
CE
96.00
80.68
80.79
81.68
84.78
0.810
0.812
1.78
2,327
DER Yan et 2021
95.32
84.23
83.70
83.48
86.15
0.833
0.834
1.38
1,822
FOSTER Wann
2022
95.95
69.58
66.90
74.82
0.637
0.645
2.20
1,225
BLC...,,1...4) w/o
CE
92.82
82.47
82.18
82.13
84.9
0.816
0.817
1.74
1,353
BLC.I.(1112.2))) w/o
CL
50.55
36.98
28.77
21.66
34.49
0.068
0.108
7.89
787
BI.L..(1..1.)) w/o
CE, B, CL
95.58
84.50
82.55
83.62
86.56
0.834
0.834
0.79
5,085
+
0,200
0.422
0.326
0.508
0.246
0.005
0.005
0.010
81.34
BLCLsp,1.2.2) W/o
CE, B, CL
95.58
84.50
83.45
82.76
86.57
0.824
0.825
0.81
4,688
0.200
0.265
0.373
0.194
0.099
0.002
0.002
0.007
99.107
Table 3: Evaluation results for the GNSS Ott al. 2024 dataset.
Loss/ Weighting
Task
Task 2
Task 3
Task 4
Task 5/
Final
Method
Acc.
Acc.
Ace.
Acc.
Acc.
Acc.
F1
F2
DB
CH
Finetune
96.06
0.04
0.04
0.06
0.02
19.24
0.033
0.048
1.48
7,958
EWC KKirkpttrick et al
2017
95.99
0.49
0.04
0.05
0.04
19.32
0.071
0.072
1.34
4,799
LwF
Li Hoiem 2018
96.40
52.39
17.49
3.50
2.52
34.46
0.026
0.037
1.97
4,565
iCarl
Rebultt et al 2017
94.75
95.43
95.23
95.14
95.14
95.14
0.601
0.651
0.81
29,076
Replay Ratelil
1507
94.55
95.97
95.49
95.50
95.75
95.45
0.594 0.661
0.96
27,622
MEMO (Zhou al 2023b w/o CE
95.90
95.41
94.99
95.14
95.19
95.34
0.579 0.651
1.15
14,066
DER Yau ct al. 2021
96.08
95.73
95.61
95.67
95.
95.77
0.562 0.642
1.15
8,828
FOSTER
Wann
2022
96.24
93.67
92.30
91.09
93.53
93.36
0.401 0.479
1.53
10,402
BLCL2322222221 W/
CE
96.39
95.87
94.89
94.37
94. 26
95.35
0.551 0,600
1.26
12,553
B1.CT.222.22222222 w/
CL
66.61
8.29
3.11
2.28
18.478
0.024 0.024
1.98
5,567
BIC.I.111..1.1II.. w/
CE, 0.9, CT.
96.03
95.99
95.54
95.61
95. 17
95.67
0.619 0.678
1.04
15,868
BLCLo)1111111] w/
CE. B, CL
95.12
95.35
94.58
94.39
93.74
94.63
0.547 0.596
0.79
98,877
BI.L222222222222 BLCI222222222 w/
CE, 0.9, CL
96.21
95.37
95.13
95.44
96.25
95.68
0.612 0.648
0.94
28,598
CE, B, CL
96.74
96.16
95.75
95.96
96.18 18
96.16
0.601
0.662
0.78
163,420
0.135
0.252
0.123
0.250
0.298
0.244
0.021
0.023
0.007
386.05
weight averaging. Figure shows results for different BLCL[aa architectures. Increasing the number of lay- ers in the specialized component from four (84.78%) to six (84.96%) and eight (85.02%) results in improved accuracy. Consequently, follow-up experiments conducted with the BLCLs|s model. Evaluation of dy- namic specialized components reveals that smaller blocks are preferable for the first task, as BL BLCLs.......) and
95
Tasi
Task 4
97
Task 1
Task2
Task
Task 6
Task
Final
Ancurany (86)
Accuracy [%)
96
95-
06.000
40,000
00.0000
82222
013333
06.12.331
M60030
00343
80003
m00.03.38..0
10...000
m.112a
N6. 1334
200
Figure& 4: Evaluation of BLCLgg) the CIFAR-10 (a) and GNSS (b) datasets.
9

---

Published in Transactions on Machine Learning Research (03/2025)
Table 4: Evaluation results for the ImageNet110 Deng et al.
2009 dataset.
Task 1
Task 2
Task
Task
Task
Task 6
Task 7
Task
Task
Task 10
Final
Method
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
Acc.
F1
F2
DB
CH
Finetune
79.60
44.30
25.07
18.50
15.32
14.90
13.29
10.88
9.36
9.20
24.04
0.02
0.38
3.99
11.34
EWC
85.20
47.30
35.13
26.25
21.20
17.60
14.20
14.25
13.93
10.96
28.60
0.06
0.07
2.74
26.87
LWF
87.00
75.40
66.40
52.20
43.24
36.23
31.06
30.52
28.91
23.24
47.42
0.18
0.19
3.24
17.18
iCarl
86.00
86,20
82.13
78.60
77.72
71.03
63. 46
59.08
55.73
54.26
71.42
0.55
0.55
2.39
48.57
Replay
86.00
86.30
81.00
78.65
77.92
70.60
63.03
59.68
66.22
54.89
71.22
0.56
0.55
2.47
43.18
MEMO
89.40
88.50
84.20
83.60
82.12
79.30
75.23
75.00
72.00
71.00
81.04
0.71 0.71
3.03
37.86
DER
80.20
81.90
76.33
73.40
70.40
69. 80
69. 23
69.92
69.78
67 62
72.86
0.72
0.72
2.47
49.29
FOSTER
81.60
81.6
71.47
67.65
62.40
61.10
62. 26
61.98
60.51
60.46
69.10
0.60
0.60
3.04
46.11
BLCL2(o..... 0
91.80
88.30
85.40
85.30
82.92
79.17
74.43
72.55
68.78
68.60
79.73
0.68
0.68
2.28
59.10
(Cound Truill
Oliss (Sooard Truthl
12
03-
8
10
(Prediction)
Class (Prediction)
(Poedlction)
(Prediction)
(a) MEMO.
(b) MEMO.
(c) MEMO.
(d) MEMO
(Gound Truth]
Cloos (Croune Twuth!
(Groand
10
10
(Prediction
Class (Prediction)
[Prediction]
[Prediction
(e) BLC
(I) BLCL
(g) BLCLoaaaz2
(h) BLCL2
Figure 5: Confusion matrices for the CIFAR-10 (a and c), CIFAR-100 (b and f), GNSS (c and g). and ImageNet100 (d and h) datasets. For confusion matrices of all methods, refer to the appendix.
BLCLs[4....] outperform BLCLsJo.... Larger layers for tasks 3 and further improve results; for instance, BLCL2JI..23344 achieves 85.11% However, simply adding the contrastive loss with naive weighting of 0.9 decreases results, motivating the adoption of automatic Bayesian weighting. For all the architectures tested, including BICCTg/..112)) BLCTT B...C.g.,.,..33) and BLC..g1,,33441 our Bayesian loss outperforms the baseline model. Consequently, our method achieves 86.57% with high robustness across all tasks, signifi- cantly outperforming MEMO with 84.78% and DER with 86.15%. On the CIFAR-100 dataset, BLCL with & = for all the 10 (asks, achieves 70.43% outperforming all stale-of-the-arl met hods On the GNSS dataset, Replay and DER achieve the highest accuracy of 95.45%, respectively 95. 95.77% although the F2-score of 0.661 is low due dataset underrepresentation. We outperform MEMO (95.34% average accuracy) with weight averaging, achieving accuracy of 95.67%. Combining this with the contrastive loss further increases accuracy to 96.1.%. The F2-score can be improved to 0.678 by reducing the false negative rate. In Table B. the Bayesian approach underperformed compared to fixed 0.9 weighting in this case but offers greater adaptability across datasels and lask complexities. A fixed weighting assumes constant loss importance, yet our analysis shows this ratio evolves dynamically. Using a fixed CL weight of 0.9 (and 0.1 for CE) may be suboptimal for complex tasks, as CE loss remains uncertain longer while CL loss stabilizes carlier.
Confusion Matrices. Figure illustrates the confusion matrices subsequent to the final task to all test classes within the CIFAR, ImageNet, and GNSS datasets, employing MEMO (Zhou al. 2023b) and our
10

---

Published in Transactions on Machine Learning Research (03/2025)
Cluss Claso
Close Closa
(a) Replay (Ratcliff 1997
(b) MEMO Zhou et al. 2023b
(c) BLCU3
(d) Replay Ratcliff 1997
(e) MEMO (Zhou et al.
2023b
(f) BLCL
Clesse
Clase Clask
Cloce
Class:
(g) Replay (Ratcliff 1997).
(h) MEMO Zhou et al. 2023b
ii) BLCLz2222223
(i) Replay RRtellff 1997
(k) MEMO
Zhou et al. 2023b
(I) BLCL20....0
Figure 6: t-SNE plots for the CIFAR-10 (a to c), CIFAR-100 (d to f), GNSS (g to i), and ImageNet100 (j
k) datasets.
proposed technique denoted as BLCL5 and BLCL222.222222) with Bayesian weighting, respectively. See the appendix, for confusion matrices on all methods. On the CIFAR-10 dataset, MEMO demonstrates significant performance superiority over Replay for specific classes 0, 1, 4, 5, 6, and 7 (refer to Figure 13e compared Figure Notably, Replay exhibits pronounced confusion between classes 8 and well as the remaining classes, misclassification mitigated by MEMO. Additionally, BLCL achieves higher true positive rate than MEMO for classes 0, 2, 4, and 9, thereby attaining average accuracy of 86.57%, surpassing
11

---

Published in Transactions on Machine Learning Research (03/2025)
MEMO's 's 84.78% (sce Figure 5el. Our approach effectively mitigates misclassifications particularly between classes 3 and 6. On the GNSS dataset, a distinct resemblance exists between classes and other classes (refer to Figure 3 resulting in misclassifications. While MEMO exhibits a higher false positive rate compared to Replay, their false negative rates remain equal, leading to Replay's superior average accuracy of 95.45% versus MEMO's 95.34% (compare Figure 16e with Figure 5 Leveraging enhanced representation learning via contrastive loss, our BLCL method effectively separates positive classes (3 to 10) into distinct clusters, thus reducing the false positive rate and yielding a higher F22score of 0.678 while simultaneously decreasing the false negative rate (refer to Figure 5g).
Cluster Analysis. Subsequently, we compare the Davies-Bouldin (DB) (Davies & Bouldin 1979: and Calinski-Harrassz (CH) (Caliáski & Harabasz 1972 scores. DB is defined as the mean similarity measure between each cluster and its most similar cluster. This measure of similarity is the ratio of within-cluster distances to between-cluster distances. Consequently, clusters exhibiting greater separation and lesser dis- persion will yield a superior score (Halkidi et al.) 2001 Ros et al. 2023 Thrun 2011). CH is a variance-ratio criterion, where higher CH score relates to a model with better defined clusters, and is defined as the ratio of the sum of inter-clusters dispersion and of intra-cluster dispersion for all clusters. Table I to Table 3 summarizes the scores attributed Finetune, EWC, LwF, iCarl, Replay, MEMO, DER, FOSTER, and BLCL. Across all three datascts, BLCI demonstrates a lower DB score and higher CH score in comparison to the baseline methods. This observation supports the assertion that BLCL results in a representation characterized by lower inter-class distances and enhanced clustering efficacy.
Embedding Analysis. The feature embedding of the last convolutional layer, with output size of 512, is depicted in Figure 6 This embedding is generated using t-distributed stochastic neighbor embedding (I-SNE) (vvan der Maaten & Hinton 2008) with a perplexity of 30, an initial moment.um of 0.5, and a final momentum 0.8. On the CIFAR-10 10 dataset, the Replay method (Figure 6a) and MEMO method (Figure 6b) demonstrate poor scparation of clusters, resulting in the overlap of classes 2, 3, and 4, thereby leading to misclassification. This observation aligns with the findings presented carlier using confusion matrices (refer to Figure 5a and 13e]. Our proposed BLCCL method (Figure (c), which incorporates contrastive loss, significantly reduces the intra-class distance and increases inter-class distance. This is also evident for the CIFAR-100 dataset (refer to Figure 6d to 6f However, some overlap persists, particularly for classes 0, 1, and 3, resulting in a higher rate of misclassification Similar trends are observed in the GNSS dataset, where both Replay and MEMO methods exhibit overlapping samples with class 0 (Figure 6g and Gh), while the BLCL method only forms a small cluster with specific samples the center of the embedding (Figure 6i).
6 Conclusion
In the context of the continual learning (ask, we proposed a method characterized by dynamic specialized component. This component combines the CE loss with a prototypical contrastive loss by utilizing Bayesian learning principles. By employing dynamic loss weighting, our method effectively achieves a fcature repre- sentation with reduced intra-class disparity and higher inter-class distance. Evaluations conducted on the CI, CIFAR-100, ImageNet100, and GNSS datasets demonstrate that our proposed technique out- performs the state-of-the-
Replay, MEMO, and DER. Specifically, our method achieves accuracy rates of up to 86.57% on CIFAR-10, 70.43% on CIFAR-100, 79.73% on the ImageNet100, and 96.16% on the GNSS dataset with improved Davies-Bouldin and Calinski- scores.
Acknowledgments
This work has been carried out within the DARCII project, funding code 50NA2401, supported by the German Federal Ministry for Economic Affairs and Climate Action (BMWK), managed by the German Space Agency at DLR. and assisted by the Bundesnetzagentur (BNetzA) and the Federal Agency for Cartography and Geodesy (BKG).
12

---

Published in Transactions on Machine Learning Research (03/2025)
References
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Woo Woo Ha, and Jonghyun Choi. Rainbow Memory: Continual Learning with Memory of Diverse Samples. In TEEE/CVE Intl. Conf. on Computer Vision and Pattern Recognilion (CVPR) Nashville, TN, June 2021. doi:
2021.00812.
Jihwan Bang, Hyunseo Koh, Seulki Park, Hwanjun Song, Jung-Woo Ha, and Jonghyun Choi. Online Continual Learning on Contaminated Data Stream with Blurry Task Boundaries. In TEEE/CVE Trall. Conf. on Computer Vision and Pottern Recognition (CVPR), New Orleans, LA, June 2022. doi:
0010226688222..00900 2022.00906.
Tobias Brieger, Nisha I.. Raichur, Dorsaf Jdidi, Felix Ott, Tobias Foigl, Johannes Rossouw van der Merwe, Alexander Rügamer, and Wolfgang Felber. Multimodal Learning for Reliable Interference Classification in GNSS Signals. In Proc. of the Intl. Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS+) pp. 3210-33000 3234, Denver, CO, September 2022. doi:
18586.
Tadeusa Caliñísk and J. Harabasz. Dendrite Method for Cluster Analysis. In Communications in Stalisties. volume 3(1), pp. 1-28, September 1972. doi:
Hyuntak Cha, Jaeho Lee, and Jinwoo Shin. Co Contrastive Continual Learning. Tn Iril. Conf. on Computer Vision (ICCV) Montreal, QC, October 2021. doi: 10.1109/ICCV48992 4BCCC.V892222222 2021.0093..
Sungmin Cha and Taesup Moon. Sy-CON: Symmetric Contrastive Loss for Continual Self- Superviid Rep- resentation Learning. In ar.:ix:2006000000 05101, June 2023.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A Simple Framework for Con- trastive Learning of Visual Reprosentations. In Intl. Conf. on Machine Learning (ICML). volume 149, pp. 1597-1607, July 2020. doi: (55655/332249383555508
Weihua Chen, Xiaotang Chen, Jianguo Zhang, and Kaiqi Huang. Beyond Triplet Loss: A Deep Quadru- plet Network for Person Re-identification. In IEEE/CVE Intl. Conf. Computer Vision and Pattern Recognition (CVPR) Honolulu, H1, July 2017. doi: 10.1109/CVPPP 20P...00.....
Qing Da, Yang Yu, and Zhi-Hus Zhou. Learning with Augmented Class by Exploiting Unlabeled Data, In Proc. of the Intl. Conf. on Artificial Intelligence (AAAT), volume 28, 2014.
David L. Davies and Donald W. Bouldin. A Cluster Separation Measure. In TEEE Trans. on Pattern Analysis and Machène Intelligence (TPAMI). volume (A)) pp. 224 227, April 1979. doi: 10.1109/TPAMI. 1979.4766909.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: Large-scale Hierarchical Image Database. In LEEE/CVF Intl. Conf. on Computer Vision and Pattern Recognition (CVPR) Miami, FL., June 2009. doi: 10. 2000//CVR..22099220000
Thanh-Toal Toan Do, Toan Tran, Ian Reid, Vijay Kumar, Tuan Hoang, and Gustavo Carneiro. A Theoretically Sound Upper Bound on the Triplet Loss for Improving the Efficiency of Deep Distance Metric Learning. In TEEE/C/EC /CVV Intl. Conf. on Computer Vision and Pattern Recognition (CVPR) Long Beach, CA, June 2019. doi:
Joff Donahua, Yangqing Jia, Oriol Vinyals, Judy Hoffmann, Ning Zhang, Eric Tzeng, and Trevor Darrell. DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition. In Journal for Machine Learning Research (JMLR) volume 32(1), pp. 647-655, 2014.
Yaron Fairstein, oren Kalinsky, Zohar Karnin, Guy Kushilevitz, Alexander Libov, and Sofia Tolmach. Class Balancing for Efficient Active Learning in Imbalanced Datasets. In Proc. of the Linguistic Annotation Workshop (LAW_XVIII) XVIII] pp. 77-86, St. Julians, Malta, March 2024.
Nishant S. Gaikwad, Lucas Heublein, Nisha L. Raichrr, Tobias Feigl, Christopher Mutschler, and Felix Ott. Foderated Learning with MMDdbased Early Stopping for Adaptive GNSS Interference Classification. In arXiv preprint or Xix::2410. 15681e2 December 2024.
13

---

Published in Transactions on Machine Learning Research (03/2025)
Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich Feature Hicrarchies for Accurate Object Detection and Semantic Segmentation. In TEEE/CVE Iull. Conf. on Computer Vision and Pattern Recognition (CVPR) pp. 580 587, 2014. doi: 10.1109/ D0.109/CCPR.2001 2V1R..004.81.
Rick Groenendijk, Sezer Karaoglu, Theo Gevers, and Thomas Mensink. Multi-Loss Weighting with Co- efficient of Variations. In IEEE/CVF Winter Conf. on Applications of Computer Vision (WACV) pp. 1468-1477 2021. doi: 0.1 M.009/WAC44630.220
Maria Halkidi, Yannis Batistakis, and Michalis Vazirgiannis. On Clustering Validation Techniques. In Journal of Intelligent Information Systems, volume 17, pp. 107 145, December 2001. doi: 10.1023/A:
1012801612483.
Arman Hasanzadeh, Mohammadreza Armandpour, Ehsan Hajiramezanali, Mingyuan Zhou, Nick Duffield and Krishna Narayanan. Bayesian Graph Contrastive Learning. In arXiiv21122000 078223 August 2022.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep Residual Learning for Image Rocognition. In TEEE/CH /CV Inll. Conf. on Compuler Vision and Pallern Recognition (CVPR) Las Vegas, NV, June 2016. doi: 10.1109/CVPR.2016..0
Lucas Heublein, Tobias Feigl, Alexander Rügamer, and Felix Ott. Achieving Generalization Orchestrating GNSS Interference Monitoring Stations Through Pscudo- Labeling.. In DGON Positioning and Navigation for Intelligent Transport Systems (POSNAV) October 2024a.
Lucas Heublein, Nisha L.. Raichur, Tobias Feigl, Tobias Brieger, Fin Heuer, Lennart Asbach, Alexander Rügamer, and Felix Ott. Evaluation of (Une)Supervieed Machine Learning Methods for GNSS Interference Classification with Real-Worle World Data Discrepancies. In Proc. of the Intl. Technical Meeting of the Satellite pp. 1293, Baltimore, MD, September 2024b.
Division of the Institute of Navigation (ION GNSS+) doi: 10.33012 2333022222241110 22//2.222...88
Lucas Houblein, Tobias Feigl, Thorsten Nowak, Alexander Rügamer, Christopher Mutschler, and Felix Ott. Evaluating ML Robustness in GNSS Interference Classification, Characterization & Localization. In Xio preprint ar XxX::2409.15114 February 2025.
Xin Huang, Yuxin Peng, and Mingkuan Yuan. MHTN: Adver--adar Hybrid Transfer Network for Cross-Modks Retricval. In Trans. Cybernetics volume 50(3), pp. 1047-1059, 2020. doi: 10.1109/ JCYB.2018.2879985
Zixuan Ke, Bing Liu, Hu Xu, and Tei Shu. CLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks. In Intl. Conf. on Empirical Methods in Natural Language Processing (EMNLP) pp. 6871-6883, November 2021. doi: 10.18653/vl /tt//2222..mmmmmmee
Alex Kendall and Yarin Gal. What Uncertaintics Do We Need in Bayesian Deep Learning for Computer Vision? Tn Advances in Neural Information Processing Systems (NTPS) Long Beach, CA, 2017.
Alex Kendall, Yarin Gal, and Roberto Cipolla. Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. In (EEE/CVI Intl. Conf. on Computer Vision and Pattern Recognition (CVPR) 2018. doi: 20000//VVP..02..00000
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Trabska-Barwinstee Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming Catastrophic Forgetting in Neural Networks. Tn Applied Mathematies volume 114(13), pp. 3521-3526, March 2017. doi: 10.1073/pman pnas. 1611835114 14.
Andreas Klaß, Sven M. Lorenz, Martin W.
David Rügamer, Bernd Bischl, Christopher Mutschler, and Felix Ott. Uncertainty-wware Evaluation of Timo-Serics Classification for Online Hand- writing Recognition with Domain Shift. In [JCAI-ECAI Intl. Workshop on Spotio- Temporal Reasoning and Learning (STRL), volume 3190, Vienna, Austria, July 2022.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Hyunsco Koh, Dahyun Kim, Jung- Woo Ha, and Jonghyun Choi. Online Continual Learning Class Incremental Blurry Task Configuration with Anytime Inference. In /nll. Conf. on Learning Represeniations (ICLR), 2022.
Alex Krizhevsky and G. Hinton. Learning Multiple Layers of Features from Tiny Images. In Report, University of Toronto, Toronto, Ontario, April 2009.
Technical
Iia Kuzborskij, Francesco Orabona, and Barbara Caputo. From to N+1: Multiclass Transfer Incremental Learning. In TEEE/CVE Intl. Conf. on Computer Vision and Pattern Recogration (CVPR). Portland, OR, June 2013. doi: 10.1109 CCVPR.201..43....
Matthias De Lange, Rahaf Aljundi, Mare Masana, Sarah Parisot, Xu jia, Ales Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A Continual Learning Survey: Defying Forgetting in Classification Tasks. In IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI) volume 44(7), July 2022. doi: 1109/PPAMI 2021 3357446.
S. Lewandowsky and S.-C. Li. Catastrophic Interference in Neural Networks: Causes, Solutions, and Data. In F. N. Dempster & C. J. Brainerd (Eds.), Interference and Inhibition in Cognition pp. 329 0011,,66,111 1995. doi: 10.1016/BB) B978- 1112208930-/5001111
Zhizhong Li and Derek Hoiem. Learning Without Forgetting. In IEEE Trans, on Pattern Analysis and Machine Intelligence (TPAMI), volume 40(12), pp. 2935-2947, December 2018 doi: 10.1109 TPAMI. 2017.2 1077.2773081.
Lilang Lin, Jiahang Zhang, and Jiaying Liu. Bayesian Contrastive Learning with Manifold Regularization for Sele-Supervised Skeleton Based Action Recognition. In IEEE Intl. Symposium on Circuits and Systems (ISCAS) Monterey, CA, May 2023. doi: N0.1109/TSCASAA
Bin Liu, Bang Wang, and Tianrui Li. Bayesian Self- Superviue Contrastive Learning. In arX January 2024.
Zilin Luo, Yaoyao Liu, Bernt Schiele, and Qianru Sun. Class Incrementlltttll Exemplar Compression for Class- Incremental Learning. In TEEE/CVE Inil. Conf. on Computer Vision and Pattern Recognition (CVPR). Vancouver, BC, June 2023. doi: D..119/CVVP52272 200.3.99202..0104...
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online Continual Learning in Image Classification: An Empirical Survey. In Neurocomputing volume 469, pp. 28-51, January 2022. doi:
2021..0.02111
Harshith Manjunath, Lucas Heublein, Tobias Feigl, and Felix Ott. Multimodal-to-Test Prompt Engincering in Large Language Models Using Feature Embeddings for GNSS Interference Characterization. In ar X preprint ar Xio:2501.05077 January 2025.
Mare Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost van de Weijer. Mlass-nncrementa Learning: Survey and Performance Evaluation on Image Classification. In IEEE Trans. on Pattern Analysis and Machine Intelligence (TPAMI), volume 45(5), May 2023. doi:
Felix Ott, David Rügamer, Lucas Heublein, Bernd Bischl, and Christopher Mutschler. Auxiliary Cross-Modal Representation Learning with Triplet Loss Functions for Online Handwriting Recognition. In TEEE Access volume 11, pp. 94148 94172, August 2023. doi: 10.1109/ACCESS 2023.3310819.
Felix Oit, Lucas Heublein, Nisha Lakshmana Raichur, Tobias Feigl, Jonathan Hansen, Alexander Rügamer, and Christopher Mutschler. Few-Shot Learning with Uncertainty- based Quadruplet Selection for Interfer- ence Classification in GNSS Data. In IEEE Intll Conf. on Localization and GNSS (ICL-GNSS). Antwerp, Belgium, June 2024. doi: 10.1109
Yiming Qin, Huangjic Zheng, Jiangehao Yao, Mingyuan Zhou, and Ya Zhang. Models. In JEEE/CVF Intl. Conf. on Computer Vision and Pottern Recognition (CVPR), pp. 18434- 18443, 2023.
Diffusion
15

---

Published in Transactions on Machine Learning Research (03/2025)
Nisha L. Raichur, Tobias Brieger, Dorsaf Jdidi, Tobias Feigl, Johannes Rossouw der Merwe, Biren- dra Ghimire, Felix Ott, Alexander Rügamer, and Wolfgang Felber. Machine Learning-assisted GNSS Interference Monitoring Through Crowdsourcing, In Proc. of the Intl. Technical Meeting of the Satellite Division of the Institute of Navigation (ION GNSS+) pp. 1151-1175, Denver, CO, September 2022. doi: 10.33012/2022.198
Nikhil Rasiwasia, Jose Costa Pereira, Emanuele Coviello, Gabriel Doyle, Gert R.G. Lanckriel, Roger Levy, and Nuno Vasconcelos. A New Approach to Cross- Modal Multimedia Retrieval. In Proc. of the ACM Intl.
Conf. on Multimedia (ACMMM) pp. 251-260, October 2010. doi: 10. 10.1145/ 00.1145/18739511 18735111187399
R. Rateliff. Connectionist Models of Recognition Memory: Constraints Imposed by Learning and Forgetting Functions. In Psychological Review volume (7))), pp. 285 308, April 1997. doi: 10.1037/0033 1t////0003322xxx990 295x.97.2. 285.
Sylvestre- Alvise Rebuffi, Alexander Kolesníkov, Georg Sperl, and Christoph H. Lampert. iCaRL: Incremental Classifier and Representation Learning. Tn TEEE Tril. Conf. on Computer Vision and Pattern Recognétion (CVPR) pp. 2011 2010, Honolulu, H1, July 2017. doi: D0.109/CVPR.2017.555
David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. Experience Replay for Continual Learning. In Advances in Neural Information Processing Systems (NIPS) 2019.
Frédéric Ros, Rabia Riad, and Serge Guillaume. PDBI: A Partitioning Davies-Bouldin Index for Clustering Evaluation. In Neurocomputing volume 528, pp. 178-199, April 2023. doi:
0..00.....
Nikolaos Sarafianos, Xiang Xu, and loannis A. Kakadiaris. Adversarial Representation Learning for Text- to-Image Matching. In TEBE/CVE Inll. Conf. on Compaler Vision (ICCV) pp. 5814-56466 5824, Seoul, Korea, 2019. doi: 10.1109/ICCY 2019.00591.
Florian Schroff, Dmitry Kalenichenko, and James Philbin. FaceNet: A Unified Embedding for Face Recog- nition and Clustering. In TEEE/CVE Intl. Conf. on Computer Vision and Pattern Recognition (CVPR). Boston, MA, June 2015. doi: (1000//CVP..021.772950
Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical Networks for Fow-shot Learning. In Advances in Neural Information Processing Systems (NIPS) pp. 4080-4090, December 2017. doi: 10.5555/335 3294996.
3295163.
Mamatha Thota and Georgios Leontidis. Contrastive Domain Adaptation. In IEEE/CVE Intl. Conf. on Computer Vision and Pattern Recognition Workshops (CVPRW) Nashville, TN, June 2021. doi: 10.1109/
CVPRW53098. 202.00000...
Michael C. Thrun. Distance-based Clustering Challenges for Unbiased Benchmarking Studies. In Scientific Report volume 11(18988), 2011. doi: 10.1038 10.1038/s41598.. /0////41119--1-1111 021-99112---111-11111
Dennis Tschee--sc Peter Reimann, Holger Schwarz, and Bernhard Mitschang. Approach to Synthetic Data Generation for Imbalanced Multi-class Problems with Heterogeneous Groups. In Leclure Notes in Informatic (LNI), Gesellschaft für Informatik, pp. 329 351, Dresden, Germany, 2023. doi: 10.18420/ BTW2023-16.
Laurens van der Maaten and Geoffrey Hinton. Visualizing Data Using t-SNE. In Journal for Machine Learning Research (JMLR) volune 9(86,, pp. 2579 2605, November 2008.
Johannes Rossouw van der Morwe, David Contreras Franco, Jonathan Hansen, Tobias Brieger, Tobias Feigl, Felix Ott, Dorsaf Jdidi, Alexander Rügamer, and Wolfgang Felber. Low-Cost COTS GNSS Interference Monitoring, Detection, and Classification System. In MDPI Sensors, volume 23(7), 3452, March 2023. doi: 10.3390/223073452.
YunYun Wang, Da- Wei Zhou, Han-Jia Ye, and Dc-Chuan Zhan. FOSTER: Feature Boosting and Compression for
Learning. In TEEE/CVF Europ. Conf. on Computer Vision (ECCV) volume 13685, pp. 398-4988 414, October 2022. doi: 10.
23.
16

---

Published in Transactions on Machine Learning Research (03/2025)
Yunchao Wei, Yao Zhao, Canyi Lu, Shikui Wei, Luoqi Liu, Zhenfeng Zhu, and Shuicheng Yan. Cross- Modal Retrieval with CNN Visual Features: A New Baseline. In Trans. on Cybernelies volume 47(2), pp. 449 460, March 2016. doi:
Ross Wightman. Pytorch image models. http://gghhhkk.om/nggaaaa/oo/oon
2019.
Jiangwei Xie; Shipeng Yan, and Xuming He. General Incremental Learning with Domain-aware Categorical Representations. In IEEE/CVF Intl. Conf. on Computer Vision and Pattern Recognition (CVPR) pp. 14351-14360, 2022.
Shipeng Yan, Jiangwei Xie, and Xuming He. DER: Dynamically Expandable Representation for Class Incremental Learning. In ]EEE/CVF Inll. Conf. on Compuler Vision and Pallern Recognition (CVPR). pp. 3014 3023, 2021.
Lu Yu, BartHomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, and Yongmei Cheng. Semantic Drift Compensation for Class-Incremeni Learning. In JEEE/CVE Intl. Conf. on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, June 2020. doi: 10.1109/CVV /CCCC/22PRO0OO..O/ CNVPR/2600.02000000000
Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Deep Class- Incremental Learning: A Survey. In ar (ir:Xx:2002.0000 03648 February 2023a.
Da- Wei Zhou, Qi-Wei Wang, Han-Jia Ye, and De-Chuan Zhan. A Model or 603 Exemplars: Towards CemmrryEEfficient Class-Incrementar Learning. In Intl. Conf. on Learning Representations (ICLR), 2023b,
Fei Zhu, Xu-Yau Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. Protolype Augmentation and Self- Supervision for Incremental Learning. In IEEE/CVE Intl. Conf. on Computer Vision and Pattern Recog- nition (CVPR) Nashville, TN, June 2021. doi: 10.1109 10.1109/CVPRRAS CVPR/4437 0TT0H4664772021.055555 2021.00581.
Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Jun Zha. Self-Sustaining Representation Expansion for Non-Exempal Ilass-lncrementll Learning. In IEEE/CV /CFF Intl. Conf. on Computer Vision and Pattern Recognition (CVPR) New Orleands, LA, June 2022, doi: 10.1109/CVPR526SS 2022.00908.
17

---

Published in Transactions on Machine Learning Research (03/2025)
A Appendix
A.1 Data Augmentation
Due to the potential for highly unbalanced Training dataset, such as the interfered classes within the GNSS dataset, we address this issue by balancing each class through data augmentation techniques (Fairstein al, augment the data to ensure uniform distribution of 1,000 samples per class. We employ techniques provided by lorckeision. transforms, including color jittering with a brightness value of 0.5 and a hue value of 0.3, Gaussian blur with a kernel size of (5,9) and sigma values of (0.1,5.00 5.0), random adjustments of sharpness with a sharpness factor of 2, and random horizontal and vertical flipping with a probability of 0.4. Values are chosen after a hyperparameter search.
2024 Qin ct al. 2023 Troder-Tschechlov et al., 2023). Specifically,
A.2 Selection of the Backbone Model
To identify the optimal backbone architecture for the generalized blocks, we trained 110 vision encoder models from Hugging Face mang 2019) on the complete GNSS classification dataset. Figure provides a summary of the results across five different metrics: all classes, only positive interference classes, the binary score, the F2-score, and the F5-score. While various model sizes were evaluated, only the primary model names are visualized for clarity. Given the highly imbalanced nature of the dataset, all models yielded similar evaluation results for both the overall class evaluation and the binary metric. However, the ResNet18 model (visualized gray) outperformed all other models on the interference classes, prompting us to select ResNet as the backbonn model.
A.3 Dynamic Adjustment of Layer Structures
In deep neural networks, the deeper layers are primarily responsible for learning task-specific features, as they capture high-lovel abstractions crucial for distinguishing between different classos. ResNet comprises four specialized ResNet blocks, each containing four convolutional layers, as illustrated in Figure 2 In our approach, we dynamically modify only the final ResNet block, as it is the most specialized for task-specific feature extraction. To determine the appropriate depth for new task, we compute its class similarity to previously learned tasks. If the new classes are similar to the previously learned ones, assume that the earlier layers have already captured the essential features, thereby allowing us to reduce the number of active convolutional layers in the last block. We progressively remove layers one by one until the accuracy begins to decline, effectively performing a slight hyperparameter search to identify the maximum number of layers that can be removed without compromising performance. Conversely, for dissimilar classes, we retain all layers to ensure the model can learn new and distinet features. This adaptive adjustment facilitates efficient knowledge reuse while preserving the model's flexibility to accommodate new tasks. This approach was demonstrated to be effective for CIFAR-10 and GNSS, where two classes were introduced per task, allowing for gradual layer adjustment, However, in CIFAR-100, where ten classes were introduced per task, it proved more beneficial to retain all layers to maintain optimal accuracy.
100
1.0
Teporferuuce Cluss
Fi Soare
Bauzy
08
0.7
0.5
0.4
MAT Huchwol CONOUe
Crpili
SCA BeTNot
BEA Reshet
DEA Rochist
ESE Nubht
4
oG Si
oc cloaate
gMiver
Bainte
Budt BotcsBoNas BoNas. Hrne
MoattOrt Noaalees SE Rea 1
Soptllaanee Begite
Lanbea Peneeaiie
Figure 7: An evaluation of 110 vision encoder backbone models on the GNSS dataset has been conducted.
The top-performing model is highlighted in gray.
18

---

Published in Transactions on Machine Learning Research (03/2025)
205
201
59
130
350
300
259
360
50
100
150
200
224
50
150
200
226
(a) Task 1.
(b) Task
(c) Task 3.
(d) Task
Figure 8: Analysis of 0 02 and the classification accuracy for each task for the CIFAR-10 datasot.
(a) Task 1.
(b) Task 2.
(c) Task 3.
(d) Task 4.
(e) Task 5.
(f) Task 6.
(g) Task 7.
(lr) Task
(i)
Task
(i) Task 10.
Figure 9: Analysis of σı, 0g and the classification accuracy for each task for the CIFAR-100 dataset.
A.4 Bayesian Learning-driven Weighting
We next present an ablation study on the Bayesian learning-driven weighting of the cross-entropy (CE) and contrastive learning (CL) loss functions. Balancing CF. and CL loss functions is essential because they capture complementary aspects of the learning process, ensuring a more comprehensive optimization strategy. CE focuses on maintaining class-specific discrimination by minimizing prediction errors within predefined categories, which helps the model align its outputs with known labels. On the other hand, CL encourages general feature extraction by promoting meaningful representations across samples regardless of explicit labels. If only the CL loss is optimized, the model may prioritize similarity relationships without anchoring them to categorical class distinctions, potentially leading to suboptimal decision boundaries and degraded performance on classification tasks. By simultaneously applying both losses, the model benefits from both structured lcarning (CE) and reprosentation enhancement (CL). Dynamic weighting becomes necessary because the importance of each objective may shift throughout the training process. Early in training, stronger emphasis on CL may help develop better feature representations, while CE becomes more important docision boundaries are refined. Therefore, dynamic balance between the two losses ensures adaptive learning, improving generalization and robustness in the final model. The CE loss quantifies performance on classification tasks, with higher weight indicating that classification accuracy is either of greater importance or subject to higher uncertainty. Conversely, the CL loss facilitates the learning of meaningful feature representations by contrasting positive and negative pairs, where a higher weight signifies focus on developing robust embeddings. The posterior distributions of the weights assigned to each loss function provide key insights. Broad postcrior distributions indicate significant uncertainty regarding the relative importance of the losses. Furthermore, shifts in the weight distributions during training may reflect for instance, emphasizing feature representation learning in the early stages and
changing task priorities classification accuracy in later stages. When the CE loss diminishes and the model demonstrates confidence in classifications, the weighting of the CE loss may decrease, reallocating focus to the CL loss. Similarly, if the CL loss plateaus, its weighting may reduce to prioritize fine-tuning classification performance.
Figure 8 Figure g. Figure 10 and Figure I1 illustrate the dynamic weighting mechanism applied to the CE and CL loss functions throughout training for all tasks on the CIFAR-10, CIFAR-100, ImageNet100, and GNSS datasets, respectively. As σi and Ø2 the noise parameters associated with the variables yı and
19

---

Published in Transactions on Machine Learning Research (03/2025)
Goodre
(a) Task 1.
(b) Task 2.
(c) Task 3.
(d) Task 1.
(e) Task 5.
2
15
(f) Task 6.
g Task 7.
(h) Task
(i) Task
(i) Task 10.
Figure 10: Analysis of 11 0g and the classification accuracy for each lask for the IetgeNet100 dataset.
4,
(a) Task 1.
(b) Task 2.
(c) Task 3.
(d) Task 4.
(e) Task 5.
Figure 11: Analysis of oi and the classification accuracy for each task for the GNSS dataset.
Y2, respectively (introduced in Section 3.3 increase, the weights assigned to Lce and LCL correspondingly decrease. This adjustment. acts as a regularization mechanism for the noise terms. This phenomenon is particularly evident during the carly stages of each task, where the introduction of new classes increases noise levels, leading to a reduction in the weighting of both loss functions. Early in training, the a value for the CE loss is high, meaning that its weighting is relatively low. In contrast, the a value for the CL loss is lower, indicating a higher weighting. This suggests that, initially, the model prioritizes learning meaningful representations through contrastive learning rather than optimizing for classification accuracy. As training progresses, the for CE loss decreases while the for CL loss increases, leading to shift where the model places more emphasis on classification accuracy. The initial fluetuation in the weights, particularly for the CE loss, may signify the Bayesian framework's 'ss attempt to model and adapt to the high uncertainty in the early stages of training. Over time, the smoother trends in the weights imply that the uncertainty has diminished as the model gains confidence its predictions and representations. The decrease in weighting for both loss functions over epochs acts as an implicit regularization mechanism. This ensures that the model does not overfit to specific aspects of the loss functions and instead balances classification performance and representation ion learning. By around 200, 150, and 200 epochs, respectively, both weights stabilize, indicating that the contributions of the CE and CL losses have reached an equilibrium. This suggests that the model has adequately learned both the task-specific and representation--relatee objectives.
A.5 Hardware & Training Setup
All experiments are conducted utilizing Nvidia Tesla V100-SXM2 GPUs with 32 GB VRAM, equipped with Core Xeon CPUs and 192 GB RAM. We use the vanilla Adam optimizer with a learning rate set to 0.1, a decay rate of 0.1, a batch size of 128, and train each task for 300 epochs.
After each task reduce the number of exemplars of previous classes. For example, the 11FAR-100 dataset has 10 tasks with each 10 classes. For the first task, the dataset contains 500 samples for cach class for 10 classes. For the second lask, the dataset contains 200 samples for each of the previous 10 classes and 500 samples for the new 10 classes. For the third task, the dataset contains 100 for each of the previous 20 classes and 500 samples for the new 10 classes. For the fourth task, the datasct contains 66 samples for each of the previous 30 classes and 500 samples for the new 10 classes. In this way, the number of samples of the previous classes decrease to accommodate samples of new classes in fixed memory.
20

---

Published in Transactions on Machine Learning Research (03/2025)
1e6
BLCL
BLCLan.
2.5
BLCLg
0.C3
B BLCL...111 1.1,
1.C1
B.CLogo.0.3,31
Parameters
2.0
1.5
1.0
Task
Task 2
Task 3
Task 4
Figure 12: Number model parameters for each task.
Table 5: Model properties.
Dataset
Method
Backbone
Model Size
Samples
Sample Size
Total
Replay (Rateliff 1997
ResNet32
1.75 MB
7,431
21.76 MB
23.51 MB
CIFAR-10 and CIFAR-100
iCarl Rebuth et aL 2017
ResNet32
1.75 MB
7,431
21.76 MB
23.51 MB
DER
Yau ct al.
2021
ResNet32
17.68 MB
2,000
5.85 MB
23.53 MB
MEMO [hou et al
2023b
ResNet32
13.83 MB
3,312
9.70
23.53 MB
BLCL (ours]
ResNet50
15.85 MB
2,000
5.85 MB
20.85 MB
Replay (Ratcliff 1997
ResNet18
42.60 MB
4,970
713.50
756.10 MB
iCarl (Rebulfi
2017
ResNet18
42.60 MB
4,970
713.50 MB
756.10 MB
GNSS
DER Yan 2021
ResNet18
468.00 MB
2,000
287.00 MB
755.00 MB
MEMO [hou al
2023b
ResNet18
362.80
2,739
393.20
755.00 MB
BLCL (ours)
ResNet18
362. 80 MB
2,000
00
649.80
Replay RRtellff 1997
ResNet.18
42.60 MB
4,970
713.50 MR
756. 10 MB
iCarl Rebulli ct al 2017
ResNet18
42. 42.60 MB
4,970
713.50 MB
756.10 MB
ImageNet100
DER Yan et al. 212
ResNet18
468. 00 MB
2,000
287.00 MB
755.00 MB
MEMO [Zhou et al.
2023b
ResNet18
362.80 MB
2,739
393 393.20 MB
755.00 MB
BLCL (ours)
ResNet18
362.80 MB
2,739
393.20 MB
755.00 MB
A.6 Parameter Counts
Figure 12 presents a detailed overview of the parameter counts for various configurations of the BLCL ar- chitecture, including BLCLs, BLCLs, and specialized configurations BLCLge] across all lasks within the CIFAR-10 dataset. The configurations differ based on the number and arrangement of dynamic specialized components. Transitioning from BLCLş BLCLs, which increases the number of blocks in the special- ized component from 6 to leads to rise in parameter count from approximately 2.1 10 to 2.7 10 This increase in model complexity correlates with an improvement. in accuracy (as reported in Table 2 highlighting the trade-off between model size and performance. The configurations BLCLg feature selective reductions in the number of specialized blocks for different tasks, as indicated in the legend. For example, BLCLs [0, 0, 0, represents the smallest architecture, with minimal or no specialized components. Despite the reduced parameter count, these configurations maintain competitive accuracy, showcasing the efficiency of dynamic specialization in balancing model size and performance. As observed in Figure [2 the param- count increases progressively with each additional task, reflecting the adaptive inclusion of specialized components. Configurations with more specialized blocks, such BLCLs[1, 1, 1, and BLCLs 1, 1, 3), ex- perience a steeper increase in parameter count compared to BLCL6. This scaling behavior demonstrates the flexibility of the architect ure to adapt to task complexity while controlling overall model growth. The com- parison of BLCLg 1, 3] with simpler configurations like BLCLg [0, 0, 0, or BLCLs [0, 0, 3. illustrates how selective reduction of parameters can retain performance. This obscrvation suggests that the dynamic allocation of specialized components effectively balances the need for task-specific learning and model gen- eralization. Table contextualizes these configurations by comparing their parameter counts, model sizes,
21

---

Published in Transactions on Machine Learning Research (03/2025)
302
D42
256
1:30
16
50
53
Class (Ground Truth]
0
618
582
Class (Ground Truth]
15
580
32
153
122
164
Class (Ground Truth]]
2
953
65
56
65
51
80
70
10F
10R
4-
452
400
162
91
162
378
440
100
306
121
74
40
760
351
6
10a
5
101
244
155
123
2337
45
12
390
72
134
-1
61
111
24
17
601
114
00
138
8
100
10c
10P
0
2
3
5
6
8
2
3
5
7
8
2
5
7
8
Classs [Prediction]
Class [Prediction]
Classs [Prediction]
(a) Finetune.
(b) EWC (Krrkptrrick al.
2017
(c) LwF (Li & Hoiem
2018
L70
RGS
0
9
49
195
790
6
0
0
60
180
1
600
2
0
18
71
Class (Ground Truth]
2-
18
41
2
37
35
Class [Ground Truth]
2-
620
48
151
78
10P
20
161
11
3
10
25
216
Closs [Ground 5n
4-
19
12
804
15
48
31
23
28
0
69
700
27
44
35
28
22
4
10
15
25
780
17
12
23
5
37
24
900
Iol
6
44
830
22
101
6
11
101
26
36
-
25
-3
16
14
100
9
0
17
21
2
11
0
2
3
6
8
9
2
4
5
6
7
9
0
2
4
5
8
9
Class [Prediction]
[Prediction]
Class [Prediction]
(d) iCarl Rebuffi et al.
2017
(e)
Replay Ratcliff
1997
(f) MEMO 7hou et al.
2023b
0-02
L82
46
0 S4c
18
24
40
450
50
18
LOD
500
1
Class (Ground Truth]
2-
15
Class [Ground Truta]
2
160
Class [Ground Truth]
2-
719
32
g.
16
266 475
132
54
19
28
102
0
163
49
390
161
78
92
69
100
3
18
31
615
36
142
83
26
17
10P
279
56
4-
12
15
37
707
20
10'
0
2
24
22
915
52
12
10r
6)
21
24
5
912
10
13
982
d
10P
10
35
10P
1G
109
0 2
3
8
3
4
9
0
2
4
5
8
Class [Prediction]
Class [Prediction]
Class [Prediction]
(g) DER
Yan et al.
2021
(h) FOSTER WWang et al. 2022).
(i) BLCLs
Figure 13: Confusion matrices of statle-ffthhe-at methods and our Bayesian BLCL approach for the CIFAR-
10 dataset.
and memory footprints with baseline methods such as Replay, iCarl, and DER. The BLCL configurations achieve competitive or superior performance while maintaining manageable model sizes.
A.7
Embedding Analysis After Each Task
Figure 13 Figure 14 Figure 15 and Figure TG present the confusion matrices for all methods on the CIFAR-
10, CIFAR-100, and GNSS datasets, respectively. Figure [7 Figure [8] Figure 19 and Figure 20 present the t-SNE visualizations for Replay, MEMO, DER, FOSTER, and BLCL these three datascts, respectively.
Figure 21 visualizes the feature embeddings after each of the tasks. Specifically for the CIFAR-10 dataset (a to d), when adding clusters from new tasks, a large inter-class distance between clusters from previous classes are maintained. However, for the GNSS dataset (c to i), the samples from classes 3 and 4 from task 2 are added in be-between the classes 0 to 2 due to a high similarity between the classes.
22

---

Published in Transactions on Machine Learning Research (03/2025)
107
Class [Ground Truth]
20
Class [Ground Truth]
20
20
Grroud
100
10"
20
20
40
[Prediction]
[Prediction]
[Prediction]
(a) Finetune.
(b) EWC (Kirkpalrick
2017
(c) LwF Li & Hoiem
2018
Class (Ground Trath1
Closs [Ground Truth]
20
Class [Grond Troth1
20
40-
60
40
101
101
101
60
60
100
10P
100
20
40
80
20
40
60 [Prediction]
80
20
40
60
[Prediction]
[Prediction]
(d) iCarl Rebuffi al.
2017
(e) Replay Ratcliff
1997
(f) MEMO Zhou al.
2023b
Clacs [Ground Truth]
Class (Ground Truth]
20
Class (Ground Drith]
20
60
40
301
100
60
60
60
80
80
107
109
107
40
80
20
40
60
Class [Prediction]
Class [Prediction]
Class [Prediction]
(g) DER Yan et al.
2021
(h) FOSTER Waag al. 2022
BLCLs(+)
Figure 14: Confusion matrices of state-of-theear methods and our Bayesian BLCL approach for the CIFAR- 100 dataset.
23

---

Published in Transactions on Machine Learning Research (03/2025)
0
20
Class [Groand Truth]
20
Class [Ground Truth)
20
IGround
40
10t
40
10:
40
109
10"
40
60
Class [Prediction]
Class [Prediction]
Class [Prediction]
(a) Finetune.
(b) EWC (Kirkpatrick
2017
LwF Li & Hoiem
2018
(Ground Troth]
20
Class (Ground Troth]
20
Class (Gronnd Troth)
20
40-
40
40
60
60
60
100
109
107
20
40
60
80
20
40
60
80
20
40
60
80
Class [Prediction]
Class [Prediction]
Class [Prediction]
(d) iCarl (Rebuffi al.
2017
(e) Replay Ratcliff
1997
(f) MEMO Zhou al.
2023b
Class (Ground Trutb]
20
Class [Ground Trutbo
Class (Ground 1nitb]
20
10t
40
60
60
80
107
109
107
20
40
60
80
20
40
60
80
20
40
60 [Prediction]
00
[Prediction]
[Prediction]
(g) DER
Yan et al.
2021
(h) FOSTER Waag al. 2022
(i) BL BLCLggo....))
Figure 15: Confusion matrices of statteof-thheart methods and Bayesian BLCL approach for the Ima- geNet100 dataset.
24

---

Published in Transactions on Machine Learning Research (03/2025)
a
1256
DWG
304
10
Class (Ground Trott)
0
30
10a
Clans (Gmunn
10P
10
10P
6
J
104
8
10
08
101
100
10
10a
10
107
2
3
6
9
10
2
3
6
7
10
3
7
8
10
Class [Prediction]
Class [Prediction]
Class [Prodiction]
(a) Finetune.
(b)
EWC (Kirkpatrick al.
2017
(c) LwF LLi & Hoiem
2018
726
304
10
10
2
0
131
Class (Ground Trutb]
Class Groond Trutn]
0
3
0
10+
0
0
0
0
10
10
(Gronnd
G
207
10P
10
G:
0
10k
10
10
0
10
100
10
10
8
3
7
Class [Prediction]
Class [Prediction]
Class [Prediction]
(d) iCarl Rebuffi et al.
2017
Replay Ratcliff
1997
(f) MEMO Zhou et al.
2023b
104
1 -74
425
36
10
1
44008500
204
Class [Ground Truth]
2-
9000
0
Class [Ground Truth)
12
29
0200
150
0
Class (Ground Truth]
2-
5
10
9000
0
50 0
0
10a
1:0
10a
5-
:07
5
10
5
10
6-
a
a
6
1001
101
101
8
a 9
6
9-
0
9-
0 0
109
10
10-
0
0
2
6
9
10
5
9
10
10
Class [Prediction]
Class [Prediction]
Class [Prediction]
(g) DER Yan et al.
2021
(h) FOSTER Wang et al. 2022
(i) BLCLs
Figure 16: Confusion matrices of state-of-the- ahe-art methods and our Bayesian BLCL approach for the GNSS dataset.
25

---

Published in Transactions on Machine Learning Research (03/2025)
(a)
Replay
Ratcliff
(b)
MEMO
Zhou (c)
DER
Yan al. (d) FOSTER
Wang
BLCLa,1,1222)
1997
et al.
2023b
2021
et al. 2022
Figure 17: t-SNE
van der Maaten & Hinton
2008 plots for the CIFAR-10 10 dataset.
(a)
Replay
Ratcliff
(b)
MEMO
Zhou (c)
DER
Yan (d)
FOSTER
(Wan
(e)
BLCLa
1997
let al.
2023b
2021
et al. 2022
Figure 18: I-SNE
van der Maaten & Hinton
2008 plots for the CIFAR-100 dalaset.
MEMO
Zhou
DER
Yan et al.
(c) FOSTER
Waag
(d) BT.CTugto....00
et al.
2023b
2021
et al. 2022
Figure 19: t-SNE van der Maaten & Hinton
2008 plots for the ImageNet100 dataset.
Replay
Ratcliff (b)
MEMO
Zhou
DER
Yan al.
(d)
FOSTER
Wang
BLC.La
1997
et al.
2023b
2021
et al. 2022
Figure 20: t-SNE van der Maaten & Hinton
2008 plots for the GNSS dataset.
26

---

Published in Transactions on Machine Learning Research (03/2025)
Choco-
Cew
CER
(a) Task 1.
(b) Task
Task
(d) Task
Dea Clei
(e) Task 1.
Task
(g)
Task 3.
(h) Task 4.
(i) Task 5.
Figure 21: t-SNE van der Maaten & Hinton
2008 plots for the CIFAR-10 (a to d) and GNSS (c to i)
datasets after each task for our BLCL method.
(a) Task
(b) Task 2.
(c) Task 3.
(d) Task 4.
(e) Task 5.
(f) Task 6.
(g) Task 7.
(h) Task 8.
(i) Task 9.
(j) Task 10.
Figure 22: t-SNE yan der Maaten & Hinton our BLCL method.
2008 plots for the ImageNet100 dataset after each task for
27


Reviews:
[{"review_id": "xXGFphBFLE", "reviewer": "Reviewer_NhZE", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n\nS1: I think the authors have done a good job communicating the main ideas in clear Figures.\n\nS2: The authors use a real-world use case with the GNSS Snapshot experiments. This greatly helps the paper in terms of impact by showing robustness beyond toy CIFAR experiments.\n\nS3: I greatly appreciate the detailed analysis and visualizations in the experiments. Good job to the authors for this.\n\nWeakness\n\nW1: Class-incremental learning for image classification on small images (e.g., 32x32 CIFAR) with rigid task boundaries and labeled data has become a low-impact problem setting due to the saturation of results and lack of adaptability to real world settings. Including the GNSS experiments has greatly strengthened the paper. However, the paper could really use some results on higher-resolution images such as ImageNet (or something in between ImageNet and CIFAR) to demonstrate it is robust to modern ML benchmarks.\n\nW2: In the method section, it would be great to clarify in section 3.2 which contributions are your own, and which part of the architecture description is from MEMO. It seems needed given that, in main contribution 1, you specific your method is similar to MEMO, and then in 3.2 you say you leverage MEMO.\n\nW3: The only architectures explored are ResNet models, which are from 2016. Could the authors explore more modern architectures, such as the Vision transformer?\n\nW4: the gains are quite slim in Tables 2 and 3. Without stdev reported, it is hard to tell whether this is statistically significant.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors propose a CIL method BLCL which builds off of the MEMO architecture and includes Bayesian weighting to balance cross-entropy and contrastive learning losses. As shown in their visualizations, the method achieves feature representations with high inter-class distances. The authors show the method has a SOTA accuracy over recent methods on CIFAR10/100 and GNSS datasets.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "R1 (related to W1): Could the authors report results on an additional dataset for image classification with higher resolution than CIFAR? in MEMO, the authors report results on ImageNet 1000.\n\nR2 (related to W2): Can you please modify 3.2 to articulate which design choices are novel contributions and which come from MEMO or other? As Bayesian CL and contrastive loss in CL are not new concepts, it would be very helpful to help distinguish how your method is novel in a more concrete manner (my belief is that the method is novel, but stating the contributions more clearly will help the reader as the method contains many existing concepts).\n\nR3 (related to W3): Could you please experiment on a more modern architecture as well? In MEMO, the authors report additional results using the vision transformer.\n\nR4 (related to W4): could you include more details such as stdev so that the reader can judge the statistical significance of the SOTA result?", "broader_impact_concerns": "I do not see any ethical concerns for this paper.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sxkUvfaduN", "reviewer": "Reviewer_acqF", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength:\n- The paper explicitly models uncertainty in losses and factor them out as weightings. This is a novel application for class incremental continual learning. \n- The paper performs experiments on real world datasets GNSS. \n\n\nWeakness:\n\n- **Interpretation of the uncertainty weighting** For both losses (CE and CL), the weighting is inversely proportional to the uncertainty parameters. In other words, higher weighting means lower uncertainty. This makes sense from the uncertainty perspective. However, this makes less sense from an optimization perspective. As shown in Fig.7, the weights continue to drop for both losses throughout training, this means that uncertainty rises during training. This interpretation seems self-contradicting. \n \n- **Effectiveness of the uncertainty weighting** From Fig.7, we can observe that the weightings for both losses seem to have a very high correlation. When balancing losses, the relative importance of the two losses are more indicative of their contributions. However, the strong correlation between the weights seem to suggest that there is no actual adjustment of importance between the losses. Furthermore, from table 3, the $BLCL_{2[1,1,1,1,1]}$ Bayesian version underperformed the regular version with a fixed 0.9 weighting. This seems to suggest that the core contribution does not provide significant improvement.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes a Bayesian approach to balance weighting between the cross-entropy loss (CE)  and constrastive loss (CL) for class-incremental continual learning.  Specifically, the paper explicitly models the uncertainty in the categorical distribution from CE and the Gaussian distribution from the CL as two hyper-parameters. Through algebraic operations, the two hyper-parameters emerge as weightings for the respective losses. The optimization objective is to jointly optimize the two losses and their uncertainty hyper-parameters, which also function as weightings for the losses.\n\nAdditionally, the paper also explores architectural variations for different datasets and tasks.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "* Could the authors make the interpretation of uncertainty and loss weights more explicit? \n\n* Could the authors demonstrate the actual relative importance between the two losses such as a ratio between the weights as a plot in addition to fig.7?", "broader_impact_concerns": "The paper does not have obvious negative social impact.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "mmsodkkrNN", "reviewer": "Reviewer_ysVo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "### Strengths\n\n1. In general, the idea makes sense and the experiments show slight better performance compared to previous approaches. \n2. The analysis regarding to the clusters and embeddings is somehow informative. \n\n### Weaknesses\n\n1. The presentation should be improved, where clearer definitions can make the paper easier to understand. \n    - For example, it's better to define explicitly $BLCL_l[s]$, where $BLCL_8[1,1,2,2]$ should also be clearly explained. If I understand correctly, this is related to the number of tasks, the number of blocks, and the five possible architectures depicted in Figure 2. \n    - Some sentences are written in a quite strange way, for example, \"As σ 1, respectively σ 2, the noise parameters associated with variable y 1, respectively y 2, increase, the weight assigned to L CE, respectively L CL, diminishes.\"\n2. The novelty is limited, mainly based on MEMO by adding a Bayesian approach to balance the two losses. \n3. In terms of the Bayesian approach itself, I have the following question: \n    - Given the cross entropy loss and contrastive loss could be different in their scales, does it really work to directly use the variance to balance the importance between them? Did you introduce any additional hyper-parameter to control this? \n4. \"Evaluation of dynamic specialized components reveals that smaller blocks are preferable for the first task.\" could you analyze the reason? Does this hold for all datasets and any task order? I feel this may be related to the complexity of the task. \n5. Will the authors release the code to promise reproducibility?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper proposes to mitigate catastrophic forgetting in class-incremental learning by (1) a Bayesian learning technique that dynamically balances the contrastive loss and cross entropy loss; (2) leveraging the MEMO architecture that trains a new specialized block per new\ntask. Experiments on CIFAR-10, CIFAR-100, and GNSS data validate the effectiveness of the proposed method.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Please address my previous concerns and  improve the presentation.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "AkUmUvJXeq", "reviewer": "Reviewer_og1a", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**\n1. The paper is well-structured and clearly written.\n2. The proposed Bayesian-driven loss weighting strategy for balancing CE loss and CL loss appears promising for continual learning scenarios.\n3. Extensive experiments on CIFAR-10, CIFAR-100, and a GNSS-based dataset validate the effectiveness of the approach.\n4. The confusion matrix and clustering visualizations provide additional insights into the effectiveness of the proposed method.\n\n**Weaknesses**\n1. The motivation for using Bayesian learning in weighting loss functions for continual learning needs clearer justification. In Section 3.3, the paper states that CL loss can incrementally enhance class prototypes. If this is the case, why is it necessary to balance CE loss and CL loss rather than optimizing CL loss alone? If applying both losses simultaneously is essential and dynamic weighting provides better results, the paper lacks an in-depth analysis of the roles of CE loss and CL loss during training. Furthermore, there is no ablation study comparing the performance when using only CE loss or only CL loss.\n\n2. Section 3.2 states: \"To maintain adaptability to varying complexities of new classes across tasks, we dynamically adjust each task’s layer structure.\" However, the paper does not clearly explain how the layer structure is dynamically adjusted during continual learning.\n\n3. It is unclear whether $\\sigma_1$ and $\\sigma_2$ in Equation (5) are trainable parameters or optimized variables with an analytical solution. The paper should provide a detailed explanation of all optimization variables in the formulation. Additionally, in the first line of Equation (5), the rightmost expression appears incorrect—based on Equation (2), it should involve addition rather than multiplication.\n\n4. The paper lacks a pseudocode representation of the proposed method, which would help readers better understand the implementation details.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces a Bayesian Learning-driven Prototypical Contrastive Loss (BLCL) for Class-Incremental Learning (CIL). The core idea is to dynamically balance cross-entropy (CE) loss and contrastive learning (CL) loss using homoscedastic task uncertainty. The method also averages the specialized component weights at the end of each task's learning process, thereby alleviating catastrophic forgetting in CIL. The approach is evaluated on CIFAR-10, CIFAR-100, and a GNSS-based dataset, demonstrating improved performance compared to state-of-the-art methods.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Clarify the motivation for Bayesian-based loss weighting and provide stronger theoretical or empirical justification for its necessity in balancing CE and CL losses.\n2. Revise and expand the formulation of Equation (5) to explicitly define all optimization variables and correct potential errors in the equation.\n3. Include algorithm pseudocode for the proposed method to improve clarity and reproducibility.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper offers a meaningful contribution to the field of class-incremental learning. The proposed BLCL method is a novel approach that combines various techniques, utilizing Bayesian learning to dynamically balance cross-entropy and contrastive losses. The experimental results are generally convincing, and the authors have effectively addressed the reviewers' concerns.

The reviewers provided constructive feedback that significantly enhanced the quality of the paper. Key improvements include:
- Clarification of the Bayesian balancing mechanism: The authors provided a more detailed explanation of the Bayesian method and its advantages.
- Inclusion of ImageNet100 results: Addressing concerns about the method's applicability to higher-resolution images.
- Provision of standard deviations: Enhancing the statistical rigor of the results.
- Ablation studies: Providing insights into the contribution of individual loss components.
- Pseudocode: Improving the clarity and reproducibility of the method.

While the paper has improved significantly during the review process, two modifications should be made before final acceptance:
1. Include the discussions between Reviewer acqF and the authors in the paper, as these discussions provide valuable insights that help explain the results.
2. To ensure the content is self-contained, lines 20 and 27 in Algorithm 1 need detailed descriptions and definitions for the functions used.

The authors have shown a strong commitment to addressing feedback and enhancing their work. The reviewers' feedback, along with the authors' responses, indicate that the study is technically sound and relevant to the TMLR audience.
Decision: Accept (poster)