L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Authors: Xiang Zhang
Run He
Chen Jiao
Di Fang
Ming Li
Ziqian Zeng
Cen Chen
Huiping Zhuang
Venue: ICML 2025 poster (2025)
Keywords: Continual Learning
Class-Incremental Learning
Multi-label Classification
Primary area: deep_learning
TL;DR: This paper proposes an exemplar-free method called L3A to address missing labels and class imbalance problems in multi-label class-incremental learning.

Abstract: Class-incremental learning (CIL) enables models to learn new classes continually without forgetting previously acquired knowledge. Multi-label CIL (MLCIL) extends CIL to a real-world scenario where each sample may belong to multiple classes, introducing several challenges: label absence, which leads to incomplete historical information due to missing labels, and class imbalance, which results in the model bias toward majority classes. To address these challenges, we propose Label-Augmented Analytic Adaptation (L3A), an exemplar-free approach without storing past samples. L3A integrates two key modules. The pseudo-label (PL) module implements label augmentation by generating pseudo-labels for current phase samples, addressing the label absence problem. The weighted analytic classifier (WAC) derives a closed-form solution for neural networks. It introduces sample-specific weights to adaptively balance the class contribution and mitigate class imbalance. Experiments on MS-COCO and PASCAL VOC datasets demonstrate that L3A outperforms existing methods in MLCIL tasks. Our code is available at https://github.com/scut-zx/L3A.

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Xiang Zhang' Run He
Jiao Chen Di Fang? Huiping Zhuang'
Ming Li Ziqian Zeng'
Cen Chen
Abstract
Training set
Training set petaan
Training set D
Class-incremental learning (CIL) enables models to learn new classes continually without forget- ting previously acquired knowledge. Multi-label CIL (MLCIL) extends CIL real-world sce- nario where each sample may belong to multi- ple classes, introducing several challenges: label absence, which leads to incomplete historical in- formation due to missing labels, and class im- balance, which results in the model bias toward majority classes. To address these challenges, we propose Lugme-Aug Analytic Adaptation (L3A), exemplar-free approach without stor- ing past samples. L3A integrates two key mod- ules. The pseudo-label (PL) module implements label augmentation by generating pseudo-labels for current phase samples, addressing the label absence problem. The weighted analytic classi- fier (WAC) derives closed-form. solution for neural networks. It introduces sample-specific weights to adaptively balance the class contribu- tion and mitigate class imbalance. Experiments on MS-COCO and PASCAL VOC datasets demon- strate that L3A outperforms existing methods in MLCIL tasks. Our code is available at https: //githubs www/co/////oo/ooo ccmt/scut--i 2x/L3A.
passon
person bike
parson, bike
person, bake,
Model
Model
Model e,
Phose
Phese
Prese
θ
Test setr
Test set Dist u Di
Test set Di U.... Di
pRESON
pessOOL lake, car
pesson
person like, car
lake car
Figure The process MLCIL. The model learns different classes such as 'person', "bicycle", and 'car" during phases 1, 2, and 'person' represents samples belong to this class lack the label. Different colors of Model θ,, O2, and O represent the changes in model parameters caused by continual learning.
CII faces the severe catastrophic forgetting problem (McCloskey & Cohen, 1989; Ratcliff, 1990), manifested as performance degradation of previously learned classes. Existing CIL methods (Zhou et al., 2024b) have achieved considerable success in single-label CIL tasks, significantly reducing the computational resources for model retraining.
1. Introduction
However, in complex real-world environments, CIL agent needs to learn complex training data with more than ob- ject in one image, termed multi-label CIL (MLCIL). Figure I illustrates the detail paradigm, where each training sample is labeled only with the classes relevant to the current learning phase, while the labels of other classes are masked. The goal of MLCIL is to enable the model eventually recognize all classes encountered throughout the learning process, even if some labels masked during training phases.
Class-incremental learning (CIL.) enables models to ac- quire knowledge new classes in a phase-by-phase man- ner, while maintaining knowledge of previously learned classes without retraining the model from scratch. Research
'Shenn-Ming School Intelligent Engincering, South China University of Technology, Guangzhou, China School of Future Technology, South China University of Technology, Guangzhou, China "Guangdong Laboratory of Artificial Intelli- genee and Digital Economy (SZ). Correspondence to: Huiping Zhuang <hpzhuang @ swu..edu..c.....
MI.II. faces three major challenges: (1) label absence: each sample is label-incomplete, containing only the labels from the current phase, with the labels from previous and future phases absent. For example, the left sample in the training set Dirain in Figure includes person riding a bicy-
Proceedings of the nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
cle in front of cars. While the sample contains three classes, only annotated the 'car' as a training label, with 'person' and "bikee being part of the background. This causes more severe catastrophic forgetting, as the current samples be- come negative samples of previously learned labels (Dong et al., 2023). (2) class imbalance: the strong co-occurrence of classes in multi-label images leads certain classes to ap- pear together frequently. For example, images with bicycles often include people. This creates imbalanced class dis- tribution, causing the model to favor high-frequency classes and neglect rare ones, leading to degraded performance. (3) privacy protection: privacy constraints often prevent access to historical sample data in real-world scenarios, rendering many replay-based methods inapplicable.
The WAC updates the classifier by recursively solving ridge regression problem while introducing sample-specific weights to balance the loss contributions of different classes. Our contributions are summarized as follows:
We propose L3A,, an exemplar-free approach that pro- vides a closed- solution to address catastrophic forgetting in MICII.
We introduce the PL module to implement label- augmented by generating labels for previously learned classes, addressing the label absence problem.
We introduce the WAC that iteratively updates the classifier by analytic learning and adaptively assigns sample-specifie weights, solving the class imbalance problem.
Existing CIL methods fail address all three challenges of MLCIL. Although many single-label CIL approaches mitigate catastrophic forgetting, they are limited by label absence and are unable to handle the MLCIL problem ef- fectively. To address label absence, KRT (Dong et al., 2023) proposes knowledge restore and transfer framework, and CSC (Du et al., 2025) introduces a class-incremental graph convolutional network to establish label relationships. Both methods rely on replaying samples to achieve optimal per- formance. MULTI-LANE (De Min et al., 2024) utilizes pre-trained large model and prompt-tuning techniques to achieve replay-free MLCIL. However, these MLCIL meth- ods fail to address the class imbalance issue and result in suboptimal performance.
Experiments on MS-COCO and PASCAL VOC datasets demonstrate that our approach achieves state- of-the-art (SOTA) performance in MLCIL tasks.
2. Related Works
2.1. Single-Label CIL
Replay-based CIL methods such as iCaRL (Rebuffi et al., 2017), ER (Rolnick et al., 2019), BiC (Wu et al., 2019), DER++ (Buzzega et al., 2020), TPCIL (Tao et al., 2020), PODNet (Douillard et al., 2020) mitigate catastrophic for- getting by revisiting old task data during training new task. Although the replay-based methods are straightforward and effective approaches, they face significant challenges under stricter privacy protection constraints.
Considering privacy protection scenarios, the exemplar- free CIL (EFCIL) methods without storing historical examples are becoming increasingly important. However, many EF- CIL methods suffer from task-recency bias due to the back- propagation algorithm, where the model focuses on optimiz- ing the most recent tasks and leads to poor performance on previous tasks. Analytic Continual Learning (ACL) (Zhuang et al., 2022) is a new branch of EFCIL that achieves supe- rior performance compared to replay-basee methods. ACL enables incremental learning by iteratively solving a closed- form solution, which obtains an equivalent solution to joint training without storing historical data. This characteristic makes ACL well-suited for protecting privacy. However, existing ACL methods are primarily effective in single-label CIL. they encounter limitations when addressing the label absence and class imbalance problem in MLCIL.
Regularization-basel CIL methods constrain certain model parameters to minimize changes to those critical for old task classification. EWC (Kirkpatrick et al., 2017). oEWC (Schwarz et al., 2018) and RWalk (Chaudhry et al., 2018) introduce regularization into the loss function and evaluate the importance of model parameter using the Fisher Information Matrix. LwF (Li & Hoiem, 2017) introduces knowledge distillation (Hinton et al., 2015) into the loss function to preserve old knowledge. However, those models can not achieve better performance compared to replay- based methods commonly.
Prompt-based CIL methods utilize pre-trained large models as backbone networks to enhance feature representation ability. L2P (Wang et al., 2022) and CODA-Prompt (Smith et al., 2023) introduce prompt tuning into continual learning, EASE (Zhou et al., 2024a) makes a task-specific subspace for each incremental task.
To overcome these limitations, we propose the Label- Augmented Analytic Adaptation for Multi-Label Class- Incremental Learning (L3A). Our approach comprises two main components: (1) the pseudo-label (PL) module and (2) the weighted analytic classifier (WAC). The PL module generates label information for previously learned classes by inputting new samples into the old classifier, enabling analytic learning to derive optimal closed-form solutions.
2.2. Multi-Label CIL
To address the label absence, class imbalance, and privacy protection problem in MICII, several approaches are pro-
2

---

L3A: Labcl-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
(a) Pseudo-Label Module
(b) Feature Extraction
(c) Weight Chass
Classifier
(d) Inference
Plaser
Label Frequency fe-1
Weight. Matrix Qt-1
Testing data
Draai
R
Frozen Backbone
Auloconelation Manix
Frozen
Backbone
W (-1
Buffer
person, bike
person bike
θ
goooo
Update
Phase
D truin
D Btrain
R₁
Buffer Layer
Autoooneaatonn Manix
penson, bike,, car
person, bike,
Label Frequency ft
Weight Matrx O2
W
Predict Labels
Figure 2. The overview of our L3A method. (a) The multi-label data stream arrives in phase, with each sample containing incomplete label information (e.g.. "persen'). The pseudo-label module generates the augmented training Di (e.g.. 'person'). (b) A frozen hackhone with buffer layer extracts sample features and projects them into higher-dimensional space. (c) The weight analytic classifier iteratively updates the weight matrix 52,, autocorreation matrix Re, and analytic classifier W. across different phases. (d) The frozen backbone with buffer layer and the current phase classifier are used for inference.
posed. PRS (Kim et al., 2020) adopts replay-based strat- egy to tackle intra- and inter-task class imbalance. OCDM (Liang & Li, 2022) introduces greedy algorithm to man- age class distribution in memory effectively. KRT (Dong al., 2023) incorporates incremental cross-attention mod- ules to restore and transfer knowledge between continual learning stages. MULTI-LANE (De Min et al., 2024) inte- grates prompt-baseeee based techniques into MLCIL, enabling an exemplar-free MLCIL paradigm. CSC (Du et al., 2025) es- tablishes cross-task label relationships and calibrates the model's over-confident output.
3. Method
3.1. Problem Definition
The goal of MLCIL is for model to incrementally learn the classification of multi-labele within input samples, while the labels introduced at each training phase are incomplete.
Let the phases as {D1, D2, Dr,...>. Dr is divided into a training set Dimin and a test set Diest, Di ((X t.1, (.,,).),, (,,,,, Vc..).... (X,,N, Vt,N,) of size No. where X', represents input samples tensor, Yt.: represents the corresponding multi-hot labels vector. Each label vector Vt,i E C and C* is the partial label space set specific for phase t, with the constraint that Vm, num * n), C n C = Ø. |C*| is the number of unique classes in phase t. The model trained on Dirmin and evaluated the performance on Diss. which defindd as joint test sets D U Diru. The cumulative label space for testing expands incrementally and is defined as Cl#t = U.....uc
2.3. Analytic Continual Learning (ACL)
ACL is an emerging paradigm in continual learning, inspired by pscudo-inverse learning (Guo & Lyu, 2004). It imple- ments an exemplar-fre free approach that uses recursive least squares find closed-form solution for network param- eters. ACIL (Zhuang et al., 2022) demonstrates that using recursive least squares is equivalent to the joint learning ap- proach firstly, enabling the application of analytic learning to the field of CII. AL (Zhuang et al., 2024b) introduces another linear classifier to improve the under-fitting limi- tation. REAL (He al., 2024) introduces a representation enhancing distillation process to strengthen the representa- tion of the feature extractor. GACL (Zhuang al., 2024a) extends the ACL generalized CIL, dealing with the mixed data categories and unknown sample size distribution. AIR (Fang et al., 2024) addresses the class imbalance in single- label CIL by introducing a re- ce-weighting module.
3.2. Proposed Framework
Our proposed Label-Augmented Analytic Adaptation (L3A) framework comprises three components: (a) pscudo-label module that completes missing label information by gen- erating pseudo-labels for historical classes, (b) feature extraction module that extracts and transforms input sam- ples into high-dimensional feature representations, and (c) a weighted analytic classifier employs closed-fome solution with sample-specifi weighting mitigate class imbalance.
3

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
with iterative updates. The detailed architecture of the L3A framework is depicted in Figure 2.
θ indicates the frozen backbone weight and the Statte (+) improves feature representation and captures more com- plex information by employing various mapping strategies (Zhuang et al., 2022; 2023). We implement the foutte(-) as a random linear projection layer followed by non-linear activation function, i.c. fuutte (X) = ReLU Wand).. The weights of projection layer W mnd are randomly initialized.
Unlike other methods that rely on backpropagation for model updates, the L3A framework can be formulated as an optimization problem. During the t-th phase of MICII, the objective is to solve for the linear classifier W; as follows:
argmin | (6 1.s5 - W W
+ +
3.5. Weighted Analytic Classifier
In this section, we calculate the solution of the linear classi- fier in Equation (I). Let X, be the feature expansion matrix obtained from Section 3.4, and Y; be the corresponding aug- mented label set obtained from Section 3.3. So X1:t and Yi indicate the feature matrix and augmented label matrix form 1 to t phase.
(1) where Sir denotes the sample-specifie weight matrix. o.. θ) represents the feature extractor function with O as the backbone. X 1s and Y1 are the input sample matrix and the true labels set from phase 1 to t. (-) refers to the pseudo-label algorithm. || |F indicates the Frobenius norm and y the regularization term.
X 1:1-1 X
91--1 Y
0 Y
X1
3.3. Pseudo-Label Module
Unlike traditional CIL methods that rely on backpropaga- tion, ACL methods use ridge regression to derive the ana- lytical expression for the classifier. Therefore, we refer to the corresponding linear classifier as the Analytic Classifier (AC). In the general case, the loss function for the analytic classifier W, is:
The problem of label absence is similar to challenges en- countered in supervseee learning, where the labels of samples from both past and future phases missing. Moti- vated by this approach, we propose the pseudo-label (PL) method to address the label absence problem.
Specifically, in the phase f, previous classifier W already retains knowledge of historical classes. We utilize this clas- sifier with the current training set Dirmin {(X,, Y,)} to generate the pseudo-label set Y, for previous classes. A confidence threshold 17 (0, 1) activates pseudo-labels follows:
argmin W
|. -Y.ell + Will
(5)
In Equation (5), each sample contributes the overall loss equally. In class imbalance scenarios, dominant classes with larger quantities introduce bias into the classifier, which tends to overfit the majority classes while underperforming on the minority classes
(k) Ve,
[1
if Pt, nn
otherwise,
(k) where A indexes classes IDs, and Pi.s denotes the predica- tion score for the k-th class. The pseudo-labe algorithm (-) results the augmented label set is defined as:
To address the class imbalance problem, we propose the weighted analytic classifier (WAC), which assigns sample- specific weight Wi, based on the frequency of each class in the training data. For a class & with frequency f(8), the class- specific weight is computed as ot (k) =1/V (()) ensuring that minority classes contribute more significantly to the loss. For sample (X Dt,i), the sample-specific weight Wa,s is the average of the weights u/ for all active classes in Sr.::
Y Y1 U.Y
The augmented training set Dirain = {(X,, {.)). which ensures the model incorporates historical class information while retaining the label set for the current phase. Section 4.4 analyses why the pseudo-label module works in method.
(k) (A) We,1 Ek=1 De. 1.
ΣK
3.4. Feature Extraction
(6)
To calculate the solution of the classifier, we first obtain the feature representation of the input sample. The fune- tion o(:, ) extracts the features from the input samples and applies a buffer layer f-ater(-), mapping features to a higher-dimensional space, which enhances the representa- tional capacity. This process results in the high-dimensional feature expansion matrix X, given by
where Wi, denotes the weight corresponding to the i-th sam- ple from the t-th phase. This weighting mechanism ensures balanced contributions across classes, mitigating bias toward dominant classes and improving the classifier's robustness. To express the weighting mechanism more concisely, we reformulate it as a matrix representation. Specifically, 2 is defined as a diagonal matrix that represents the sample- specific weight matrix for all samples across the phases from
X, + ((Xx, A)
(4)
4

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
1 to 1:
inverse of feature matrix product from past phase samples. However, it is impossible to reverse-engineer the informa- tion of individual samples from Rg so the L3A method retains R2 instead of storing any historical samples, achiev- ing exemplar-free learning.
11: = diag(SI S22,-... .2e)
(diag(W1,1) W2,11 w....).
When the weighting mechanism is incorporated into the loss function Equation (5), the original formulation can be rewritten as follows:
Algorithm shows the pseudo-code of L3A, which utilizes the PL module to generate overall labels, extracts the sample features, and recursively updates the classifier by WAC.
argmin || 1/2 (X - -Y:)) + +al Willi,
(8)
Algorithm Training process of L3A
Input: Training set Di ... Dipiin with Dirain (K,, Y,), the frozen backbone O
which is equal to the optimization problem Equation (I).
2// denotes the element-wis wise square root of this diagonal matrix, where each diagonal element is defined as the square root of the corresponding weight.
Initialization: Rg + YI, W +0 0 for phase t. = to to T do b Pseudo-label module. Dyran (X,; Y.) + fr 1/5 O W,-1) D Count the labels frequency.
To minimize the loss function of Equation (8), we compute
its derivate with respect to We:
fx Count(Y; (Y, fi-1)
a 1/2 [x 1 11 Will))
> Calculate the sample-specific weight.
aw,
(9)
Dt.s) for all (X, :) E DDrrin do
2X S W. - 2X S.: + W.
We, + -Weight(fr, Dt..)
Ωr + -diag(wi,, W,,,,,,,,,,,,"'"""" ,
Let the derivate equal to zero, we can get the closed-form solution of WN:
D Feature extraction.
4
X, < o6(x;,)
Update R, with Equation (13)
W =
+ X
X Sia &
(10)
Update W, with Equation (12)
Return: W.
The goal of L3A is to utilize Wi1 to update W, recur- sively without other historical samples or data. We define the autocorrelation matrix in the phase:
4. Experiments
R = X [ 11 + +11
(11)
4.1. Setting
The iterative analytic solution process in WAC is shown in
4.1.1. DATASETS
Theorem 3.1.
We follow previous works (Dong et al., 2023; De Min et al., 2024) in MLCIL and evaluate our method on MS-COCO 2014 (Lin et al., 2014) and PASCAL VOC 2007 (Evering- ham et al., 2010) datasets. MS-COCO dataset contains 120k images with 80 category labels, PASCAL VOC dataset con- tains 10k images with 20 category labels. Those datasets are suitable for MICII,, but they both exhibit issues with class imbalance. Following previous research (Dong et al., 2023), we evaluate our method using the following pro- tocols on both MS-COCO and PASCAL VOC datasets: (1) MS-COCO B0-C10: the model is trained across all 80 classes, divided into 8 continual learning phases, each learn- ing 10 classes. (2) MS-COCO B40-C10: the model is ini- tially trained on 40 classes, followed by incremental learn- ing phases the remaining 40 classes. (3) VOC BO- C4: the model is trained on all 20 classes across 5 continual learning phases, with 4 classes learned in each phase. (4) VOC B10- C2: the model has a base training on 10 classes, with the subsequent 10 classes trained continually over 5 phases. The order of incremental training the lexicographical order of
Theorem 3.1. The optimal solution of We can be calculated
recursively by
W. W R,X, R.X, W.WWW. R,X, 2.
(12)
where the autocorrelation matrix can be calculated recur- sively by
R R R--1-R1-1X R (2, X(R(-IX,)) R,R
(13)
Proof. The proof is provided in Section A.
0
Theorem 3.1 represents the update of classifier W. at phase
I relies solely on the previous classifier W,-1 and the cur- rent phase data. The solution in Equation (12) is equivalent to the joint-learning formulation presented in Equation (10). The autocorrelation matrix R, assists in this computation. As shown in Equation (11), stores knowledge with the
5

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Table Comparison of the results among 1.3A and other methods MS-COCO dataset. Memory represents the numher of replay samples, where 0 indicates that the method is examplar -ree. Data in bold represents the best results and data underline represents the second best results.
MS-COCO BO-C10
MS-COCO B40-C10
Method
Type
Memory
Avg.
Last
Avg. MAP
Last
mAP
CFI
OFI
mAP
CFI
OFI
mAP
Upper-bound FT (Ridnik al., 2021)
Baseline
76.4
79.4
81.8
76.4
79.4
81.8
0
Baseline
38.3
6.1
13.4
16.9
35.1
6.0
13.6
17.0
TPCIL (Tao al., 2020)
63.8
20.1
21.6
50.8
25.3
25.1
53.1
PODNet (Douillard et al., 2020)
CIL.
65.7
13.6
17.3
53.4
65.4
24.2
23.4
57.8
DER+ (Buzzega et al., 2020)
CIL
S/class
68.1
33.3
54.6
41.9
43.7
59.0
KRT-R (Dong et al., 2023)
MLCII.
75.8
60.0
61.0
68.3
78.0
66.0
65.9
74.3
CSC-R (Du et al., 2025)
MLCIL
79.2
67.3
68.1
73.7
78.4
68.1
69.0
75.6
ICaRL et al., 2017)
CII.
59.7
19.3
22.8
43.8
65.6
22.1
25.5
55.7
BiC (Wu ct al., 2019)
CIL
65.0
31.0
38.1
51.1
65.5
38.1
40.7
55.9
ER (Rolnick et al., 2019)
CII.
60.3
40.6
43.6
47.2
68.9
58.6
61.1
61.6
TPCIL (Tao al., 2020)
CIL
69.4
51.7
52.8
60.6
72.4
60.4
62.6
66.5
20/class
PODNet (Douillard et al., 2020)
CIL
70.0
45.2
48.7
58.8
71.0
46.6
42.1
64.2
DER++ (Buzzega et al., 2020)
CIL
72.7
45,2
48.7
63.1
73.6
51.5
53.5
66.3
KRT-R (Dong et al., 2023)
MLCIL
76.5
63.9
64.7
70.2
78.3
67.9
68.9
75.2
CSC-R (Du et al., 2025)
MICII.
79.6
67.8
68.6
74.8
78.7
68.2
69.4
76.7
PRS (Kim al., 2020)
MLCIL
48.8
8.5
14.7
27.9
50.8
9.3
15.1
33.2
OCDM (Liang & Li, 2022)
MLCIL
49.5
8.6
14.9
28.5
$1.3
9.5
15.5
34.0
1000
KRT-R (Dong al., 2023)
MLCIL
75.7
61.6
63.6
69.3
78.3
67.5
68.5
75.1
CSC-R (Du al., 2025)
MLCII
79.3
67.5
68.5
73.9
78.5
67.8
69.7
76.0
PODNet (Douillard et al., 2020)
CIL.
43.7
7.2
14.1
25.6
44.3
6.8
13.9
24.7
oEWC (Schwarz et al., 2018)
CIL
46.9
6.7
13.4
24.3
44.8
11.1
16.5
27.3
LWF (I.i & Hoiem, 2017)
CII.
47.9
9.0
15.1
28.9
9.5
15.8
29.9
KRT (Dong et al., 2023)
MLCIL
74.6
55.6
56,5
65.9
77.8
64.4
63.4
74.0
CSC (Du et al., 2025)
MLCIL
78.0
64.9
66.8
72.8
78.2
65.7
67.0
75.0
L3A
MLCIL
0
81.5
69.7
72.5
77.6
79.9
69.5
72.9
78.8
class names.
tion term 7 in Equation (8) to 1000, and the bufferr layer size to 8192 for MS-COCO and PASCAL VOC.
4.1.2. EVALUATION METRICS
4.2. Comparison Methods
Following previous MLCIL works (Dong et al., 2023;
De Min et al., 2024), we adopt the mean average precision (mAP) as the primary evaluation metrie for each MI.CII. phase, and report both the average mAP (the average of the mAP across all phases) and the last mAP (the mAP of the last phase). We also report the per-class Fl score (CF1) and overall F1 score (OF1) for performance evaluation.
We select several advanced methods in single-label class incremental and multi-label class incremental for compre- hensive comparison.
Baseline methods. Upper-bound represents the ideal per- formance result, assuming model has access to the data from all tasks simultaneously. Finetune (FT) is the simplest baseline method, where the model is trained directly on the current task with catastrophic forgetting.
4.1.3. IMPLEMENTATION DETAILS
We adopt ImageNet-21k pre-trained TResNetM (Ridnik al., 2021) as our backbone (All compared methods adopt pre-trained TResNeMM or ViT-B/16 as the backbone). The batch size is set 64 for MS-COCO and 256 for PASCAL VOC. In all experimental protocols, set the regulariza-
CIL methods. We select several classical CIL methods for comparison, including replay-based approaches such as PODNet (Douillard et al., 2020) and regularization based methods like oEWC (Schwarz al., 2018), with the over-
6

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Table Comparison of the results between L3A and prompt-based CIL methods on MS-COCO dataset. Data in bold represents the best results and data underline represents the second becond-best results.
MS-COCI BO-C10
MS-COCO B40-C10
Method
Backbone
Param.
Avg. mAP
Last mAP
Avg. mAP
Last MAP
Upper-bound L.P (Wang et al., 2022)
83.2
83.2
73.4
68.0
73.7
71.1
ViT-B/16
86.0M
CODA-P (Smith et al., 2023)
74.0
65.4
73.9
67.5
MULTI-LANE (De Min et al., 2024)
79.1
74.5
78.8
76.6
L3A
TResNet-M
29.4M
81.5
77.6
79.9
78.8
85
100
80
95
Overall mAP(%)
75
90
85-
70
70-
85-
80-
80-
75-
50
75
70-
70
10
20
10
40
50
70
80
40
50
60
70
80
16
20
10
12
14
16
18
20
Number of
Number of Classes
Number Classes
Number Classes
(a) MS-COCO BO-CTO
(b) MS-COCO B40-C10
(c)
(d) VOC H10-C2
(CaRL
ER
TPCIL
PODNel
DER
KRT-R
MULTILANE
L3A(Ours)
Figure Comparison results (mAP%) on MS-COCO and PASCAL VOC datasets under different protocols against competitive methods.
all methods presented in Table 1. Additionally, we include prompt-based methods like L2P (Wang et al., 2022), which leverage the more powerful pre-trained large model VIT- B/16, with their results shown separately in Table 2.
based methods, L3A achieves competitive results, outper- forming the best replay-based approach CSC-R in the last mAP by 2.8% and 2.1%.
Table 2 compares the performance of L3A with prompt- based methods the MS-COCO B0-C10 and B40-C10 benchmarks. While prompt-based methods benefit from more powerful backbones with stronger representation ca- pabilities, enabling competitive classification performance without replay, L3A achieves superior results with a less powerful backbone. Specifically, L3A outperforms best prompt-based MLCIL method MULTI-LANE in the last mAP by 3.1 % and 2.2% the two benchmarks. This demonstrates that our method achieves stronger classifica- tion performance with fewer network parameters.
Multi-label CIL methods. We select five best-performing MLCIL methods for comparison, including two online raplay-based methods PRS (Kim al., 2020) and OCDM (Liang & Li, 2022), the regularization-base and CSC, as well as their replay-based extensions KRT-R (Dong et al., 2023) and CSC-R (Du et al., 2025), and the prompt-based method MULTI-LANE (De Min al., 2024), which currently represents the SOTA in exemplar-free ML- CIL method.
methods KRT
4.3. Comparison Results
4.3.2. RESULTS IN PASCAL VOC
4.3.1. RESULTS IN MS-COCO
Table 3 shows the performance comparison on the VOC BO-C4 and B10-C2 benchmarks. L3A achieves results close to the upper-bound, with the last mAP of 94.1 % and 94.0% the VOC BO-C4 and B B10-C2 benchmarks respectively, surpassing the current SOTA methods MULTI-LANE by 5.3% and 5.7% on these two benchmarks.
Table shows the performance comparison on the MS- COCO BO-C10 and B40-C10 benchmarks. L3A demon- strates significant advantage over other exemplar-fr free ap- proaches across all metrics, achieving the highest last mAP of 77.6% and 78.8% on the MS-COCO B0-C10 and B40- C10 benchmarks respectively. L3A outperforms the best exemplar-fre free method CSC in the last mAP by 4.8% and 3.8% on these two benchmarks. Even compared with replay-
Figure shows the detailed comparison result between I.3A and other competitive methods on the four protocols. L3A

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
achieves more significant performance in every MLCIL phase. Moreover, these performance improvements become increasingly pronounced as the learning phase increases.
for each sample at phase / is the average of its constituent class-specific weights e(()) = 1/ f(,), where f(() is the frequency of class k. This adaptive design of 1 1/v ((A) en- sures that rare labels (with small f(k) contribute dominantly to wr. providing smoother scaling and preventing excessive weighting.
The comparison experiments on both MS-COCO and PAS- CAL VOC datasets demonstrate the remarkable perfor- mance of L3A. As an exemplar-free method, L3A achieves competitive advantages over over replay-based and prompt-based methods, highlighting its effectiveness in addressing the challenges of MLCIL.
Theoretically, this weighting mechanism acts as a soft con- straint on the closed-form solution Equation (10), redis- tributing the residual errors in the least squares objective to penalize misclassifications of rare labels more severely. Cru- cially, the fusion of class- dlass-ppecific weights ensures a balanced representation of both rare and frequent labels, mitigating the risk of overfitting to dominant classes while maintaining sensitivity to underrepresented ones.
Table 3. Comparison of the results on PASCAL VOC dataset. Memory represents the number of replay samples, where in- dicates that the method is examplar-free. Data in bold represents the best results and data underline represents the second-best re- sults.
4.6. Ablation Study
VOC BO-C4
VOC B00-C2
Method
Memory
Avg. mAP
Last
Avg.
Last
The ablation study in Table 4 compares the performance of two key modules in L3A. We use the method of freezing the backbone and fine-tuning the classifier as the baseline,
mAP
mAP
mAP
Upper-bound
94.7
94.7
0
FT
82.1
62.9
70.1
43.0
the update of the classifier is based back nack-propagation.
iCaRL
87.2
72.4
79.0
66.7
We introduce two additional variants of L3A. (1) L3A w/o WAC: This variant optimizes using only the PL module. (2) L3A w/o PL: This variant optimizes using only the WAC.
BiC
86.8
72.2
81.7
69.7
ER
86.1
71.5
81.5
68.6
TPCIL
87.6
77.3
80.7
70.8
2/class
Removing the WAC (L3A w/o WAC) results in a signifi- cant performance drop, it indicates the model without WAC fails to keep the knowledge of past knowledge. The removal of the PL module (L3A w/o PL) also causes performance degradation. By integrating both two modules, the L3A can effectively acquire new knowledge while retaining previ- ously learned information, with benefits in addressing class imbalance. These results demonstrate both the strengths and effectiveness of the PI. module and WAC within the proposed L3A framework.
PODNet
88.1
76.6
81.2
71.4
DER+
87.9
76.1
82.3
70.6
KRT-R
90.7
83.4
87.7
80.5
CSC-R
92.4
87.9
91.6
87.8
CSC
90.4
85.1
89.0
83.8
CODA-P
0
90.6
84.5
90.2
85.0
MULTI-LANE
93.5
88.8
93.1
88.3
L3A
0
96.7
94.1
95.6
94.0
4.4. Pseudo-Label Addresses the Label Absence
Table 4. Performance comparison with pseudo-label (PL) module and weighted analytic classifier (WAC).
The PL module addresses the label absence problem by estimating missing labels from past phases. Specifically, it constructs an augmented label set Yiz, which combines the true labels from the current phase with pseudo-labels for previous phases. This augmentation enables the ana- lytic learning method in Equation (8) to update W, more accurately by leveraging complete representation.
COCO BO-C10
COCO B40-C10
Model
PL
WAC
Avg. mAP
Last
Avg.
Last
mAP
mAP
mAP
Baseline
x
x
48.5
25.6
45.9
26.3
(I) w/o WAC
/
x
70.9
56.7
77.7
72.1
Without the label augmentation, the true label set Y; only contains current phase labels, model progressively loses label information learned in earlier phases. This limitation undermines the precision of the analytic solution for Wt.
(2) w/o PL.
x
81.4
77.3
79.6
78.4
L3A
/
81.5
77.6
79.9
78.8
Regularization term. As shown in Table 5, observe that the performance of L3A in a wide range of y value. Setting 7 = 1000 provides good results across MS-COCO benchmarks.
4.5. Weighted Analytic Classifier Addresses the Class Imbalance
The WAC addresses the class imbalance problem through a weight fusion strategy. The sample-specific weight wi,i
Buffer layer size. Table 6 shows that the accuracy of L3A improves with an increase buffer layer size, but the im-
8

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Weighting mechanism. We evaluate different weighting strategies for the WAC module and find that the pro- posed class-specific weights of = 1 [(k) formulation achieves the best performance in Table 8. Its smoothing effect effectively alleviates the impact of extreme class im-
Table 5. The ablation study regularization term (7)-
COCO BO-CI0
COCO B40-C10
7
Avg. mAP
Last mAP
Avg. mAP
Last mAP
0.1
80,41
76.83
79.36
78.22
80.39
76.90
79.37
78.27
balance.
10
80,49
76.90
79.38
78.28
100
80.86
77.09
79.51
78.29
1000
81.38
77.56
79.89
78.75
Table 8. Different weighting mechanism the WAC module.
10000
81.22
77.61
79.67
78.69
COCO BO-C10
COCO B40-C10
Weighting mechanism
Avg. mAP
Last
Avg. mAP
Last
mAP
mAP
provement becomes negligible once the size reaches 8196.
The size of 8196 is sufficient and larger size entails more computation on matrix inversion.
1/V/00
81.45
77.57
79.91
78.77
1/f(0)
81.13
77.30
79.60
78.27
1((0g(f())++)
81.36
77.41
79.87
78.27
Table The ablation study on buffer layer size.
COCO BO-C10
COCO B40-C10
Buffer layer size
Avg. mAP
Last mAP
Avg. mAP
Last
5. Conclusions
mAP
In this paper, we focus on the multi-lahel class-incremental learning (MLCIL). We propose L3A to address label ab- sence, class imbalance, and privacy protection. L.3A an exemplar-free approach comprising two key modules: the pseudo-label (PL) module, which solves label absence by generating pseudo-labels for previously learned classes in current phase samples, and the weighted analytic classifier (WAC) adaptively balances class distributions by introduc- ing sample-specific weights while leveraging a closed-form analytical solution. Experimental results demonstrate that L3A achieves SOTA performance on MS-COCO and PAS- CAL VOC datasets.
512
79.33
75.24
77.78
76.43
1024
80.47
76.52
78.89
77.67
2048
81.08
77.20
79.58
78.44
4096
81.38
77.55
79.91
78.77
6144
81.43
77.60
79.94
78.75
8192
81.43
77.56
79.96
78.79
12288
81.40
77.56
79.93
78.82
Confidence threshold. We analyze the effect of the confi- dence threshold 1) in the PL. module. As shown in Table 7, the best performance is consistently observed when 17 is set around 0.7. Motivated by (Dong et al., 2023), we further ex- plore dynamic thresholding strategy (denoted as Dynamic in Table 7) that adaptively adjusts 1) based on the number of pseudo-labels generated each continual learning phase. However, this method introduces additional hyperparam- eters and offers no notable performance gain, suggesting that a fixed threshold of 0.7 remains more effective and practical choice.
A Acknovledgements
This research was supported by the National Natural Science Foundation of China (62306117 17, 62406114, 62472181), the Guangzhou Basic and Applied Basic Research Foun- dation (2024A04J3681), GJYC program of Guangzhou (2024D03J0005), National Key R & D Project from Minister of Science and Technology (2024YFA1215500) the Fundamental Research Funds for the Central Uni- versities (2024ZYGX/R074) Guangdong Basic and Ap- plied Basic Research Foundation (2024A1515010220, 2025A1515011 1413), and South China University of Technology- TCL Technology Innovation Fund.
Table 7. The ablation study confidence threshold (r)).
COCO BO-C10
COCO B40-C10
7
Avg. mAP
Last mAP
Avg. mAP
Last mAP
0.55
71.25
67.21
78.30
76.12
0.6
78.63
74.21
79.52
78.09
0.65
80.97
77.08
79.85
78.68
Impact Statement
0.7
81.45
77.57
79.91
78.77
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences our work, none which we feel must be specifically highlighted here.
0.75
81.36
77.58
79.80
78.62
0.8
81.34
77.57
79.69
78.47
Dynamic
81.12
76.56
79.86
78.45

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
References
He, R., Zhuang, H., Fang, D., Chen, Y., Tong, K., and Chen,
C. REAL: Representation enhanced analytic learning for exemplar-free class-incremental learning, March 2024.
Buzzega, P., Boschini, M., Porrello, A., Abati, D., and CALDERARA, S. Dark experience for general contin- ual learning: a strong, simple baseline. In Advances in Neural Information Processing Systems, volume 33, pp. 15920-15930. Curran Associates, Inc., 2020.
Hinton, G., Vinyals, O., and Dean, J. Distilling the knowl-
edge in neural network, March 2015. URL https:
/ / //aaxiiv..o org. / 1103. 02531.
Chaudhry, A., Dokania, P. K. Ajanthan, T., and Torr, P. H. S. Riemannian walk for incremental learning: Understand- ing forgetting and intransigence. In Computer Vision ECCV 2018, pp. 556-572, Cham, 2018. Springer In- ternational Publishing. ISBN 978-3- 978-3-030-01122-6. doi: 10. 10.1007997--3-00--11322--
Kim, C. D., Jeong, J., and Kim, G. Imbalanced continual learning with partitioning reservoir sampling. In ECCV, pp. 411-428. Springer, 2020.
Kirkpatrick, J., Pascanu, R.,, Rabinowitz, N., Veness, J., Des- jardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Trabska-Barwinska. A., Hassabis, D., Clopath, C., Ku- maran, D., and Hadsell, R. Overcoming catastrophic for- getting in neural networks. Proceedings of the National Academy of Sciences, 1 114(13)3321-33526 2017. doi: 10. 10.1073/pnas. 161.7///ma..1111855555... 183155114
De Min, T., Mancini, M., Lathuilière, S., Roy, S., and Ricci, E. Less is more: Summarizing patch tokens for efficient multi-label class-nncremental learning. In Conference on Lifelong Learning Agents. PMLR, 2024.
Li, Z. and Hoiem, D. Leaninn without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelli- gence, 40 40(12))2933-299) December 2017. ISSN 1939- 3539. doi: 10. 10911T/AM....11.2733301
Dong, S., Luo, H., He, Y., Wei, X., Cheng, J., and Gong, Y. Knowledge restore and transfer for multi-label class- incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 1871 18720, October 2023. doi: 10.1 N0.1109//CCV510000.222 01715.
Liang, Y.SS. and Li, W.-. Optimizing class distribution in memory for multi-label online continual learning, 2022. URL https: / arxiv. org/ aba 2209. 11469.
Douillard, A., Cord, M., Ollion, C., Robert, T., and Valle, E. PODNet: Pooled outputs distillation for small-tasks incremental learning. In Computer Vision ECCV 2020, pp. 86-102, Cham, 2020. Springer International Publishing. ISBN 978-3-030-58565-5, doi: 10.1007/ 978-3- 778-3-030-58565-5.6
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft COCO:
Common objects in context. In Computer Vi- sion - ECCV 2014, pp. 740-755, Cham, 2014. Springer International Publishing. ISBN 978-3-319-10602-1 doi: 10. 10.1007/978-3-11 1-112-1060221.1
Du, K., Zhou, Y., Lyu, F., Li, Lu, C., and Liu, G. Con- fidence self-calibration for multi-label class caassincrementll learning. In Computer Vision - ECCV 2024, pp. 232-252, Cham, 2025. Springer Nature Switzerland. ISBN 978-3- 031-72751-1. doi:
McCloskey, M. and Cohen, N. Catastrophic interfer- ence in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, volume 24, pp. 109-165. Academic Press, 1989. doi: 10. 00.1016//0079--42220000990333390
14.
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., and Zisserman, A. The paseal visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2):303-338, Jun 2010. ISSN 1573-1405. doi: 10.10077/112233-00--27755
Ratcliff, R. Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions. Psychological Review, 97(2):285-33) 1990. doi: 10.
001000033-299.99.22
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5533-5542, July 2017. doi: 10. 1000///VP..01...555
Fang, D., Zhu, Y., Lin, Z., Chen, C., Zeng. Z., and Zhuang, H. AIR: Analytic imbalance rectifier for continual learn- ing, 2024. URL 10349.
abs/248.. 2408.
Guo, P. and Lyu, M. R. A pseudoinverse learning algorithm for feedforward neural networks with stacked general- ization applications to software reliability growth data. Neurocomputing. 56:101-121, Januarr 2004. ISSN 0925-
Ridnik, T., Ben-Baruch, F., Zamir, N., Noy, A., Friedman, I., Protter, M., and Zeinik-Mandn L. Asymmetric loss for multi-label classification. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 82-91, 2021. doi: 10. 10.110//CCV489922.0000
2312. doi:
10

---

L3A: Label-Augmented Analytic Adaptation for Multi-Label Class Incremental Learning
Rolnick, D., Ahuja, A., Schwarz, Lillicaa, T., and Wayne, Lillierap, G. Experience replay for continual learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., December 2019.
Zhuang. H., Weng, Z., He, R., Lin, Z., and Zeng. Z. GKEAL: Gaussian kernel embedded analytic learning for few-shot class incremental task. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 7746-7755, June 2023. doi: D0.100/CVPR55729.2222 00748.
Schwarz, J., Czarnecki, W., Luketina, J., Grrassk--Barwinge A., Teh, Y. W., Pascanu, R., and Hadsell, R. Progress & compress: A scalable framework continual learning. In International Conference on Machine Learning, pp. 4528 4528-4537. PMLR, 2018.
Zhuang, H., Chen, Y., Fang, D., Hc, R., Tong, K., Wei, H., Zeng, Z., and Chen, C. GACL: Exemplar-free general- ized analytic continual learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc., December 2024a.
Smith, J. S., Karlinsky, L.., Gutta, V., Bescantee-oniiley P., Kim, D., Arbelle, A., Panda, R., Feris, R., and Kira, Z.
CODA-Prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Pro- ceedings of the IEEE/CVF Conference on Computer Vi-
Zhuang. H., He, R., Tong, K., Zeng, Z., Chen, C., and Lin, Z. DS-AL: A dual-stream analytic learning for exemplar- free class-incremental learning. Proceedings of the AAAI Conference on Artificial Intelligence, 17237 17244, March 2024b. doi: 10. 10009//aii.33811... 296670..
sion and Puttern Recognition (CVPR), pp. 11909-11919, June 2023, doi: 10.1109/VYR657222200001010
Tao, X., Chang, X., Hong, X., Wei, X., and Gong, Y.
Topology- proboeriirrs dass-incremental learning. In Com- puter Vision ECCV 2020, pp. 254-270, Cham, 2020. Springer International Publishing. ISBN 978-3-030- 58529-7. doi: 10. 10.1007/978-3-030-5855558
Wang, Z., Zhang, Z., Lee, C.-Y., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., and Pfister, T. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 139-149, June 2022. doi: 10.
NU9/CVVPR52688.2222.0000
Wu, Y., Chen, Y., Wang. L., Ye, Y., Liu, Z., Guo, Y., and Fu, Y. Large scale incremental learning. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 374-382, June 2019. doi: N0.110//CVR.2011.000
Zhou, D.- Sun, H.-L., Ye, H.-J., and Zhan, D.-C. Expand- able subspace ensemble for pre-trained model-based class- incremental learning. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 23554-23564, June 2024a. doi: 10.1 CVPR552333.2024
Zhou, D.-.., Wang, Q.- Qi. Z.HH. Ye, H.-J., Zhan, D.- C., and Liu, Z. Class-incremental learning: A survey. IEEE Transactions on Pattern Analysis and Machine In- telligence, (16(12)99811-88717 December 2024b. ISSN 1939-3539. doi: 10. 10.11099PAMM..22443299995
Zhuang, H., Weng, 7., Wei, H., Xie, R., Toh, K.-A., and Lin, Z. ACIL: Analytic class inas-innremmenal learning with abso- lute memorization and privacy protection. In Advances in Neural Information Processing Systems, volume 35, pp. 11602-11614 Curran Associates, Inc., 2022.

---

L3A: Label-uggmentedd Analytic Adaptation for Multi-Label Class Incremental Learning
A. Proof of Theorem 3.1
Proof. The representation of t can be written as
(
Wi-1 = ② x
N,X, -
-
R. 21111-11111
(14)
and W, can be written as
W = Σ
[T ... X xT228..
(15)
E
The autocorrelation matrix in phase 1 can be defined as
R
x
(16)
We can define Rg from R through Equation (16)
[-1 R. = Σ +
RR + x, 2.X,)-1
(17)
According to the Woodbury matrix identity, we have
(A+
-A--
Let A R U X. C = Ω2 and V = X, in Equation (17), we have
R = R R R (o,
+ X,R X,R:11.
(18)
Now we can obtain R, from last phase R.- and other current phase data (e.g., X,) recursively. The update of the autocorrelation matrix is proved.
Substituting Equation (17) into Equation (15), we have
W, R
x 22 P
x 2
R
R t-11 x, S.
Rs
[R
x,
X XT
W - R2X, 2xx,WX RXT2881
(19)
which completes the proof.
12


Reviews:
[{"review_id": "pvAv0m9zs7", "reviewer": "Reviewer_VTRd", "summary": "This paper addresses the challenges of multi-label class-incremental learning (MLCIL), specifically label absence, class imbalance, and privacy constraints. The proposed method, L3A, introduces two key modules: 1) Pseudo-Label (PL) Module: Generates pseudo-labels for historical classes using the previous classifier, addressing label absence, 2) Weighted Analytic Classifier (WAC): Uses a closed-form solution with sample-specific weights to balance class contributions, mitigating imbalance. L3A achieves state-of-the-art (SOTA) performance on MS-COCO and PASCAL VOC datasets, outperforming existing exemplar-free and replay-based methods.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your review and valuable comments. Here we address your concerns individually as follows:\n\n---\n**W1**: Recursive updates in Eq. 13 may be computationally intensive.\n\n---\nThe computational cost of our method primarily comes from the matrix inversion in Eq. 13 with the complexity of $\\mathcal{O}(d^2N_t + N_t^2d + N_td|C^{1:t}|)$ where `N_t` is the number of samples at stage *t* and `|C^{1:t}|` is the total number of classes up to stage *t*. The complexity scales up linearly with the dataset size and L3A provides training with one-epoch **closed-form solution**. This makes L3A more efficient than existing iterative BP optimization. As shown below, L3A is 10x faster than KRT.\n\n**Table1**: Comparison of methods in terms of runtime and memory usage in COCO B40C10 benchmark.\n|Method|Time(s)|GPU Memory(MB)|\n|-|-|-|\n|**L3A**|**1108**|**5750**|\n|KRT(BP-based)|16971|8170| \n\n---\n**W2**: Fixed confidence threshold in PL.\n\n---\nTo address this weakness, we explore and validate a dynamic threshold approach, where the threshold is adjusted based on the number of pseudo-labels to be generated in each phase. The results of this approach is denoted as \"Dynamic\" and is not better. It can add additional hyperparameters of target number of pseudo-labels, which may be unavailable in practice. We will further explore to address the fixed threshold in future.\n\n**Table2**: The threshold studies on confidence threshold ($\\eta$) in COCO dataset.\n|$\\eta$|B0-C10 Avg. mAP|B0-C10 Last mAP|B40-C10 Avg. mAP|B40-C10 Last mAP|\n|-|-|-|-|-|\n|0.55|71.25|67.21|78.30|76.12|\n|0.6|78.63|74.21|79.52|78.09|\n|0.65|80.97|77.08|79.85|78.68|\n|0.7|**81.45**|77.57|**79.91**|**78.77**|\n|0.75|81.36|**77.58**|79.80|78.62|\n|0.8|81.34|77.57|79.69|78.47|\n|*Dynamic*|81.12|76.56|79.86|78.45|\n\n---\n**S1**: Test on long-tailed datasets.\n\n---\nWe follow [1] to construct a long-tailed dataset (LT-COCO) by applying a power-law decay sampling method on the COCO dataset then validate on it. Although L3A's performance slightly declined under the long-tailed distribution, it still outperformed other methods, demonstrating its robustness in multi-label continual learning scenarios.  \n\n**Table 3**: Comparative results on the LT-COCO dataset (classes are randomly shuffled).  \n|Method|LT-COCO B0-C10 Avg. mAP|LT-COCO B0-C10 Last mAP|LT-COCO B40-C10 Avg. mAP|LT-COCO B40-C10 Last mAP| \n|-|-|-|-|-|\n|KRT|74.41|66.89|73.77|70.54|\n|L3A|**76.92**|**72.61**|**74.58**|**73.40**|\n\n---\n**S2**: Clarify the buffer layer. \n\n---\nThe buffer layer is used to project the model's output to a higher-dimensional space. According to Cover's theorem [2], projecting features into a higher-dimensional space via nonlinear projection functions can improve linear separability. Following ACIL [3], we use a randomly initialized linear projection in this paper and it is simple and effective. We will further clarify this in the text and explore other type of buffer in future.\n\n---\n**Q1**: Choosing weighting strategy.\n\n---\nIn this paper, we use the 1/sqrt(f^(k)) to compute the weight. To validate its superiority, we compare different weighting mechanisms and weighting strategy using 1/sqrt(f^(k)) achieved the best performance. Due to smoothing effct of 1/sqrt(f^(k)), WAC can maintain a good balance against extreme class imbalances. We will include this study in the manuscript.\n\n**Table4**: Different weighting mechanism in WAC module.\n|Weighting mechanism|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP| \n|-|-|-|-|-|\n|1/sqrt(f^(k))|**81.45**|**77.57**|**79.91**|**78.77**|\n|1/f^(k)|81.13|77.30|79.60|78.27|\n|1/(log(f^(k))+1)|81.36|77.41|79.87|78.27|\n\n---\n**Q2**: How Theorem 3.1 helps MLCIL.\n\n---\nOur work focuses on the MLCIL where we cannot directly apply standard ridge regression (Eq. 10) to optimize all data at once. Here, we extend standard ridge regression into the recursive analytical learning method presented in Theorem 3.1. This allows us to update the classifier through analytical solutions at each incremental learning phase, effectively addressing the MLICL problem.\n\nAlso, Theorem 3.1 incorporates two key components of our approach to address issues in MLCIL:  \n1. **PL** (Eq. 3) generates pseudo-labels for past phases, solving the label absence issue.\n2. **WAC** uses weighted iterative least squares to update classifier, addressing both class imbalance and privacy protection concerns. Theorem 3.1 provides theoretical support for L3A by proving optimal closed-form solution sample-specific weighting for MLCIL.\n\n[1] W. Tong, et al, \"Distribution-Balanced Loss for Multi-Label Classification in Long-Tailed Datasets,\" in EECV 2020.\n\n[2] K.-A. Toh, et al, \"Between classification-error approximation and weighted least-squares learning,\" TPAMI, 2008.\n\n[3] H. Zhuang, et al, \"ACIL: Analytic class-incremental learning with absolute memorization and privacy protection,\" in NeurIPS 2022.", "claims_and_evidence": "The claims (addressing label absence and class imbalance) made in the submission are well-supported as follows,   \n\n* Section 4.4 supports the label absence.\n* Section 4.5 supports the class imbalance.", "methods_and_evaluation_criteria": "Yes, the manuscript employs reasonable methods and evaluation criteria, including common datasets scenarios (MS-COCO and PASCAL VOC), well-recognized metrics (Avg. mAP and Last mAP).", "theoretical_claims": "Yes, I have reviewed them. The manuscript includes good theoretical claims, such as the process of optimal solution in Theorem 3.1.", "experimental_designs_or_analyses": "Yes, I have checked them. The overall design of experiments and analysis are quite good (e.g., sec. 4.4 and 4.5)", "relation_to_broader_scientific_literature": "The submission provides a very good discussion of literature. The branches of CL are well partitioned, and can provide reasonable guidance to broader group of readers.", "essential_references_not_discussed": "It seems the references cover all the important ones (just a feeling).", "other_strengths_and_weaknesses": "**Strengths**   \n* Exemplar-Free Privacy: L3A’s analytic updates avoid storing historical data, addressing privacy concerns.\n* Comprehensive Evaluation: Rigorous experiments across datasets and protocols validate L3A’s robustness.\n* Modular Design: The PL and WAC modules are independently justified and validated.\n\n**Weaknesses**   \n* The recursive updates of the autocorrelation matrix (Equation 13) may be computationally intensive for large-scale datasets?\n* The PL module’s confidence threshold (η) is fixed;", "other_comments_or_suggestions": "* Test L3A on datasets with extreme class imbalance (e.g., long-tailed distributions) to further validate its robustness. \n* Clarify the buffer layer’s role in feature extraction (e.g., why random linear projections are used).", "questions_for_authors": "* How to choose the weighting strategy? \n* How does the claim in Theorem 3.1 helps the multi-label incremnetal learning scenario?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zn4174IyW5", "reviewer": "Reviewer_x334", "summary": "This paper proposes Label-Augmented Analytic Adaptation (L3A), an exemplar-free approach without storing past samples, for Multi-Label Class Incremental Learning (MLCIL). It integrates two key modules: the pseudo-label (PL) module implements label augmentation by generating pseudo-labels for current phase samples, and the weighted analytic classifier (WAC) derives a closed-form solution for neural networks. It also introduces sample-specific weights to adaptively balance the class contribution and mitigate class imbalance. Experiments demonstrate the effectiveness of the proposed method. The main contributions of this paper are:\n\n- It proposes L3A, an exemplar-free approach that provides a closed-form solution to address catastrophic forgetting in MLCIL.\n- It introduces the PL module to implement label-augmented by generating labels for previously learned classes.\n- It introduces the WAC that iteratively updates the classifier by analytic learning and adaptively assigns sample-specific weights.\n\n## update after rebuttal\nThank you for your response. However, in my opinion, the motivation is still somehow confusing. Although the authors indicate some of MLCIL's challenges, the approach they take, especially in tackling class imbalance, is not specially designed for MLCIL's unique challenges. Therefore, I keep my rating.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback and constructive suggestions. Below we provide point-by-point responses to your comments:\n\n---\n**W1 & Q1**: Uniqueness of challenges in MLCIL problem.\n\n---\nFollowing prior research [1,2], we consider label absence as a unique challenge in MLCIL. Unlike single-label class-incremental learning (SLCIL), MLCIL inherently suffers from missing labels of previous and future labels during training. This increases the difficulty of learning new classes while preserving knowledge of previous ones since learning with only current labels could erase the previous knowledge, making MLCIL fundamentally different from SLCIL in handling label distribution shifts.\n\nFor class imbalance, while it is a common issue across machine learning tasks, it has a more severe impact in MLCIL due to catastrophic forgetting. In MLCIL, class distributions are imbalanced not only within each learning phase but also across different incremental stages, where multiple labels coexist per sample, making the imbalance problem even more complex. This leads to the severe under-representation of certain classes as training progresses. As a result, simple class weighting methods often fail to mitigate forgetting effectively in MLCIL, necessitating additional analysis of the class distribution shifts across different incremental learning phases.\n\nFor privacy protection, it is a challenge across CIL and existing MLCIL approaches rely on data replaying [1,2], which are incompatible with real-world scenarios when data storage is restricted. This makes exploring exemplar-free methods particularly crucial in MLCIL.\n\nWe will further clarify these challenges in the manuscript.\n\n---\n**W2 & Q2**: Parameter sensitivity analyses of $\\gamma$ and buffer layer size.\n\n---\nWe include the parameter analysis as below. \nFor $\\gamma$, as it increases, the performance of L3A first improves and then declines in **Table 1**. A value of $\\gamma$ (1000) achieves consistently good results. \n\n**Table1**: The ablation study on regularization term $\\gamma$. \n|$\\gamma$|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP| \n|-|-|-|-|-|\n|0.1|80.41|76.83|79.36|78.22| \n|1|80.39|76.90|79.37|78.27|\n|10|80.49|76.90|79.38|78.28|\n|100|80.86|77.09|79.51|78.29|\n|1000|**81.38**|77.56|**79.89**|**78.75**|\n|10000|81.22|**77.61**|79.67|78.69|\n\nFor the buffer size, in **Table 2**, the accuracy of L3A improves with an increase in buffer layer size, but the improvement becomes negligible once the size reaches 4096. The size of 4096 is sufficient and a larger size entails more computation on matrix inversion.\n\n**Table2**: The ablation study on buffer layer size.\n|Buffer layer size|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP|\n|-|-|-|-|-|\n|512|79.33|75.24|77.78|76.43|\n|1024|80.47|76.52|78.89|77.67|\n|2048|81.08|77.20|79.58|78.44|\n|4096|81.38|77.55|79.91|78.77|\n|6144|**81.43**|**77.60**|79.94|78.75|\n|8192|**81.43**|77.56|**79.96**|78.79|\n|12288|81.40|77.56|79.93|**78.82**|\n\n---\n**W3 & Q3**: PL module effectiveness analysis.\n\n---\nWe include the analysis as follow. \n\"The PL module to address the label absence problem. However, notably, we found that the improvement of using PL upon WAC is less significant in our experiments, while directly using PL upon baseline introduces large improvement. The reason could be that, during the training of WAC, it retains information of previous classes via $R_t$. Then adding the PL is less effective. Nonetheless, the use of the PL module could be still beneficial when using upon both baseline and WAC.\"\n\n---\n**S1**: Refine the threshold strategy and analyze limitations.\n\n1. **Threshold strategy**. To refine this threshold strategy, we explore and validate a dynamic threshold approach, where the threshold is adjusted based on the number of pseudo-labels to be generated in each phase. The result of this approach is denoted as \"Dynamic\" and is not better than current results. It can add additional hyperparameters of target number of pseudo-labels, which may be unavailable in practice. We will further explore to improve the threshold strategy in future.\n\n**Table3**: Studies on $\\eta$\n|$\\eta$|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP|\n|-|-|-|-|-|\n|0.55|71.25|67.21|78.30|76.12|\n|0.6|78.63|74.21|79.52|78.09|\n|0.65|80.97|77.08|79.85|78.68|\n|0.7|**81.45**|77.57|**79.91**|**78.77**|\n|0.75|81.36|**77.58**|79.80|78.62|\n|0.8|81.34|77.57|79.69|78.47|\n|*Dynamic*|81.12|76.56|79.86|78.45|\n\n2. **Limitations analysis**. L3A utilizes a frozen backbone and updates only the WAC. This could limit the improvement of feature extraction. We will explore to strengthen the backbone's representation to further improve L3A.\n\n[1] Dong, S., et al. Knowledge restore and transfer for multi-label classincremental learning. in ICCV, 2023.\n[2] K. Du, et al, \"Confidence self-calibration for multi-label class-incremental learning,\" in ECCV 2024.", "claims_and_evidence": "The claims are supported by evidence. The proposed method's superiority is evident in its consistent outperformance of existing methods.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense for the problem of MLCIL. The proposed L3A is designed to handle the specific challenges of MLCIL, and the evaluation metrics are appropriate for assessing the performance of the proposed method.", "theoretical_claims": "The theoretical claims in the paper are correct. However, Theorem 3.1 is not exactly called a \"Theorem\", but a computational method.", "experimental_designs_or_analyses": "The experimental designs and analyses are overall reasonable. However, the parameter sensitivity analyses are missing. It is not clear how the regularization term $\\gamma$ and the buffer layer size affect the final performance.", "relation_to_broader_scientific_literature": "The key contributions of the paper are related to the broader scientific literature. The authors discuss the related works in single-label CIL, multi-label CIL, and analytic continual learning (ACL). They clearly position their work within the context of existing research and highlight the novelty of their approach.", "essential_references_not_discussed": "There are no essential references that are missing from the paper.", "other_strengths_and_weaknesses": "### Strengths\n\n- The problem studied in this paper is interesting.\n\n- This paper is well written and in good sharp, which is easy to follow.\n\n- The experimental results are somehow promising.\n\n  \n\n### Weaknesses\n\n- This paper mainly tackles the **label absence**, **class imbalance**, and **privacy protection** challenges in MLCIL. However, the uniqueness of these challenges in the MLCIL problem is not clear. For example, the class imbalance problem, which can be solved using common weighting means, does not seem to be special in the MLCIL problem. The authors should further explain the uniqueness of these challenges in the MLCIL problem to highlight the significance of the proposed approach.\n- The parameter sensitivity analyses are missing. It is not clear how the regularization term $\\gamma$ and the buffer layer size affect the final performance.\n- From Table 4, one can observe that the effectiveness of PL in the proposed method is not obvious. The authors should analyze the reason for this phenomenon.", "other_comments_or_suggestions": "The chosen strategy of the threshold values used for generating pseudo-labels needs to be further refined. Moreover, discussing the limitations of the proposed method and potential future research directions can better facilitate the reader's understanding.", "questions_for_authors": "1. What are the uniqueness of the **label absence**, **class imbalance**, and **privacy protection** challenges in the MLCIL problem?\n2. How do the regularization term $\\gamma$ and the buffer layer size affect the final performance?\n3. The effectiveness of PL in the proposed method is not obvious. The authors should analyze the reason for this phenomenon.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Y6OzDUzkek", "reviewer": "Reviewer_ybc7", "summary": "This paper focuses on the multi-label class-incremental continual learning (ML-CICL) task. To address the challenges of missing historical labels and class imbalance in this task, the authors propose an exemplar-free L3A method. Specifically, L3A utilizes a pretrained language model (PLM) to supplement historical labels for samples, mitigating the issue of missing classes. Additionally, it employs weight adaptive calibration (WAC) to obtain sample-adaptive weights, thereby alleviating the class imbalance problem. Experimental results validate the effectiveness of the proposed model.\n\n### update after rebuttal\nBased on the author's response, I don't have any other questions, and I will keep my rating.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your constructive feedback and insightful suggestions. Below are our point-to-point responses:\n\n---\n**W1**: Comparison between WAC and learnable parameter-based methods.\n\n---\nTo clarify, the Weighted Analytic Classifier (WAC) is a recursive analytical learning method, where we address class imbalance using a weighting mechanism. Since analytical learning relies on least-squares solutions, it operates **without loss functions or gradient computations**. This differs from many parameter-based weighted methods using gradient-based optimization, for example: **KRT**, which adds an *Asymmetric Loss* [1] in the loss function to handle class imbalance. **CSC**, which incorporates *maximum entropy regularization* [2] in the loss function.\n\nIn contrast, L3A directly computes the **closed-form solution** for network parameters (without backpropagation), requiring less computational time and lower memory than most parameter-based methods. L3A outperforms the existing learnable parameter-based methods in terms of performance as shown in the manuscript and we further validate the efficiency here. L3A can also improve over existing methods in terms of efficiency.\n\n**Table1**: Comparison of methods in terms of runtime and memory usage in COCO B40C10 benchmark.\n|Method|Time (s)|GPU Memory (MB)|\n|-|-|-|  \n|**L3A**|1108|5750|\n|KRT (BP-based)|16971|8170|\n\n---\n**W2**: Lack of comparison with class imbalance methods.\n\n---\nWe have selected several class imbalance methods for comparison in the manuscript, including BiC and PRS, which are originally designed for single-label long-tailed continual learning. However, these methods perform poorly in multi-label scenarios. Another compared method KRT is designed for the class imbalance issue in MLCIL which L3A also outperforms. Detailed experimental results can be found in Table 1 of our paper. If you believe we need to include more relevant techniques, could you please provide some methods we need to include? We are open to include the experiments accordingly.\n\n---\n**W3**: Unclear pretrained parameters for comparison methods.\n\n---\nIn this paper, all the backbones including TResNetM and ViT-B/16 are pretrained on ImageNet-21k. We will clarify this in the manuscript. \n\n---\n**W4**: Large performance gap between baseline and mislabeling in Table-4.\n\n---\n1. **Performance Gap**, the baseline method in Table-4 of paper is a continuous learning method that does not employ any anti-forgetting method. Only the classifier is trained with a frozen backbone in the baseline and the baseline could suffer from catastrophic forgetting, resulting in a very low Last mAP. With the WAC and PL, L3A can achieve high performance with less forgetting.\n\n2. **Label Correction**, thank you for pointing the mislabeling problem. We will correct \"WAC\" to \"L3A\" in the manuscript.\n\n---\n**S1**: The motivation behind each module.\n\n---\nOur paper proposes two main modules, and their motivations are mentioned in the Introduction part. We provide additional clarification:  \n\n1. **To address the challenges of class imbalance and privacy protection**, we introduce a replay-free recursive analytic continual learning method. However, since this approach struggles with class imbalance, we further propose the Weighted Analytic Classifier (WAC), which employs a sample-specific weighting mechanism that adaptively adjusts the importance of samples weights based on class frequency. This correction ensures balanced least-squares optimization\n\n2. **To tackle the label absence problem**, we adopt a pseudo-labeling strategy (PL module) to generate labels for past tasks. This allows WAC to utilize complete label information during training.  \n\n---\n**S2**: Lack of experiments with different backbones.\n\n---\nWe add experiments with L3A on pre-trained ViT-B/16. The performance of L3A on ViT-B/16 is slightly worse. The reason could be that vanilla ViTs achieve the less local attention than CNN networks when dealing with complex multi-label images (e.g., scenes with many small objects) [3].\n\n**Table2**: Performance results on different backbones\n|Method|backbone|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP|\n|-|-|-|-|-|-|\n|L3A|TResNetM|81.45|77.57|79.91|78.77|\n|L3A|ViT-B/16|78.32|73.89|73.35|71.66|\n\n[1] T. Ridnik, et al, \"Asymmetric loss for multi-label classification,\" in ICCV 2021.\n\n[2] K. Du, et al, \"Confidence self-calibration for multi-label class-incremental learning,\" in ECCV 2024. \n\n[3] Z. Liu, et al, \"Swin Transformer: Hierarchical Vision Transformer using Shifted Windows,\" in ICCV 2021.", "claims_and_evidence": "The paper presents a approach to addressing the challenges in multi-label class-incremental continual learning. The proposed L3A method is supported by logical reasoning, proof reasoning, and the experimental results are provided to validate its effectiveness.", "methods_and_evaluation_criteria": "The evaluation criteria, including benchmark datasets and performance metrics, appear appropriate for assessing the effectiveness of the method. The experimental setup provides comparisons, supporting the validity of the proposed approach.", "theoretical_claims": "The paper provides a theoretical proof for WAC, which is intended to mitigate class imbalance by assigning sample-adaptive weights. The provided proof appears to be logically sound and aligns with established principles in adaptive weighting strategies.", "experimental_designs_or_analyses": "The experimental design is sound, as the proposed method is evaluated on two public benchmark datasets, COCO and PASCAL VOC, making the results reliable. However, the paper includes limited ablation studies, and a more detailed analysis of different backbone setting would further strengthen the evaluation.", "relation_to_broader_scientific_literature": "This method contributes to the broader scientific literature by proposing an exemplar-free approach for multi-label class-incremental continual learning, which reduces memory consumption compared to traditional exemplar-based methods.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "WAC is a dynamic weighting-based method. Can it be compared with learnable parameter-based methods? What are the key differences between WAC and such methods?\n\nIn other tasks, such as long-tailed classification, there are existing methods to address class imbalance. Would it be possible to compare WAC with these approaches?\n\nRegarding pretrained parameters, L3A uses pretrained weights from ImageNet-21k, but the paper does not specify what pretrained parameters are used for the comparison methods. Does this affect fairness?\n\nIn Table 4, why is there a difference of more than 50 between the baseline and the last row? Additionally, the last row should correspond to the L3A method, but it is incorrectly labeled as WAC.", "other_comments_or_suggestions": "The motivation behind each module is not sufficiently explained. It would be helpful to provide more detailed justifications.\n\nThe paper lacks experiments with different backbones. It would be beneficial to include such experiments.", "questions_for_authors": "None", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X4bHBKpFGW", "reviewer": "Reviewer_jGY3", "summary": "This work studies the task of Multi-Label Class Incremental Learning (MLCIL), where a model is tasked to incrementally learn to assign multiple labels for each image during each incremental session. The goal is to not forget previously leanred classes while learning novel classes. This task is an extension of class incremental learning.\n\nTo tackle MLCIL, the authors proposed Label-Augmented Analytic Adaptation (L3A), an exemplar-free approach without storing past sample. Exemplar-free based approach has advantage of protecting privacy, since no past data will be stored.\n\nL3A has two modules: i)  the Pseudo-Label (PL) module, which generates pseudo-labels for previously learned classes to address label absence, and the ii) Weighted Analytic Classifier (WAC), which uses a closed-form solution with sample-specific weights to mitigate class imbalance. \n\nExperiments on MS-COCO and PASCAL VOC (multi-label, or say tags, benchmarks) shows that L3A achieved SOTA performance.\n\nThe main contribution of this paper is an exemplar-free MLCIL solution L3A that avoids storing past samples, making it suitable for privacy-sensitive applications, and provides a theoretical framework for recursive updates of the classifier using an autocorrelation matrix.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your insightful feedback and constructive suggestions. Below, we provide a point-by-point response to your questions and concerns.\n\n---\n**Q1**: Ambiguous confidence scores and refinement mechanisms in the PL module.\n\n---\n1. **Ambiguous confidence scores**, To clarify this issue, we validate our L3A with different thresholds. The results are shown below and we find that the change of results from 0.65 to 0.80 is not significant. This demonstrates that PL could be robust to ambiguous confidence scores since varying thresholds do not have large effect. \n\n**Table1**: The threshold studies on confidence threshold ($\\eta$) in COCO dataset.\n| $\\eta$ | COCO B0-C10 Avg. mAP | COCO B0-C10 Last mAP | COCO B40-C10 Avg. mAP | COCO B40-C10 Last mAP |\n|-|-|-|-|-|\n|0.55|71.25|67.21|78.30|76.12|\n|0.60|78.63|74.21|79.52|78.09|\n|0.65|80.97|77.08|79.85|78.68|\n|0.70|**81.45**|77.57|**79.91**|**78.77**|\n|0.75|81.36|**77.58**|79.80|78.62|\n|0.80|81.34|77.57|79.69|78.47|\n\n2. **Self-Correction**, During the learning, pseudo-labels are not treated as static annotations. Instead, they are continuously re-evaluated as new data is introduced. This iterative correction mechanism helps mitigate the impact of early-stage pseudo-labeling errors.\n\n---\n**Q2**: The choice of different weighting mechanism in WAC module.\n\n---\nIn this paper, we use the 1/sqrt(f^(k)) to compute the weight. To validate its superiority, we compare different weighting mechanisms and weighting strategy using 1/sqrt(f^(k)) achieved the best performance. Due to smoothing effect of 1/sqrt(f^(k)), WAC can maintain a good balance against extreme class imbalances. We will include this study in the manuscript.\n\n**Table2**: Different weighting mechanism in WAC module.\n|Weighting mechanism|COCO B0-C10 Avg. mAP|COCO B0-C10 Last mAP|COCO B40-C10 Avg. mAP|COCO B40-C10 Last mAP| \n|-|-|-|-|-|\n|1/sqrt(f^(k))|**81.45**|**77.57**|**79.91**|**78.77**|\n|1/f^(k)|81.13|77.30|79.60|78.27|\n|1/(log(f^(k))+1)|81.36|77.41|79.87|78.27|\n\n---\n**Q3**: Analyze the generalization of the L3A method in more diverse multi-label settings.\n\n---\nRegarding dataset selection, we followed previous multi-label class incremental learning studies [1,2] by using COCO and VOC datasets as benchmarks. The data for multi-label classification needs to contain multi-classes within a single sample, and \"multiple objects in every single image\" is easy to meet this requirements. We acknowledge that real-world multi-label data can extend beyond object tags to include attributes like time, mood, action, or fine-grained details, but we do not find a dataset having such features while meeting the requirement of multi-classes in a sample (ImageNet-C performs different noises on an image to generate several samples). \n\nFor the scenario that multi-labels from different aspects, our L3A can also work in theory. When denoting the fine-grained tags as distinct labels, there is less difference between the labels coming from different objects in the training of WAC. Moreover, the buffer layer could enhance separability of features [3] then help distinguish originally similar features during fine-grained classification. \n\nWe are willing to include the validation of L3A on such a dataset, but unfortunately, we did not find a suitable dataset which has different aspects and meets the requirement of multi-classes in a single sample with limited time. Since we may overlook any relevant datasets, could you please designate a suitable dataset? We are open to include the validation accordingly.\n\n---\n**Q4**: Comparison with open-vocabulary detectors and taggers methods.\n\n---\nIn this paper, we focus on the **class-incremental continual learning**, where the goal is for the model to learn new classes while retaining previously learned knowledge. We need to achieve good performances on both old and new datasets. In contrast, open-vocabulary detectors and taggers typically evaluate on data outside their training distribution (unseen data) to test generalization ability. The motivation is quite different from the continual learning and it may be not appropriate to directly transfer L3A to this task. \n\n[1] S. Dong, et al, \"Knowledge restore and transfer for multi-label class-incremental learning,\" in CVPR 2023.\n\n[2] K. Du, et al, \"Confidence self-calibration for multi-label class-incremental learning,\" in ECCV 2024.\n\n[3] K.-A. Toh, et al, \"Between classification-error approximation and weighted least-squares learning,\" TPAMI, 2008.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes.", "experimental_designs_or_analyses": "Yes, the experimental design is fair and well-established.", "relation_to_broader_scientific_literature": "The main contribution of this paper is an exemplar-free MLCIL solution L3A that avoids storing past samples, making it suitable for privacy-sensitive applications, and provides a theoretical framework for recursive updates of the classifier using an autocorrelation matrix.", "essential_references_not_discussed": "To the best of my knowledge, this paper sufficiently discussed related literature.", "other_strengths_and_weaknesses": "Strengths:\n1) The proposed appraoch is, as a whole, novel to a certain degree, with a solid theoretical framework.\n2)  The Pseudo-Label (PL) module generates pseudo-labels for previously learned classes, effectively addressing the challenge of label absence in multi-label incremental learning.  However, this has been widely used in examplar-free pipelines.\n3) The paper is well-written\n4) The performance of the proposed method L3A looks good, consistenly outperforming the compared methods across two benchmarks.", "other_comments_or_suggestions": "See below.", "questions_for_authors": "Q1: How does the Pseudo-Label (PL) module handle cases where the confidence scores for pseudo-labels are ambiguous (e.g., close to the threshold)? Does the method incorporate any mechanisms to refine or correct pseudo-labels over time?\n\nQ2: The WAC uses a weighting mechanism based on class frequency ((v^{(k)} = 1/\\sqrt{f^{(k)}})). Why was this specific weighting function chosen, and how does it compare to other possible weighting schemes (e.g., inverse frequency or log-based weighting)?\n\nQ3 (main):  Do the authors think the two studied benchmarks are sufficient? MS-COCO and PSACAL VOC indeed have multiple labels. However, the labels are mainly object tags, since these images have multiple objects in every single image. In reality, \"Multiple-label\" should come from different pespectives about the visual content, such as: time of the day, mood, action, color scheme, even noises type (ImageNet-C). Will the proposed method also work in the settings where the multiple-lables from different aspects? In addition, when the labels are from different granularity, such as: {person, dog, cat} is a common granularity, however, the label can also be: {green T-shirt with a dog,  black dots, yellow ear} where the labels are from more fine-grained visual details. How can we prove the proposed method also effective in such scenarios? \n\nQ4: It would be interesting to compare the proposed method with open-vocabulary detectors and taggers.\n\nMy main concern is about the insufficient evaluation of the propsoed method.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: In this paper, the authors propose a new task, multi-label class-incremental learning, which extends class-incremental learning to a real-world scenario where each sample may belong to multiple classes. They also introduce a new method, label-augmented analytic adaptation, for this new task. 

This paper was reviewed by four expert reviewers. After the rebuttal period, it received mixed ratings. Two reviewers supported this paper, while the remaining two reviewers gave negative ratings. The authors provided many additional results during the rebuttal period. I checked the discussion and believe most of the remaining concerns have been addressed during the rebuttal period. Therefore, I tend to accept this paper. However, Reviewer x334's remaining concerns, such as "this approach is not specially designed for multi-label class-incremental learning's unique challenges", should be discussed in the final version.
Decision: Accept (poster)