POC: Preventing the Over-Collapse of Classes for Class-Incremental Learning

Authors: Shengchao Zhou, Bolin Ni, Youzhi Zhang, Zhaoxiang Zhang, XIAOJUAN QI, Gaofeng Meng
Primary area: transfer learning, meta learning, and lifelong learning
Keywords: Class-Incremental Learning, Over-Collapse, Catastrophic Forgetting

Abstract:
Deep neural network-based classification models often suffer from catastrophic forgetting during class-incremental learning (CIL). Previous studies reveal that it results from the overlap between seen and future classes after being mapped by model to its feature space through extracting the features. In this paper, we analyze that this overlap mainly results from the $\textit{over-collapse}$ of seen classes, where the model tends to map originally separated one seen class and its adjacent regions in input space to be mixed in the feature space, making them indistinguishable. To this end, we propose a two-step framework to $\textbf{P}$revent the $\textbf{O}$ver-$\textbf{C}$ollapse (POC). During training, POC first learns and applies a set of transformations to the training samples of seen classes. Based on our theoretical analysis, the transformation results will locate in the adjacent regions of the seen classes in the input space so that we can let them represent the adjacent regions. Then, the model's optimization objective is modified to additionally classify between the seen classes and the adjacent regions, separating them in model's feature space so that preventing the over-collapse. To retain the model's generalization on the seen classes, a deterministic contrastive loss that makes the separate features of seen classes and adjacent regions close is further introduced. Since POC uses the adjacent regions exclusively for classification, it can be easily adopted by existing CIL methods. Experiments on CIFAR-100 and ImageNet demonstrate that POC effectively increases the last/average incremental accuracy of six SOTA CIL methods by 3.5\%/3.0\% on average respectively.

Decision: reject
Reviewer scores: 5, 6, 5

Reviews and discussion:
<thread>
creation date:
1729349301751

thread_type:
official_review

<thread_object>
creator: reviewer_uhwh

<content>
summary:
This paper aims to improve the generalization on seen classes in CIL by preventing the over-collapse (POC) of seen classes. To this end, the authors generate samples in adjacent regions by some learnable transformations and making the classification model predict them with a modified loss, which is much similar to IL2A where it predicts auxiliary classes generated by mixup. The authors use theories in OOD detection to prove such transformations generate samples in adjacent regions. Furthermore, the author claims the generated samples in adjacent regions are prone to be far away from the seen class, thus introduces the deterministic contrastive loss (DCL) to make them closer to the seen class. Finally, the authors perform performance comparisons with SOTAs and similar works, verify the effectiveness of DCL and POC with ablation study on them and plotting ICD and ICG metrics during the incremental training.

strengths:
- The reported performance of the method out-performs similar works like IL2A.
- The proposed method is compatible with various methods in CIL.

weaknesses:
- The reason why over-collapse leads to forgetting is not clear enough. It seems to assume the samples in the future class are in the adjacent area of the previous samples.
- The DCL is somewhat not well-motivated, there is no empirical or theoretical evidence provided about the _far away_ projection. Instead, the authors state that the distance of the transformed sample is upper-bounded in Proposition 3.2.

questions:
See weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1729837353941

thread_type:
official_review

<thread_object>
creator: reviewer_rqbh

<content>
summary:
This paper tackles the over-collapse phenomenon in class incremental learning (CIL) that makes it difficult to distinguish the seen and unseen classes.
To address this issue, the authors suggest distinguishing the seen classes and their transformed versions.
To generate samples close yet adequately distant from the original ones, the authors suggest rotation and a learnable affine transformation and theoretically demonstrate the effectiveness of these transformations.
The authors argue that the proposed method can prevent over-collapse phenomenon, thereby enhancing generalization ability to unseen classes.

strengths:
The most intriguing aspect is generating samples that are close yet distinct from the original ones.
To prevent the rotated samples from being overly similar to the original ones, the authors propose learning a set of affine transformations, ensuring the generated samples are adequately adjacent but maintain a sufficient distance from the originals.
The theoretical analysis further supports the effectiveness of the proposed transformation for generating adjacent samples.

weaknesses:
It seems that there are no significant issues on the paper.
One minor concern may be the generalization ability of the proposed method.
Can the proposed method be applied to other tasks over the image classification task?
The reviewer thinks it is somewhat difficult to directly generalize the proposed method to other tasks, which may limit its value.

questions:
Please refer to the weakness part.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730700174518

thread_type:
official_review

<thread_object>
creator: reviewer_dvqr

<content>
summary:
the authors propose a two-step framework called Prevent the Over-Collapse (POC) for class incremental learning. During training, POC applies transformations to training samples of seen classes, maintaining their distinction in the feature space. It also introduces an expanded classifier to separate seen classes from adjacent regions. In the testing phase, the expanded classifier is masked, allowing classification of seen classes without extra computational costs. POC incorporates a deterministic contrastive loss to keep adjacent regions close to their original classes, enhancing generalization. Experimental results on CIFAR-100 and ImageNet show that POC improves the last and average incremental accuracy of several state-of-the-art CIL methods by 3.5% and 3.0%, respectively.

strengths:
-	The proposed POC framework effectively prevents the overlap between seen and future classes in the feature space as shown in fig.3. This innovative approach might enhance the model's ability to generalize across tasks.
-	The experimental results show that POC can robustly enhance the performance of various CIL approaches across several approaches.
-	The article provides sufficient evidence for some of its claims in the appendix.

weaknesses:
-	An important assumption of the POC is that it addresses the issue of over-collapse, which can lead to catastrophic forgetting. However, there is insufficient literature to prove that over-collapse is the cause of catastrophic forgetting. The citations provided in the article, such as Masana et al., 2022 on line 184, do not offer relevant explanations, and the article also does not sufficiently analyze the over-collapse phenomenon as claimed in its contributions. This results in the article appearing to lack a reasonable motivation for its claims.
-	The POC requires inference on multiple augmented images, which may lead to a significant increase in training costs. However, the article does not discuss this issue.
-	I’m not certain that the primary reason POC is effective is due to its backbone having seen multiple augmented images. I believe it is necessary to conduct an experiment where all images augmented by learnable augmentations are used as positive samples corresponding to their categories for direct training. I think this approach could also yield some performance improvement, and it might not perform worse than the results shown by POC.

questions:
All my concerns are mentioned in the weakness. And I think the experiments in the third line of weakness should be conducted.
</content>
</thread_object>

</thread>
<thread>
creation date:
1731922943170

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Gratitude to All the Reviewers

comment:
We would like to express our sincere gratitude to all the reviewers for their careful and thoughtful review of our manuscript, as well as for the valuable feedback they have provided. Their comments have contributed to the improvement of our paper and have deepened our understanding of the subject. We have uploaded our responses to each reviewer's comments, along with a revised version of the paper, which we hope adequately addresses the points raised.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732499842330

thread_type:
official_comment

<thread_object>
creator: authors

<content>
comment:
Dear Reviewers,

Thank you once again for your insightful comments and suggestions—they have been immensely helpful in improving our work. We have carefully addressed each of your points and included additional experimental results to clarify and strengthen our arguments.

We understand this may be a particularly busy time, and we deeply appreciate your taking the time to review our responses. Please let us know if our revisions adequately address your concerns. Should you have any further feedback, we will do our utmost to address it promptly.

Best regards,

The Authors of Submission 5728
</content>
</thread_object>

</thread>


Metareview:
metareview:
This submission received two negative sores and a positive score after rebuttal. After carefully reading the paper, the review comments, the AC can not recommend the acceptance of this submission, as the average score is under the threshold bar and the concerns about the motivation and justification of the proposed approach remain. The AC recognizes the contributions confirmed by the reviewers, and encourages authors to update the paper according to the discussion and submit it to the upcoming conference.