Addressing Imbalance for Class Incremental Learning in Medical Image Classification

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-04-02 05:25:37

Abstract: Deep convolutional neural networks have made significant breakthroughs in medical image classification, under the assumption that training samples from all classes are simultaneously available. However, in real-world medical scenarios, there's a common need to continuously learn about new diseases, leading to the emerging field of class incremental learning (CIL) in the medical domain. Typically, CIL suffers from catastrophic forgetting when trained on new classes. This phenomenon is mainly caused by the imbalance between old and new classes, and it becomes even more challenging with imbalanced medical datasets. In this work, we introduce two simple yet effective plug-in methods to mitigate the adverse effects of the imbalance. First, we propose a CIL-balanced classification loss to mitigate the classifier bias toward majority classes via logit adjustment. Second, we propose a distribution margin loss that not only alleviates the inter-class overlap in embedding space but also enforces the intra-class compactness. We evaluate the effectiveness of our method with extensive experiments on three benchmark datasets (CCH5000, HAM10000, and EyePACS). The results demonstrate that our approach outperforms state-of-the-art methods.

Main paper: 
 related work 2.1 class incremental learning: 
 Class incremental learning aims to train a model from a sequence of classes, ensuring its performance across all the classes. Existing class incremental learning methods can be roughly divided into three groups: regularization-based, structure-based, and memorybased.Regularization-based methods [10, 13, 18, 36] apply additional constraints to prevent the existing model from forgetting previous knowledge. LUCIR [18] constrains the orientation of the features to preserve the geometric configuration of old classes. PODNet [13] introduces a novel spatial distillation not only for the outputs of the final layer but also for the intermediate features to mitigate representation forgetting. However, regularization-based methods still suffer from feature degradation of old knowledge due to the limited access to old data [46] .Structure-based methods [14, 19, 27, 40, 46] aim to preserve the learned parameters associated with old classes while incrementally creating modules to enhance the model's capacity to acquire new knowledge. Recently, DER [46] adds a new feature extractor at each step and then concatenates the extracted features for classification. DyTox [14] applies transformer [11] to incremental learning and dynamically expands task tokens when learning new classes. Nevertheless, dynamically adding new modules will lead to an explosion in the number of parameters and an increase in the independence between each feature extractor to harm performance in new classes [40] .Memory-based methods [3, 4, 34, 42, 45] address the challenge of forgetting by storing a limited number of representative samples from old classes in a memory buffer. iCaRL [34] learns the exemplarbased data representation and makes predictions using a nearestmean-of-exemplars classifier. GEM [4] uses exemplars for gradient projection to overcome forgetting. Additionally, some approaches employ generative models to synthesize old class samples for data rehearsal [31, 35, 41] while other works consider saving feature embeddings instead of raw images [20] . In our work, we follow the memory-based approach to directly store a small subset of old class data for rehearsal.Class imbalance is a key challenge for class incremental learning [18] . Due to the only access to the classes of the current step, the classifier is severely biased, and there is an inevitable overlap and confusion between the feature space representations of old and new classes [23] . Even with the limited size of the memory buffer, the biased optimization by imbalanced data between old and new classes is still a crucial problem that causes catastrophic forgetting [28, 34] . To cope with it, SS-IL [1] isolates the computation of the softmax probabilities on old and new classes for bias compensation. BiC [45] introduces a bias correction layer to address the bias in the last fully connected layer.In real-world medical scenarios, most existing datasets contain highly imbalanced numbers of samples [32] , which leads to a more severe forgetting. To the best of our knowledge, LO2LN [6] is the first attempt to address the problem of class incremental learning in medical image classification. First, they utilize the class-balanced focal loss [8] to avoid the classifier bias. However, the class-balanced focal loss is not specialized and efficient for incremental learning. Second, they introduce the margin ranking loss [18] to separate old and new classes. We argue that this constraint may not be sufficiently robust, resulting in large clusters within classes (intra-class) and potential overlaps between classes (inter-class). By contrast, in this paper, we propose two simple yet effective plug-in loss objectives: (i) a CIL-balanced classification loss to alleviate prediction bias by adjusting the logits, and (ii) a distribution margin loss that can push the distributions of old and new classes away and provide more compact intra-class clustering simultaneously. 
method: 
 In this section, we first outline the setting of class incremental learning in medical image classification (Sec. 3.1). Then, we provide a detailed description of the two proposed loss objectives: CILbalanced classification loss (Sec. 3.2) and distribution margin loss (Sec. 3.3).Class incremental learning aims to train a model from a sequence of data incrementally. Specifically, we denote the sequence of tasks asD = {D 1 , D 2 , D 3 , ..., D 𝑇 }, where D 𝑡 = (X 𝑡 , Y 𝑡 ) = (𝑥 𝑡 𝑖 , 𝑦 𝑡 𝑖 ) 𝑛 𝑡 𝑖=1represents the training set from step 𝑡 with 𝑛 𝑡 instances. Here, 𝑥 𝑡 𝑖 ∈ X 𝑡 is a sample and 𝑦 𝑡 𝑖 ∈ Y 𝑡 is the corresponding label. The label space of the model is all seen classes Y 𝑖:𝑡 = 𝑡 𝑖=1 Y 𝑖 , where Y 𝑡 ∩ Y 𝑡 ′ = ∅ for all 𝑡 ≠ 𝑡 ′ . Inspired by memory-based methods [3, 34, 45] , our method consistently samples 𝑚 representative instances from each old class and store them in a memory buffer M 𝑡 , which is updated after the training step 𝑡 is completed. It should be mentioned that only data from D𝑡 = D 𝑡 ∪ M 𝑡 -1 is available for training during the 𝑡-th step.Classically, the model at step t can be written as the composition of two functions:𝑓 𝑡 = 𝑓 𝑡 𝜃 • 𝑓 𝑡 𝜙 (•), where 𝑓 𝑡 𝜙 represents a feature extractor, and 𝑓 𝑡 𝜃 represents a classifier. For an input sample 𝑥 𝑖 , its feature representation is denoted as ℎ 𝑡 𝑖 = 𝑓 𝑡 𝜙 (𝑥 𝑖 ). We employ cosine normalization [18] as the classifier 𝑓 𝑡 𝜃 . Consequently, the predicted logit 𝑝 𝑡 𝑖,𝑐 for class 𝑐 at step 𝑡 can be calculated from ℎ 𝑡 𝑖 as:EQUATIONwhere 𝑤 𝑐 are the weights for class 𝑐 in the classifier layer, 𝜂 is a learnable scalar, and ⟨•, •⟩ denotes the cosine similarity between two vectors.As claimed in previous works [28, 32] , the inherent imbalance in medical datasets and the imbalance in class incremental learning can lead to a biased classifier. Inspired by [30] , we aim to mitigate this issue by adjusting the logits according to category frequency. However, for a memory-based method in class incremental learning, only the data from D𝑡 is available at step 𝑡, which consists of the memory buffer M 𝑡 -1 and the training set D 𝑡 . Hence, we define the category frequency as follows:EQUATIONwhere 𝑞 𝑐 is the number of training samples for class 𝑐, and |•| is the cardinality of a given set. After that, we add 𝑙𝑜𝑔 𝑟 𝑐 to the output logits during training. Thus, the logit-balanced classification loss can be formulated as:EQUATIONTo explain how our method works, we reformulate Eq. 3 into Eq. 4 by introducing 𝑣 𝑦 𝑖 ,𝑗 := 𝑙𝑜𝑔 𝑟 𝑗 -𝑙𝑜𝑔 𝑟 𝑦 𝑖 , which are defined as follows:EQUATIONwhere:EQUATIONIt is known that traditional softmax loss necessitates 𝑝 𝑡 𝑖,𝑦 𝑖 > 𝑝 𝑡 𝑖,𝑗for the accurate classification of sample 𝑥 𝑖 . In order to prioritize the learning of old and rare classes, we employ the following logit adjustment strategy. Specifically, when 𝑦 𝑖 ∈ Y 1:𝑡 -1 and 𝑗 ∈ Y 𝑡 (the first line in Eq. 5), we instead require𝑝 𝑡 𝑖,𝑦 𝑖 > 𝑝 𝑡 𝑖,𝑗 + log 𝑞 𝑗 /𝑚 [> 0]. Hence, it is clear that we require a larger 𝑝 𝑡 𝑖,𝑦 𝑖 , which makes the training process place more emphasis on old class 𝑦 𝑖 than previously. However, if both 𝑦 𝑖 and 𝑗 are within Y 1:𝑡 -1 (the second line in Eq. 5), the logit remains unchanged, since they are both old classes with the same memory size.For the other two cases when 𝑦 𝑖 ∈ Y 𝑡 . If 𝑗 ∈ Y 1:𝑡 -1 (the third line in Eq. 5), the term log 𝑚/𝑞 𝑦 𝑖 < 0 suggests that old class 𝑗 receives more emphasis. If 𝑗 ∈ Y 𝑡 (the fourth line in Eq. 5), more emphasis is placed on the class 𝑦 𝑖 when it has fewer instances, and conversely, the focus is on class 𝑗 when the size 𝑞 𝑗 is smaller. Therefore, the logit-balanced classification loss can effectively reduce the bias towards new and frequent classes.To further control the balance between the old and new classes, we introduce a scale factor 𝛾:EQUATIONwhere 𝛼 ∈ [0, 1] is a trade-off coefficient for each dataset. With the help of this scale factor, the CIL-balanced classification loss can be written as:EQUATIONwhich reduces the output values for the old classes while maintaining the outputs for the new classes unchanged, thereby encouraging the model to produce larger logits for these old ones. Consequently, this scaling strategy further mitigates the issue of imbalance in class incremental learning. In this context, although a decrease in 𝛼 improves the significance of old classes, it may affect the model's learning of new ones. Thus, determining the optimal 𝛼 becomes crucial for achieving a better trade-off (see Sec. 4.4). Notably, when 𝛼 is assigned a value of 1, the current CIL-balanced classification loss degrades to the logit-balanced classification loss (Eq. 4).In class incremental learning, the representations of the old and new classes would be easily overlapped in the deep feature space [47] .To address this issue, margin loss [5] is introduced to avoid the ambiguities between old and new classes. In detail, the vanilla margin loss aims to ensure that the distance from the anchor to the positive (embedding of the ground-truth old class) is less than the distance of the anchor from the negative (embedding of the new class) to meet a predefined margin 𝑚, which can be computed as:� � � � � � � � , � � � � , � � ∟ ∟ (a) � � , � � � � , � � � � � � � � ∟ ∟ (b) � � � � � � � � , � � � � , � � ∟ ∟ (c) � � � � � � � � , � � � � , � � ∟ ∟ (d)EQUATIONwhere ⟨•, •⟩ denotes the cosine similarity and the margin 𝑚 is set to 0.4 for all experiments. However, the vanilla margin loss exhibits two limitations. First, it only focuses on the triplet: anchor, positive, and negative embeddings. Even if the distance from the anchor to the negative exceeds that to the positive by a margin 𝑚, their distributions may remain close or even overlap, thereby introducing potential ambiguities in classification (shown in Fig. 2a ). Second, while the vanilla margin loss aims to separate the ground-truth old class from new classes (maximizing inter-class distance), it fails to adequately address the minimization of intra-class distance, often leading to large intraclass clustering (shown in Fig. 2c ).To address the above limitations, we try to restore the class distribution and design a novel distribution margin loss that contains two loss terms. The first term optimizes the triples by ensuring that the distance from the anchor to the positive embedding is less than its distance to the negative class distributions by the margin L 𝑐𝑏𝑐 ← Eq. 7;⊲ CIL-Balanced Classification Loss 5:L 𝑑𝑚 ← Eq. 9;⊲ Distribution Margin Loss 6:L 𝑘𝑑 ← Eq. 11;⊲ Knowledge Distillation Loss L 𝑎𝑙𝑙 ← Eq. 12;⊲ Overall Loss 9: until reaches predefined epoch 𝑚, rather than merely to the negative embeddings. By optimizing this term, the distribution margin loss can push the samples of old classes away from the new class distributions to facilitate the interclass separation (shown in Fig. 2b ). The second term attempts to maintain the anchor embedding within the distribution range of its corresponding class, thus obtaining compact intra-class clustering (shown in Fig. 2d ). Accordingly, the distribution margin loss can be formulated as:EQUATIONwhere ŵ𝑐 represents the distribution range of class 𝑐. Specifically, we model the data distribution of each class in the feature space by applying a Gaussian distribution around their centroids. However, due to the imbalanced number of samples across different classes, the features of classes with limited instances may get squeezed into a narrow area in the feature space [43] . As a result, we assign a larger distribution range to the majority classes and a more restricted range to the minority classes:EQUATIONwhere r𝑐 represents the inherent ratio of class 𝑐 among all seen classes, and 𝜂 ∼ N (0, 1) is a Gaussian noise which has the same dimension as the classifier weight.To prevent forgetting and maintain the discrimination ability, we also apply knowledge distillation loss [17] to build a mapping between the old and the current model:EQUATIONTherefore, the overall loss is defined as:EQUATIONwhere 𝜆 𝑑 and 𝜆 𝑘 are the hyper-parameters for balancing the importance of each loss. We show the guideline of our method at incremental step t in Alg. 1. 
experiments 4.1 experimental setups: 
 Datasets. Following the benchmark setting [6] , we evaluate the performance on CCH5000 [21] , HAM10000 [38] , and EyePACS [7] .• CCH5000: consists of histological images in human colorectal cancer. This dataset contains 8 different classes with 625 images per class: tumor, stroma, complex, lympho, debris, mucosa, adipose, and empty. Evaluation protocols. Following the experimental protocols in [6] , we evaluate our method for different scenarios, such as 4-1, 4-2, 3-1, and 3-2. In each scenario, the numbers indicate the number of base and new classes, respectively. For example, considering the HAM10000 dataset with 7 classes, the scenario of 3-2 corresponds to learning 3 classes at the initial step and subsequently adding 2 new classes at each incremental step, requiring a total of 3 training steps.Metrics. Following previous work [6] , we evaluate our method based on two standard metrics: Average Accuracy (Acc) and Average Forgetting (Fgt). Let 𝑎 𝑡,𝑖 be the accuracy of the model evaluated on the test set of classes in Y 𝑖 after training on the first 𝑡 steps. The Average Accuracy is defined as:EQUATIONwhich measures the average classification accuracy of the model until step 𝑇 . The Average Forgetting is defined as:EQUATIONwhich measures an estimate of how much the model forgets by averaging the decline in accuracy from the peak performance to its current performance. Compared methods. To demonstrate the superiority of our method, we first compare it to classical incremental learning approaches: iCaRL [34] , UCIR [18] , PODNet [13] , and DER [46] . Besides, we also compare to the current state-of-the-art method: LO2LN [6] . Implementation details. As in [6] , we adopt a cosine normalization classifier with a ResNet-18 [16] backbone pre-trained on the ImageNet [9] . Our method is implemented in PyTorch [33] , and we employ SGD with a momentum value of 0.9 and weight decay of 0.0005 for optimization. During training, the batch size is set to 32 for the CCH5000 and HAM10000 datasets and 128 for the EyePACS dataset in each learning step. Note that, for a fair comparison, we [44] and stored in memory M. Furthermore, we conduct all experiments on three different class orders and report the means ± standard deviations over three runs.As shown in Tab. 1, 2, and 3, we report the experimental results on three benchmark datasets: CCH5000, HAM10000, and EyePACS, respectively. CCH5000. We can see that our method achieves state-of-the-art performance in terms of Acc and Fgt on both settings. Specifically, our method surpasses LO2LN by 1.7% on the 4-2 setting and 1.8% on the 4-1 setting in terms of Fgt, indicating the effectiveness of our method in overcoming forgetting. HAM10000. Different from the CCH5000 dataset, the HAM10000 dataset is a highly imbalanced dermoscopy dataset. Experimental results demonstrate that our method significantly improves the performance on the HAM10000 dataset, benefiting from the strong ability to address class imbalance. To be more specific, compared with the SOTA method, we improve the accuracy from 82.0% to 85.0% on the 3-2 setting. On the 3-1 setting, we achieve an overall performance of 80.9%, which is 2.8% higher than LO2LN's 78.1%. Moreover, the average forgetting is also reduced by 4.8% (3-2 setting) and 4.9% (3-1 setting).EyePACS. Furthermore, we present a comparison of different methods on the challenging EyePACS dataset. Our proposed method not only demonstrates significantly higher average accuracy but also achieves lower average forgetting than the other baselines. Notably, it surpasses LO2LN by 0.9% in terms of Acc and outperforms PODNet and DER by substantial margins of 19.5% and 24.1%, respectively.Accuracy. As shown in Fig. 3 , we display the average incremental performance of each step for three datasets. According to these curves, it is evident that the performances of all methods are similar in the first step, but the baselines suffer from a significant drop as the learning steps increase. In contrast, our method effectively slows down the drop, leading to an increasing gap between the baselines L 𝑐𝑏𝑐 L 𝑑𝑚 Acc Fgt 67.6±1.6 30.4±11.9 80.2±2.8 8.6±4.0 83.6±2.9 13.1±6.0 85.0±3.1 8.0±3.5 Table 4 : Performance contribution of each component on the HAM10000 3-2 setting. and our method over time. This demonstrates that our method benefits class incremental learning in medical image classification and outperforms prior works. Forgetting. Fig. 4 depicts the average forgetting across each incremental step for three datasets. The forgetting of most methods increases rapidly as new classes arrive, while our method consistently outperforms the SOTA methods, indicating improved resilience to catastrophic forgetting.Impact of each component. In Tab. 4, we present an ablation analysis on the HAM10000 3-2 setting to evaluate the effect of each proposed component. The first row refers to the baseline, which is w/o margin loss 83.6±2.9 13.1±6.0 Margin ranking loss [18] 83.9±2.4 12.6±5.1Distribution margin loss (Eq. 9) 85.0±3.1 8.0±3.5Table 6 : Performance of different margin losses on the HAM10000 3-2 setting.trained with the cross-entropy loss (CE) and the knowledge distillation loss L 𝑘𝑑 . Firstly, we observe that the distribution margin loss L 𝑑𝑚 brings a significant contribution when applied alone, improving the performance by 12.6% in terms of Acc. Secondly, when we replace CE with the CIL-balanced classification loss L 𝑐𝑏𝑐 , the average accuracy is improved from 67.6% to a notable 83.6%. Finally, the combination of L 𝑑𝑚 and L 𝑐𝑏𝑐 further improves the performance, demonstrating the effect of both proposed components. Effect of CIL-Balanced Classification Loss. We investigate the impact of different classification losses on the HAM10000 3-2 setting when combined with the knowledge distillation loss L 𝑘𝑑 and our distribution margin loss L 𝑑𝑚 . As shown in Tab. 5, we present results of using cross-entropy loss (CE), focal loss (Focal) [25] , classbalanced focal loss (CB Focal) [8] , and our proposed methods (logitbalanced and CIL-balanced). It can be observed that both of our proposed methods consistently outperform the other classification loss objectives, indicating the effectiveness of them to address the imbalance issue. Furthermore, the CIL-balanced classification loss (Eq. 7) achieves an additional 1.9% improvement compared to the logit-balanced classification loss (Eq. 4), benefiting from the scale factor 𝛾 to strengthen the learning of old classes. Effect of Distribution Margin Loss. To verify the effectiveness of our distribution margin loss objectives, we conduct experiments on the HAM10000 3-2 setting combined with the knowledge distillation loss L 𝑘𝑑 and our CIL-balanced classification loss L 𝑐𝑏𝑐 . The results presented in Tab. 6 demonstrate that our distribution margin loss brings significant improvements compared to cases without the margin loss and with the margin ranking loss [18] . To further illustrate the advantages of our method, we present t-SNE [39] visualizations of feature distributions with different margin loss objectives for the CCH5000 dataset, as shown in Fig. 5 . In the absence of margin loss, we observe large intra-class clusters (blue rectangle) and significant inter-class overlap in feature space (red circle). When employing the margin ranking loss, the issue of overlap is mitigated to some extent (red circle) compared to the method without margin loss. Finally, by optimizing our distribution margin loss, we achieve a more pronounced separation between the distributions of old and new classes (red circle), while simultaneously ensuring that the representations of old classes become more compact (blue rectangle). Ability to integrate with other existing methods. Our proposed methods can be easily integrated with other existing CIL methods. To demonstrate this, we conduct experiments utilizing iCaRL [34] and UCIR [18] on the HAM10000 3-1 setting. Specifically, we replace the classification loss in each baseline with our CIL-balanced classification loss and incorporate our distribution margin loss. As shown in Tab. 7, the accuracy (Acc) for both baselines can be improved by about 3% with the integration of our methods. More notably, our approaches effectively reduce forgetting (Fgt) by 5.7% and 14.8% for iCaRL and UCIR, respectively. Sensitivity study of hyper-parameters. In this paper, there are three hyper-parameters during training: the weight of the distribution margin loss 𝜆 𝑑 , the weight of the knowledge distillation loss 𝜆 𝑘 , and the trade-off coefficient 𝛼. We first conduct experiments to explore the impacts of different 𝜆 𝑑 and 𝜆 𝑘 on the HAM10000 3-2 setting. As shown in Fig. 6a , we vary 𝜆 𝑑 within the range of {0.1, 0.2, 0.3, 0.4, 0.5}, and 𝜆 𝑘 within the range of {0.1, 0.25, 0.5, 0.75, 1.0}, resulting in a total of 25 compared results. From the results, we consistently observe satisfactory performance from our model, demonstrating its robustness to the selection of 𝜆 𝑑 and 𝜆 𝑘 .To investigate the impact of different values of 𝛼 on addressing the imbalance between the old and new classes, we evaluate the accuracy by varying 𝛼 from {0.1, 0.3, 0.5, 0.7, 0.9} on three benchmark datasets. As shown in Fig. 6b , the results indicate that the accuracy gradually improves as 𝛼 grows larger initially, while it starts to decline when 𝛼 is close to 1. Since the data distribution differs across datasets, the selection of the trade-off coefficient 𝛼 also varies. Specifically, the optimal values of 𝛼 for the three datasets are 0.7, 0.5, and 0.3, respectively. Longer incremental learning. In class incremental learning, a key challenge is catastrophic forgetting, which becomes more pronounced as the number of learning classes increases [12, 18, 34] . To quantify the robustness of our method in overcoming catastrophic forgetting, we evaluate it on two longer-step protocols: 50-10 (6 steps) and 50-5 (11 steps), employing the more challenging CIFAR100 dataset. Following the experimental protocol outlined in [26] , we conduct experiments under both conventional (Conv) and long-tail (LT) scenarios. In the conventional scenario, each class has 500 training samples for training. Conversely, the longtailed scenario follows an exponential decay in sample sizes across classes, where the ratio between the least and the most frequent class is 0.01. As illustrated in Tab. 8, our method achieves superior results in all settings. Specifically, we observe a more significant improvement in the long-tail scenario, further validating the effectiveness of our method in addressing the class imbalance problem in class incremental learning. Furthermore, we present the dynamic performance changes during the incremental learning process in Fig. 7 . It is evident that with more learning steps, the gap between the baselines and our method widens, and our method's performance remains superior across different scenarios (conventional and long-tailed) throughout most of the learning steps. 
conclusion: 
 In this paper, we propose two simple yet effective plug-in loss functions for class incremental learning in medical image classification. First, to address the challenge of classifier bias caused by class imbalance, we introduce a CIL-balanced classification loss via logit adjustment. Second, we propose a novel distribution margin loss that aims to enforce inter-class discrepancy and intra-class compactness simultaneously. Our extensive experimental evaluation demonstrates the state-of-the-art performance of our method across various scenarios on medical image datasets: CCH5000, HAM10000, and EyePACS.  
: 
 instead of retraining the model with all data available [24] . Therefore, in this work, we focus on class incremental learning in the medical domain.Fig. 1 illustrates the setting of class incremental learning in medical Image classification. Taking the EyePACS dataset [7] as an example, the model is initially trained to classify three classes (i.e., No DR, Mild, and Moderate). Subsequently, incremental classes (e.g., Severe, and Proliferative DR) arrive in sequential steps to update the model. The classes introduced in different steps are disjoint, and the model must be able to predict all classes seen over time. However, when updating the model with only new classes, new data tends to erase previous knowledge. This phenomenon is known as catastrophic forgetting [15, 22] .For class incremental learning, imbalanced data between old and new classes is one of the primary reasons for catastrophic forgetting [23, 48] . To this end, numerous approaches have been proposed to store a small proportion of previous training data in memory and rehearse them when learning new classes [1, 18, 45] . However, the limited size of memory can also lead to an imbalance between old and new classes [28, 34] . Under this circumstance, the class imbalance will lead to (i) a classifier biased towards the new and majority classes; and (ii) the embeddings of new classes inevitably overlap with the old ones in the feature space (i.e. the ambiguities problem). In addition to the class incremental learning imbalance, many reallife medical datasets exhibit significant class imbalance [32] , with some classes having notably higher instances in training samples than others, e.g., HAM10000 [38] , and EyePACS [7] , which further aggravate the catastrophic forgetting. Therefore, addressing data imbalance is crucial for class incremental learning in medical image classification.In this paper, we propose two simple yet effective plug-in loss objectives to tackle two challenges caused by imbalance in class incremental learning. First, we propose a CIL-balanced classification loss instead of the traditional cross-entropy (CE) loss to avoid the issue of classifier bias. Specifically, we first adjust the logits based on the category frequency to place more emphasis on rare classes and then introduce a scale factor to further achieve a balance between old and new classes. Second, to alleviate the overlap of classes in the feature space, we propose a distribution margin loss, a novel improved margin loss, which not only facilitates to push away the distributions of old and new classes but also obtains the compact intra-class clustering. Extensive experiments on benchmark datasets under various settings verify the superiority of our method.To summarize, the main contributions of this paper are:• To reduce the classifier bias towards new and majority classes, we propose a CIL-balanced classification loss that emphasizes rare ones via logit adjustment. • We introduce a novel distribution margin loss that can effectively separate the distributions of old and new classes to avoid ambiguities and realize the optimization of the intraclass compactness.• Extensive experiments demonstrate that our method can effectively address the issue of data imbalance with the state-ofthe-art performance achieved on three benchmark datasets: CCH5000, HAM10000, and EyePACS. 
 
 Appendix:  


Decision: This paper proposes a method to address class imbalance in class incremental learning for medical image segmentation. It identifies the imbalance between old and new classes as a major concern and proposes two strategies to mitigate this issue: CIL-balanced classification loss and distribution marginal loss. Experimental results demonstrate the effectiveness of these strategies. This paper will provide valuable insights into the field of class imbalance in class incremental learning. 

The authors should incorporate some of the rebuttal content into the revised paper based on the reviewers' suggestions. Based on both the reviewers' comments and the author's rebuttal, I recommend accepting this paper.

Reviews: review: paper_summary: paper_summary: The paper addresses the challenges of class incremental learning in medical image classification, specifically focusing on issues caused by class imbalance. The authors introduce two novel loss functions to mitigate these challenges. The first is a CIL-balanced classification loss that uses logit adjustment to reduce classifier bias towards majority classes. The second is a distribution margin loss designed to enhance inter-class separation and intra-class compactness. Two loss functions can be incorporated in existing training methods in a plug-and-play manner. Experiments on benchmark datasets show that the proposed methods outperform current state-of-the-art approaches.
strength_weakness: strength_weakness: * The paper has a clear motivation and is well organized. 
* Proposed loss objectives are very straightforward and easy to implement, and can be incorporated to other prior works in plug-in way. 
* Extensive experiments are delivered on three benchmark datasets, and the proposed method outperforms previous state-of-the-art methods with considerable amount of gap. 
* Sufficient amount of visualizations and ablation studies are presented to validate each component of the proposed loss objectives.
limitations: limitations: * Overall, it seems like the training methods and pipeline is heavily adopted from the LO2LN [6]. For instance, the combined loss form of margin loss, distillation loss, and class-balanced classification loss is first introduced and validated in the LO2LN. In my perspective, this work proposes a little expanded version of baseline margin loss and class-balanced loss, while maintaining all the rest identical as LO2LN. The added materials on top of baseline margin loss and class-balanced loss are both very straightforward, which are based on category frequency values. However, the performance improvement that stems from them is non-negligible. 

* Ablation study on the sample size 'm' of representative instances in memory buffer needs to be explored, since it could be the most important parameter that divides majority and minority class during class incremental training.
paper_summary: paper_summary: The authors propose two methods to improve the loss function, aiming to address class imbalance and the overlap of old and new classes in continual learning.
strength_weakness: strength_weakness: The experimental analysis in this work is thorough, with comprehensive main experimental results and ablation studies.

The motivation for this work is reasonable.
limitations: limitations: I am not familiar with the current state of research in medical image recognition. For a researcher studying imbalanced learning on natural images, the core idea of the class-balanced loss proposed in this work has already been introduced. I am unsure whether the loss function proposed by the authors to address the problem of catastrophic forgetting in continual learning has already been presented by others.
paper_summary: paper_summary: This paper addresses class imbalance in class incremental learning for medical image segmentation. It identifies the imbalance between old and new classes as a major concern and proposes two strategies to mitigate this issue: CIL-balanced classification loss and distribution marginal loss. Experimental results demonstrate the effectiveness of these strategies.
strength_weakness: strength_weakness: - The motivation is clear, and the proposed design seems reasonable.
- The experimental results, including comparisons with state-of-the-art methods and ablation studies, demonstrate the effectiveness.
limitations: limitations: - Please justify why \(\alpha\) is restricted to the range [0-1] in Row 321. In my understanding, this is a hyperparameter. Furthermore, the range of [0-1] for the old class seems too close to the weight for the new class, indicating a balanced weight between the old and new classes, which contradicts the motivation of this paper.