Paper: Imbalanced data robust online continual learning based on evolving class aware memory selection and built-in contrastive representation learning (ICLR-2024)

Reviewer comment (Evaluation):
There is no discussion of the computational cost of the algorithm.

Author response:
Thank you for your valuable suggestion.

$Q1$ : **There is no discussion of the computational cost of the algorithm.** Our approach primarily incurs computational costs in calculating the matrix distance. For each input batch, it's necessary to compute the distance between the batch and the memory set. To optimize this process, our method employs the following strategies:

- **Efficient Distance Matrix Computation:** We do not recalculate the entire distance matrix \( D \) (the distance between memory points) for each input data. Instead, we update only the parts associated with retrieved memory points. This approach significantly reduces the computational overhead.
  
- **Linear Memory Cost:** The memory cost scales linearly with the size of the memory set, avoiding exponential increases.

- **Computational Cost Evaluation:**

To provide a clearer picture of the computational efficiency, we conducted tests on a single RTX3070 using ResNet18 with a memory size of 1000 in a balanced PACS setting. The results are as follows:

| Method | Training Time (hours) |
| ------ | --------------------- |
| ER     | 0.18                  |
| GSS    | 0.85                  |
| OCS    | 0.26                  |
| Ours   | 0.68                  |

As illustrated in the table, our method requires more computation time than ER and OCS, but is less computationally intensive than GSS. This positions our method as a viable option in terms of computational efficiency, particularly when considering the enhanced performance it offers in more complex and realistic learning scenarios.

Reviewer comment (Evaluation):
Lack of discussions on the framework's applicability and performance in large-scale scenarios, which remain unaddressed in the current version of the paper.

Author response:
$Q3$ : **The authors acknowledge the substantial difficulties encountered when dealing with imbalanced data across various classes ...** 
Our method's adaptability to both balanced and imbalanced scenarios, and its scalability in terms of memory size, demonstrate its potential applicability in large-scale scenarios. Although we did not explicitly test on extremely large datasets due to resource constraints, the principles underpinning our method – particularly the efficient memory management and contrastive learning components, are designed to be scalable and applicable in large-scale settings. The scalability of our method is demonstrated by its application from the smaller PACS dataset to the larger DomainNet dataset.

Reviewer comment (Experiments):
The authors mentioned Split ImageNet-1k in section 5. Experiment as one of the benchmarks but there are no mention of this dataset in the result section.

Author response:
$Q2$ : **The authors mentioned Split ImageNet-1k**. Sorry, there was a typo error. It should be the split mini-ImageNet.

Reviewer comment (Experiments):
The authors acknowledge the substantial difficulties encountered when dealing with imbalanced data across various classes and domains, citing works by Wu et al. (2019) to underscore this challenge. However, the evaluations suffice to mid-sized datasets. This raises questions about the framework's applicability and performance in large-scale scenarios, which remain unaddressed in the current version of the paper.

Author response:
$Q3$ : **The authors acknowledge the substantial difficulties encountered when dealing with imbalanced data across various classes ...** 
Our method's adaptability to both balanced and imbalanced scenarios, and its scalability in terms of memory size, demonstrate its potential applicability in large-scale scenarios. Although we did not explicitly test on extremely large datasets due to resource constraints, the principles underpinning our method – particularly the efficient memory management and contrastive learning components, are designed to be scalable and applicable in large-scale settings. The scalability of our method is demonstrated by its application from the smaller PACS dataset to the larger DomainNet dataset.

Reviewer comment (Experiments):
From the title and abstract of the paper, it looks like your method is supposed to work for all settings of online incremental learning, so it should achieve the best performance so far for different settings, but for experiments with balanced datasets, it looks like OnPro[1], GSA[2], and OCM[3] have achieved much better experimental results with the same resnet18 and M=5K methods, so you should add the latest methods into it to make your experiments more convincing.

Author response:
Thank you for your valuable suggestion. We fully understand the importance of including the latest methods in our experiments to enhance the credibility and relevance of our work. However, we would like to highlight the differences in terms of the positioning and focus of our method compared to OnPro[1], GSA[2], and OCM[3], although these works represent important contributions to the field of online class incremental learning.

1. **Distinct Scenarios and Setting**: OnPro[1], GSA[2], and OCM[3] have brought significant contributions on continual learning but they are primarily proposed under class incremental learning scenario and tested on the Split CIFAR100 and Split mini-Imagenet  under data balanced setting. For instance, in section 3.1 on problem definition, OnPro[1] assumes that different tasks D_t are mutually disjoint in terms of classes. Therefore,  their method assumes that learning data for a given class can only arrive once through a number of successive batches within a given task. Such an assumption moves away from the original goal of continual learning where the learned knowledge is supposed to be consolidated anytime, including already learned concepts.   In contrast, our paper does not make such an assumption and proposes a novel online continual learning method operating under much more diverse and challenging scenarios, including Domain-incremental, Class-incremental, and Domain-Class incremental learning under both balanced and imbalanced settings. This broader scope addresses a wider range of practical applications and challenges in the field.
2. **Methodological Differences**: our method primarily concentrates on the efficient management and utilization of the memory buffer which is the cornerstone of all rehearsal-based CL methods. This approach is  fundamentally orthogonal to the methods mentioned above, emphasizing different aspects of the learning process. Therefore, it is more appropriate for our method to be compared with other memory selection methods, which align more closely with our research's core focus:   
    - OnPro[1] does a great work in feature learning to avoid shortcut learning, using in particular the contrastive learning loss on prototypes from the memory set and the current batch and data augmentation. It employs Experience Replay (ER) to select memory samples. In contrats, our method introduces a new, more efficient memory selection technique compared to ER. Their method does a great work on calculating the contrastive prototype learning loss within the memory set and within current batch. In contrast, our method generates the contrastive learning loss between the memory set and current batch without data augmentation.
    - GSA[2] introduces a new training objective with a gradient-based self-adaptive loss to tackle gradient imbalance (GI).
    - OCM[3] maximizes mutual information between input data and holistic feature representations, using local and global rotations for data augmentation.

Experimentally, we have shown in our paper that our memory selection method (FDBS) can be combined with any other rehearsal-based CL methods to obtain improved performance and robustness to unbalanced data. In the following section, we further showcase this property of complementarity of our proposed memory selection method when it is combined with OnPro[1].    

3. **Comparative Analysis with OnPro[1]**: 
    - In balanced Class-incremental Learning (CIL) on CIFAR100, OnPro shows superior performance with a 35.98% accuracy compared to our 21.13%. This result suggests the effectiveness of feature learning through online prototype learning in OnPro[1] under data balanced setting; 
    - In the more complex and realistic scenario of imbalanced Domain-incremental Learning (DIL) on the PACS dataset, our  proposed method (FDBS+IWL) outperforms OnPro, achieving 31.25% accuracy against their 26.54%.
    - If we replace OnPro's memory selection method with our FDBS, we observe that in a balanced scenario, the resultant method increases slightly the performance. However, in the context of imbalanced domain-incremental learning, combining OnPro with our method achieves the best performance, further demonstrating the effectiveness and the versatility of our memory selection approach.

These results along with those in the submission across different scenarios suggest that OnPro and similar methods are highly specialized for balanced CIL. In contrast, our proposed method, focusing on efficient memory management, demonstrates greater versatility and effectiveness in these diverse and more challenging learning environments, and its ability to be combined with any rehearsal-based CL method.


|       | Balanced CIL (Cifar100) | Imbalanced DIL (PACS) |
|-------|-------------------------|-----------------------|
| OnPro | 35.98                  | 26.54                 |
| Ours  | 21.13                   | 31.25                 |
|OnPro + FDBS(ours)| 36.2|  33.14|