Progressive Prototype Evolving for Dual-Forgetting Mitigation in Non-Exemplar Online Continual Learning

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-11 10:29:37

Abstract: Online Continual Learning (OCL) aims at learning a model through a sequence of single-pass data, usually encountering the challenges of catastrophic forgetting both between different learning stages and within a stage. Currently, existing OCL methods address these issues by replaying part of previous data but inevitably raise data privacy concerns and stand in contrast to the setting of online learning where data can only be accessed once. Moreover, their performance will dramatically drop without any replay buffer. In this paper, we propose a Non-Exemplar Online Continual Learning method named Progressive Prototype Evolving (PPE). The core of our PPE is to progressively learn class-specific prototypes during the online learning phase without reusing any previously seen data. Meanwhile, the progressive prototypes of the current learning stage, serving as the accumulated knowledge of different classes, are fed back to the model to mitigate intra-stage forgetting. Additionally, to resist inter-stage forgetting, we introduce the Prototype Similarity Preserving and Prototype-Guided Gradient Constraint modules which distill and leverage the historical knowledge conveyed by prototypes to regularize the one-way model learning. Consequently, extensive experiments on three widely used datasets demonstrate the superiority of the proposed PPE against the state-of-the-art exemplar-based OCL approaches. Our code will be released.

Main paper: 
 related work 2.1 continual learning: 
 Current Continual Learning methods can be broadly categorized into rehearsal-based, regularization-based, and architecture-based models. Among them, rehearsal-based approaches [27, 28, 34] concentrated on preserving knowledge by actively selecting and replaying representative data from earlier classes. Moreover, regularizationbased methods [21, 24, 39, 40] aimed to address forgetting by stabilizing model parameters or regulating feature adjustments. The architecture-based CL models [18, 43, 44, 50] adapted dynamically to evolving data streams by either modifying network structures or incorporating specific parameters tailored for each learning stage. However, the aforementioned methods usually have to store historical data, raising critical concerns about data privacy.Consequently, recent efforts have been made to address CL without retaining historical exemplars, leading to Non-Exemplar Continual Learning. In this context, the challenge of catastrophic forgetting intensifies due to the absence of previous data. To address this, [10, 11, 24] proposed knowledge distillation to resist forgetting. [51, 52] emphasized data augmentation to broaden the classification boundary and diminish the representation bias in continual learning. Several methods maintained prototypes of each class as surrogates for prior knowledge. Most of them [31, 33, 37, 53] computed prototypes as the mean feature of each class after the training process and proposed various prototype augmentation and reminiscence techniques to retain past knowledge. The latest research [3] treated prototypes as learnable parameters during the training process. However, these methods typically utilized previously learned prototypes solely to mitigate knowledge forgetting in the following learning stages, without fully harnessing the information from the prototypes of the current stage.Recently, a more challenging but realistic CL scenario, named Online Continual Learning, has been investigated, where each sample can only be accessed once. Existing OCL methods suffered from not only inter-stage forgetting when learning unseen classes but also intra-stage forgetting of previously learned knowledge within a learning stage. Therefore, various OCL-specific approaches have emerged that predominantly rely on retaining previous exemplars as the rehearsal. [2, 20] focused on storing balanced and representative exemplars from the learned stages, and [1, 38, 41] proposed to select valuable samples for replay, based on the gradient or class information. In addition to preserving exemplars, several approaches [4, 47] concentrated on enhancing the learning capability of models by either incorporating supervised contrastive learning [26, 30, 46] or employing mutual information [13, 15] to facilitate OCL. Considering that it is infeasible to directly compute the average features of all samples as prototypes in OCL, besides keeping exemplars, CoPE [7] also maintained prototypes as extra knowledge through momentum updating, but the obtained prototypes might be misled by the latest samples.The aforementioned methods all need to preserve historical exemplars, which obviously hinders data privacy and contradicts the online setting. In this paper, we focus on tackling OCL without preserving any exemplars, in which rare works have been proposed. To preserve knowledge without assessing to previous data, [8] employed knowledge distillation for old classes but it ignored the intra-stage forgetting. [14] concentrated on adopting orthogonal projection to mitigate forgetting, but its learning capability is severely restricted, resulting in unsatisfactory performance. The latest approach [19] focused on learning a pre-trained model in an offline manner on the base stage and subsequently froze the pretrained backbone during online learning. Though the forgetting problem can be addressed with the frozen backbone, the learning ability is limited especially when the training data of the base stage is insufficient. In contrast, our method entails continual learning from scratch without relying on an offline training stage. The proposed PPE model leverages a progressive evolving strategy to learn informative and discriminative prototypes that convey helpful knowledge to mitigate intra and inter-stage forgetting issues. 
the proposed method 3.1 problem formulation: 
 NEOCL considers learning a model continually from a single-pass stream of 𝑇 stages D = {𝐷 1 , 𝐷 2 , ..., 𝐷 𝑇 } without storing any exemplars of previous data. The model Θ = {Φ, Ψ} consists of a feature extractor Φ and a classifier Ψ. Data of the 𝑡-th stage𝐷 𝑡 = {𝑋 𝑡 , 𝑌 𝑡 } consists of an image set 𝑋 𝑡 = {𝑥 𝑡,𝑖 } 𝑛 𝑡 𝑖=1 and a class label set 𝑌 𝑡 = {𝑦 𝑡,𝑖 ∈ 𝐶 𝑡 } 𝑛 𝑡 𝑖=1, where 𝑛 𝑡 is the number of data in stage 𝑡 and 𝐶 𝑡 represents the set of classes in stage 𝑡. Notably, each input sample 𝑥 𝑡,𝑖 can only be used once, and labels of different stages are disjoint.Progressive Prototype Evolving. Firstly, we introduce the acquisition of our progressively learned prototypes to further demonstrate the collaboration between prototype evolution and intrastage forgetting mitigation. Since no previous data can be accessed in NEOCL, we propose to handle catastrophic forgetting by employing learnable prototypes as the representative of essential knowledge for each class. Specifically, the prototypes are treated as a set of learnable parameters and supervised with the features of respective classes. At the beginning of the 𝑡-th learning stage, we Then given a batch of data, we optimize the prototypes by minimizing the distance between the features and the prototypes as follows:EQUATIONwhere 𝑛 𝑏 represents the batch size, 𝑝 𝑦 𝑖 denotes the prototype for class 𝑦 𝑖 , and stop refers to the operation that stops the gradient backward. Through this optimization process, prototypes in 𝑃 𝑡 can progressively acquire class-specific information within this batch. As a result, the prototypes 𝑃 𝑡 progressively evolve over the whole learning process instead of solely relying on the latest batch, they effectively accumulate knowledge of previously encountered samples within a stage.To mitigate the critical intra-stage forgetting issue in OCL and fully leverage the knowledge compressed in 𝑃 𝑡 , we incorporate prototypes into the model training process of the 𝑡-th learning stage. Specifically, 𝑃 𝑡 are fed to the classifier:EQUATIONwhere L CE and 𝑌 𝑝,𝑡 denote the cross-entropy loss and labels associated with prototypes 𝑃 𝑡 . Through this operation, the accumulated knowledge within 𝑃 𝑡 is readily replayed to mitigate intra-stage forgetting and enhance the training of a more robust model. Consequently, the learning of the model and the progressive evolution of prototypes are mutually reinforcing. A more robust model contributes to the evolution of prototypes, resulting in more informative prototypes.Prototype Similarity Preserving. To alleviate the commonly mentioned inter-stage forgetting problem, existing NECL methods [37, 52] always apply a hard loss function that minimizes the Euclidean distance between the features of input data extracted by the current and the old models. However, such a paradigm severely limits the learning capability of the model and leads to unsatisfied performance in NEOCL, where the model suffers from insufficient training. Thus, to mitigate inter-stage forgetting while maintaining the plasticity of knowledge acquisition, we propose to distill the knowledge of similarity between prototypes and input data. Specifically, following [22] which shows that a model updated with model fusion can better represent the accumulated knowledge, we maintain a fusion model denoted asΘ 𝑒 = {Φ 𝑒 , Ψ 𝑒 }.Then similarity between the prototypes and the features of input data, extracted by the current model Θ and the old model Θ 𝑒 , is calculated via cosine similarity. The proposed Prototype Similarity Preserving loss can be formed as:EQUATIONwhere ⟨•, •⟩ represents the cosine similarity, 𝑛 𝑏 represents the batch size, and C 𝑡 = 𝑡 -1 𝑖=1 𝐶 𝑖 represents labels of classes that have been seen during previous stages. By employing L 𝑝𝑠𝑝 , the semantic relationships between data features and prototypes can be distilled to resist inter-stage forgetting.Prototype-Guided Gradient Constraint. To further preserve more informative knowledge of historical stages, motivated by [36] , we design a Prototype-Guided Gradient Constraint scheme to mitigate the inter-stage forgetting by orthogonal gradient projection. When training on the 𝑙-th batch of the 𝑡-th stage, we remove the projection of the gradient 𝑔 𝑡,𝑙 into the space of gradients from previous stages to mitigate its influence on these stages:EQUATIONwhere M 𝑡 -1 = [𝑢 1 , 𝑢 2 , ...] represents the orthogonal bases of subspace which contain the gradients of previous stages. To obtain M 𝑡 , we leverage the proposition that the gradient update of each layer lies in the span of inputs as illustrated in Supplementary. After the training process of each stage, an input matrix R 𝑡 of each layer is constructed. The part of R 𝑡 that can be represented by M 𝑡 -1 is removed:R𝑡 = R 𝑡 -M 𝑡 -1 M 𝑇 𝑡 -1 R 𝑡 .Then the SVD operation is performed on R𝑡 = Û𝑡 Σ𝑡 V𝑡 where Û𝑡 = [ û𝑡,1 , û𝑡,2 , ...] and a minimum value of top-𝑘 rank approximation ( R𝑡 ) 𝑘 is chosen to meet the following criteria given a hyperparameter 𝜖 𝑡 :EQUATIONThen M 𝑡 is updated as [M 𝑡 -1 , û𝑡,1 , ..., û𝑡,𝑘 ]. Existing methods [25, 36] construct R 𝑡 via sampling data from the entire dataset which is infeasible in NEOCL. Therefore, we handle this constraint by utilizing the latest batch of a learning stage to obtain R 𝑡 .In fact, the knowledge of different periods of a data stream in NEOCL is always unpredictable and may lead to varying degrees of model parameter drifting during learning. Therefore, instead of relying on a high threshold 𝜖 𝑡 to mitigate forgetting, we argue that an adaptive threshold parameter is essential to guide the update of orthogonal bases effectively. To do so, we leverage the obtained progressive prototypes to measure the newly acquired knowledge of the current stage and guide the update of orthogonal bases. Specifically, after the training process of the 𝑡-th stage, we calculate the average cosine similarity between the set of prototypes for classes in stages 𝑡-1 and 𝑡 as:EQUATIONA lower average similarity implies that the features of new classes are significantly different from those of previously seen classes, indicating that the model has learned more novel knowledge during this stage. Consequently, a larger 𝜖 𝑡 is assigned to preserve more information about the current stage. Thus, we choose a linear function with negative scope to model this negative correlation between the 𝜖 𝑡 in Equation ( 5) and 𝑠 𝑡 as:EQUATIONwhere 𝛼 ≥ 0, 𝛽 ≥ 0 are hyperparameters.The overall pipeline of our proposed PPE method is shown in Figure 3 . For optimization, following [26, 30] , a base loss L 𝑏𝑎𝑠𝑒 consisting of a cross-entropy loss and a supervised contrastive learning is adopted to train the model. The proposed L 𝑝𝑝𝑒 in Equation ( 1) is incorporated for progressive prototype evolving, and then the learned prototypes are fed to the classifier based on the prototype classification loss L 𝑝𝑐𝑒 to mitigate the intra-stage forgetting. Moreover, a prototype similarity preserving loss L 𝑝𝑠𝑝 is also involved to mitigate the problem of inter-stage forgetting. Finally, the overall loss function in the proposed method can be formed with weight parameters 𝜆, 𝛾, 𝜇:EQUATIONTo be noted, the gradient of the model is constrained by the Prototype-Guided Gradient Constraint. At the end of each learning stage, the orthogonal bases are updated with the adaptive threshold in Equation (7).Maintaining prototypes is a kind of rising method in the area of NECL and OCL. In this section, we aim to elucidate the distinctions between our PPE method and existing techniques. (1) . We propose progressive prototype that treat the prototypes as learnable parameters and optimize them during training. This prototype fits the online learning scenarios where conventional methods, such as computing the mean feature of samples from the same class, are impractical. (2). To mitigate intra-stage forgetting, we propose the L 𝑝𝑐𝑒 which incorporates the accumulated knowledge obtained by our evolving prototypes. As discussed in Section 1, apart from the well-known inter-stage forgetting, we demonstrate that NEOCL also meets with the intra-stage forgetting that the model may forget the knowledge of previously seen samples during a learning stage. The effectiveness of our proposed L 𝑝𝑐𝑒 is verified in Section4.3 and Supplementary. (3) The knowledge of prototypes is fully explored to mitigate inter-stage forgetting. Firstly, drawing inspiration from knowledge distillation techniques, we introduce L 𝑝𝑠𝑝 to distill knowledge using our progressive prototypes. Secondly, to address the limited learning capability of the Gradient Projection Memory (GPM) methods and strike a balance between knowledge acquisition and forgetting, we design an adaptive threshold for GPM based on the estimated knowledge shift between different stages, as indicated by the similarity of prototypes. 
experiments 4.1 experiment settings: 
 Datasets. The evaluation of our method is conducted on three widely used datasets, CIFAR-10 [23] , CIFAR-100 [23] , and MiniIm-ageNet [42] . We follow previous works [26, 30] to build the OCL setting. Specifically, CIFAR-10 is divided into 5 stages, each comprising 2 classes, while CIFAR-100 and MiniImageNet are divided into 10 stages, each encompassing 10 classes.Comparison Methods. We compare our PPE with various methods including twelve OCL models (ER [6] , MIR [1] , CoPE [7] , SCR [30] , OCM [15] , RAR [48] , DVC [13] , GSA [16] , PCR [26] , CBA [45] , On-Pro [46] , SSD [12] ) and four NECL approaches (PASS [52] , AOP [14] , PRAKA [37] , DSR [19] ). For OCL methods that need to store and replay data, we set the memory size with 10/20/100 for CIFAR-10, as well as 100/200/500 for CIFAR-100 and MiniImageNet which are widely adopted by OCL methods1 . To be noted, NECL methods and PPE do not need to store exemplars, thus the memory size is 0. Evaluation Metrics. Following [13, 38, 46] , the Final Accuracy and Average Forgetting [5] are adopted for evaluation. Final Accuracy is computed as the accuracy of all seen classes and Average Forgetting calculates the average accuracy degradation of different classes, which represents the anti-forgetting ability of the model. Implementation Details. For a fair comparison, the reduced ResNet18 [17] trained from scratch is used as the backbone. Following [16, 46] , we use resized-crop, random flip, and gray-scale as data augmentation strategies. At each training stage, the classification head related to the current stage are optimized. During testing, prototypes are treated as class means and the nearest class mean classifier [30, 32] is used. For hyperparameters, in Equation ( 7), 𝛼 = 2.65, 𝛽 = 2 . The weighting parameters of different losses are 𝛾 = 0.5, 𝜆 = 1, and 𝜇 = 10. Optimization of model parameters and progressive prototypes are performed using the SGD optimizer with learning rates of 0.02 and 60 respectively. Experiments of hyperparameters are shown in the Supplementary. The batch size is 10 for all methods. For the comparison methods, we reproduce their results with their official source code and all methods use the reduced ResNet18 backbone except for AOP which uses AlexNet. All results are the average performance across 15 runs.1XPEHURI6WDJHV $FFXUDF\ &,)$5 1XPEHURI6WDJHV $FFXUDF\ &,)$5 1XPEHURI6WDJHV $FFXUDF\ 0LQL,PDJH1HW 2XUV 3&5 2Q3UR '9& 6&5 66'Comparison of Final Accuracy. Table 1 reports the Final Accuracy of the comparison methods. Across various scenarios, without storing any exemplars, our PPE significantly outperforms existing OCL methods and achieves comparable results when they preserve a large number of exemplars. Specifically, the performance of existing OCL methods will dramatically degrade with limited exemplars. For example, on the CIFAR-100 dataset, the performance of the latest method SSD drops from 21.9% to 9.5% when the memory size goes smaller while our PPE achieves superior results of 22.0% without reusing any previous exemplars. Compared with NECL methods, our PPE outperforms them by 0.5%, 10.8%, and 9.5% on CIFAR-10, CIFAR-100, and MiniImageNet. Specifically, although AOP utilizes the AlexNet which is more suitable for CIFAR-10, our method can still beat it. This can be attributed to the comprehensive utilization of our progressively evolved prototypes. Different from the prototypes in these NECL methods which are solely used to resist the inter-stage forgetting of classification heads, our prototypes are able to jointly mitigate the intra and inter-stage forgetting.Comparison of Average Forgetting. In Table 2 , we present the Average Forgetting results of the state-of-the-art exemplar-based OCL methods that achieve promising Final Accuracy results. The memory sizes are 20/200/200 for CIFAR-10 /CIFAR-100 /MiniIma-geNet. It is obvious that, despite the extensive use of exemplars in previous methods, they exhibit higher forgetting rates compared to our PPE. Since the existing OCL methods primarily rely on reusing abundant exemplars without designing specific components to mitigate forgetting. Our approach, on the one hand, introduces Progressive Prototype Evolving and Prototype Similarity Preserving loss to address the dual-forgetting, and the proposed Prototype-Guided Gradient Constraint aids in achieving a more balanced trade-off between knowledge retention and acquisition.Accuracy of Different Stages. To present results in detail, the classification accuracy of different methods on sequential learning stages are shown in Figure 4 . The memory sizes are 20/200/200 for CIFAR-10/CIFAR-100/MiniImageNet. Notably, during the early stages, certain exemplar-based OCL methods achieve higher accuracy than our approach. This is because these methods leverage a fixed-size memory buffer, resulting in more exemplars per class in the early stages, thereby enhancing the performance. However, as training advances, the stored exemplars per class continue to decrease which exacerbates the problem of catastrophic forgetting causing their accuracy to decline. Eventually, our proposed PPE method outperforms these counterparts, yielding superior Final Accuracy results in the long run. Comparison of Training Time. In Figure 6 , we conduct a comparative analysis of training times among various methods using CIFAR100 with a memory size of 200. As can be observed, our PPE demonstrates superior performance, although the training time is slightly higher than ER due to the gradient projection operation introduced in Section 3.3. Notably, our approach significantly outperforms RAR, OCM, and OnPro, which introduce repeated sampling from memory buffers or employ strong data augmentation techniques. This efficiency advantage makes our approach highly adaptable to the demands of the online scenario.Effectiveness of Different Components. Ablation results are presented in Table 3 . Our method incorporates four losses L 𝑏𝑎𝑠𝑒 , To further illustrate the effectiveness of PGC, we present results adopting fixed thresholds 𝜖 𝑡 on CIFAR-100 in Table 4 . Observably, performance degrades with fixed high or low thresholds. This is because a high threshold imposes strict constraints, limiting knowledge acquisition, while a low threshold compromises anti-forgetting ability. In contrast, employing thresholds calculated by prototypes achieves the best results, emphasizing the crucial role of the prototype's knowledge in guiding the model to strike an appropriate balance between knowledge acquisition and forgetting. Visualization of Progressive Prototypes. In Figure 5 , we present the t-SNE visualization results of prototypes and extracted features of images in test sets at different batches. The results of different batches within an online continual learning stage show the evolution of our progressive prototypes alongside the prototypes obtained through the momentum updating method CoPE [7] . For a fair comparison, no memory buffer is used in this experiment. It can be observed that, as the training progresses, the features of distinct classes become increasingly distinguishable and the prototypes evolve along with features. Our learnable prototypes demonstrate a superior ability to approximate the center of each class's features, while the prototypes acquired through momentum updating in CoPE are significantly influenced by later encountered samples and deviate from the true class centers. Notably, the superior discriminative ability of feature representations learned by our method is mainly because of the exploration of the knowledge encoded in the progressive prototypes to mitigate intra-stage forgetting, enhancing the overall capability of the model. Intra-stage Forgetting Mitigation. Intra-stage forgetting refers to the phenomenon that the model gradually loses knowledge of previously encountered samples belonging to the same class within a specific training stage. Thus, a direct method to illustrate intrastage forgetting is testing the model's performance on previously seen training samples during subsequent training batches. For a training stage with 500 batches on CIFAR-100 datasets (a total of 10 classes with 5000 samples, 10 samples per batch), given the potential insufficient training in the initial batches, where the model may not sufficiently learn from the early samples, we opt to use samples from batches 250 to 300 for constructing the test set. In Figure 7 we demonstrate the effectiveness of our anti-intra-stage forgetting module, L 𝑝𝑐𝑒 , by showcasing the accuracy improvement brought by L 𝑝𝑐𝑒 on our constructed test set. The results reveal a consistent enhancement in accuracy with the incorporation of L 𝑝𝑐𝑒 . Furthermore, the magnitude of this improvement increases with the progression of training batches. This is because as the training progresses, forgetting becomes more severe. L 𝑝𝑐𝑒 can effectively mitigate intra-stage forgetting by leveraging the accumulated knowledge acquired through our progressive prototypes, thereby contributing to the observed performance enhancement. More verification of intra-stage forgetting mitigation is provided in Supplementary. Collaboration between PPE and Intra-stage Forgetting Mitigation. In this part, we delve into the synergistic relationship between our proposed PPE and the mitigation of intra-stage forgetting. To demonstrate the enhancement brought by L 𝑝𝑐𝑒 to the learning process of prototypes, we compare the average cosine similarity between the class mean feature on test sets and prototypes at each batch in Figure 8 . A higher similarity signifies closer alignment between learned prototypes and their appropriate positions. Compared with other prototype acquisition methods, the momentum updating-based CoPE [7] and solely learning prototypes without L 𝑝𝑐𝑒 , our method achieves superior similarity results indicating that the learned prototypes are closer to their appropriate position. This result demonstrates the positive impact of our mitigating intra-stage forgetting on prototype learning. 
conclusion: 
 In this paper, we target the challenging yet crucial NEOCL problem and propose a novel method, named the Progressive Prototype Evolving, to tackle the dual-forgetting issues in NEOCL. To mitigate intra-stage forgetting, our approach learns class-specific progressive prototypes as surrogates for previous knowledge and leverages a prototype feedback design to utilize the accumulated knowledge of prototypes within a learning stage. Additionally, to resist inter-stage forgetting, our method incorporates Prototype Similarity Preserving and Prototype-Guided Gradient Constraint modules which explore prototypes to distill previous knowledge and constrain the model update. This work offers a fresh perspective on forgetting mitigation in OCL which highlights the knowledge of prototypes, getting rid of the reliance on exemplars.  
: 
 from old stages [9] . Of particular note is that during a continuously non-stationary data stream, a unique dilemma emerges. Each sample can be accessed only once for learning which will further exacerbate the catastrophic forgetting problem. Consequently, there is a rising focus on a more practical but challenging CL scenario known as Online Continual Learning (OCL) [29] where each sample can only be accessed once, as depicted in Figure 1(c) and (d) .Due to the one-pass data stream in OCL, besides the catastrophic forgetting of knowledge between different learning stages, named inter-stage forgetting, deep models may also forget the previously learned knowledge within a learning stage and thus suffer severely from insufficient training. As depicted in Figure 2 , the model may severely forget knowledge of data encountered earlier within a learning stage, and such a phenomenon is named intra-stage forgetting in this paper. As a result, the performance of existing CL methods is greatly limited in this challenging scenario. Recently, various OCL methods [1, 2, 20, 41] follow the same manner as previous exemplar-based CL approaches to store exemplars from previous learning stages for rehearsal. Nevertheless, apart from raising critical concerns about data privacy, these approaches stand in contrast to the essence of the online setting where each sample can only be accessed once.Therefore, how to effectively address OCL without using any previous exemplars still remains challenging and unsolved, named as Non-Exemplar Online Continual Learning (NEOCL), as shown in Figure 1(d) . Indeed, there are rare works focusing on this challenging NEOCL problem. [14] introduced a gradient projection strategy to constrain the updating of model parameters during OCL. The latest work [19] pre-trained an offline model on half of the datasets and subsequently froze the backbone during the online learning phase. While these strict constraints on model updating alleviate the catastrophic forgetting issue to some extent, they also impose significant limitations on the learning capacity. This restriction becomes particularly apparent when a substantial portion of the pre-training data is inaccessible. Recently, to resist forgetting without exemplars, various prototype-based methods have been investigated in Non-Exemplar Continual Learning (NECL) [52, 53] , which compute the mean feature of all samples from the same class after a learning stage, serving as the prototypes. However, these approaches only tackle inter-stage forgetting and are infeasible in NEOCL due to the single-pass data stream.To handle both the intra and inter-stage forgetting issues, we propose a novel NEOCL method called Progressive Prototype Evolution (PPE). Our PPE approach leverages learnable class-specific prototypes for each class as surrogates of previously acquired knowledge to facilitate the learning of OCL. Specifically, the prototypes, treated as learnable parameters, are progressively learned and evolved along with the OCL process to convey more informative knowledge. By involving them in the learning of the current OCL stage through a prototype feedback design, the intra-stage forgetting issue can be greatly alleviated. Additionally, to tackle inter-stage forgetting, two vital components, the Prototype Similarity Preserving and Prototype-Guided Gradient Constraint modules, are proposed based on the obtained prototypes. The former aims to distill the similarity knowledge between the learned prototypes and features of input data, regularizing the model and resisting forgetting from the data perspective. Furthermore, the inter-stage forgetting caused by model updating will inevitably deteriorate the representation ability of the prototypes of previous stages. Therefore, the Prototype-Guided Gradient Constraint is explored to dynamically control model updates according to changes in knowledge measured by the learned prototypes, achieving a better balance between preserving acquired knowledge and learning new information.To sum up, the main contributions of this work are three-fold: (1) To tackle the challenging and critical NEOCL problem, we propose a novel Progressive Prototype Evolving method that effectively learns prototypes and guides the model to mitigate both intra and inter-stage forgetting. (2) The prototypes are progressively learned and concurrently fed back to the current learning stage to mitigate intra-stage forgetting. (3) Moreover, Prototype Similarity Preserving and Prototype-Guided Gradient Constraint are introduced to counter inter-stage forgetting, striking a balanced approach between knowledge retention and acquisition. 
 
 Appendix: For example, a memory size of 100 for CIFAR-10 means keeping 10 exemplars per class. 


Decision: This paper proposes a non-exemplar method for online continual learning. It involves progressively learning and evolving class-specific prototypes during the online learning phase to mitigate both intra-stage and inter-stage forgetting. 

In the meantime, it leverages prototype similarity preservation and prototype-guided gradient constraint modules to distill and leverage historical knowledge. The novelty is ok and the experiments are basically complete.

Most reviewers support this paper, and I believe some of its merits can outweigh its disadvantages. Therefore, I also hold a positive stand for this paper.

Reviews: review: paper_summary: paper_summary: The paper introduces a Progressive Prototype Evolving (PPE), to address the challenges of catastrophic forgetting in non-exemplar online continual learning (NEOCL). Previous methods often rely on storing exemplars from previous tasks, which raises privacy concerns and is impractical in settings where data can only be accessed once. The proposed PPE method does not store past data; instead, it learns class-specific prototypes during the online learning phase. These prototypes serve as accumulated knowledge, which is used to mitigate both intra-stage and inter-stage forgetting. The effectiveness of PPE is demonstrated through extensive experiments on three benchmark datasets, showing its superiority over state-of-the-art exemplar-based and other non-exemplar continual learning approaches.
strength_weakness: strength_weakness: The paper presents thorough experimental results, comparing the PPE method against multiple baselines and existing state-of-the-art methods across different settings and datasets. The method is also detailed and clear, with explanations of the mechanisms like Prototype Similarity Preserving and Prototype-Guided Gradient Constraint
limitations: limitations: - The method appears to be computationally intensive due to the additional steps of learning and updating prototypes. There is a lack of discussions on the impact on computational scalability.
- The method involves multiple hyperparameters (like the weighting parameters of different losses), and there is a lack of sensitivity analysis.
- There is a lack of limitation discussions in this paper.
paper_summary: paper_summary: This paper proposes a novel online continual learning (OCL) method called Progressive Prototype Evolving (PPE) to tackle the specific dual-forgetting problem. Different from existing OCL methods, PPE effectively addresses the forgetting issue without using any previous exemplars. Specifically, PPE mitigates intra-stage forgetting using the progressive prototype evolving module, as well as mitigates inter-stage forgetting using the Prototype Similarity Preserving module and Prototype-Guided Gradient Constraint module. Extensive experiments on three widely used datasets demonstrate the superiority of the proposed PPE against the state-of-the-art exemplar-based OCL approaches.
strength_weakness: strength_weakness: 1. This paper proposes a non-exemplar method for online continual learning (OCL), and I like this idea. Since existing OCL methods are almost all based on the replay of old data, these methods are susceptible to restrictions on accessing old data.
2. The elaboration of the paper is clear and reasonable.
3. The related work does help readers to understand the essence of the idea and to follow the work easily.
4. The experimental results are well presented and prove that the model is indeed effective. Many baselines are used.
limitations: limitations: 1. In my opinion, this paper's biggest issue is its contribution to the multimedia community. Specifically, the proposed framework consists of four different loss functions, each of which is not very innovative and lacks theoretical depth for a paper on machine learning theory research. As for multimedia applications, the image classification task the paper selected is too simple.

2. The concept of prototypes has been explored in the continual learning community [1][2][3], and there is existing work on prototypes. What distinguishes the prototypes in this paper?

3. Why not take the weights in the classifier as the prototypes? In your proposed PPE method, you denote and optimize the prototypes based on the weights of the classifier. We generally take the weights of the classifier as the prototypes. 

4. Is the learning rate 0.02 for all baseline methods? Why not set the learning rate as 0.02? In most research works, the learning rate is usually set to 0.1.

5. Show me the performance of the PPE method on both new knowledge and old knowledge at each learning stage.

6. Prove with experimental results that the Progressive Prototype Evolving module addresses the issue of intra-stage forgetting.

[1] Asadi N, Davari M R, Mudur S, et al. Prototype-sample relation distillation: towards replay-free continual learning[C]//International Conference on Machine Learning. PMLR, 2023: 1093-1106. [2] Shi W, Ye M. Prototype Reminiscence and Augmented Asymmetric Knowledge Aggregation for Non-Exemplar Class-Incremental Learning[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023: 1772-1781. [3] Zhu K, Cao Y, Zhai W, et al. Self-promoted prototype refinement for few-shot class-incremental learning[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 6801-6810.

The following content does not affect my evaluation: I am one of the reviewers for the CVPR2024 submission version of this paper.
paper_summary: paper_summary: This paper introduces a novel approach to Online Continual Learning without saving exemplars. The proposed method, Progressive Prototype Evolving (PPE), progressively learns and evolves class-specific prototypes during the online learning phase to mitigate both intra-stage and inter-stage forgetting. The PPE approach leverages Prototype Similarity Preserving and Prototype-Guided Gradient Constraint modules to distill and leverage historical knowledge.
strength_weakness: strength_weakness: 1. The prototype Similarity Preserving and Prototype-Guided Gradient Constraint modules are somewhat novel and technically sound to address the dual-forgetting problem effectively.
2. The paper is well-structured, with clear explanations, visual aids, and detailed experimental results, which enhance understanding.
3. The paper provides extensive experiments on three datasets (CIFAR-10, CIFAR-100, and MiniImageNet), showing that PPE outperforms these methods.
limitations: limitations: 1. This work does not show the combined hyperparameter sensitivity analysis for the overall optimized target, especially considering there are three hyperparameters in equation 8. It is essential to prove robustness. In the supplementary file, you only show the influence of each one by fixing the values of the other two.
2. The idea of utilizing prototype similarity/relation is not novel enough [1,2]. It would be better if there were more discussions to differ yours from the existing ones.
[1] Multi-granularity knowledge distillation and prototype consistency regularization for class-incremental learning. Neural Networks, 2023.
[2] Prototype reminiscence and augmented asymmetric knowledge aggregation for non-exemplar class-incremental learning. ICCV, 2023.
3. In the evaluation, the memory size seems too small for memory-based methods, failing to show their advantages. Meanwhile, you only compared your method with one method (DSR) that is specifically designed for exemplar-free OCL, which is insufficient.
4. I think there should be more comparisons and discussions to show the novelty and effectiveness more deeply.
paper_summary: paper_summary: This paper proposes a non-exemplar online continual learning method that progressively learns class-specific prototypes. These prototypes as surrogates of previous knowledge of different classes can mitigate intra-stage forgetting when fed back to the model during training. Authors further propose the Prototype Similarity Preserving and Prototype-Guided Gradient Constraint so as to address the inter-stage forgetting.
strength_weakness: strength_weakness: This work researches the online continual learning problem which is more common than the traditional continual learning in real-world applications. The propose method achieves higher performance than existing OCL methods with relatively low memory size in various datasets.
limitations: limitations: 1.	What is the “stage” defined in the online continual learning? How are the different stages divided? 
2.	The proposed prototype-guided gradient constraint is inspired by an individual category of continual learning methods, namely gradient projection memory [1]. How does the gradient projection memory perform in the online setting?
3.	For the prototype-guided gradient constrain method, a memory unit $M_t$ is utilized to store bases of subspaces of previous stages. How much memory is consumed by this memory unit? 
4.	How is the hyper-parameter $\lambda$ in Equation (8) chosen? Authors are encouraged to provide some results to show its influence.

[1] Gobinda Saha,Isha Garg,and Kaushik Roy. 2021. Gradient Projection Memory for Continual Learning. In International Conference on Learning Representations. https://openreview.net/forum?id=3AOj0RCNC2