PBES: PCA Based Exemplar Sampling Algorithm for Continual Learning
Authors: Sahil Nokhwal
Nirman Kumar
Venue: Submitted to ICLR 2023 (2023)
Keywords: Continual Learning
Incremental Learning
Machine Learning
PCA
principal directions
principal component analysis
Class-incremental learning

Abstract: Traditional machine learning is both data and computation intensive. The most powerful models require huge quantities of data to train and the training is highly time-consuming. In the streaming or incremental model of machine learning, the data is received and processed in a streaming manner, i.e., the entire data stream is not stored, and the models are updated incrementally. While this is closer to the learning process of humans, a common problem associated with this is “catastrophic forgetting” (CF), i.e., because the entire data is not stored, but just a sketch of it, as more and more data arrives, the older data has invariably a smaller representation in the stored sketch, and this causes models to perform badly on tasks that are closer to older data. One of the approaches to solve this problem stores an “exemplar set” of data items from the stream – but this raises the central question: how to choose which items to store? Current approaches
to solve this are based on herding, which is a way to select a random looking sample by a deterministic algorithm. We propose a novel selection approach based on Principal Component analysis and median sampling. This approach avoids the pitfalls due to outliers and is both simple to implement and use across various incremental machine learning models. It also has independent usage as a sampling algorithm. We achieve better performance compared to state-of-the-art methods.


Under review as a conference paper at ICLR 2023
PBES: PCA BASED EXEMPLAR SAMPLING ALGO- RITHM FOR CONTINUAL LEARNING
Anonymous authors Paper under double-blind review
ABSTRACT
Traditional machine learning is both data and computation intensive. The most powerful models require huge quantities of data to train and the training is highly time consuming. In the streaming or incremental model of machine learning, the data is received and processed in a streaming manner, i.c., the entire data stream is not stored, and the models are updated incrementally. While this is closer to the learning process of humans, a common problem associated with this is "catastrophic forgetting" (CF), i.e., because the entire data is not stored, but just sketch of it, as more and more data arrives, the older data has invariably smaller representation in the stored sketch, and this causes models to perform badly on tasks that are closer to older data. One of the approaches to solve this problem stores an "exemplar set" of data items from the stream but this raises the central question: how to choose which items to store? Current approaches to solve this are based on herding, which is way to select a random looking sample by a deterministic algorithm. We propose novel selection approach based on Principal Component analysis and median sampling. This approach avoids the pitfalls due to outliers and is both simple to implement and use across various incremental machine learning models. It also has independent usage as a sampling algorithm. We achieve better performance compared to state-of-the-art methods.
INTRODUCTION
In traditional machine learning, one usually trains model from given data that serves for training and testing purposes. The model can then be applied to various queries. On the other hand, humans learn and build "mental models" incrementally over time from series of experiences and information sources. The corresponding machine learning scenario is known as continual learning (CL). In this scenario, machine learning model continually keeps learning and adapting to new data; in effect, the data is viewed as stream rather than batch. Both humans and continual learning systems suffer from the problem of forgetting. Humans can forget information they learned in the past, whereas a machine learning model's adaptation to new incoming data suffers from so- called catastrophic forgetting (CF) due to inaccessibility of the data of carlier tasks in CL setups. In machine learning, trained model depends heavily on the class distribution seen in the data. The reason for CF is that the class distributions of incoming data changes over time. Newer classes are introduced with time and this invariably changes the class distribution. Conceptually the stream of data can be thought of as a sequence of tasks where each task usually consists of data about few new classes. Thus, different tasks have different class distributions and the model needs to update itself in order to recognize the new classes. It is impractical to retrain a model over all the past data, and thus one "remembers" either just a sample of the older data or else one tries to make the model itself more complex so that it can hope to just incorporate the newer data into the additional model complexity. In either case, there is some forgetting involved as the entire old data cannot be stored or remembered in the model. The challenges posed by CF cause difficulties for AI models to adapt to practical systems across a multitude of fields (Prabhu et al. 2020 sem] Li et al. 2020). In particular, different streams of data have different inter-task distributional gaps and thus pose challenges of different natures. While there have been continual learning studies on various domains such image classification commonly found day-to-day objects (CIFARIO, CIFAR100, ImageNet datasets), food images (FoodlK dataset), but datasets with high diversity (large inter-class gaps) have not been studied previously. Such data usually has outliers, and it

---

Under review as a conference paper at ICLR 2023
becomes crucial to remember data that is not outlier. An example domain that has this large data variance is the sports domain. In the sports domain, there is high diversity and a large inter-class gap, and thus existing approaches do not perform well while classifying images of various sports. Another difficulty associated with learning systems is handling class imbalance. Labeled data for classes may be highly imbalanced and this affects the model trained so that its performance can be bad. This problem is perhaps exacerbated in a continual learning scenario where only a sample of the older data can be remembered. In case the data is highly imbalanced, it possible that the rare classes get no representation in the sample at all. while the dense classes occupy all of the samples. This can potentially lead to overfitting or underfitting of the trained model. The goal of this paper is to address these problems. First, we want to design a more robust sampling scheme so that the data remembered less prone to outliers. Perhaps an example real-life scenario will serve to convey the problem. Imagine attending party at a large company you just started working for. At the party, you meet many people, but you want to retain the names of the "key" people. In particular, if you cannot remember everyone's name, perhaps it is good to remember those of the people in your group or closely related ones as well as the top company officials, but compromise on those of their friends or family members they may have invited. Second, our learning approach should be able to handle class imbalance gracefully. There have been intriguing findings in recent rehearsal-bass CL papers approaches that maintain fraction of previously seen data when training new incoming classes [Prabhu et al.] 2020) in inasemnnrre scenarios, hence, mitigating CF (Goodfellow et al.. 2013). However, as mentioned, in rehearsal- based slass-incrementll scenarios an important question arises: how should the representative memory be managed optimally? Due to the limited number of stored data points compared to the number of incoming data points, during training the stored data points could either suffer from over-fitting or be disregarded because of the large quantity of incoming data points. A naive approach would be to progressively raise the storage size as new tasks are coming; however, this technique neglects an important representative memory constraint, i.e., to store fixed number of data points. Hence, an approach is required that can preserve enough information about the previous class while using modest number of data points.
The literature (Castro et al., 2018 Rebuffi et al., 2017 Wu et al., 2019 Zhao et al. 2020 He et al, 2020 Hou al.) 2019) mainly uses a herding algorithm (Welling 2009) for choosing data points, also known as exemplars, that is based only on the class mean. As per Javed & Shafait (0018) herd- ing algorithm is better than random selection of data points. Many researchers have proposed other effective algorithms to select data points or exemplars in rehearsal based method to mitigate
the CF (Kim al., 2020 Aljundi al. 2019a Chen & Lin 202T: Wiewel & Yang 202T). As
mentioned before however, none of the current approaches performed well in our experiments when the data variance is very large, such as in the sports domain. We propose a novel sampling algorithm that performs better than state-of-ther -f--te--rttttttt methods in CL., Our proposed continual learning system is effective for both class-balanced and class imbalanced datasets. The proposed system is effective even when the dataset is sparse and intra-class variation is high. To test the performance of our system in a class imbalanced scenario, we use it for the image classification problem in the sports domain. For our experiments we have used Sports73, Sports100 and Tiny ImageNet datasets. See Figure T for a sample of images from Sports100 dataset. Our main contributions are as follows: (1) A novel sampling algorithm PBES to select exemplars is proposed that is robust even when out- liers are present. (2) We show how to mitigate class imbalance issue in continual learning settings by using KeepAugment (Gong al., 2021
data augmentation approach. (3) We demonstrate the ef- ficiency of our proposed method using two class class-imbalee imbalannede Sports and Sports 100 image datasets and one balanced Tiny ImageNet dataset. We demonstrate that our overall continual learning system outperforms existing state-of-the -ftte--r- approaches in all cases.
2 RELATED WORK IN CONTEXT
In this section, we mention some related work and how the work in this paper fits in the context. Intuitively, CL can be viewed as learning where the data is presented as a stream during the learning phase. Even so, it can be divided into tasks where each task is a group of related data. For concrete- ness, let us consider a classification scenario. In this case, task is a data for a sequence of classes. Data for a class in a task is not repeated in future tasks. There are two main situations under which continual learning works (van de Ven et al. 2021) (i) task-incrementu (TIL): here task ID is present along with class data. This comes of use later in testing, where during a classification query, the task ID of the (unknown) class is available us, and, (ii) class-incremente (CIL.:: no such task
2

---

Under review as a conference paper at ICLR 2023
ID is present. CIL is thus more challenging than TIL, and this paper is about CIL. Even though there is no task ID, we may conceptually divide the data stream into into chunks where each chunk has data for a fixed number of classes. Each such chunk can be termed methods can be divided into three approaches (Mai et al. 2022 He & Zhu 2021 Luo et al. 2020): 1) rehearsal-based approaches: where representative memory
task. Incremental learning
be used retain some exemplars
of previous tasks, which can subsequently be replayed in the current task (Wan et al. 2022 Yan
ct al. 2022 Chaudhry et al. 2018: Castro al., 2018 Rebuffi et al. 2017). 2) regularization-bann approaches: where no exemplars parrs from previous tasks are kept, but the information from the model parameters is used to train model on new incoming tasks (Kirkpatrick et al., | Zenke et al. 2017
Liu et al. 2018: Li & Hoiem 2017 Lee et al. 2017 Mallya et al.) 2018), and, 3) architecture- based approach which involves continuously expanding the model to the new incoming tasks without affecting the prior network parameters. As per literature (Mai et al. 2022: He & Zhu [02T] Luo ict al., 2020). oftentimes, replay-based approaches outperform regularization baaed approaches in CIL setups. Assuming representative memory exists, the aim of this paper is to improve represen- tative memory management for storing exemplars. We also examine the significance of exemplars memory size using novel algorithm for exemplars selection. We now look at two important issues that arise during CIL using a rehearsal-based approach.
The techniques in rehearsal-based approaches have documented significant catastrophic forgetting as a result of an imbalance in the class's data points (Wu et al. 2019). Models are sensitive to the most prominent memory buffer classes. The gradient information is used to update the model's weight from the previous tasks in MER (Riemer et al. 2018), GSS (Aljundi al. 2019b), and GEM (Lopez-Paz Ranzato 2017). In the recent past, MEGA Guuo et al.] 2020) has proposed loss-balancing strategy to alleviate forgetting by combining the loss of prior and present classes. Chaudhry et al. (2020) suggest using the most damaging examples from previous tasks as anchor points to solve the CF problem (the HAL approach), whereas Balaji et al. (2020) suggest retaining more details by saving intermediate activations conjunction with the raw images (the CAL approach). Nonetheless, these methods disregard the significance of representative memory management and often employ Ling-buffer (Chaudhry et al.] 2020) or reservoir sampling
Class Imbalance in training data:
(Riemer et al. 2018), or simple random sampling (Balaji et al. 2020 Guo et al. 2020).
Representative Memory Management: Here memory management refers to managing the content of the memory, i.e., which exemplars store. Various techniques are found in the literature (Parisi) jet al. (219)) for this purpose. Intriguingly, considering their computational complexity, a number of suggestions (Rebuffi et al.] 2017 Chaudhry et al., 2018 Castro et al., 2018) demonstrate only minor accuracy improvements above regular random sampling. Discriminative sampling (Liu et al., 2020). herding selection (Welling 2009), and samples based on entropy (Chaudhry al., 2018) are a few examples of these approaches. Herding selection selects samples proportionate to the number of data points in each class, depicting each data point's 's distance from its class mean. Discriminative sampling selects data points which establish decision boundaries between classes. In entropy- based sampling, data points are selected based on the randomness of their softmax distributions in the final layer. Each class distribution's 's mean and boundary may be accurately represented by Liu et al. (2020) suggested sampling approach. Using a cardinality-constrainee bilevel optimization approach, representative memory technique generation of coreset is given by Borsos et al. (2220). A GAN- based memory is suggested by Cong et al. (2020) that can be perturbed for exemplars in incremental learning. These papers address the effectiveness of the data points kept in memory; nonetheless, training a sample GAN for exemplars' memory would be computationally intensive or complex (Borsos al. 20200). Because of the instability of GAN-based sample generation, we must instead rely on effectively sampling exemplars.
NOTATION, FORMAL SETUP, AND BASIC ARCHITECTURE
During CIL training, stream of data is presented to the learning system and this can be divided into T' tasks. Conceptually, each task is a pairt = (C, D) of a set of classes C and data about them D, so overall the stream looks like, (C1, D)). (C2, DD) .D.,,.., "C",..,",, DT"), where f. = (C', D') is the ith task. We let C = {ci, (ei,c2.... .m, where the oj are the m, classes in the set C². The dataset De is the collection of points, {(1 vf), (x), v&)..... (2h, vo.)) where ar, is a data point and v} E C is its
3

---

Under review as a conference paper at ICLR 2023
pubo Tootboll
Bebsled
Tock Niycle
Folo
Hammer Thrww
10C
130
200
200
200
200
730
A0D
203
200
200
200
200
200
200
200
High jump
Fiold hockey
Roure akalin man
10C
I0C
200
236
J03
150
120
Figure I: Random images from Sports 100 dataset
label. Thus, the total number of data points is, Σ n1 and the total number of classes E mi- Notice the classes are disjoint across tasks, so C nci 0 for +
Here we will assume for CIL that the number of classes m, does not change across the tasks so my same for all however the total number of data points can vary from task to task, so n, may not equal ny for f j. Moreover, the number of data points in DD' that have y cj. i.e., are of the kth class, may not be the same for all & between 1 and m,. This termed class imbalance.
We now describe briefly the architecture of most offline learning systems, including for example iCaRL (Rebuffi et al., 2017). The main components are (1) Representative memory which used to store and manage data points that best represent what the model has learned in previous tasks. Storing prior data points may be impracticable due to storage issues. To alleviate CF in class- incremental settings, rehearsal-ber based methods retain small amount of previously seen data termed exemplars. Two important aspects of representative memory are selection and discarding techniques which refers to which examples are stored as exemplars and which of them discarded, and the memory size which refers to how many exemplars are stored; feature extractor this is usually deep neural network that has been trained on two different loss functions, such distillation loss and cross-entropy loss, and it is typically called a cross-distilled loss function. It extracts a fixed number of features from every data point; (3) weighting policies in real-world CL systems, some tasks may have more importance than others, whereas, during training, distribution techniques fail take into account the significance of each task. Thus, important consider all tasks with some weighting policy; (4) classifier this is an algorithm that classifies test data point into one of the classes seen so far; (5) data augmentation this refers to increasing the number data points by modifying the original dataset to alleviate the problem that only small amount of data is available in representative memory for previously seen classes. This increases inference accuracy.
OUR APPROACH
Our proposed method to mitigate CF includes novel technique of selecting exemplars. This tech- nique is coupled with an effective data augmentation technique to produce a balanced training dataset from imbalanced dataset. The overall workflow is shown in Figure Here, F.E. and D.A. stand for Feature Extractor and Data Augmentation. respectively. Rather than choosing exemplars according the class mean, used in herding (Welling) 2009), develop custom sampler
4

---

Under review as a conference paper at ICLR 2023
Classifier
Classifier
Classifier
B.
Medelt
Model
Model
Sampler
M exemptars
Sampter
M exeimptare
D.A.
D.A
Initial clnssns
New chasses
Training
Training task
Figure 2: Architecture of Our Continual Learning Approach
which is based on Principal Component analysis and median sampling. The KeepAugment (Gong al., 2021) algorithm is used to supplement each new incoming class, creating balanced data stream. We apply the classification loss to this balanced stream and the distillation loss to the exem- plars stored. the next section, we will discuss each component's 's specifics.
4.1 SELECTION OF EXEMPLARS
In this section we present our algorithm PBES to select the exemplars. The pseudocode for the algorithm shown Algorithm The input to the algorithm a training set X , ... Fn of n images of a particular class and target integer m I of images to select as exemplars. Henceforth we simply call the images "points" as they are treated as points in a high-dimensional space. The algorithm starts by computing only when n is odd and m is even. Then, it runs the Principal Component Analysis (PCA) algorithm on X and computes the first p principal directions. Next, it initializes set R to X. The set R is the set of remaining points, none of which have been selected. The sequence of selected points is stored in P, of which the first y will be returned the end. It repeatedly does the following: projects and sorts the remaining set R of points on the ith principal direction (for = 1, 2...... .) and select either 1 (when R] is odd) or 2 (when |R] even) median points, and appends them P. removing them from R. The following is elementary. For completeness a proof is included in Appendix A
[m/2] or mm/2] + 1, the latter choice being taken
Lemma 4.1 The sampling algorithm appends exactly m or m + points to (before returning its first m points), depending on whether m, n. have the same or different parity.
Intuition There are two key ideas involved Algorithm [ The first one is the selection of the median points along projection to some direction (this happens to be principal direction). The reason for this is to make the selection exemplars robust to outliers. common approach is to select the exemplars by distance from the mean. However, since the mean be affected greatly by the presence of outliers, this strategy does not perform well for datasets with outliers, for example Sports73 and Sports 100 datasets, where variance is pretty high given the nature of sports data and outliers are to be expected. The second key idea is to project the data along the principal direc- tions before doing this. To why, imagine dataset with intrinsically low dimensions in a high dimensional space. Suppose it lies on -ww-dienniioal affine subspace. Projecting the data to direction orthogonal to this subspace will map all the points the same point and the median will not have any meaning. Thus, only directions lying in the subspace really matter. For such directions, the "best" ones are those that expose the maximum variance in the data, as intuitively the outliers "cause" the high variance. The first principal direction is the best for this, and as the subsequent principal directions maximize the "residual" variance they are preferred. There is recent work in statistical sampling using similar ideas (Deng et al.).
Another idea, that we use as a control, is to project on few random directions. The idea of reducing the dimensionality of data set using random projections goes back to the lemma (Johnson & Lindenstrauss 1984). which says that the pairwise distances between n points are approximately preserved upto + e, when they are projected to random Ollog n/e²) dimensional
5

---

Under review as a conference paper at ICLR 2023
subspace. This projection be achieved essentially by projecting to those many randomly chosen directions. In general, random projections are an important tool to preserve geometry and random directions serve good control set of directions. We implement the idea of using median sampling along a few random directions. We call this RandP in experimental results section (Section 6
4.2 GENERATING BALANCED DATASET USING DATA AUGMENTATION
Since predicting Sports73 and Sports 100 datasets class distribution real challenge, the model finds it hard to retain knowledge learned so far. The problem is exacerbated the datasets are class imbalanced. The most popular continual learning research focuses on experiments with balanced datasets such MNIST, CIFAR 100, ImageNet, etc. In those datasets, the number of data points is equal between classes, whereas the number data points in Sports73 and Sports not equal. Therefore, have proposed to use Keep/uugmentttion (Gong et al. 2021 algorithm to make each task t of Sports73 and Sports 100 balanced during training the feature extractor. Although a substan- tial amount of work has been devoted to enhancing the variety of exemplars kept in memory through data augmentation, but less work has been devoted to improving the robustness of feature extrac- tor using the technique. Data augmentation improves the classifier's generalization performance to achieve the highest efficiency on learned tasks. First, from a task t; (C D'), identify the class c¹ C that has the highest number of images among all the classes C". Let lcf. denote its size. Then, apply the Keep/uugmenttioon algorithm on images of all other classes in the task f, except for class c) and generate the required number of images ri to make a class cj balanced with respect to the class cfj, where & E [1, me], k * j. Therefore, the required number of additional augmented data points to balance the classes can be mathematically expressed as r le,l - k E [1, k + j. Let 7/ denote the r new images generated for class cf and DÍ represent the original training dataset of class d. Then, the augmented dataset for class o is T U Dj. The augmented datasets for each class og for k E [1,m;], + j and of are then used to train the feature extractor.
Algorithm 1: Sampling of exemplars using median points
Function PRES__ [mmple e (X.m). m)
Input:
Image training set X (f1,......, Fn) of class c, m number of exemplars output
Output: P exxmplaass
i is odd and m is even then
L p [m/2] +
else
L p p=[mm/2]
Compute the p principal directions of X using PCA
R + X
for + 1 to p do
L; Sort points of R along ith principal direction
10
if [ is even then
Let æ1. be the lower and higher median points from L
12
P + + (21,
R < R {21,02)
14
else
Let , be the median point from
16
P + P
17
R R (
18
return first m points of P
EXPERIMENTAL SETUP
Experiments used NVIDIA RTX 3070 GPU, with 8 GB VRAM, 16 GB RAM and PyTorch 1.8.0.

---

Under review as a conference paper at ICLR 2023
5.1 DATASETS
For our work, we used the Sports73 and Sports 100 datasets imported from Kaggle. These datasets have images from the sports domain and have 73 and 100 sports classes respectively. They have classes such as swimming, badminton, etc. Sports images were taken indoors and outdoors in diverse lighting conditions. The datasets are divided into train, test and val datasets, and contain 11,365 and 13,572 training images; 365 and 500 testing images; and 365 and 500 validation images respectively. All images are 224x224 pixel sized. Randomly selected classes with task sizes of 10 and 20 (number of randomly selected classes at each task) were used in the experiments. Both the datasets are imbalanced and a histogram visualizing this imbalance for the Sports73 dataset displayed in Appendix C Apart from these two datasets, we have done experiments on the Tiny ImageNet dataset to show that our approach also works on a balanced dataset.
5.2 IMPLEMENTATION DETAILS
For feature extraction and representation learning ResNet32 (He et 2016) model was used. We start our experiments with a learning rate of 2.0 (Rebuffi et al. 2017). and at each epoch, the learning rate will be decreased by 20% 49th and 63rd epochs, and the weight decay will be 0.0001 (Rebuffi it al. 2017). The memory size was 550 for Sports73 and 800 for Sports Exemplars account for about 4% of the highest number of images among all the classes (Rebuffi al.] (017). except for the experiments where we test the effect of varying the number of exemplars on performance (Section (6.3.. In order to reduce the training and representative memory requirements the datasets were resized to 96x96 pixels, and batch size of 32 was used for each experiment.
EVALUATION METRICS
After each task, we evaluate the model by testing on all the test classes we observed up to that point. Let a denote the accuracy obtained in the test after task We use the metries, (1) Average accuracy, which is defined as the average of accuracies upto task t, i.e., Average Accuracy(A:) -E=10 0: (If t T, then the average accuracy Ar denotes the average training accuracy over all the tasks.), and, (2) Last accuracy which is the final accuracy observed after training all the tasks T, i.e., Last Accuracy = 0T. To determine these numbers, three trials were conducted, and the average of each value defined above was computed over the three trials.
EXPERIMENTAL RESULTS
In our experiments, we first compare our approach with the well-known state-of-the-ard approaches such as Rainbow Memory(Bang et al. 2021), GDUMB(Prabhuu et al. 2020), and iCaRL. al., 2017). Further, we compare our approach to two other approaches, Fine-tune and Upper bound. As part of the Fine-tune approach, only the new class images are used for training, and exemplars are ignored. Cross-entropy loss is applied to the new class images, and distillation loss is not taken into consideration. Thus, the number of exemplars is zero. The fine-tune approach is taken as a lower bound for our problem (in terms of performance). In the lower bound, the exemplars not stored in memory, and the training for all the tasks happen without considering any exemplars. The Upper bound approach considers all the images seen so far at every task and applies cross-entropy loss. In short, the number of exemplars per class is equal the number of images for that particular class. In other words, it represents the upper bound for our problem (in terms of performance). We have also compared our results with random projections. By conducting ablation studies, we discuss the effectiveness of our exemplar selection in Section 6.2 of this paper.
6.1 COMPARATIVE STUDY WITH OTHER EXISTING TECHNIQUES
The final (Last) accuracy and average (Avg) accuracy for each increment task are summarized in Table I We observe that the accuracy of the CL approach changes depending the task size. For a certain number of training classes to learn, a lower task size will result in more incremental tasks, increasing the likelihood of catastrophic forgetting. In contrast, training many classes for every incremental task is likewise a tough challenge for bigger task sizes. In particular, we discover that Fine-tune suffers from severe catastrophic forgetting problem, with Last and Avg accuracy being
7

---

Under review as a conference paper at ICLR 2023
drastically worse than they are with the Upper bound. This is because there is limited training data for every learned task throughout the incremental learning process.
In Figures 3 and and 4 we have displayed the accuracy examined at the end of every task with task sizes of 10, 15, and 20, Our approach achieves better results than state-of-the-ar methods in continual learning. The performance difference between approach and the Upper bound is minimal. The results for the Tiny ImageNet dataset are shown in Appendix
Table 1: Average and Last task accuracy of Sports73 and Sports 100 datasets with task size 5, 10 and
20
Datasets
Sports73
Sports 100
Task size
10
13
20
I0
13
20
Accuracy
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
Avg
Last
RM
29.01
16.71
30.87
24.65
34.95
25.47
46,67
25.2
48.44
31.6
55.68
45
iCaRL
36.34
16.25
45.79
20.26
44.54
20
33.9
19.6
30.09
11.80
44.86
30.8
GDUMB
25.83
11.23
24.51
23.49
14.79
42.72
23.4
39.83
23.6
38. 19
23.2
RandP (Ours)
54.49
37.5
52.43
62.43
39.6
51.74
38.1
55.40
37.52
58.93
46.4
PBES (Ours)
62.51
41.5
62.68
41.87
68.27
46.75
59.44
58.22
42.09
63.97
Upee
Upper and
Vpee soun
COLMO
GOLPNE
GELLM
Fine
Numberat
Number chinee
Number
Figure 3: Testing accuracy of Sports73 with task size 10, 15, and 20
Uppur sound
Uppnr bound
Upow bound
Konet ('urs
Rosie Frand
Number
Number
Mumbeer
Figure 4: Testing accuracy of Sports with task size 10, 15, and 20
6.2 ABLATION STUDY
Now, we will study our approach's components and illustrate their influence the final accuracy. The number of exemplars is kept fixed for these studies. Our approach has two modules 1) module- 1: proposed algorithm PBES, and, 2) module-2: our approach to integrate the KeepAugment algorithm (Gong et al.) 202T) for data augmentation. In particular, we compare and contrast the following approaches: baseline: select exemplars using the herding algorithm. (2) PBES
8

---

Under review as a conference paper at ICLR 2023
sampler without KeepAuumeentation PBES sampler for exemplar selection but data augmen- tation. (3) baseline and KeepAuugeenattion method: Herding algorithm for exemplar selection and KeepAugmentatioo for data augmentation. (4) Ours: PBES sampler for exemplar selection and Kegmenttnnnniion for data augmentation. Each component (PEES and KeepAugmentation) discussed in this paper, when compared to the baseline, improve performance. We find that when integrate all the modules of the recommended strategy, our results significantly outperform the base- line. Since there is significant class imbalance present in the Sports73 and Sports 100 datasets, where the number of data points in training data varies in the range [98, 191] among sports classes, we also conclude that our training regime gives significant performance improvements for such datasets. The datasets also have huge variance and the proposed strategy performs well even in this case.
Table 2: Average accuracy of different methods on Sports73 and Sports datasets with task size 20
Method
Sports73
Sports T
baseline
44,86
44.54
PBES sampler without KeepAugmentation
61.2
63.56
baseline with KeepAugmentation
60.67
61.28
PBES sampler with KeepAugmentation
63.97
68.27
6.3 INFLUENCE OF EXEMPLARS SIZE
This section shows the effect of exemplars size the performance. To test this vary the total number of stored exemplars M (550, 700, 850, 1000) for Sports73 and M in (800, 1000, 1200, 1400} for Sports100, Our average accuracy using the Sports73 dataset with task size of 20 is displayed in Table 3a and in Table 3b for the Sports 100 dataset. For both ap- proaches, improved performance is seen when more exemplars are used. Exemplar memory capac- ity is amongst the most crucial components for a continual learning systems, especially in an offline environment. We discover that our proposed technique performs better than the baseline for any given number of exemplars. This improvement over baseline becomes more apparent as the number of allowed exemplars decreases, i.e., our method responds much better under a memory crunch.
Table Influence of Exemplars Size for (a) Sports73 dataset & (b) Sports dataset
(a)
(b)
M=550
M/700
M=850
M=1000
M=800
M=1000
M/=1200
M=1400
68.27
71.43
75.1
78.3
63.97
66.4
69.2
72.23
CONCLUSIONS
This study devised a unique sampling technique that selected representative data from a stream of incoming training data in effort to reduce the catastrophic forgetting problem of continual learning. Further, a learning regime was presented, employing the data augmentation method, which aids in creating balanced training from unbalanced dataset. Both complex Sports73 and Sports 100 datasets, we discovered that our proposed strategy improves performance over state- th-the-art methods, particularly when the number of new sports classes included each iteration increased. This points to the feasibility of continual, large-scale learning in practical systems.

---

Under review as a conference paper at ICLR 2023
REFERENCES
[PDF]
Parameter-Ffficient Transfer Learning for NLP Semantic Scholar
semanticscholar.org
https://www semant. www.hclar..rgg/paaa.....eee.
Drrameterr-Iffiiiet----eeeee Learning for Houl--ooolyy--uuuggglg 2aGe__13220077666660222222666663
[Accessed 03-Jun-2022).
Rahaf Aljundi, Lucas Caccia, Eugene Belilovsky, Massimo Caccia, Min Lin, Laurent Charlin, and Tinne Tuytelaars. Online continual learning with maximally interfered retrieval. In Proceedings of the 33rd International Conference on Neural Information Processing Systems. Curran Associates Inc., 2019a.
Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. Advances in neural information processing systems, 32, 2019b.
Yogesh Balaji, Mehrdad Farajtabar, Dong Yin, Alex Mott, and Ang Li. The effectiveness of memory replay in large scale continual learning. arXiv preprint arXiv:2010.02418, 2020.
Jihwan Bang, Heesu Kim, YoungJoon Yoo, Jung-Was Woo Ha, and Jonghyun Choi. Rainbow mem-
ory: Continual learning with memory of diverse samples. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8218-8227, 2021.
Zalán Borsos, Mojmir Mutny, and Andreas Krause. Coresets via bilevel optimization for continual learning and streaming. Advances in Neural Information Processing Systems, 13:44999-44890, 2020.
Francisco M Castro, Manuel J Marín-Jiménez, Nicolás Guil, Cordelia Schmid, and Karteek Alahari. End-to-end incremental learning. In Proceedings of the European conference computer vision (ECCV), pp. 233-248, 2018.
Arslan Chaudhry, Puneet K Dokania, Thalaiyasingam Ajanthan, and Philip HS Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 532-547, 2018. 2018...001..
Arslan Chaudhry, Albert Gordo, Puneet Dokania, Philip Torr, and David Lopez-Paz. Using hind- sight to anchor past knowledge in continual learning. arXiv preprint arXiv:2002.08166 3, 2020.
Zhiyi Chen and Tong Lin. Principal gradient direction and confidence reservoir sampling for contin- ual learning. In International Conference on Artificial Neural Networks, pp. 421-432. Springer, 2021.
Yulai Cong, Miaoyun Zhao, Jianqiao Li, Sijia Wang, and Lawrence Carin. Gan memory with no forgetting. Advances in Neural Information Processing Systems, 33:16481-16494 2020.
Lih- Yuan Deng, Ching-Chi Yang, Dale Bowman, Dennis K.J. Lin, and Henry Horng-Shing -hing Lu. Big data model building using dimension reduction and sample selection. Personal communication.
Chengyue Gong, Dilin Wang, Meng Li, Vikas Chandra, and Qiang Liu. Keepaugment: A simple information- preeeovvngg data augmentation approach. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 1055-1064, 2021.
Ian Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empiri- cal investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint 1122v:1312.6211, 2013.
Yunhui Guo, Mingrui Liu, Tianbao Yang, and Tajana Rosing. Improved schemes for episodic memory-based lifelong learning, Advances in Neural Information Processing Systems, 33:1023- 1035, 2020.
Jiangpeng He and Fengqing Zhu. Online continual learning for visual food classification. In Pro- ceedings of the IEEE/CV International Conference Computer Vision, pp. 2337-2346, 2021.
Jiangpeng He,, Runyu Mao, Zeman Shao, and Fengqing Zhu. Incremental learning in online sce- nario. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13926-13395 13935, 2020.
10

---

Under review as a conference paper at ICLR 2023
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning unified classifier incrementally via rebalancing. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattem Recognition, pp. 831-839, 2019.
Khurram Javed and Faisal Shafait. Revisiting distillation and incremental classifier learning, In Asian conference on computer vision, pp. 3-17. Springer, 2018.
W. B. Johnson and J. Lindenstrauss. Extensions of lipschitz mappings into hilbert space. Contem- porary Mathematics, 26:189-206, 1984.
Chris Dongjoo Kim, Jinseo Jeong, and Gunhee Kim. Imbalanced continual learning with partition- ing reservoir sampling. In European Conference on Computer Vision, pp. 411-428, Springer, 2020.
James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska- Barwinska, al. Overcom- ing catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521-355 2017.
Sang-Woo Woo Lec, Jin-Hwa Kim, Jachyun Jun, Jung-Wop Woo Ha, and Byoung- Tak Zhang. Overcoming catastrophic forgetting by incremental moment matching. Advances in neural information pro- cessing systems, 30, 2017.
Yuanpeng Li, Liang Zhao, Kenneth Ward Church, and Mohamed Elhoseiny. Compositional lan- guage continual learning. In ICLR, 2020.
Zhizhong Li and Derek Hoiem. Learning without forgetting. JEEE transactions on pattern analysis and machine intelligence, 40(12):2935-2941 2017.
Xialei Liu, Marc Masana, Luis Herranz, Joost Van de Weijer, Antonio M Lopez, and Andrew D Bagdanov. Rotate your networks: Better weight consolidation and less catastrophic forgetting. In 2018 24th International Conference on Pattern Recognition (ICPR), pp.
IEEE, 2018.
Yaoyao Liu, Yuting Su, An-An Liu, Bernt Schiele, and Qianru Sun. Mnemonics training: Multi- class incremental learning without forgetting. In Proceedings of the IEEE/CVF conference on Computer Vision and Pattern Recognition, pp. 12245-12254, 2020.
David Lopez-Paz and Mare' Aurelio Ranzato. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30, 2017.
Yong Luo, Liancheng Yin, Wenchao Bai, and Keming Mao. appraisal of incremental learning methods. Entropy, 22(11) (1(1))1119 1990, 2020.
Zheda Mai, Ruiwen Li, Jihwan Jeong, David Quispe, Hyunwoo Kim, and Scott Sanner. Online continual learning in image classification: An empirical survey, Neurocomputing, 469:28-51, 2022.
Arun Mallya, Dillon Davis, and Svetlana Lazebnik. Piggyback: Adapting a single network to multi- ple tasks by learning mask weights. Proceedings of the European Conference on Computer Vision (ECCV), pp. 67-82, 2018
German I Parisi, Ronald Kemker, Jose L Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54-71, 2019.
Ameya Prabhu, Philip HS Torr, and Puneet K Dokania. Gdumb: A simple approach that questions our progress in continual learning. In European conference on computer vision, pp. 524-540. Springer, 2020.
Sylvestre- Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pp. 2001-2010, 2017.

---

Under review as a conference paper at ICLR 2023
Matthew Riemer, Ignacio Cases, Robert Ajemian, Miao Liu, Irina Rish, Yuhai Tu, and Gerald Tesauro. Learning to learn without forgetting by maximizing transfer and minimizing interfer- ence. arXiv preprint arX1v:1810.11910 2018.
Gido M de Ven, Zhe Li, and Andreas S Tolias. Class-incremental learning with generative clas- sifiers. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3611-3620, 2021.
Timmy ST Wan, Jun- Jun-Cheng Chen, Tzer-Yi Yi Wu, and Chu-Song Chen. Continual learning for visual search with backward consistent feature embedding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 16702-16711, 11, 2022.
Max Welling. Herding dynamical weights to learn. In Proceedings of the 26th Annual International Conference Machine Learning, pp- 1121-1128, 2009.
Felix Wiewel and Bin Yang. Entropy-based sample selection for online continual learning. In 2020
28th European Signal Processing Conference (EUSIPCO), pp. 1477-1481. IEEE, 2021.
Yue Wu, Yinpeng Chen, Lijuan Wang, Yuancheng Ye, Zicheng Liu, Yandong Guo, and Yun Fu. Large scale incremental learning. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 374-382, 2019.
Qingsen Yan, Dong Gong, Yuhang Liu, Anton den Hengel, and Javen Qinfeng Shi. Learning bayesian sparse networks with full experience replay for continual learning. Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 109-118, 2022.
Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In International Conference on Machine Learning, pp. 3987-3993 PMLR, 2017.
Bowen Zhao, Xi Xiao, Guojun Gan, Bin Zhang, and Shu-Tao Xia. Maintaining discrimination and fairness class incremental learning. Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 1220-112111 2020.
12

---

Under review as a conference paper at ICLR 2023
A PROOF OF LEMMA 4.1
Proof: When n is even, the algorithm repeatedly selects points from each of the principal direc- tions as [T'] remains even throughout. Thus points are selected in all. On the other hand, when n is odd, the algorithm will select point from the first principal direction but 2 from each of the subsequent ones, i.e.. for 2, 3, ................................ ........t........... t, since [T'] becomes even after the first iteration and remains so in subsequent iterations. So, 1 + 2( - 1) = 2t points are selected in all. The claim can now be verified by examining all the four possible cases, i.e., even/odd, m even/odd.
B EXPERIMENTAL RESULTS FOR TINY IMAGENET (BALANCED) DATASET
GanP Dull
AOOL NOOLV00
CaNAA
Figure 5: Testing accuracy of Tiny ImageNet with task size 10, 15, and 20, and memory size 2000
13

---

Under review as a conference paper at ICLR 2023
C HISTOGRAM TO VISUALIZE CLASS IMBALANCE IN SPORTS73 DATASET
Dasse uisiib.iio
15
31
25
25
Nrterliss
Figure 6: Histogram of Sports classes
14


Reviews:
[{"review_id": "_Bibmw0kLq", "reviewer": "Reviewer_RPyF", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": " In this paper, the other proposes a novel selection approach based on\nPrincipal Component analysis and median sampling. This approach avoids the\npitfalls due to outliers and is both simple to implement and use across various\nincremental machine learning models. It also has independent usage as a sampling\nalgorithm.\n", "summary_of_the_review": "Overall, the article has a complete structure and clear thinking. But the content needs to be further improved, including experiment and theory.", "strength_and_weaknesses": "Strength：\nIn general, the article has a complete structure and clear thinking.\nThe experimental setting is reasonable, experiments verified the proposed algorithms on Sports  datasets.\n\nWeaknesses：\nThe description of the proposed method is not detailed enough, and the innovation point is not outstanding enough.\nThis paper mainly focuses on the effect of the strategy, and it is suggested to add more theories to support the strategy, so that the method can be more explainable.\nThe paper points out that the median is more suitable than the mean. It is suggested to carry out relevant experimental demonstration in the experimental part.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "NBvilHA9vx", "reviewer": "Reviewer_JcYz", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper addresses the problem of continual learning with a novel exemplar selecting algorithm.\nThe authors claim that selecting the exemplars based on PCA-based direction well addresses the catastrophic forgetting problem of the datasets with high intra-class variance.\nThe experimental results on various benchmarks show the effectiveness of the proposed framework.", "summary_of_the_review": "Despite the paper studies some novel concepts with respect to PCA-based exemplar sampling, I hardly agree that the current version of the submission meets the bar of ICLR, since there are some missing experiments as stated in the weakness section and the writing quality should be improved.", "strength_and_weaknesses": "- Strengths\n    - The proposed framework demonstrates better performance than other baselines for the newly conducted benchmarks, Sports 73 and Sports 100.\n    - The idea of sampling the most unaffected data along the principal axis seems novel and interesting.\n    \n- Weaknesses\n    - The writing quality of the paper is far behind the bar of top-tier conferences like ICLR.\n        - For instance, what is the point of the first paragraph of the introduction section? The paragraph consists of intro to CF, data distribution and class imbalance, the goal of this research, and etc. The paragraph should be split into multiple paragraphs to represent the crucial points for the authors' claim.\n        - Formatting issues\n            - Missing numbering for some subsections (e.g., Evaluation Metrics) and sections (e.g., Conclusion).\n            - Inconsistency of the format for the references.\n    - Datasets\n        - Is there any quantitative evidence that sports datasets have higher variance than other common datasets, such as CIFAR-100 and ImageNet, which are widely used for continual learning benchmarks? For me, the claim is not that convincing since many sports-based video datasets, such as UCF-101, or Fine-gym, do not have large intra-class variance. While I’m not an expert on the Sports 73 or Sports 100 datasets, I’m not sure why such datasets have high variance and are expected to have many outliers. Actually, when I schemed the Sport 100 datasets on the Kaggle page, it seems the datasets have less variance than CIFAR or ImageNet, since the viewpoint of camera and poses of actors seem pretty similar to each data in the same class. The claim that “Sports images were taken indoors and outdoors in diverse lighting conditions” in the Section 5.1 also seems inappropriate since the augmenting contrast is one of the widely used augmentation strategies, and thus the diverse lighting will be considered when we train datasets like ImageNet with strong data augmentation.\n    - Possible discussion in the Methods section\n        - Selection of exemplars\n            - As far as I understand, the sampled exemplars by $i$th principal direction and $j$th (where $i < j$) direction are not related to each other much. Then, the median points sampled by $j$th direction could be the outliers for $i$th direction, since two different principal axes are orthogonal. While it might be the extreme case, I think it might happen quite often when $m$ increases since the data embeddings used for sampling lie in the high-dimensional space. The claim of the paper could be improved if the authors provide any mathematical proof or intuitive explanation for this.\n            - In the Abstract section, there is a sentence \"herding, which is a way to select a random looking sample by a deterministic algorithm\". However, the proposed PCA-based framework is also almost deterministic given the dataset except for the case that some directions share the same singular values. Moreover, what do the authors mean that herding selects a random looking sample, while it selects the samples closest to the mean class vector?\n        - Generating balanced dataset using data augmentation\n            - What is the reason for generating data only for $r_k^i = \\|c_j^i\\| - \\|c_k^i\\|$? It might be better to generate $\\|c_j^i\\|$ data for each class using KeepAugment and replace $\\|c_k^i\\|$, rather than just generating $\\|r_k^i\\|$ to boost the regularization effect.\n    - Experimental results are not sufficient to support the effectiveness of the proposed framework.\n        - The combination of other sampling algorithms and the diverse data augmentation strategy must be studied more deeply.\n            - According to Table 2, it seems KeepAugment highly affects the performance. Then, it is worth checking the performance of other baselines (RM and GDumb) equipped with KeepAugment. Moreover, the good results on TinyImageNet in Figure 5 in Appendix B also might be due to the effect of KeepAugment, since the difference is marginal. Therefore, the effectiveness of the proposed algorithm for the balanced datasets is not convincing. It might be better to show the results on CIFAR-100 datasets, which is a widely used benchmark for class incremental learning. Training details for other baselines on Sports 73 and Sports 100 datasets are also missing.\n            - What if the model employs different data augmentation strategies (e.g., CutMix, etc., ), rather than KeepAugment?\n        - It is unclear whether improvement in performance is driven by class-balancing or augmented data.\n            - What is the reason for generating data only for $r_k^i = \\|c_j^i\\| - \\|c_k^i\\|$? It might be better to generate $\\|c_j^i\\|$ data for each class using KeepAugment and replace $\\|c_k^i\\|$, rather than just generating $\\|r_k^i\\|$ to boost the regularization effect.\n    - Minor\n        - Need consistency for the term “Principal Component **a**nalysis” and “Principal Component **A**nalysis”\n        - It seems the columns of Table 2 should be exchanged with each other, according to Table 1.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}, {"review_id": "8k4eJEjjwu", "reviewer": "Reviewer_Gq7Z", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Training machine learning on streaming data sets has been an important problem with the widespread use of online systems. This often requires selecting specific examples to update your model. The paper proposes a PCA-based exemplar sampling algorithm. \n", "summary_of_the_review": "I don't think that the paper is strong enough to be at ICLR. From experimentation to justifying the importance of the problem, the novelty in the proposed solution, there are many gaps. ", "strength_and_weaknesses": "The proposed technique includes a simple PCA-based dimension reduction and mapping the data along these dimensions and sample data with enough diversity. With this technique, outliers are not included in the training data. There has been a decent amount of experimentation and details provided.  I have seen some drawbacks in the paper, though.\n\n1. How big of a problem is this? What is the real-life use case and significance of the problem, and the innovation aspect (scope/impact) is not clear? \n2. The proposed solution is simple, but I would be surprised if something along these lines has never been tried (I did not do a thorough literature survey, but it seems more like a simple use case of PCA). With this regard, I think the paper lacks novelty. \n3. There can be more depth and rigor in the experimentation. For example, PCA provides good strength in eliminating outliers. It sort of smooths out the training data—like a filter. However, it comes up with removing more edge/nuance cases in a data set as well. Usually, these samples are less in population and lost in the PCA analysis—often considered as noise. However, they are very valuable with respect to providing unique information for the training. It is not clear what the trade-off is here in this case and how we can address selecting such under-represented groups in this framework.  ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "2: Several of the paper’s claims are incorrect or not well-supported.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "1: The contributions are neither significant nor novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject