Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a deep class conditional model for the online task's incremental learning. The primary motivation of the paper is to learn the feature embedder since a fixed feature extractor is not useful in the dynamic environment. In this work, the model leverages the replay buffer to overcome the catastrophic forgetting and the buffer samples are selected by maximizing the posterior between the subset of samples and task samples. They used jointly the buffer and batch samples to update the embedder network parameter. Finally, the samples are classified by learning the prototype/mean of the class.
strength_and_weaknesses: Pors:

1: The model is simple and requires a small memory buffer to overcome the catastrophic forgetting. 

2: Easy to follow and ablation for the sample selection strategy w.r.t. reservoir shows the improvement. Sample selection method is intuitive and it may be useful in the other replay based approach. However, practical time complexity may be a bottleneck.

3: For the reproducibility, they provided the code.

Cons:

1: The paper handles the easiest setting of the continual learning (CL) where task id are provided during inference, which is not practically useful. In this setting, a simple expansion based approach [a,b,c,d] without any replay, model achieves the performance near the upper-bound without using the pretrained backbone. In this paper, none of these methods are compared. Replay is required for the comparatively much complex setting, like class incremental learning or task-free continual learning.

2: The model is not end-to-end, it learns the embedder and classifier separately. The embedder learns the class specific cluster representation and class conditional Gaussian learns the prototype classifier. Also, learning only the mean is too simple and it may not handle the complex scenarios. 

3: The sample selection strategy requires finding the subset of samples to maximize the posterior. Is there any optimization based model to select this subset of samples? If not, it’s like a brute-force model and sample selection is too costly.

4: Novelty is limited conditional marginal likelihood was explored in earlier work, sample selection approach seems novel but the complexity of it may be a bottleneck.

5: The baselines are weak and no recent model are included. How the author created the baseline for the previous work and upper-bound? The implementation details of the baseline are missing, do the author did the proper hyperparameter search for the baseline? 

[a] Calibrating CNNs for Lifelong Learning, NeurIPS-20

[b] Ternary Feature Masks: zero-forgetting for task-incremental learning, CVPRW-21

[c] Supermasks in Superposition, NeurIPS-20

[d] Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors, AISTATS-21


clarity,_quality,_novelty_and_reproducibility: The paper has clear writing, novelty is limited and proper baseline of close work and recent works are missing.
The code is provided hence reproducibility may not be an issue.
summary_of_the_review: The paper uses the simplest setting of CL and without replay recent model shows the result close to the upper-bound.

Recent comparison with the similar and recent work is missing 

Paper has some novelty, but that is not significant for the publication.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper proposes using an empirical Bayesian framework to simultaneously learn an embedding model ($f_\phi$) and a probabilistic model (parameterized w/ $\theta$) for the continual learning setting. To that end, $f_\phi$ is updated using a variant of experience replay through a running memory of samples, where the memory samples are updated with each task such that the KL divergence between the true posterior for $\theta$ and the posterior with the new memory is minimized. While updating the embedding network, the authors use marginal log-likelihood wrt $y$ of a subset of data conditioned on the rest. The paper presents one instance of the proposed framework referred to as DeepCCG which uses conditional Gaussian model as the probabilistic model. Several experiments show the DeepCCG performs better than a few selected baselines in two different (disjoint and shifting window) task incremental settings. 
strength_and_weaknesses: Strengths

- The proposed DeepCCG seems to do reasonably well in the task-incremental setting compared to some of the considered baselines.
- The experiment design of using a moving window to simulate task overlap has not been explored before.

Weakness

- *Missing related work*: There is an entire class of methods that do expansion-based [1-5] continual learning that have not been discussed in the related work. [1-4] also use a Bayesian framework (though for expansion) for continual learning. These methods also learn the entire model without using any pre-trained/frozen embedding network. This class of methods should be discussed in the related work.
- *Replay Baseline*: Replay based on generative models [5-6] has not been considered as a baseline.
- The writing and the presentation in the paper can be improved significantly. In terms of presentation, section 4 seems like an over-generalization of the methodology presented in section 5. Since the paper only discusses a simple variant of the general framework, which uses a class conditioned Gaussian to define the probabilistic model, section 4 seems a bit redundant. 
- Some of the claims over-state the contribution, for e.g. Pg 5 section 4 (last para): "our general approach can be used in task-agnostic settings as well" -- The task-agnostic continual learning is a much more challenging setting than task incremental. Without any empirical evidence, this claim is not well supported. 
- The sample selection strategy based on the KL divergence of the two posteriors in equation 11 assumes that the training data in each task is same across the tasks? From equation 11, it seems that given a new task with large number of samples, the new memory would be biased towards the new task, which could easily cause catastrophic forgetting of previous tasks. However, the paper only assumed settings with equal number of samples across all tasks.
- The paper only considers the task-incremental setting, which is known to be a relatively easier setup than other challenging continual learning settings, including class-incremental and task-free continual learning. 


Other Minor concerns:
- The notation in the paper can be improved:
  - In equation 1, the marginalized likelihood is a function of the embedding network. Making this precise in the notation (e.g. $p_\phi (y|x,t_{x,y})$) would make the discussion that follows in section 4 clearer.
- In section 6.1 under benchmarks: "in shifting windows ... there is no overlap between any two of the task" -- Is this a typo? There is overlap with shifting windows, right? 


[1] A Neural Dirichlet Process Mixture Model for Task-Free Continual Learning, (Lee et al., (2020))
[2] Continual Learning using a Bayesian Nonparametric Dictionary of Weight Factors (Mehta et al., (2021))
[3] Bayesian structure adaptation for continual learning, (Kumar et al. (2020))
[4] Hierarchical Indian Buffet Neural Networks for Bayesian Continual Learning, (Kessler et al, (2020))
[5] Efficient Feature Transformations for Discriminative and Generative Continual Learning, (Kumar et al., (2021))
[6] Deep Generative Replay, (Shin et al. (2017))

clarity,_quality,_novelty_and_reproducibility: Clarity and Quality: The writing and presentation in the paper can be improved significantly. The section 4 and section 5 can be combined to clarify that the probabilistic model and the embedding network are updated separately.  There are also many grammatical errors that can be fixed.

Novelty: The novelty seems limited given that the probabilistic model is based on a simplified class-conditional Gaussian model (LDA) which was also used in Ostapenko et al. (2020). Moreover, the idea of using a subset of samples to determine the conditional marginal log-likelihood (Equation 3) was also explored in Lofti et al., (2022), although its application in continual learning seems new.


summary_of_the_review: Based on the above weaknesses and concerns, my recommendation is to reject the paper in its current state. While the idea of using DCCG w/ experience replay is interesting, there are some major concerns in terms of clarity and presentation. The baselines and related work can also be improved significantly.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper addresses the problem of lifelong learning, specifically without a fixed backbone by using a Bayesian approach. The approach shows good results in a challenging setting.  
strength_and_weaknesses: Strengths:
- The paper is well written and easy to follow
- The approach is relatively novel. The main focus in continual learning is on fixed backbone, also the shifted setting got relatively little attention.
- The results seem promising compared to other baselines.

Weaknesses:
- Not enough comparisons, cifar10 with only 10 classes is not a good benchmark for this method and cifar100 is a relatively easy benchmark. Would be better to compare on other benchmarks like CUB. Also use only a scaled-down version of resnet18
- Overall results aren't that good, maybe because the difficult setting but also could be due to the network used. Would be more convincing if it would work with a larger network
clarity,_quality,_novelty_and_reproducibility: The method is clear, novel, and reproducible.
summary_of_the_review: The paper addresses an important challenge and gets improved results compared to other approaches
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The authors proposed an approach for learning embedding function and a probabilistic model on top of it for task-incremental online learning. The method has two update steps, first updating the model parameters based on the log-marginal likelihood of the probabilistic model and then update a memory bank of class representative examples by optimizing the kl divergence between a current posterior and a candidate posterior which considers data from the current task. The authors presented an instance of this algorithm with Gaussian distributions. The authors presented results of their method on CIFAR-10/100 and miniImageNet datasets in two settings, disjoint tasks (in terms of classes), and a shifting window.
strength_and_weaknesses: Strengths:
* A simple method that seems to work well compared to the baselines.
* The method was evaluated in two settings, disjoint tasks and a shifting window which shows that it is relatively robust.
* The paper is easy to follow and understand.
* Code was provided, which is noteworthy.

Weaknesses/Questions:
* Although I am not an expert in the field, it seems that the true novel part is the KL divergence loss for selecting the memory bank. I would imagine that there were other strategies for making this selection in the continual, life-long, or incremental learning setups, so why this strategy is better than others? 
* Following the last question, Given that the authors chose a Gaussian class conditional distribution with unit covariance, I can imagine that the resulting selection strategy appeared in other papers as a frequentist loss function. Have the authors checked that?
* Lastly on the same subject, why did the authors assume a unit covariance? is it a realistic assumption? I would imagine that even an isotropic covariance matrix that requires inference on an additional parameter per class will be more suitable.
* I wonder what is the computational overhead of the two-step learning algorithm (i.e., learning the embedding and learning the probabilistic model on of them at each iteration), can you quantify that using wall-clock time or FLOPS?
* To me it seems that most baselines are outdated, how does the method compare to newer ones?
* It seems that the authors missed an important related work [1, 2] that share some of the assumptions of this work (for instance, [1] also assume that classes are grouped in a Gaussian ball in the feature space). In general, I wonder how this method or alternative methods that use GPs will work on the tasks presented in this paper. This seems like an important baseline.
* Which of the baselines learn the embedding function and which isn't? It seems like important information and I didn't find an answer to that.

[1] Achituve, I., Navon, A., Yemini, Y., Chechik, G., & Fetaya, E. (2021, July). GP-Tree: A Gaussian Process Classifier for Few-Shot Incremental Learning. In International Conference on Machine Learning (pp. 54-65). PMLR.

[2] Titsias, M. K., Schwarz, J., Matthews, A. G. D. G., Pascanu, R., & Teh, Y. W. (2019, September). Functional Regularisation for Continual Learning with Gaussian Processes. In International Conference on Learning Representations.
clarity,_quality,_novelty_and_reproducibility: * The paper is written clearly for the most part.
* The novelty of the paper is limited.
summary_of_the_review: I think that the paper is nice, but I am not sure that in terms of novelty, the approach, and the compared baselines it is good enough to be accepted. I am willing to change my mind in light of new information from the authors.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a method for online continual learning for the setting when the task ids are known (the task-incremental setting). The basic idea is to model each class using a class-conditional Gaussian in an embedding space which is learned in an online fashion (instead of using a pre-trained embedding function).

The paper received detailed reviews from reviewers who are experts in continual learning. While the reviewers appreciate the idea, there were also several concerns, one of which being the fact that the paper considers the task-incremental setting which is considered much more easy (and less practical) than the class incremental/task agnostic setting. In fact, there is a lot of recent interest and recent work on online class-incremental learning. Although the authors in the response say that their method can be extended to this setting by having a task-id predictor, it needs to be seen how such an approach would perform.

The reviewers also had some concerns about missing baselines. The authors responded to these concerns but the reviewers remain unsatisfied from the response.

In the end, after considering the reviews, the author response, and the discussions, it was felt that the paper, despite its interesting ideas, still falls short of the acceptance criterion. Also, none of the reviewers championed the paper or argue for acceptance. 

I think the paper does have a nice, principled Bayesian framework; however, the above-mentioned issues also weaken it. Overall, it has makings of a good paper but is not there yet. The paper would be stronger if the proposed method can be adapted to work for the (online) class-incremental learning setting as well, in addition to empirically showing that it is competitive to recently proposed online class-incremental learning methods. The authors are encouraged to explore this possibility to improve their method and submit at another venue. 

Although, unfortunately, the paper fell below the acceptance bar, I also deeply appreciate the authors for engaging in a constructive and open discussion with the reviewers.
justification_for_why_not_higher_score: The paper does have the makings of a nice paper but, due to the reasons given in the meta-review, the paper falls short of the acceptance criterion.
Decision: Reject