Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 1: Reject
review_content: Summary: The paper uses a Gaussian Processes framework previously introduced in [1] to identify the most important samples from the past for functional regularization. For evaluation authors report their average accuracy on Permuted MNIST, Split-MNIST, and CIFAR10-100 and achieve superior performance over EWC, DLP, SI, VCL-Coreset, and FRCL.

Pros:
(+): The paper is well-written, addressed the prior work quite well despite missing a few important work from the past (more on this later)
(+): The paper is well motivated

Cons that significantly affected my score and resulted in rejecting the paper are as follows:

1- lack of support for “scalability”:
Authors claim their method is scalable in several parts of the paper (abstract in line 7, Section 3 in the 1st paragraph, and Section 5 in Discussion). However, this claim is not supported in the experimental setting as the benchmark used are only toy datasets (Permuted MNIST, Split MNIST, and CIFAR10 followed by CIFAR100) where the maximum # of task considered is 10 and the maximum size of the datasets is 60K which is not convincing for ability to scale. There is also no time complexity provided. 

2- Incremental novelty over the prior work (FRCL by Titsias et al 2019):
This baseline is the closest prior work to this work which according to the experiments shown in Table 2 are slightly outperformed by the proposed method. (for example for P-MNIST the gain is 0.6%+-0.1) where there is a lack of complete discussion on how the two methods are different. Particularly I suggest that the authors elaborate more on their claimed differences stated on page 4, paragraph 5 such as “tractability of the objective function only when we assume independence across tasks”. Do authors mean assuming clear task boundary between tasks? If so, have they considered a “no-task” or an "overlapping” task boundary in their experiment? Isn't it necessary to back up this if it is stated as a shortcoming of FRCL? Also, how are these methods differ in their computational expenses?

3- Lack of measuring forgetting: 
This is the most important drawback in the experimental setting. Authors indicate on page 3 “Our goal in this paper is to design methods that can avoid such catastrophic forgetting.” and reiterate on this on other parts of the paper yet there is no forgetting evaluation to support this claim. Authors can simply report the initial performance of the model on each task so that readers can compare it with the reported accuracy after being done with all tasks. Having a method with high average accuracy does not necessarily mean it has minimum forgetting. You can use forgetting measurements such as Backward Transfer (BWT) introduced in [1] or forgetting ratio defined in [4] for this assessment. 

4- Ambiguous claims about prior work:
(a) On page 1, paragraph 3, when authors mention that methods such as GEM or iCaRL use random selection to pick previous samples, I think the line of follow-up work on these methods should be mentioned as well that have explored different techniques for sample selection and have provided benchmark comparisons (ex. [2,3]). In fact it would be beneficial if authors could compare the samples selected by their method versus other sampling techniques. 
(b) On page 1, paragraph 3, they mention some prior work such as GEM and iCaRL “do not take uncertainty of the output into account”. While it is true, there have been methods proposed that use uncertainty of the output for parameter regularization [5]. It appears to be a parallel work to this but it’s worth mentioning to prevent false claims.

5- Claim on the state of the art should be double-checked:
	Although the results shown for the experiments are superior to the provided baselines, there is an important baseline missing which has achieved higher performance than the reported ones. Also missed to be cited in the prior work list. Serra et al [4] proposed a method at ICML 2018 called HAT, which is a regularization technique with no memory usage that learns an attention mask over parameters and was shown to be very effective on small and long sequence of significantly different tasks. They do not use samples from previous task but yet achieved good average ACC as well as minimum forgetting ratio. Note that 5-Split MNIST is not reported in [4], but a recent work has reported HAT’s performance on this dataset (https://openreview.net/forum?id=HklUCCVKDB) that achieves 99.59%. I recommend authors provide comparison of their own on the given benchmarks with the original HAT’s implementation (https://github.com/joansj/hat) before claiming to be SoTA. In my opinion, it is not an issue if a novel method achieves a slightly lower performance to the sota because I think it still adds value and proposes a new direction. However, a false claim should not be stated.

Less major (only to help, and not necessarily part of my decision assessment):

1- Providing upper bound?
It is common to show an upper bound for any continual learning algorithm by showing joint training performance which is considered to be the maximum achievable performance. I also recommend showing the naive baseline of fine-tuning for the proposed method  which often can give insight to maximum forgetting ratio.

2- Forward transfer?
Regularization techniques combined with memory might have an ability to perform zero-shot transfer or so called FWT. I recommend authors provide such metric to further support their method.

3- Hyper parameter tuning?
It is also worth mentioning how the tuning process was performed. In continual learning we cannot assume that we have access to all tasks' data, hence authors might want to shed some light on this.

References: 
[1] Khan, Mohammad Emtiyaz, et al. "Approximate Inference Turns Deep Networks into Gaussian Processes." arXiv preprint arXiv:1906.01930 (2019).

[2] Chaudhry, Arslan, et al. "Continual Learning with Tiny Episodic Memories." arXiv preprint arXiv:1902.10486 (2019). (https://arxiv.org/abs/1902.10486)

[3] Aljundi, Rahaf, et al. "Gradient based sample selection for online continual learning." arXiv preprint arXiv:1903.08671 (2019). (https://arxiv.org/abs/1903.08671)

[4] Serrà, J., Surís, D., Miron, M. & Karatzoglou, A.. (2018). Overcoming Catastrophic Forgetting with Hard Attention to the Task. Proceedings of the 35th International Conference on Machine Learning, in PMLR 80:4548-4557

[5] Ebrahimi, Sayna, et al. "Uncertainty-guided Continual Learning with Bayesian Neural Networks." arXiv preprint arXiv:1906.02425 (2019).


----------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
POST-REBUTTAL Response from R1:

Thank you for taking the time and replying to comments. Here are my responses to authors' replies:

[Authors' response:] 1. Scalability: Our algorithm only adds a small computational overhead on top of Adam on a standard neural network. This is what we mean by scalable. The additional complexity scales cubically in M, the coreset size. This is due to the inversion of the kernel in fr_grad. Another overhead is the computation of Jacobian which is order PKM, where K is the dimensionality of the output and P is the number of network parameters. Both of these additional costs are small for small coreset sizes M. We will add these details to make these points clear in the paper.

[Reviewer's response:] I still insist on the fact that simply explaining the overhead of a method is not a support for scalability claim versus showing the performance on a large scale dataset and comparing it with other CL methods that also have high scalability given the fact that authors only use MNIST and CIFAR datasets.

[Authors' response:] 4. Prior work: We discuss other works in Section 1 (“two separate methods are usually used for regularisation and memory-building”), and we will expand upon this sentence, going into more detail, and also referencing iCaRL and other works (including [3]). Note that our method of choosing a memorable past follows directly from the theory in Section 3.1, and is achieved with a single forward-pass through the trained network (as mentioned in the paper). Other techniques for sample selection do not integrate so naturally with the framework, and are not as straightforward to understand or implement either.

[Reviewer's response:] I disagree with authors on this because GEM, its faster version (A-GEM (Chaudhry et al. 2018)), and all other methods explored in the recent study which I mentioned in my review (Ref#2) use the single epoch protocol and are perfect match to be compared with this method but there is no memory-based baseline except for VCL with coreset and FRCL (only for MNIST variations) which makes it difficult to measure this method's capabilities (performance, memory size, and computational time) against methods which only require one epoch to be trained.

Authors have provided FWT for their method as 6% which is unbelievably large for this metric (see GEM paper) and hence does not make sense to me. Please double check whether you computed this value right. 

While I accept the response for the remaining questions from authors but I am still concerned about the weak experiments and an issue brought up by R3 regarding lack of enough comparisons with FRCL on any other datasets besides split MNIST and P-MNIST. Also in  CIFAR experiment, what is the architecture used across the baselines? More importantly in results reported for VCL on CIFAR, it is not clear to me how authors obtained this results. Did they use a conv net? VCL was originally shown on MLPs only and it is one of the downside of this method that was never shown to be working in convolutional networks. Therefor, it is important to mention how they are obtained. This might explain the reason for the huge forgetting reported for VCL with coreset (−9.2 ± 1.8) as opposed to −2.3 ± 1.4 for EWC which is really strange as VCL even without coreset (on permuted mnist for example) is reported superior to EWC by a large margin (6%) in the original VCL paper. Overall I am concerned about the experimental setup and some of the reported results and hence intend to keep my score.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: Summary
The paper proposes a method for continuous learning called Functional Regularization of Memorable Past (FROMP) which maintains the output distribution of models on memory samples. FROMP uses the Laplace approximation and Gaussian process with neural tangent kernel (NTK) to approximate the output distribution. According to the leverage score strategy, the sample to be stored is selected. The leverage score strategy tends to select the sample of highest variances. 
 
Strengths
To some extent, I think the proposed method is novel, although there is a similar work named as Functional Regularisation for Continual Learning (FRCL). FROMP first uses NTK in Gaussian process for continual learning and proposes a new strategy of selecting memory samples.
The strategy of selecting samples to be stored is simple and effective.
The method achieves a good performance.
The paper is clearly written and easy to follow.
 
Weaknesses
It needs more experimental comparisons between FROMP and FRCL, like adding comparison results of FROMP and FRCL for Split-Cifar. Currently, this paper only shows the performance on Permuted MNIST and Split MNIST but those two benchmark are quite simple and also the improvement is limited.  
The experimental section needs more detailed analysis. At least, in current version, it is not clear how many tasks in Permuted MNIST. The setting of hype-parameters for dropout are not provided.
 
Other comments 
In this paper, for Split MNIST experiment with multi-head, it shows that the method of EWC achieves worse results than SI. However, in my experiment, the precision of EWC is at least larger than 97%. In theory, I think they should have the similar performance and at least the discrepancy of accuracy between them is not as big as shown in this paper. I expect authors could explain this point.
review_title: Official Blind Review #3

final_score: rating: 1: Reject
review_content: The paper proposed a new functional regularization method with gaussian process which has similar direction with recent two works (khan et al, titsias et al).
To perform functional regularization, they introduce small coreset which are selected from previous dataset instances, called memorable past. They select most memorable samples depends on eigenvalue. The model FROMP outperforms baselines and their ablations. However, the experiments are only performed on shallow networks, it is required to apply on much deeper networks, such as ResNet. Also, in the experiment results, I feel the performance of the FROMP largely depends on the number of the coreset, while 'important' selection just shows marginal effects even on split CIFAR. 
FROMP show higher performance than FRORP with only a few of examples, but it isn't meaningful results that anyway the performances are too poor that are even worse than old baseline, EWC. 

I have several wonderings on the paper.

- How about of training time on FROMP? I wonder if utilizing or selecting memorable pasts requires much time for training.

- Is there an analysis like figure 1 on real dataset, such as MNIST or CIFAR?




review_title: Official Blind Review #2

Meta-review: This work tackles the problem of catastrophic forgetting by using Gaussian processes to identify "memory samples" to regularize learning.

Although the approach seems promising and well-motivated, the reviewers ultimately felt that some claims, such as scalability, need stronger justifications. These justifications could come, for example, from further experiments, including ablation studies to gain insights. Making the paper more convincing in this way is particularly desirable since the directions taken by this paper largely overlap with recent literature (as argued by reviewers).

Decision: Reject