Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: This paper considers the stability-plasticity dilemma in continual learning. The authors develop an algorithm which addresses catastrophic forgetting while trying to improve generalisation on new tasks.

The paper introduces a new cost function for continual learning, which takes into account the model's performance on previous, current and future tasks. An approximation, H, to this cost function, which involves only the data that has been encountered so far, is optimised. The optimisation is phrased as a 2-player game, where player 1 tries to maximise H by creating worst-case dissimilar inputs for future tasks. Player 2 minimises the H by optimising the model's parameters. The authors show that the two players would eventially reach a single solution (an equilibrium).
Catastrophic forgetting is prevented by maintaining a subset of previously encountered datasets and using it when optimising H.

Contributions:
- A novel formulation of the cost function of continual learning (CL)
- A theoretically justified approach to CL.
- An implementation of their approach
- Experiments which show that the resulting method outperforms the many baselines.

main_review: Disclaimer: I did not verify the proofs in the appendix.

It was interesting for me to read the new CL cost function. The analysis of the cost function and the proposed 2-player approach was also informative.
The experiments evaluate different CL settings (class-incremental, domain-incremental and task-incremental), which I think is useful for determining the overall worth of a proposed new method.


Weaknesses: 
[Experiments] I'd be curious to see experiments with mixed datasets (e.g. CIFAR10 and MNIST), which would better illustrate whether your method can generate to diverse input domains

Note: I'm not sure that the baselines used in the experiments are state-of-the-art.

Note: [Clarity] It appears that Algorithm 1 has j=0 and i=0 swapped. Needs to be corrected.

Questions regarding the cost function H on line 206: 
q1: When approximating H for player 2, you copy over the parameters in a new network \theta_B and optimise them. Afterwards, you use the predictions of this optimised network to calculate H. However, these predictions are not differentiable wrt the original \theta , are they? If this is the case, why calculate it at all?
q2: H includes J_k(\theta) a number of times: H = \beta * J_k(\theta) + ... - J_k(\theta) + ... -J_k(\theta) = J_k(\theta) *(\beta - 2) + ... .
If \beta < 2,  when player 2 is minising H, wouldn't it try to increase J_k(\theta) in order to minimise H? In this case, I find it a bit confusing why the method works so well?
I looked at your code as well to try to find the answers to the above questions, but it's not clear to me. Looking at the code, it appears that you are using \beta = 1.

q3: How computationally intensive would you say your method is, compared to others?

limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper theoretically investigates the trade-off between generalization and forgetting in continual learning (CL). The authors prove the existance of a saddle point in the generalization/forgetting space; they show that it is stable under certain conditions; and propose a practical algorithm called balanced continual learning (BCL) for finding it. Finally, the authors compare their approach experimentally against the state of the art in CL.
main_review: In my opinion, the paper is original and contains some very interesting ideas (i.e., the generalization/forgetting trade-off and using gradient ascent to improve generalization).

The text is mostly clear, but I think the authors should try to simplify their mathematical notation, as it is very dense and very difficult to wrap your head around it.

The theoretical results and the algorithm presented in the paper are significant for non-task-free CL settings. 

I feel that the experiments of the paper are insufficient. Two-thirds of the experimental work are based on the MNIST dataset which is extremely simplistic and cannot be considered to give trustworthy benchmarks. The Permuted MNIST benchmark, in particular, has been (in my opinion, justly) criticized by other works (e.g., Farquhar and Gal, 2018) for not being realistic enough. I recommend that the authors include at least one or two more split-type benchmarks in their experimental work (e.g., CIFAR-10 and/or mini-ImageNet). I would also recommend studying the effect of memory size on the behavior of the proposed algorithm using one of the 'harder' datasets.

Finally, I think that the paper will benefit a lot by if the authors were to describe some intuitive interpretations of the main theoretical results of their work.


Minor Issues:

- Line 96: There is a "Third" left there (probably after re-writing the paragraph) that needs to be removed.
- Line 134: In line 3 of Eq. (3), insert a large enough gap between the min and max operators so that their corresponding subscripts are clearly seperated.
- Line 210: Regarding the expectation operator, you have used an italic E in the formula but a regular E in the text. In any case, I would recommend using a blackboard bold typeface for the expectation operator. 


Questions:

- What is the effect of memory size on the BCL game?
- Do you think BCL can be expanded for task-free CL?
- Do we really need to prioritize tasks? Why not use the expectation of all task losses (which will be bounded given the conditions stated in the paper)?
- Why define the loss function to be a weighted sum of H, where H is a weighted sum of task losses? Why not just use a weighted sum of task losses directly?
- What is the intuition behind the selected order of play (Δx first, θ afterwards)? 
- Have you experimentally tried using the reverse order of play? I am curious if the results would be different as Theorem 2 only discusses the selected order of play but not the reverse.
limitations_and_societal_impact: Yes
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper proposed a novel idea of formalizing the generality-forgetting trade-off in continual learning via a two-player sequential game. Theoretical and empirical results are represented to support the authors' claims. The main contribution of the paper is to formally discuss the forgetting and generalization trade-off theoretically.
main_review: The idea of formalizing continual learning problem as a two-player game is novel and interesting. The theorems are interesting, but the flow of the paper is not quite clear, which makes the details hard to fully understand. The paper has put a lot of emphasis on its theoretical contribution, however, it is not quite clear in their experiments. Showing the average prediction accuracy in benchmark datasets, although necessary, could not fully support the theoretical side. When talking about trade-off theoretically, authors should think about ways to reveal it experimentally also.  For example, several metrics such as average forgetting and learning curve area [1] could be leveraged to depict the actual trade-off between forgetting and generalization. Also, in Table 2, showing more recent methods instead of earlier methods that are not particularly designed for incremental class learning might be a better choice.


[1] EFFICIENT LIFELONG LEARNING WITH A-GEM, Chaudhry et al.
limitations_and_societal_impact: Yes.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper attempts to present a theoretical framework for modelling the trade-off between generalization (adaptation to future tasks, plasticity) and forgetting (stability on previous tasks). Towards this, the paper proposes an objective function that minimizes the weighted loss on all the past (implemented using episodic memory), present (implemented using the dataset of the current task) and future tasks (approximated using preturbing the datasets of the past and current tasks). The perturbation to this loss is then approximated in the local neighbourhood of the current task. The optimization of the loss is posed as a minimax game between the two players, where player 1 maximizes the generalization by maximizing the discrepancy between the successive tasks, and player 2 minimizes the forgetting by adapting the model parameters against that discrepancy. The authors show that if player 1 starts first, the equilibrium solution of this game exists in the local neighbourhood that is defined using the current task. The proposed approach achieves strong results against several baselines on MNIST and CIFAR-100 variants used for continual learning. 

main_review: Positives:
- The paper is written very well and the generalization-forgetting trade-off is well-motivated. 
- There is a dearth of theory papers in continual learning and this presents a very nice theory for analyzing the trade-off between forgetting and generalization. 
- Empirical results are quite strong. 

Negatives:

While I don’t have many problems with the paper, I do like to ask a couple of questions. 

- Modelling generalization: The paper argues that the degree of generalization depends on the discrepancy between the previous and new tasks. Hence in maximizing that discrepancy, and subsequently training your model to account for that discrepancy, we maximize the generalization. I could see why as an optimization objective such an adversarial training would encourage the model to adapt to potentially different tasks. But you can’t say that a model would generalize better to more different tasks, in fact it is the opposite. If the two tasks are similar then the model has a better chance of generalizing (it's a test time concept). Maybe the authors would want to rephrase the lines 121-125 to correctly reflect this. 

- Time constraints: It seems that the proposed approach BCL is quite time consuming. Could the authors please provide the timing comparison of their approach with the other baselines?

limitations_and_societal_impact: Yes.
review_title: null

Meta-review: The paper addresses the balance between stability-plasticity tradeoff as a two player sequential game and​​ show theoretically that a balance point between the two players exists for each task and that this point is stable. This leads to a theoretically justified approach, balanced continual learning (BCL), which has empirical performance that is comparable to or better than the state of the art CL approaches.

Overall, the paper addresses an important question, provides a new perspective of formulating the cost, provides theoretical justification as well as empirical evaluation for the proposed method. The reviewers raised some points that were satisfactorily addressed in the rebuttal, which helped improved their understanding of where the paper stands. 

Decision: Accept (Poster)