Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 5: Marginally below acceptance threshold
review_content: The research question of this paper is the existence of an extremely sparse network with an initial weight assignment that can be trained online to perform multiple tasks to compete with a dense network, in a lifelong continual learning configuration. Another research question of this paper is how to identify this sparse network and achieve competitive performance. To address these questions, the authors proposed to incrementally introduce new non-zero weights when learning incoming tasks (Figure 2 and Equation 1). The network considered by the authors has a common base for all models and a head for individual tasks.

While the topic that combines lifelong learning and network sparsity is definitely interesting, the development of this paper is incremental,  and there lacks some theoretical justification on why introducing a new task will both keep the network sparse and reuse weights of the previous networks. So the paper is slightly below the acceptance threshold for now.

Pros:

+ Interesting topic.
+ The proposed schema works, as shown in the experiments.

Cons:

- There needs work to satisfactorily define the new lifelong winning ticket framework. For example, in a multi-tasks configuration, why can a new task be learned by adding sparse non-zero weights (+retraining) with the performances of previous tasks preserved? If we simply rely on restarting from scratch when adding sparse new weights doesn't do the work, then what is the contribution? In what situation do we need to restart from scratch and in what situation can new tasks be learned from the current sparse network incrementally? 
- Lack of theoretical and experimental justification about how the proposed concept will hold or fail.

 To improve the paper, I would like to see more experiments, and preferably theoretical discussions.
review_title: interesting topic, incremental work

final_score: rating: 7: Good paper, accept
review_content: ##########################################################################

Summary:

 
The paper provides an interesting extension of the lottery ticket hypothesis in the lifelong learning setup, showing the existence of these tickets for class incremental learning. The paper also explores top-down and bottom-up tickets. The authors performed experiments on CIFAR10,CIFAR100, and Tiny-ImageNet datasets showing the effectiveness of the proposed ticket strategy.   


##########################################################################

Reasons for score: 

 
Overall, I am leaning positive. I find the idea of investigating lottery tickets in continual learning is valuable to study and understand. My major concern is about the clarity of the paper and some additional experiments (see cons below). Hopefully, the authors can address my concern in the rebuttal period. 

 
##########################################################################
Pros: 

 
1. The paper focuses on one of the most important machine intelligence tasks, continual learning and more specifically class incremental learning, and studies how to find winning lottery tickets inspired from lottery ticket hypothesis paper (Frankle, Carbin, 2019). 

 
2. The authors proposed top-down and bottom-up tickets as strategies to find these tickets and showed that can perform on bar and sometimes better compared to the full mode.
 
3. This paper provides experiments on CIFAR10, CIFAR100, min-ImageNet including quantitative results, to show the effectiveness of the proposed initialization. 

 
##########################################################################

Cons: 


1. The paper uses episodic memory, a small number of examples from the previous setting. It is not clear whether this observation would generalize to regularization or generative approaches that do not require so. Meaning, lifelong learning methods are desired not to assume access to previous task data (i.e., regularization based approaches like EWS[R1], LWF[R10], Intelligent Synapses[R9], MAS[R3], UCB[R0], generative approaches includes [R6, R13])

2. Related work section can be enriched. There is a lot of work that has been done in continual learning. 

a) I attach below some representative references but it will be position this work within regularization, memory-based, and structural continual learning methods. 

b) it will be good to also show experimentally how these CIL tickets may generalize in regularization approaches and/or generative approaches mentioned above. Currently, experiments are restricted to iCaRL and IL2M. 

3) It seems that iterative CIL pruning requires retraining of the entire sequence. Does not this seem to break the natural setup of revisiting previous task data, again and again even if we prune only once?  I understood that however the goal is to probably show the existence of these CIL tickets but it is still not so clear how to make this more practical and realistic in CIL setting. 

 
##########################################################################

Questions & Clarifications
--------------------------------------
1) continual pruning algorithm A may assign -1,0,1 but some of these weights may not make sense based on m^(i-1). For example, if a mask in some location is already 0, producing -1 is not valid. In the context of this paper, do the authors observe the case of -1, when? will be good to analyze these learning dynamics during the continual learning process. 

2) "we find that the schedule of IMP over sequential
tasks, in terms of fn(i)g, is critical to make pruning successful in lifelong learning" could you elaborate on this experimentally? what went wrong in some of your experiments for other choices and why?   


3) it could be cleared to update Algorithm 1 and 2 to include the tasks loop. 

minor
--------
1) SA is sometimes confusing. In some cases, it is spelled out. sometimes not, will be good to fix it. 
2) "Compared BU with TD pruning, TR-BU tickets surpass the best TD tickets." 
Comparing? TR-BU abbreviation is not defined before this point but defined later. It will be nice to clarify. 

[R1] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., ... & Hassabis, D. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521-3526.

[R2]Zenke, Friedemann, Ben Poole, and Surya Ganguli. "Continual learning through synaptic intelligence." Proceedings of machine learning research 70 (2017): 3987.

[R3]Memory Aware Synapses: Learning what (not) to forget. (ECCV’18)
 R Aljundi, F Babiloni, M Elhoseiny, M Rohrbach and T Tuytelaars 

[R4]Exploring the Challenges towards Lifelong Fact Learning.  (ACCV’18)
M Elhoseiny, F Babiloni, M Paluri, R Aljundi, M Rohrbach and T Tuytelaars

[R5]David, and Marc'Aurelio Ranzato. "Gradient episodic memory for continual learning." In Advances in neural information processing systems, pp. 6467-6476. 2017.

[R6]Shin, Hanul, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim. "Continual learning with deep generative replay." In Advances in Neural Information Processing Systems, pp. 2990-2999. 2017.

[R7]Rusu, Andrei A., Neil C. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. "Progressive neural networks." arXiv preprint arXiv:1606.04671 (2016).

[R8]Efficient Lifelong Learning with A-GEM  (ICLR’19)
        Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, Mohamed Elhoseiny

[R9]Uncertainty-guided Continual Learning with Bayesian Neural Networks (ICLR’20)
        Sayna Ebrahimi, Mohamed Elhoseiny, Trevor Darrell, Marcus Rohrbach

[R10] Li, Zhizhong, and Derek Hoiem. "Learning without forgetting." IEEE transactions on pattern analysis and machine intelligence 40.12 (2017): 2935-2947

[R11] Kemker, Ronald, and Christopher Kanan. "Fearnet: Brain-inspired model for incremental learning." arXiv preprint arXiv:1711.10563 (2017).

[R12] Hayes, Tyler L., and Christopher Kanan. "Lifelong machine learning with deep streaming linear discriminant analysis." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.

[R13] Liu, Xialei, et al. "Generative Feature Replay For Class-Incremental Learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.



 
 

review_title: Long Live the Lottery: The Existence of Winning Tickets in Lifelong Learning

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: [EDIT AFTER DISCUSSIONS] I thank the authors for their answers to my comments. Having read the various threads, I confirm my score and see interesting work in this paper.
[/EDIT]

##########################################################################
Summary: this paper extends the lottery ticket hypothesis to life-long learning. The paper proposes a top-down and bottom-up approach to network pruning and shows that the bottom-up pruning reaches SOTA performance on several datasets while reducing the network size to a few percent of the full model size. The paper also shows higher performance against SOTA for class-incremental learning. 

##########################################################################
Reasons for score: To the best of my knowledge, this paper brings novel contributions to the community.  The approach is sound, well-explained, and evaluated rigorously.  The Open Questions section presents open questions but these do not represent a blocker to publication in my opinion.

##########################################################################
Pros: The paper has the following advantages:

- Novelty: to the best of my knowledge, the paper brings a novel contribution to the problem of LTH for life-long learning

- Clarity: the paper is well-structured, clear, and easy to read

- Rigor: the work presented in the paper is rigorous. An ablation study is included and several in-depth analysis are presented. Due diligence has been done on experimental setup. The appendix contains numerous experimentation details (providing code would be even better)

- Impact: the paper brings a significant contribution to the literature by beating or reaching SOTA on lifelong learning.

There are several open questions in the Rebuttal section, which, according to me, should not challenge my score.  However, I am interested in the opinion of the authors and other reviewers on these questions.

##########################################################################
Cons: I do not see major limitations to this paper, apart from the code not being released, which reduces the reproducibility of the paper.  This section contains only minor editorial recommendations.

The first sentence of the abstract should read "The lottery ticket analysis states that..." instead of "demonstrates that..." since a hypothesis cannot "demonstrate" something.  It is a minor detail but since this is the first sentence of the paper, it has a significant impact on the reader's impression of the paper.

Typo page 3 in "can be trained same well in isolation" (this phrase does not make sense)

Typo page 4 in "Why we need beyond top-down pruning" (this phrase does not make sense)

Graphs on Figure 3 (and in the Appendix) are hard to read for small values of remaining weights.  Many scaling the x-axis differently would help.

Typo page 6 in "learning the rest three tasks" (does not make sense)

Page 6, the sentence "Therefore, the bottom-up lifelong pruning debuts, ..." does not make sense

The Related work section is put at the end of the paper, which can make sense (some paper do this regularly).  However, for this particular paper, I would tend to think that moving it up in the paper (close to the beginning) could make sense too.


##########################################################################
Open questions

My understanding when reading page 4 section "Curriculum schedule" is that TD pruning requires the knowledge of the number of tasks.  Is that correct?  How would it extend to an unlimited or unknown number of tasks?

The rewinding point approach seems to require maintaining the full model in parallel to the optimized one.  If that is true, it seems to defeat the purpose of optimizing the model.  Am I missing something?  Also, in the real world, this could have memory implications that could make the approach less practical.

The ticket size seems not to have a theoretical upper bound in this approach.  Is this correct?

Results seem out of noise for most experiments, but it would be nice to have confidence intervals, in particular for the claim that TR-BU outperform the dense model by 0.52% (page 6)

From Figure 4, it seems that the ticket sizes seem to converge for TR-BU and TD as the number of tasks grows.  Is that what is happening?  Any theoretical analysis of this?

#########################################################################

review_title: Good contributions on the lottery ticket hypothesis to life-long learning with a few open questions

Meta-review: This work extends the lottery ticket hypothesis to lifelong learning and, in particular, it tackles the problem of class incremental learning. This is an important and difficult problem, and of great interest to the community. The authors considered top down and bottom-up pruning strategies. The proposed approaches were validated on existing benchmarks (CIFAR10,CIFAR100, and Tiny-ImageNet), reaching state-of-the-art results, and showing that catastrophic forgetting could be alleviated. While some questions remain in terms of practical relevance, they authors showed the existence of winning tickets in the continual setting. There were concerns regarding clarity and requests for additional experiments, but all were convincingly addressed and the clarifications provided by the authors in their rebuttal further strengthened the paper.
Decision: Accept (Poster)