Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper introduces sparse continual learning, a continual learning framework that overcomes catastrophic forgetting of deep networks in the learning of a task stream and accelerates the training and inference. 
The efficiency and effectiveness are achieved through three key components that encourage sparse network weight connection, replay buffer selection, and sparse gradient truncation. 
The effectiveness is validated on both public benchmarks on real-world edge deivce. 
strengths_and_weaknesses: Strengths:

- The presentation of the paper is clear and easy to follow. 

- The results on mobile devices are a plus


Weakness:

- I see sparsity as a universal and generic way of improving deep model efficiency. Therefore it is not surprising to see introducing sparsity to CL can improve efficiency.  

- Among the three techniques proposed by this paper, only the weight sparsity part seems to have a direct impact on overcoming catastrophic forgetting, which is the key to improving continual learning. 

- In Table 1, the latest method included in the comparison is from 2020. The quantitative comparison are clearly insufficient to support the claim that the proposed framework 'further improves the SOTA accuracy by at most 1.7%.'

- Overall, this paper delivers a useful message that sparsity can improve the efficiency of CL, and even slightly improve the accuracy. However, the novelty and soundness of the paper do not meet the standard of NeurIPS. 
questions: The proposed method relies on the usage of a reply buffer. As also discussed by the authors in Appendix Section A, the proposed method can potentially be applied to more advanced methods that do not rely on any previous data. Showing the proposed method to be more 'model agnostic might help improve the paper. 


limitations: The authors discussed the limitations in Appendix Section A
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper  proposes Sparse Continual Learning (SparCL in which they wanted to do cost-effective continual learning on edge devices by leveraging the  sparsity. Author's also claim that they achieve both training acceleration and accuracy preservation through the synergy of three aspects: weight sparsity, data efficiency, and gradient sparsity. Including TDM process for dynamic data removal (DDR) to remove less informative training data, and dynamic gradient masking (DGM) to sparsify the gradient updates. Authors also claim that the proposed method is uses 23Xless training FLOPs, and, achieving the SOTA accuracy by at most 1.7%.  The authors also tested the method on two datasets split cifar-10, spilt tiny Image-net in Class Incremental and Task Incremental setting.
strengths_and_weaknesses: 

Strengths:

+ Inclusion of Dynamic data removal along with sparsity is good benefit to overall system.
+ Sparsity in gradient and propose dynamic gradient masking 
+ Method also considers the  importance of weights w.r.t. data saved in rehearsal buffer.

Con's:
- While reading the paper, I really felt that some sections of the paper are not clearly written specifically Sec 4.1 Task-aware Dynamic Masking.

- I am not really clear/convincing about the authors claim about the memory foot print and training FLOPS. Consider one of the longback papers basic paper on CL using Neural Pruning (https://arxiv.org/abs/1903.04476)(You can choose any other regular CL on pruning paper) and some other CL using sparsity papers which does CL on split CIFAR-10/CIFAR-100 using pruning and they achieve the CL on them without using memory buffer of 500 images. Similarly in the method proposed by authors in which it is also showed that that they need to do extra steps of dynamic Gradient Masking,Dynamic Data Removal which add the overhead too. Clarify this if I am wrong in my understanding on how these additional overheads won't effect the training FLOPS and memory footprint.

- I am also not clear with authors point of view of choosing the "Rehearsal based methods also" for comparing for  "memory foot print and training" as it is obviously clear that rehearsal based methods does need  additional amount of training and memory.

- I am also very curious to know what happens when buffer size of the proposed method is 0. What happens to accuracies as shown in the Table 1 when buffer size increased from 200--->500 the acc increased from 65-->72

- The improvement in the acc w.r.t to SOTA minimal as PackNet shown in table 1 does show packnet, LPF achieves 93.73%,94.50% acc when compared to best SparCL-DER75(95.19±0.34)(500 buffer) and   SparCL-DER75(94.06±0.45)(200 buffer).

- One more limitation I see regarding the proposed approach is towards the scalability as you can see current papers such as ANML(https://ecai2020.eu/papers/939_paper.pdf), of CL does show CL >100's of tasks. But how would given proposed scale comparing the overhead of buffer, memory masks while trying to learn huge no of tasks.



Overall the paper is generally well written and easy to follow apart from few sections., and the experiments are thorough and well-executed but limited to fewer datasets, lesser tasks. and minimal improvement on SOTA There are extensive ablation experiments demonstrating some of the key components the proposed approach. 


questions: 
Please refer "Strengths And Weaknesses"
limitations: Please see "Strengths And Weaknesses"
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: The authors discuss whether continual learning can be investigated under the perspective of training efficiency and propose combining network sparsification techniques in a CL setting. As a result, they notice that sparser network are not necessarily worse continual learners and that they can possibly surpass some recent non-sparse approaches. The effectiveness of the proposed SparCL method is investigated by means of ablative studies and a conclusive test on an edge device.
strengths_and_weaknesses: Strenghts
- At the core of this work is a very well-thought intuition: the learning process is noisy/overparameterised and pruning can be beneficial in CL, where we need to cram knowledge in waves into a model and possibly retain generalisation capabilities to facilitate upcoming tasks.
- the presentation of this work is clear, the exposition is easy to follow.

Weaknesses (in decreasing order of importance)
- While the authors claim that they are using a SOTA roster of competitors, I believe that this is not exactly the case. Some recent methods that are missing and could provide an interesting comparison are: LUCIR [a]/BiC [b] (somewhat stronger baselines w.r.t. iCaRL that additionally manage the bias problem), MIR [c]/GSS [d] (which similarly to DDR operate a selection on data which is memorised), CO2L [e]/DualNet [f] (recent works focussing on representation learning, which could or could not benefit from sparsification, as their inner working is rather different from standard replay methods).
- In line with the previous point, the authors take DER as one of their reference baselines. In doing so, however, they are omitting the improved DER++ method proposed in the same paper that appears to be better in all respects w.r.t. DER++. Is there a reason for this exclusion? If not, I take that DER++ should be included instead.
- It would be very interesting to have an additional study on where the sparsification takes place in the network. The authors describe very practical routines to establish which weights to prune. Linked to the recent trend discussing the effect of catastrophic forgetting on different layers of the networks [g], it would be very interesting to have an analysis of at which level the network is pruned the most and whether this changes at all within or between tasks.
- an experimental comparison could be made between DDR and the approach for buffer reduction delineated in [d] (gradient based) or the one described in [h] (loss based)
- there seem to be some problems with Alg.1: the second if-then (if $t=\delta$ k) is missing its end and I cannot understand whether the following $t mod \delta k = 0$ also applies for tasks > 1. If this is the case -- as I understood from the text -- then this is not an else-if.
- I recommend that the authors include figures in this paper in vectorial format so as to facilitate reading.
- as per neurips policies, limitations and societal impact should be in the main paper, not in the appendix
- Typos: line 95 (thus not applicable - verb missing), line 161 (missing escape of space after w.r.t. - write w.r.t.\ in latex), line 282 (contributes -> contribute), line 288 (as much as informative -> as much informative)

[a] Hou, Saihui, et al. "Learning a unified classifier incrementally via rebalancing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.   
[b] Wu, Yue, et al. "Large scale incremental learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.  
[c] Aljundi, Rahaf and Belilovsky, Eugene and Tuytelaars, Tinne and Charlin, Laurent and Caccia, Massimo and Lin, Min and Page-Caccia, Lucas, "Online Continual Learning with Maximal Interfered Retrieval", NEURIPS 2019  
[d] Aljundi, Rahaf, et al. "Gradient based sample selection for online continual learning." Advances in neural information processing systems 32 (2019).  
[e] Cha, Hyuntak, Jaeho Lee, and Jinwoo Shin. "Co2l: Contrastive continual learning." Proceedings of the IEEE/CVF International Conference on Computer Vision. 2021.  
[f] Pham, Quang, Chenghao Liu, and Steven Hoi. "Dualnet: Continual learning, fast and slow." Advances in Neural Information Processing Systems 34 (2021): 16131-16144.  
[g] Ramasesh, Vinay V., Ethan Dyer, and Maithra Raghu. "Anatomy of catastrophic forgetting: Hidden representations and task semantics." arXiv preprint arXiv:2007.07400 (2020).  
[h] Buzzega, Pietro, et al. "Rethinking experience replay: a bag of tricks for continual learning." 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021.  
questions: Please refer to the weakness section above, I am mostly interested in seeing the following point addressed:

- Why is DER++ not among competitors?
- Can any hypothesis be made on whether representation learning approaches CO2L/Dualnet can also benefit from sparsification?
- Where are weights pruned the most in the backbone, is there a pattern?

limitations: The authors adequately addressed the limitations and potential negative societal impact of their work. It is recommended to do so in the main paper, not in the appendix.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses continual learning in edge devices. It claims that previous work on CL didn't report the actual FLOPs in training which is very important in real life applications on edge devices. The paper includes adaptations of sparse training to CL . It applies three techniques:  1)dynamic weight sparsification based on the magnitude of the weights and magnitude of the gradients on the novel task and on the buffer (it combines it with collapse and expand approach on class boundaries). 2)It reduces the data based on misclassification. 3)It applies less weight updates by not updating parameters with low gradients. Applying these techniques results in significant time and memory savings without loss of accuracy when integrated with rehearsal methods for CL. It even shows slight improvement in forgetting. 


strengths_and_weaknesses: Strength: reduces the resources in CL, while previous work focused on not increasing them. Useful in real-life applications. Novel in terms of focus. 
Weakness: The proposed techniques  are straightforward and  do not offer technical novelty.
However, suggesting a straightforward solution in a paper that raises a new problem/challenge (to the best of knowledge) could still be OK.
questions: There is no section 6 in the Appendix. 
limitations: Yes
review_title: null

Meta-review: metareview: This paper introduces a new continual learning scheme whose efficiency and effectiveness are achieved through three key components that encourage sparse network weight connection, replay buffer selection, and sparse gradient truncation. After the author-review discussion phase, a majority of reviewer suggest acceptance. Only one negative reviewer did not respond to the authors' rebuttal, but AC thinks that it is convincing enough to resolve her/his concerns. AC thinks that investigating sparse networks for continual learning is novel, and demonstrating it under edge-device level is a big plus. Overall, AC is happy to recommend acceptance. AC strongly recommend the authors to incorporate all additional results and discussion-with-reviewers into the final draft.
confidence: Certain
Decision: Accept