Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: In this work, the author focused on developing the solution for mitigating the catastrophic forgetting problem in the continuous learning problem. The proposed framework is mainly composed of two components, the first component is exclusive and non-overlapping subnetwork weight training, The second component is the KNN knowledge transfer module. The authors have evaluated the proposed pipeline on several benchmarks and demonstrate the improvement over listed benchmarks. 
strength_and_weaknesses: Strength:
1.	The framework has adopted and expanded previous works’ solution and achieved better performance than listed benchmarks.
2.	The idea is straightforward and easy to reproduce.

Weakness:
1.	The listed image benchmark is relatively simple, hard to justify the effectiveness of the proposed solution for dealing with big dataset. 
2.	One of the most important steps in this pipeline is the supermask learning. It needs to be learned when adding new task. The author has listed the operations after 3 tasks in figure 1. What will be the parameters complexity after N tasks? Will that be a problem in the long run of continuous learning. 

clarity,_quality,_novelty_and_reproducibility: The writing quality is ok. I think the results can be reproducible. 
summary_of_the_review: My rating is between the borderline and weakly reject since the evaluation data is relative small. Not sure if it can be generalized to more complicated task with big data.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a mask-based approach to obtain subnetworks for specific tasks in continual learning (CL). It is an improvement of the previous method SupSup, which finds fixed supermask for each task to alleviate forgetting. This paper proposes to perform exclusive and nonoverlapping subnetwork training to avoid conflicting updating on the shared parameters. A specific KNN based method is proposed to enhance parameter sharing. The experiments show that the proposed method can produce better performance than other methods. 
strength_and_weaknesses: Strength
- The work is well motivated. It starts from the limitation of the SupSup method and designs well-motivated and well-justified techniques to improve the supermask based method. 
- The proposed method performs better than the compared methods with a similar setting. 
- The paper conducted carefully designed analyses and discussions on the experiments. 

Weakness
- The whole framework is based on SupSup with similar systematic limitations and benefits, which influences the significance at some level. Specifically, the task identifier is required in both training and testing, restricting the methods on the task-incremental setting. 
- As a result, all the experiments are restricted to the task-incremental setting. Please correct me if I overlooked other settings.
- Some related works are not discussed, such as (but not limited to) the following ones. There are a series of approaches to generate subnetworks at a module or neuron level to alleviate the forgetting and parameter interference issue in CL. Although the settings may not be the same, they may be discussed. 

    - Hurtado, Julio, Alain Raymond, and Alvaro Soto. "Optimizing reusable knowledge for continual learning via metalearning." Advances in Neural Information Processing Systems 34 (2021): 14150-14162.

    - Veniat, Tom, Ludovic Denoyer, and Marc'Aurelio Ranzato. "Efficient continual learning with modular networks and task-driven priors." arXiv preprint arXiv:2012.12631 (2020).

    - Yan, Qingsen, Dong Gong, Yuhang Liu, Anton van den Hengel, and Javen Qinfeng Shi. "Learning Bayesian Sparse Networks with Full Experience Replay for Continual Learning." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 109-118. 2022.
    - Golkar, Siavash, Michael Kagan, and Kyunghyun Cho. "Continual learning via neural pruning." arXiv preprint arXiv:1903.04476 (2019).

- How is the efficiency of the KKT module? It needs to run training and testing on multiple splits. Can the running time on the vision dataset be reported? How does the setting for the KKT module (like number of splits) influence the running time and performance? 


clarity,_quality,_novelty_and_reproducibility: The whole paper is written clearly, and the work is with good reproducibility, especially considering the relationship with SupSup. 
Although the relationship with SupSup may influence the significance of the novel, this work may still provide enough novel insights and observations. 

summary_of_the_review: As discussed above, the work is well motivated and justified, and the paper is well-written. 
The paper provides enough novel insights and observations and some empirical experimental results and analyses on the proposed method. The significance is influenced by its relationship with SupSup and that it is restricted to the task-incremental setting, where task identifiers are required in both training and testing. 

---
I appreciate the authors' response, which addressed part of my concerns, especially on the KKT module. I am still concerned by the novelty (mainly about the relationship with SupSup) and technical contributions. After further discussions, I am further convinced that the experiment part also needs to be improved, especially in comparison with other parameter isolation-based methods, instead of only SupSup.  
The work may be further improved by fixing the issues and conducting some more work, such as some attempts to extend the method to other settings beyond the task-incremental setting.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes an Exclusive Supermask Subnetwork Training framework for continual learning of both text classification and vision tasks. Compared to the previous method SupSup, the proposed ExSSNet makes fixed weights trainable thus facilitating the knowledge transfer from previously learned tasks to new tasks. The further proposed KKT module can be regarded as a better Initialization mechanism, which helps to learn new tasks faster and better. Extensive experiments and impressing performance validate the effectiveness of the proposed method.
strength_and_weaknesses: Strength:
- The idea of the overall framework is novel compared to previous work SupSup. It is a natural idea to expend fixed weights to trainable weights but it brings a large boost and more advantages, e.g., under the condition of sparse masks.
- The experiments are sufficient and can effectively validate the claimed points. The informative figures and tables are also helpful for illustration and understanding.
- The paper is well organized and comprehensive in terms of experimental settings and details.

Weakness:
- Although the idea is novel, most of the specific implementation of the framework is based on existing work. For example, the supermask learning follows Ramanujan et al.(2019), the training mechanism using exclusive mask is similar as Learn to Grow[A].
- The description of the paper can be polished. Some long sentences interfere with reading, and there are also a few typos, e.g., the first sentence of the fourth paragraph, “We overcome the aforementioned issues, we propose our method,...”.
- The main concern is about the claim that ExSSNet can learn 100 tasks. First, as listed in Table 4, SupSup actually achieves a good results (90.34%) and the improvement of ExSSNet (91.21%) is not noticeable. Therefore, the ability to learn 100 tasks is what SupSup have achieved, but not unique for ExSSNet. Second, I wonder that this kind of ability is related to the size of the model since the extreme case is there are no free weights. How is the performance of using a small model like LeNet for SplitMNIST and ResNet18 for SplitCIFAR100. Furthermore, how is the performance of using a larger model like Resnet101? Can larger models learn more than 100 tasks since they have more parameter spaces? 

Reference:
- [A] Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher. "Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting." International Conference on Machine Learning. PMLR, 2019.
clarity,_quality,_novelty_and_reproducibility: The originality in terms of framework is good while the originality of specific method is not so creative. The overall quality and clarification of the work meets the standard of the conference. 
summary_of_the_review: I think this paper can be regarded as an good improvements of the previous method SupSup thus I tend to accept. But my main concern is about the claim of learning a large number of tasks. If the author can give good feedback, I will improve the score.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper is proposing a model for continual learning based on subnetwork isolation and training. Specifically, the authors build on top of SupSup, a prior method relying on a randomly initialized backbone where task-specific subnetworks are discovered and isolated, showcasing good performances on a stream of tasks to be learned. This paper extends the method by i) allowing subnetwork parameters to be updated by gradient descent, and therefore achieving better performance on their task and ii) introducing a KNN-based mask initialization method for bootstrapping subnetwork selection on the next task. Experiments are carried out for natural language processing and computer vision tasks.
strength_and_weaknesses: + +The paper is very clear and well written, one read suffices to the interested reader to grasp the main ideas and technical contributions.
+ +Experiments showcase encouraging results on two different data modalities and tasks, for a total of 5 different datasets among NLP and vision.
+ +Experimental results are very encouraging, as the model is able to outperform a number of recent works, including some relying on replay buffers.
+ +An ablation study grounds the effectiveness of KKT, grounding its beneficial effect.
- -The biggest weakness of the paper is its technical novelty, that I'll discuss in the next point.
- -The model, needing to separate the relevant task-specific subnetwork during inference, can only be employed in task-incremental learning settings, which represent the easiest and more controlled protocol in CL literature. This raises questions about the suitability of the model for real problems.
clarity,_quality,_novelty_and_reproducibility: - -The technical contributions of the paper are quite limited. Indeed, the authors build on top of a previous work, SupSup, and add a few incremental updates to improve performance. The first improvement relies on allowing parameters of subnetworks to be updated by the optimization (and freezing the ones used by previous tasks). This contribution is very intuitive and effective, but its addition doesn't constitute sufficient novelty per se. Moreover, the model resembles a number of parameter isolation methods that were proposed in the last few years [1,2,3,4]. Only a couple of them are discussed and the authors claim ExSSNet avoids their shortcomings. However, none of these methods is compared against. More specifically, [1,2] rely on task-specific subnetworks, whose weights and masks are learned jointly, and have similar mechanisms to block gradient updates on parameters used in previous tasks. In this respect, differences between these methods and the proposed work is in details. Finally, the KKT component is interesting and effective, but its employment is a bit contextual to SupSup.
- -Section 2 is not necessary, as it reiterates the motivations of the paper in a more detailed way. In my opinion, motivations are convincing enough in the introduction.
- -Difference between SSNet and exSSNet does not emerge clearly in Figure 2. I however acknowledge it emerges in other experiments.
- -I struggle understanding the notation paragraph in section 3. Specifically, a node is defined as $\mathcal{Z}_v=\sigma(\mathcal{I}_v)$, where $\mathcal{I}_v$ are inputs, $\mathcal{Z}_v$ are outputs, and $\sigma$ is defined as an activation function. This way, it seems that a node is the same as an activation function.

[1] Serra, Joan, et al. "Overcoming catastrophic forgetting with hard attention to the task." International Conference on Machine Learning. PMLR, 2018.

[2] Abati, Davide, et al. "Conditional channel gated networks for task-aware continual learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020.

[3] Mallya, Arun, Dillon Davis, and Svetlana Lazebnik. "Piggyback: Adapting a single network to multiple tasks by learning to mask weights." Proceedings of the European Conference on Computer Vision (ECCV). 2018.

[4] Kim, Jangho, Jeesoo Kim, and Nojun Kwak. "StackNet: Stacking feature maps for Continual learning." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.

summary_of_the_review: I appreciate the quality of the research and of the experimental section. However, in my opinion, the technical contribution of the paper is not worth for publication in a venue such as ICLR. I really think this work could benefit more discussion and comparisons with the parameter isolation methods I mentioned above.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: The paper builds on top of an existing continual learning algorithm dubbed as SupSup,  which uses a randomly initialized, fixed base network (model) and finds a supermask for each new task that selectively keeps or removes each weight to produce a subnetwork. The authors observe a limitation in that the performance of supermask is sub-optimal due to the fixed weights restriction, and hence proposed ExSSNeT (Exclusive Supermask SubNEtwork Training), which performs exclusive and non-overlapping subnetwork weight training. The authors also propose a KNN-based Knowledge Transfer (KKT) module that dynamically initializes a new task's mask based on previous tasks for improving knowledge transfer. The transfer is disabled if the nearest task is so far. 

Strengths
--------------

- The method is well written (Reviewer jWtC), easy to reproduce, and achieved better performance than the listed benchmarks (Reviewer XDRc, Reviewer 7z7s, Reviewer jWtC).
- The work is well-motivated, starting from the limitation of SupSup method (Reviewer 7z7s,Reviewer fPix,  )
-The paper conducted carefully designed ablations(Reviewer jWtC),  analyses, and discussions on the experiments ( Reviewer fPix) 

Weaknesses
---------------
	
-  The listed image benchmark is relatively simple, hard to justify the effectiveness of the proposed solution for dealing with big dataset (Reviewer XDRc) 
- More analysis is needed to test the scalability of the method as both the number of tasks and the number of classes increase over time (Reviewer XDRc). 
- The technical contributions of the paper are quite limited (Reviewer fPix, Reviewer jWtC). The framework is based on SupSup with similar systematic limitations and benefits, which influences the significance (Reviewer 7z7s).
- The approach is mainly explored in the incremental task setting rather than class incremental. This limits the evaluation of the method's ability to distinguish similar yet different classes that appeared in different tasks (Reviewer 7z7s, Reviewer jWtC,  AC).  
- There are a series of approaches to generate subnetworks at a module or neuron level to alleviate the forgetting and parameter interference issue in CL. Although the settings may not be the same, they may be discussed and experimented against (see references provided by Reviewer 7z7s). 
- The description of the paper can be polished (Reviewer fPix).
- Not well-established claim that ExSSNet can learn 100 tasks is mainly thanks to SupSup have achieved (Reviewer fPix), also Limited Experimental validation: what happens in the extreme case where there are no free weights. How is the performance of using a small model like LeNet for SplitMNIST and ResNet18 for SplitCIFAR100. Furthermore, how is the performance of using a larger model like Resnet101? Can larger models learn more than 100 tasks since they have more parameter spaces? (Reviewer fPix),


In conclusion, the reviewers recommended 
5: marginally below the acceptance threshold	
5: marginally below the acceptance threshold	
6: marginally above the acceptance threshold	
3: reject, not good enough

leaning overall not to accept the paper in its current version. 

justification_for_why_not_higher_score: See Summary Of AC-reviewer Meeting
justification_for_why_not_lower_score: N/A.
Decision: Reject