Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper introduces a probabilistic framework for modular continual learning.
They built on the idea in [1] that modules can be chosen locally based on their input distribution.
Contrarily to [1], this paper opts for a Bayesian treatment of the module composition problem.

The paper proposes two transfer learning settings for compositionality: perceptual vs non-perceptual transfer.
In the former, we assume that most of the drift in p(x,y) happens in p(x), whereas the in the latter, it occurs in p(y|x).
They prescribe one probabilistic method for both settings.

Some experiments in CTRL [2] and a modified version show that the method can outperform some baselines.

[1] Continual Learning via Local Module Composition. Oleksiy Ostapenko, Pau Rodríguez, Massimo Caccia, Laurent Charlin
[2] Efficient Continual Learning with Modular Networks and Task-Driven Priors. Tom Veniat, Ludovic Denoyer, Marc'Aurelio Ranzato
strength_and_weaknesses: **Strengths**

The proposed method is sensible and adequately scales w.r.t. the number of training modules, which was a limitation of [1].
This work can inspire future work in continual learning.

**Weaknesses**

I wish the paper offered a more profound contrast with [1], both technically and empirically.

The idea of using the modules' input distribution as a signal for activating the modules was proposed in [1], but section 4 does not acknowledge that. Section 4 should explain how previous work has solved the problem above and then explain how their Bayesian treatment relates. The Section should further explain the pros and cons of such treatment compared to the previous solution.

The paper could instead (or additionally) benchmark their method against [1], which is the most relevant baseline for the paper.
I understand that running new experiments during rebuttal is not ideal. 
I'm not expecting the authors to do that.
At least the others might want to add the reported results of [1] for the CTRL benchmark while mentioning that [1] operates in a much more challenging setting, i.e., **class**-incremental learning and not task-incremental.

  
clarity,_quality,_novelty_and_reproducibility: great on all accounts.
summary_of_the_review: I think the paper is worthy of a publication if the weakness mentioned above are addressed.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: The paper tackles the idea of doing inference over a set of modules such that the resulting model can generalize as well as adapt quickly. Different approaches differ as to how they do search over an ensemble of modules. The proposed paper deals with both "perceptual" transfer and "non-perceptual" transfer. They introduce a probabilistic framework for selection of pre-trained modules. The key idea for driving the selection (i.e., which module to select) is  how well the module can transform the input. Each module is parameterized as a neural network. The paper define a posterior and prior, where the posterior is a function of how well the module can explain the input, and prior gives preference to modules which help achieve a higher accuracy. 
strength_and_weaknesses: Strengths:

-  The paper tackles an important problem i.e., exploring better ways of searching through possible combinations of modules.

Weaknesses:

- It would be useful to study how the proposed method scales as a function of number of modules.
- It would also be useful how the proposed method compares to methods which does a "soft" selection of the output of modules (like in LMC). 
- The paper seems a bit hard to parse. It may be useful to give a high level "overview"of the proposed method before jumping to describe how the paper works. 

clarity,_quality,_novelty_and_reproducibility: Clarity: Different sections are well written.

Quality: The paper tackles an important problem. 

Novelty: The underlying idea seems interesting, and to the best of my knowledge has not been explored in this context before. 


summary_of_the_review: The paper proposes a probabilistic framework to do inference over an ensemble of modules.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: This paper extends prior work on Continual Learning in two directions: 1) It introduces new learning sequences designed to evaluate non-perceptual transfer (transfer the last l layers) and few-shot transfer (transfer all layers) as well as more challenging sequences for the evaluation of perpetual transfer (transfer the first l layers). 2) Building on modular approaches to CL, e.g., (Venital et al. 2020), a probabilistic framework is proposed to tackle the exponential space of module combinations. One probabilistic model is proposed for perceptual (and few-shot) transfer and a separate model is proposed for non-perceptual transfer.

Experiments compare the proposed approach to some recent methods and the results are somewhat favorable to the proposed method.

strength_and_weaknesses: ## Strengths

**S1.** The paper identifies limitations of prior work and seeks to address them.

## Weaknesses

**W1.** The contributions of the submission are very incremental. I see it as an extension of the work of (Venital et al. 2020) in the two directions outlined in the summary. Further, the theoretical contributions are obfuscated by the rough presentation (more on this below) and the validation is, in part, designed for the proposed method (i.e., some of the evaluation sequences are designed to not be handled properly by prior work).

**W2.** The presentation has many issues:

1. often symbols are introduced without definition and sometimes never get defined, e.g., Z in eq. (10), d in eq. (10) does not match the d previously defined; A at the end of page 4;

2. evaluation metrics like those in Table 1 are never defined; etc.

3. some explanations do not make much sense, e.g., “gives preference to modules which helped achieve a higher accuracy, on the problems which they have been trained on,” “minimum number of transfer which need to be transferred,” “can be explored by applying this search strategy to each sequentially,” etc.

4. the probabilistic models proposed are not explained in detail, e.g., eq. (5) is simply stated with no explanation and the Gaussian Process in section 5.1 is not even specified.

**W3.** The paper disregards other benchmarks in CL like (Lin et al. NeurIPS 2021.) Such a benchmark should be discussed, and results on it should either be included or there should be a discussion explaining why those would not be relevant.

## References

Lin et al. The CLEAR Benchmark: Continual LEArning on Real-World Imagery. NeurIPS 2021.

clarity,_quality,_novelty_and_reproducibility: In my view this work has potential but is work in progress, and is not quite ready for publication.
summary_of_the_review: The paper’s contribution is limited as outlined above and the quality of the presentation is very poor.
review_title: null

final_score: rating: 3: reject, not good enough
review_content: summary_of_the_paper: The authors propose a probabilistic framework to model modular architectures for the problem of continual learning (CL). They divide the problem of choosing modules in two parts: perceptual transfer (PT), and non-perceptual transfer (NT). In PT, the first l layers are assumed to be pre-trained and the model must choose between those modules to maximize performance when training new modules on top. NT works the other way around. From this, they derive 3 CL algorithms: (i) PeCL, which does perceptual and few-shot transfer, (ii) NoCL, which does non-perceptual transfer, and (iii) CCL, which combines both strategies. To evaluate their model, the authors introduce a compositional version of CTrL and evaluate plasticity, backward transfer, forward transfer, perceptual transfer, few-shot transfer, and non-perceptual transfer. In experiments they show that their models perform favourably compared to SA, MCL-RS, HOUDINI, MNTDP.
strength_and_weaknesses: Strengths
=======
* The method is sound and it tries to approach modularity from first principles with a divide-and-conquer approach
* The proposed method outperforms MNTDP

Weaknesses
=========
* The model description is not clear, which makes it difficult to understand. Overall the submission looks unpolished. I highly encourage the authors to carefuly rework sections 3, 4, and 5, to make it clearer.
* The authors cite LMC but do not compare with it, is there any reason?
* The divide-and-conquer approach has a clear disadvantage: it introduces the additional complexity of establishing where to split the preceptual part from the non-perceptual part. Maybe the authors could include a limitations section to talk about that.
clarity,_quality,_novelty_and_reproducibility: Clarity
=====
I found this work difficult to read and understand. Here are some unclear points I found while reviewing the submission:

* To my understanding, only one module is chosen per layer. Then, at the beginning of Section 3, you mention that you split $\Pi_t$ in subsets $\Pi_t^i$. Does that mean that you choose modules layerwise? If so, why do you call $\Pi_t^i$ a subset if it is a single module? Maybe I am missing something.
* In equation 3, what is the j subscript after the parenthesis?
* In equation 6, q is introduced without any definition. I could get it from the context but a more explicit definition would help. The same with $\mathbf{A}$
* In section 4.2: "we freeze this selection and reuse the same l modules for PT paths which transfer more modules" what do you mean?
* "We define the augmented search strategy" why augmented?
* It took some time for me to parse equation 7, I think some explanation or guidance to the reader (particularly about $\pi^{*PT,l-1}[l-1]$) could make it much easier to understand.
* "The difference between two NT paths ... of length l is that their last l pre-trained modules compute different functions." If they are of length l, the last l pre-trained modules is the whole path no?
* I understand equation 10 but how it is used in your algorithm is not clear to me.
* "number of transfer which need to be transferred in order to improve the performance." what do you mean by the transfer that needs to be transferred?
* Eq 11. What is the $j$ after the parenthesis?
* You introduce the term LCB but I could not find an explanation for what it is. The search brought me to the appendix, where I could find what the letters mean as well as many "??" for the references.

Besides these points there are some minor typos like "as it does not does not involve" in Section 3.

Quality
=====
* For the most part, the technical quality is good, however, the lack of clarity hinders my understanding of the method. In experiments,  it is not clear why the authors did not compare to LMC.
* It is not clear whether the number of parameters of their method to put them in comparison to the other baselines (if it is the same I did not find it clearly stated in the text, maybe appendix C.2 last paragraph?)

Novelty
======
The proposed method is novel to the best of my knowledge.

Reproducibility
===========
The authors did not provide the code neither a reproducibility statement, however, the appendix contain some implementation details.
summary_of_the_review: The proposed method is sound and it tackles an interesting problem for continual learning. However, the lack of clarity of the text makes it difficult to properly asses the quality of this work. In its current form, this work does not meet the standards of ICLR.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper addresses the scalability issue in modular continual learning(MCL). It presents a Bayesian approach to selecting pre-trained modules based on how well the module can transfer the input.
Two different search strategies are proposed to tackle perceptual and non-perceptual transfer separately. The experiments show that the proposed framework performances comparable to state-of-the-art CL baselines.

Strengths:
- The addressed problem is challenging and important in MCL.
- The proposed method combines two search strategies to address perceptual and non-perceptual transfer differently.
- It is more efficient comparing with LMC, requires less memory and more scalable with the increase of modules, and easier to apply to new experimental settings.

Weaknesses:
- Limited novelty: The paper is built upon previous work LMC[1] which proposed using the modules' input distribution as a signal for activating the modules.
- Lack comprehensive comparison to closely related work such as Lin et al. NeurIPS 2021.
- Missing details: the probabilistic models proposed are not explained in detail.
- Limited clarity: some symbols and evaluation metrics are introduced without definition.

justification_for_why_not_higher_score: The reviewers noted that, although the paper addresses a challenging and important problem, it has limited novelty, lacks comprehensive comparison to closely related work, and misses details on the proposed probabilistic models which makes it difficult to reproduce. The authors clarified some points in their response, but the paper would still require some more modifications to be ready for publication.
Decision: Reject