Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The work couples an RNN with a latent space representation that can infer the current task in a continual learning setup. The latent space is inspired by the role of thalamus, and enables the RNN to adapt to new tasks by changing the latent representation rather than the internal dynamics of the RNN. 
strength_and_weaknesses: ### Strengths

A clear algorithm for continual learning that involves a thalamus-like part and offloads some of the weight updates in the main network to that part.

Good experimental performance.

A clear way to learn from an unlabeled stream of tasks (by triggering learning of z with large accuracy deviations) that also enforces stability in the RNN weights (as z is updated first in those cases).

### Weaknesses

As the goal of the paper is biological plausibility, it needs a discussion on how Thalamus could be implemented in the brain. As I can see it, the main issue would be credit assignment to the latents, requiring backpropagation through the RNN.

Presentation is lacking in parts (mostly the plots) but can be easily improved.

No code attached.

Comparison with other methods could be improved (see in bold below).


clarity,_quality,_novelty_and_reproducibility: ### Clarity

The paper is well-written and easy to follow. Some additional discussion could improve the presentation/place it better in the context of biologically plausible learning.

Fig. 4C: is the average done over random seeds? How many, and what’s the average latent update count? Ideally it would be plotted on a separate graph (in the appendix?).

### Quality 

The technical side of the paper seems solid. The experiments are done on a many tasks and are compared with other continual learning algorithms.

Question about prompting with a training batch (Tab. 2): are there any updates in the weights during prompting, or only in the latents? I understand the need to do it, but **I think the fair way to compare Thalamus to other algorithms would be by freezing the RNN weights completely during evaluation** (I'm not sure if it's done or not).

**Minor issues:**

Overall, the graph captions/labels could be made more clear and aligned among subplots.

Eq.5: (t) should not be a subscript (to be consistent with the rest). Maybe replacing relu with \phi (or replacing the \phi earlier with relu) would improve readability.

End of page 5: random square in the middle of the page

Fig. 4A: labels on top are not readable. I think removing them completely, and in addition only plotting the final accuracy for each task would massively improve the plot. The running accuracy plot (Fig. 4A) could be then moved to the appendix as learning curves don't tell us much here.

Fig. 11 is cut short at the bottom.

### Novelty

The work is, to my knowledge, novel. The related papers on the computational role of thalamus are mentioned, but the authors should have a look at the following paper:

Laureline Logiaco, L.F. Abbott, Sean Escola,
Thalamic control of cortical dynamics in a model of flexible motor sequencing, 2021

It introduces a different learning algorithm but is still very related to the current work.

### Reproducibility

The code is not provided, although Sec. 3 implies it will be. The authors can attach it as a zip to the submission.

**UPDATE**: the code is now provided.

summary_of_the_review: Good paper, but several aspects (presentation, discussion on plausibility, maybe better comparison with other methods, and adding code) could be improved. I put the current score to 6 but I can raise it if my concerns are addressed.

**UPDATE**: updated the score from 6 to 8 as all of my concerns have been addressed.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: This paper presents a model for continual learning that is inspired by thalamocortical networks in the brain. Neuroscience studies have revealed that (i) the prefrontal cortex (PFC) shows representation of relevant task variables, and (ii) the thalamus shows representations of the task being performed. Further, is thought that the connections from the thalamus to the PFC gate the computations performed by the PFC by selecting the relevant representations. 

In the proposed model, a RNN and a latent embedding vector serve as abstractions of the PFC and the thalamus, respectively. The projections from the latent embedding to the neurons in the RNN correspond to thalamocortical projections. The model is trained on a sequence of tasks with alternating RNN weight and latent embedding updates, resulting in a network capable of parsing a sequence of inputs into the appropriate contexts. 

Finally, the performance of the model was evaluated on cognitive tasks from (Yang et al., 2019) and on the split MNIST task. 
strength_and_weaknesses: The proposed model is a simple RNN with projections from a latent embedding vector. But by training this model using a simple procedure of alternating RNN weight and latent embedding updates, we obtain a network that is capable of learning tasks in the absence of any explicit task labels. These results are quite interesting, and as the authors mention in the conclusion there are a lot of interesting future directions (e.g., using a latent space that allows for compositionality over computational primitives). 

The following are some comments/questions I had:
-  Is there a specific reason why the weights from the latent embedding are drawn from a Bernoulli distribution? 
- Figure 3 shows that the latent embeddings cluster according to task id with the pretrained network. I suppose the same could be expected in the thalamus network trained from scratch? 
- What is the loss function used to train the networks? 
- I wonder if there is a better criterion for switching to latent updates than just a drop in accuracy. This requires the network to always know how well it is performing at the task? 
- Would simultaneously optimizing both the RNN weights and the latent embedding vector do worse than the alternating updates? 
clarity,_quality,_novelty_and_reproducibility: Overall, the paper is clearly written and easy to follow. The following are some minor comments/questions I had:
- The symbols $t$, $\mathrm{t}$, $\mathbf{t}$ are all used;  it is a bit confusing as to which one corresponds to time and task
- The network output is described as a linear layer projection from the RNN neurons, but eq 5 also includes a relu activation function? 
- What do the round colored markers in figure 2C correspond to?
- The loss function used to train the network is not described in the main paper or the appendix
- At the end of section 3.5, is the learning rate for SGD $10^{-4}$ ? 
summary_of_the_review: The paper is well written and the main concepts are clearly conveyed. The experimental results also highlight the potential of this approach. Overall, the proposed model is promising and could potentially lead to more interesting avenues of future work.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes a brain-inspired algorithm for continual learning and disentangled representations. However, the contributions are generally not clear. 
strength_and_weaknesses: The paper is somewhat well-written. The results may show the effectiveness of proposed method. 
clarity,_quality,_novelty_and_reproducibility: I am not a researcher in the field of continuous learning, but I am a researcher in brain-like computing, this writing and organization of paper makes me unable to judge its novelty and value. Generally, I feel the technical details and contributions of this paper are very vague, especially since the authors claim that their work benefits from brain mechanisms.
summary_of_the_review: Weakness: 
1、	The connection between brain mechanisms such as thalamic-cortex interaction is weak. The background description is very confusing. I could not get the clear message from the description. I feel that they purposely link their algorithm with a brain mechanism. Unfortunately, authors don't tell a complete and feasible story.
2、	The description for method is simple (e.g., the Section 2 contains very short paragraphs) but I can not see any connections between the model and the motivation mentioned above.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors proposed a RNN with latent embedding that uses optimization at inference time to generate internal contextual signals allowing the agent to parse its temporal experience into discrete events and organize learning about them. They showed that the model trained on tasks sequentially using weight updates with task identifiers provided, can be used to identify tasks dynamically by taking gradient steps in the latent space. Moreover, the model shows generalization to novel tasks and can discover temporal events at arbitary time-scale and does not require a pre-specified number of events or clusters. 
strength_and_weaknesses: The proposed model is simple and effective on the benchmark. The author claimed that the model shows generalization to novel tasks. However, with such simple network, I am wondering if this is a real generalization, i.e., how different the novel tasks is from those tasks the model has seen during training? Do they share the same underlying task distribution?

There are several minor questions here: 

1, Why is the multiplicative effect consistent with the dis-inh. thalamocortical projections?

2, What is r(t) in Eq.5?

3, During training, why do you keep $W^z$ fixed?

clarity,_quality,_novelty_and_reproducibility: The paper is clearly written and the logic is clear. However, the task should be introduced in a clearer way since many readers are not familiar with the cognitive tasks such as working memory and decision making.
summary_of_the_review: The paper borrow some idea from how the brain carries out continual learning (especially relating to the Thalamus). But how the simple RNN model is related to the PFC and the Thalamus is not that clear. Even though the authors showed that the proposed model works well on the cognitive benchmark, it is still not clear to what extent the model can be applied to more realistic task (those tasks considered in the ML community).
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper presents a model for continual learning that is inspired by thalamocortical networks in the brain. There is consensus that the paper should be accepted on the ground that the approach is interesting and it opens up novel avenues for future work and that the experimental evaluation is solid. There was two reviewers with more of a neuroscience background who found that the claims regarding the biological plausibility were overblown and I would encourage the author to tone these down.
justification_for_why_not_higher_score: While there is general consensus that the paper should be accepted some of the claims regarding neuroscience were not adequately backed up.
justification_for_why_not_lower_score: There is general consensus that this should be accepted.
Decision: Accept: poster