Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: **General**

The paper presents a very intriguing hypothesis, and I believe that its publication will benefit the community and stimulate fruitful discussion. The model seems to offer a compelling and fairly novel explanation of cerebellar deficits (including non-motor) with a broad significance across the neuroscience and deep learning communities. That said, I think that there are opportunities for improving clarity and filling in details that will be lost on readers who aren't strongly familiarity with Jaderberg et al.

The paper currently presents the model primarily in the feedforward setting, and the results in the recurrent setting. This becomes confusing, since the two settings can be associated with different locking problems in neuroscience (i.e. bio-plausible alternatives to backprop vs. learning from delayed rewards / bio-plausible BPTT). For instance, the abstract and main text intro ("a given layer has to wait for all the layers above to finish computing its gradients") seems to propose a solution for feedforward bio-plausible backprop. In the results, however, it focuses primarily on the ability to learn from delayed signals with BPTT models. I think it may help to focus earlier on the recurrent setting with references to the delayed feedback problem. 

In Section 2, the cortical network uses the backward synthesiser to avoid needing to wait for the loss signal - however, this seems to merely shift the problem since now the synthesiser will need to wait for the loss signal to train. Reading Jaderberg et al. (and the SM), it's clear that the synthesiser is instead continually trained on bootstrapped estimates. I think that a reference to bootstrapping in the main text (and a reference to the analogy with bootstrapped value functions in RL, as in Jaderberg) would make the model much clearer.

Since Jaderberg et al. have already shown that DNIs improve on truncated LSTMs, it seems like a more interesting comparison in the results would be DNI vs. CC-DNI.  If the authors are proposing that CC-DNI is a competitive deep learning approach than I would like to see something like the original DNI architecture included as a baseline. Otherwise, I would at least like to see a clearer description of the architectural differences between DNI (as previously implemented) and CC-DNI/sCC-DNI. 

**Details**

- Do the predictions/consequences in Fig. 1, g, roughly correspond to dL/dh_i? If so it would help to make this explicit.
- Figure 2(f)(g) are labelled in the caption as (e)(f).
- Unlike Jaderberg et al., the paper combines update and backwards locking under the same label. A clarification, either in the SM or a footnote, would help.

review_title: Sginificant and original hypothesis linking ML and neuroscience; some clarity issues.

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors consider the architecture of the cerebellum as the predictive component of a decoupled neural interface (DNI). Using this framework, they perform experiments training networks with BPTT on several temporal tasks. 

The paper is exceptionally clear and the experimental investigations are well-done. 

However, it does not offer any new insight into either the cerebellum or DNI; rather it simply juxtaposes the details of two existing bodies of knowledge. The two statements that might most closely constitute new insight—that DNI is helpful on temporally challenging tasks and that sparsity-induced de-correlation can be helpful—were both established within their respective research domains. The logical induction that the authors make "predicting that the cerebellum is particularly important for more temporally challenging tasks" does not require the DNI to be established. That enforcing the architectural constraints of sparsity within a DNI might be helpful, although curious, does not constitute sufficiently extensive findings for a publication, and (although aided by) does not require connecting DNIs and the cerebellum.

Much like the platitude "the brain is like a computer" offers no new insight into either computers or brains, here, I do not believe that authors' investigations, although amusing to follow-along with, have added significant insight into either the cerebellum or DNIs. Therefore, I cannot offer strong support for acceptance.
review_title: A clever mixture of existing ideas

final_score: rating: 3: Clear rejection
review_content: The authors proposed that cerebellum in the brain computes synthetic gradients, as used in decoupled neural interfaces (DNI), to enable learning in neural circuits without waiting for gradients to propagate backwards. The authors incorporated several architectural properties of biological cerebellum into their cerebellar model. They showed that a LSTM trained with such synthetic gradients can learn a variety of tasks, from motor reaching to caption generation. The paper is clearly written, and the link between DNI and cerebellum is a novel idea. However, the authors made little attempt to actually compare their model with experimental findings from cerebellum (except for the cerebellar properties built into the network), limiting its scientific impact. Meanwhile, it is not clear whether the cerebellum-inspired DNI provides concrete advantages over DNI proposed in Jaderberg 2017.

Major comments:
(1) The authors made little attempt to actually compare their model with experimental findings, or at least make concrete testable predictions.

Main results (Fig. 2-4) are mostly showing that their core models, CC-DNI and sCC-DNI, can successfully reduce losses across a variety of tasks, and do so better than heavily truncated BPTT gradients.

Except for the architectural features incorporated into the model, and some loose arguments that cerebellum is involved in sensory, motor, and cognitive tasks, the link between the model and biological cerebellum appears somewhat weak.

The authors claimed that “our work makes numerous predictions and opens the exciting door to explicit comparison…” without actually spelling out any key prediction. One prediction of the model is that when a task is very well-learned, the gradients (both real and predicted) should be close to 0, and cerebellum output neurons in the deep cerebellar nuclei should be somehow silent. Another prediction is that cerebellum lesioning should only impact learning, but not performance of well-learned tasks. I’m not sure these predictions are supported by empirical evidences. 

I would feel much more comfortable supporting this manuscript if the authors provide more comparisons with experimental data, and make more concrete testable predictions.

(2) Critical questions about how real gradients are computed and transmitted to inferior olive is not answered.

For the brain, the backward lock may not be the most acute issue when searching for approximated gradient descent in brains. DNI relies on computing the real gradients, and using it to train the generation of synthetic gradients. Both computations are challenging in the brain. The authors completely circumvent this problem by saying it is “outside of the scope of the current paper”. However, I think this issue is critical for considering the feasibility of the proposed mechanism. For example, how can cerebellum learn to predict the gradient of individual cortical neurons when there are many fold less Purkinje cells than cortical neurons? There are of course more granule cells than there are cortical neurons, but in the model, granule cells are not the ones representing the synthetic gradient, \hat{g}_M, right?

Minor
(1) The references are at a number of places somewhere between inaccurate and incorrect. Here are a few that I noticed.

In the introduction, the authors wrote “These observations suggest that the cerebellum implements a universal function across the brain (Diedrichsen et al., 2019)”. However, if I’m not mistaken, the Diedrichsen review is arguing the exact opposite that cerebellum is not implementing a universal function.

In section 2.1.1, the authors wrote “Here we use LSTMs (Hochreiter and Schmidhuber, 1997) as a model of cortical networks, which have recently been mapped onto cortical microcircuit (Costa et al., 2017) ”. Costa 2017 provided a potential way to link cortical microcircuit to a LSTM-like structure. I don’t think it’s fair to say that LSTMs are "mapped" onto cortical microcircuits.

In section 2.1.2, the authors wrote “On the other hand Bellec et al. (2019) showed that temporal gradients as used in BPTT are equivalent to using eligibility traces that transmit gradient information forward in time”. The eligibility-trace-based algorithm proposed by Bellec 2019, namely e-prop, is not “equivalent” to back-prop. It is an approximation that works well empirically in the cases studied in that paper.

(2) Fig. 2 panels are mislabeled.

review_title: Interesting idea but relatively weak connection to cerebellum

Meta-review: Reviewers split on this paper with one arguing that it is an intriguing and significant paper for both neuroscience and deep learning, whereas others argued that it fails to answer some key questions and stops short of offering testable predictions or novel findings. In particular Reviewer 2 questioned the limited experimental predictions and their experimental backing, as well as the plausibility of gradient calculations.  Reviewer 4 raised more fundamental concerns about the significance of the paper's contributions. All reviewers appreciated the paper's clarity. Overall, though, Reviewers 2 and 4 raised enough significant concerns that I cannot recommend acceptance. 
Decision: Reject