Cortico-cerebellar networks as decoupled neural interfaces
Authors: Joseph Pemberton
Ellen Boven
Richard Apps
Rui Ponte Costa
Venue:  (2021)
Keywords: systems neuroscience
cerebellum
neocortex
decoupled neural interfaces
deep learning
decorrelation
inverse models
forward models

Abstract: The brain solves the credit assignment problem remarkably well. For credit to be correctly assigned across multiple cortical areas a given area should, in principle, wait for others to finish their computation. How the brain deals with this locking problem has remained unclear. Deep learning methods suffer from similar locking constraints both on the forward and backward phase. Recently, decoupled neural interfaces (DNI) were introduced as a solution to the forward and backward locking problems.
Here we propose that a specialised brain region, the cerebellum, helps the cerebral cortex solve the locking problem closely matching the computations and architecture of DNI. In particular, we propose that classical cerebellar forward and inverse models are equivalent to solving the backward and forward locking problems, respectively. To demonstrate the potential of this framework we focus on modelling a given brain area as a recurrent neural network in which the cerebellum approximates temporal feedback signals as provided by BPTT. We tested the cortico-cerebellar-DNI (CC-DNI) model in a range of sensorimotor and cognitive tasks that have been shown to be cerebellar-dependent. First, we show that the CC-DNI unlocking mechanisms can facilitate learning in a simple target reaching task. Next, by building on the sequential MNIST task we demonstrate that these results generalise to more complex sensorimotor tasks. Our cortico-cerebellar model readily applies to a wider range of modalities, to demonstrate this we tested the model in a cognitive task, caption generation. Models without the cerebellar-DNI component exhibit deficits similar to those observed in cerebellar patients in both motor and cognitive tasks. Moreover, we used CC-DNI to generate a set of specific neuroscience predictions. Finally, we introduce a CC-DNI model with highly sparse connectivity as observed in the cerebellum, which substantially reduces the number of parameters while improving learning through decorrelation.
Overall, our work offers a novel perspective on the cerebellum as a brain-wide decoupling machine for efficient credit assignment and opens a new avenue of research between deep learning and neuroscience.

Under review as a conference paper at ICLR 2021
CORTICO-CEREBELLAN NETWORKS AS DECOUPLED NEURAL INTERFACES
Anonymous authors Paper under double-bling review
ABSTRACT
The brain solves the credit assignment problem remarkably well. For credit to be correctly assigned across multiple cortical areas given area should, in principle, wait for others to finish their computation. How the brain deals with this locking problem has remained unclear. Deep learning methods suffer from similar locking constraints both on the forward and backward phase. Recently, decoupled neural solution to the forward and backward locking
interfaces (DNI) were introduced problems. Here we propose that a specialised brain region, the cerebellum, helps the cerebral cortex solve the locking problem closely matching the computations and architecture of DNI. In particular, we propose that classical cerebellar forward and inverse models are equivalent to solving the backward and forward locking problems, respectively. To demonstrate the potential of this framework focus modelling a given brain area as a recurrent neural network in which the cerebellum approximates temporal feedback signals provided by BPTT. We tested the cortico-cerehellar- DNI (CC-DNI) model in a range of sensorimotor and cognitive tasks that have been shown be cerehellar-dependent CC-DNI unlocking mechanisms can facilitate learning in a simple target reaching task. Next, by building on the sequential MNIST task we demonstrate that these results generalise to more complex sensorimotor tasks. Our certic-ccerbelllee model readily applies
First, show that the
wider range of modalities, to demonstrate this we tested the model in a cognitive task, caption generation. Models without the cerebellar- DNI component exhibit deficits similar to those observed cerebellar patients in both motor and cognitive tasks. Moreover, we used CC-DNI to generate a set of specific neuroscience predictions. Finally, we introduce a CC-DNI model with highly sparse connectivity as observed in the cerebellum, which substantially reduces the number of parameters while improving learning through decorrelation. brain-wide
Overall, our work offers a novel perspective the cerebellum decoupling machine for efficient credit assignment and opens a new avenue of research between deep learning and neuroscience.
INTRODUCTION
Efficient credit assignment in the brain is a critical part of learning. However, how the brain solves the credit assignment problem remains mystery. One of the central issues of credit assignment across multiple stages of processing is the need to wait for previous stages to finish their computation before others can proceed (Rumelhart al. 1986 Schmidhuber [9900 Lee et al. 20155] Marblestone et al. 2016: Jaderberg et 2017). In deep artificial neural networks these constraints are explicit. During the forward phase given layer has to wait for all its previous layers to finish before it can proceed, a constraint known as the forward lock. Similarly, during the backward phase given layer has to wait for all the layers above to finish computing its gradients- - backward lock. Recently, a framework was introduced to decouple artificial neural networks decoupled neural interfaces (DNI; [Jadeberrg et al. 2017) effectively breaking forward and/or backward locks.
Here, we propose that specialised brain area, the cerebellum, performs similar role in the brain. In the classical view the cerebellum is key for fine motor control and learning by constructing internal models of behaviour. (Mar) 1969
Albus 1971 Raymond and Medinal 2018 Wolpert al. 1998
DNIs related earlier work using network crities to train neural networks (Schmidhuber 1990s.

---

Under review as a conference paper at ICLR 2021
Miall et al. 1993). More recently, however, the idea that the cerebellum is also involved in cognition
has gained significant traction (Schmahmann et al., [019] Wagner and Luo| 2020 Brissenden and
Somers 2019). An increasing body of behavioural, anatomical and imaging studies points to
role of the cerebellum in cognition in humans and non-human primates (Schmahmann et al.] 2019;
Brissenden and Somers 2019: Guell et al.] 2015: 2018). Impairments in cerebellar patients occur across a range of tasks including language [Guell et al., 2015). working memory (Deverett et al.
2019). planning (Baker al. 1996), and others (Fezz et al. 1992). These observations suggest that the cerebellum implements a universal function across the brain (Marr 1969; Albus 1971 Raymond and Medina 2018; Diedrichsen al., 2019). Moreover, experimental studies looking cortico- cerebellar interactions have demonstrated that cerebellar output is crucial for maintaining neocortical representations in order to drive behaviour (Chabrol et al. a [2019] Gao et al. (018). However, to the best of our knowledge, no theoretical framework has considered what might be the function of such interactions between the cerebellum and cortical areas.
In attempt to reduce the existing gap between experimental observations and existing computa- tional approaches we introduce DNI as a cortico-cerebellar model - cortico-cerebellar DNI (CC-DNI). Consistent with the cerebellar universal role we theorise that the cerebellum serves to break the locks inherent to both feedforward and feedback information processing in the brain, akin to DNI. In partic- ular, we posit that the two classical internal models of the cerebellum, forward and inverse models, are equivalent to DNI-mediated unlocking of feedback (gradients) and feedforward communication, respectively. Following this view the cerebellum not only provides motor or sensory estimates, but also any other modality encoded by particular brain region. Inspired by neuroscientific studies, we test our model on sensorimotor tasks: (i) a target reaching task (Sanes et al. 1990 Butcher et al.) 2017. Nashef et al., 2019) and (ii) a set of more complex temporal tasks based on the MNIST dataset, but also (iii) on a cognitive task caption generation (Guell al. (015). Our results support the cortico-cerebellar DNI models we study and show that they generally speed up learning by unlocking the main network, qualitatively consistent with a wide range of behavioural observations (Guell et al.
2013 Sanes et al. 1990 Butcher et al. 2017 Nashef et al., 2019).
Two defining features of the cerebellum are the large expansion at the granule cell input layer with 50 billion neurons (the most numerous cell in the brain) and the highly sparse connectivity (each granule
cell receives ~ 4 synapses) (Sanger et al., 2020). These observations have been long suggested to
help up learning in the cerebellum through decorrelation (Albus 1971 [Sanger et al. 2020 Sayco-Gaja et al. (017)) Building on these studies we introduce a new DNI model, sparse CC-DNI. al., (017) we show that input sparsity can improve learning in the presence of high correlations. We finish with a discussion on the implications and predictions of this new brain-wide model of the cerebellum.
Consistent with classical cerebellar models (Albus 1971 Cayco-Gajic
2 CEREBELLUM AS A DECOUPLING MACHINE
We first describe DNIs following Jaderberg et al. (2017) and then establish the link to cortico- cerebellar networks. Assume that a feedforward neural network consists of N layers, with the äth layer (1 S i < N) performing a "computational step" Sa with parameters 0,. Given input 2 at layer I, the output of the network at its final layer is therefore given by ff(fN-i-- use Fi denote the composition of steps from layer to layer j (inclusively). Finally, let h2 denote the (hidden) activity at layer i, so that h; = f((h,-1) with fgg = x.
(x)) We
To illustrate the locking constraints of standard artificial neural networks used in deep learning suppose that a network is in the process of learning via backpropogation, with current input-targe pair (z, ). To update the layer parameters 6, the gradient 8L required, where L L(y, Starg) 29,
is the loss which compares the target value against the model output = J. (x) under some loss öL function L; we then apply gradient descent on the parameters, 0, + 0 a 30, with learning rate a > O. Suppose however that the network has only recently received the input x and is currently only at module i of the forward computation. In order to update the corresponding parameters of that layer, 0,, the layer must first wait for all remaining layers to finish fq (j > i) for the loss to be computed. Only then the various gradients of the loss are backpropogated and aL is finally available. 88,
These two characteristics of backpropogation make layer i "backward locked" to Fl enforcing a strong dependence of the layer's learning to the speed of forward and backward propagation through
2

---

Under review as a conference paper at ICLR 2021
: Area
b
Approximating temporal feedback/gradients
Area 3
error
Inferior Olive
L
L
L
9
L++3
h2
00
9
ht
92
Area 2
activtty
M
feeaback
xt
1n
T
5 91
9,43
9
Cerebellum
944
Area 1
.... Fecdlack/gradient approvimaiitttooiittattioooooooo
9443
Input
hera
bootstraping modules
sti..
Figure 1: Cortico-cerebellar networks as decoupled neural interfaces (CC-DNI), (a) The cerebellum approximates forward (black arrows) or feedback (blue arrows) cortical processing, effectively decoupling the respective brain areas as done by DNIDaderberg et al. (2017) Here highlight the case of unlocking feedback processing. The cerebellum attempts to approximate feedback signals (ă2 {gg. }23) which can be purely spatial (g2) temporal 9 This means that area does not need to wait for the exact feedback signals. The cerebellum receives neural activity from area 2, which project (sparsely) through mossy fibers (cyan) onto granule cells (PC; orange) and Purkinje cells (PC; orange). Cerebellar learning is mediated by the inferior olive, which compares estimated feedback Φ2 with real feedback !2, computing Le l|92 او2وl (see main text). Sparse connectivity onto GCs (connections and pruned connections denoted by solid and dashed cyan lines, respectively) represents the sparseDNI model. (b) Illustration of the cerebellum approximating temporal feedback signals provided by BPTT. By approximating future feedback (c.g. at 9. and 91++3) the cerebellum reduces the need for strong BPTT. Adapted from Jaderberg et al. (2017). (c) To trained with temporal feedback the cerebellum bootstraps its own teaching signals. This is consistent with the modular structure of the cerebellum, where the each module's 's predictions are used by the cerebellum itself to speed up cerebellar learning.
the rest of the network. Similarly the the network is forward locked during the forward pass, DNI can be used to unlock both backward and forward locks Jaderberg et al. (2017).
To illustrate the model here we focus on backward DNI, which goal is break the backward lock by feeding the hidden layer i activity hę to a separate neural network, a backward synthesiser C.a. that learns to produce synthetic gradient ÿ; an estimate of the real gradient expected by layer i, (haa) = Ji ab, This synthetic gradient can then be used to update the weights of layer as soon as h2 is available, following
al
Oh, 0, t 0, acgi 00
= C.b)))
More specifically, the parameters of C.5 (h+) are learned by comparing the estimated gradient with target gradient so as to minimise Le = llø. gill- Ideally we set the target gradient as the true gradient, D. 3L 30, as can be implemented without difficulty in the case of feedforward networks. However, if we consider the temporal case (see sections 2.1.1 and 3) where we wish to estimate gradients of future losses many timesteps ahead, = =. Lt. the true backpropagated gradient is computational expensive to obtain. Jaderberg et al. (0117) counter this potential problem by applying a bootstrapping principle in which DNI module itself is used guide learning; explicitly, the target gradient at timestep is ğt = Er>t Tr the interesting resemblance to the n-step return used in reinforcement learning algorithms.
#L= + CT (hT) Th. where defines some limited horizon; note
2.1 CORTICO-CEREBELLAP DNI MODELS
Building directly DNI introduce model of cortico-cerebellar computation (CC-DNI). These models use a simple feedforward neural network, consistent with the mostly feedforward architecture of the cerebellum (Mar 1969 Albus 1971 Raymond and Medinal 2018). The input layer of the
3

---

Under review as a conference paper at ICLR 2021
cerebellum module C models the mossy fiber input onto granule cells (GCs; Fig. IT. which the most numerous neurons in the brain (> 50 billion in humans (Herculano- Houzel 2009)). Consistent with this cerebellar dimensionality expansion, in our models use M > N, where M is the number of GCs and N the number of neurons in the main (cortical) network (Fig. D. In particular, we use 2009 This is different
ratios M N. 4, consistent with experimental observations (Herculano- to DNI, in which the synthesizer uses a single hidden layer with the same number of units as LSTM (for comparison in performance see ref to Fig. S2). In addition, the hidden layers and the output of C approximate the role of GCs and Purkinje cells, and the cerebellar output nuclei, respectively.
Cerebellar granule cells receive sparse input connections (K) with only around 3-7 synapses per GC (Herculano-Fe Houzel 2009; Eccles et al.) 2013). These architectural constraints have led to sparse
encoding and decorrelation theories of cerebellar function (Albus] 1971; Cayco-Gajic et al., 2017 7, Schweighofer et al. 2001] [Broomhead and Lowe| 1988) [Blllings et a Kumar et al.
J017/Sanger et al., 2019p. Inspired on these features, we introduce a new model sparse CC-DNI
(sCC-DNI), for which we set small number of incoming input connections K 4 (Fig. D.
To measure decorrelation achieved by SCC-DNI we use the Pearson correlation and a population correlation metric Tpop that has been shown to better capture cerebellar effects (Cayco-Gajic et al.
maxx (vill
where Z are the number of neurons being considered and A; are the eigenvalues of the covariance matrix of the neuronal activity (e.g. h2r) To manipulate the hidden correlations of the model, we adjust the variance of its input and recurrent weights by scaling each by a factor with b # 1 (see SM).
2017) Pppp z-r Σ. VA.
There is strong similarity between CC-DNI models and the flow of information in standard internal models of the cortico-cerebellar networks (Fig. Table [ST). Below we draw a parallel between classical cerebellar internal models and our CC-DNI models. For simplicity, below and in our results we focus on the link between forward internal models and backward DNI (but see [ST] for our interpretation of inverse internal model as forward DNI).
2.1.1 FORWARD CEREBELLAR MODELS AS BACKWARD DNI
We propose that classical forward cerebellar models are equivalent to backward DNL. In the forward model of sensorimotor control, the cerebellum receives an efferent copy of the motor command from the motor cortex (Miall et al. [993] Ito 1970), and sensory feedback from motor centres. With these two inputs the forward model learns to predict the sensory consequences of motor commands. We argue that similar predictive model can be applied predict cerebral activity in brain regions such as the prefrontal cortex and the cemporo-parietae cortex which are involved in planning of cognitive behaviour and decision making (Schmahmann et al.) 2019; Wagner and Luo [0200] Briissendnn and Similarly to the forward model backward-DN also receives an efferent copy, which is represented by the hidden activity from given layer (or brain area) h;- From this input the synthesiser learns to predict the gradient with respect to the activity of layer h (Fig. [b.c). In addition, we suggest that the cost function is computed by the inferior olive (e.g. C= |1 GM [l; Fig. [,), which mediates learning in the cerebellum via climbing fibres, consistent with existing cerebellar theoretical frameworks (Marr 1969 Albus 1971 Raymond and Medina 2018). Finally, the prediction 9M is sent back to the brain.
Somers, 2019 Ito 2008)
Such backward DNIs are of particular importance when using recurrent neural networks (RNNs). which learn estimate future gradients from timestep t 1 given current state at time (Jaderberg et al,, 2017) [p; see SM for more details). Our tasks use RNNs with weak (i.e. with low truncations) backpropagation through time (BPTT) as it allow to more clearly demonstrate the potential of the CC-DNI models. As is the case in DNIs the cerebellum relies on a bootstrapped cost function (Fig-D-). We interpret such bootstrapping as being provided by the current other cerebellar modules, consistent with the highly modular cerebellar architecture (Apps et al. 2018). Here we use LSTMs (Hochreiter and Schmidhuber 1997) as model of cortical networks, which can be potentially linked to cortical microcircuit (Costa et al. 2017
2.2 ENCODING GRADIENTS IN THE BRAIN
Recent developments have introduced biologically plausible solutions to how the brain encodes gradients (Lillicrap and Santoro 2019 Guerguiev al. 2017 Sacramento al. 2018 Richards
4

---

Under review as a conference paper at ICLR 2021
and Lillicrap 2019 Payeur et al. 2020 Ahmad et al. 2020). Regarding spatial backpropagation
of gradients (i.e. across layers), Sacramento et al. (2018) 8) demonstrated that biological networks do not need to explicitly send gradient information, but rather that this can be reconstructed locally in dendritic microcircuits. On the other hand Bellec et al. (019) showed that temporal gradients as used in BPTT be approximated by eligibility traces that transmit information forward in time. Both of these solutions can be incorporated into our framework, in which CC-DNI would predict feedback activity originating from upstream brain areas (as Sacramento
2018) and/or eligibility traces in RNNs (see (S2 for schematie; Bellec et al. (2019). However, this is outside of the scope of the current paper and we leave this future work (see more detailed discussion in SM).
2.2 COMPARISON WITH EXISTING CEREBELLAR COMPUTATIONAL MODELS
Over the past decades the regular circuitry of the cerebellum has driven theorists model the cerebellar cortex in order to understand its function (Kaiser et al., 2018). The Marr-Albus models are particular popular models which view the local cerebellar circuitry as multi-layer perceptrons (Albus) 197I). Another modelling framework that has been used is based on Kalman filters, which are also used to model the local circuitry to generate sensorimotor predictions (Porrill et al. (013). These models have enabled significant theoretical and experimental advances. However, they do not consider potential interactions with the wider brain, in particular the neocortex, which are known to exist and appear to be critical for cerebellar function (Gao et al. 2018). Moreover, these models have so far only captured relatively simplistic (sensorimotor) tasks (e.g. eye-blink conditioning). In contrast, framework provides solution to both of these issues, while being consistent with existing local models of the cerebellum.
3 RESULTS
In this paper we focus our experiments on backward-CC-DD models (cerebellar forward model; Fig. l which we test on range of task domains that have been shown to be cerebellar-dependdnt. Below we provide summary of these different tasks and the respective results (see details in SM).
3.1
SIMPLE SENSORIMOTOR TASK: TARGET REACHING
Figure 2: Target reach- ing task. (a) Trajectory produced by LSTM (grey), CC-DNI (orange) and sCC- DNI (cyan) models to- wards targets when using BPTT = 2. (b) Models MSE. (c) Distance between model output and the tar- gets. (d) MSE for multiple truncations T over the last 5 epochs. (e) MSE and cor- relation of hidden activity (rx) epoch 1. (f) Correla- tion of CC-DNI estimated gradients (ra) over the first 100 epochs of learning.
100
LSTM
CC-DNI
SCC-DNI
0
error (MSE)
75
50
0
25
0
100
200
300
C
epoch
dst from target
10
error (MSE)
50
limestps
truncation
citer (MSE)
0.
50
0.4
20
40
60
100
epoch
Inspired by classical sensorimotor studies in the cerebellum we first test a simple target reaching task (Sanes al.) 1990; Butcher et al., 2017 Nashef al.) 2019). in which given input time t the network needs to reach a target (Fig. 2 and see SM for more details). In this task error signals are only available at the end of task, which must be associated with the initial input. In line
5

---

Under review as a conference paper at ICLR 2021
with cerebellar experiments (Sanes et al. 1990: Butcher al., 2017; Nashef et al. 2019). only the CC-DNI models learn to reach the target with good accuracy, while LSTM still learns but much more slowly (Fig. bb. Interestingly. sparse CC-DNI not only learns more quickly than CC-DNI (Fig. 2... and S3p. but also reaches the target more quickly (i.e. in fewer time steps) than the two
other models (ig.. 2b). Consistent with cerehellar theories (Sanger al., 2020 Cayco-Gajic et al.. 2017 [Bilings et al. 2014) SCC-DNI helps when high correlations are present in the main network activity h (Fig. 2b), due to decorrelation of its estimated gradients 9 (Figs. 2 see S6 for pop. corr.. CC-DNI properties are particularly important when learning with low BPTT truncations (Fig. Di), predicting that the cerebellum particularly important for more temporally challenging tasks and consistent with DNI theory (Jaderberg et al. 2017).
3.2
ADVANCED SENSORIMOTOR TASKS
b
SEGMNIST
C-SEQMNIST
SeqMNIST
C-SeQMNIST
0
60%
JiT
8
ena-1 (%9)
CC-ONE
enorr (MSE)
80
40%
SOC-ONI
60
4
6
5
20%
40
dd-segMNIST
Id seqMISST
dd ddNseTMNSST
Id-seqMNIST
MUN
9
0
30
2
error (MSE)
ervor (MSE)
20
30G
3
10
6
5
4
10
10
20
10
20
d
epoch
40%
error (98)
35%
0.16
cror
30%-
20%
08
0
25%
0.3
0.8
error (%)
10%
0%
0.6
-00%
0.2
5 6
8
9 10
truncation
epoch
Figure 3: Sequential MNIST tasks. (a) Illustration of the four variants of sequential MNIST (seqMNIST). Top left: Standard classification sequential MNIST coow-by-row.. Top right: given seqMNIST input the model is asked to reach a target coordinates (C-seqMNIST); Bottom left: given seqMNIST input the model asked to draw the digit following a template (dd-seqMNIST); Bottom right: as in dd-seQMNIST, but the model is asked to draw straight line (Id-seqMNIST), Target drawings dotted grey and model output coloured by digit. (b) Validation learning curves for the four tasks with gradient truncation T 5. For IN-Se differrent degrees of loss sparsity ? are shown (see main text). (c) Model performance across for seq.NNIST (top) and respective differences between DNI and LSTM (bottom). (d) Model performance after epochs function of the Pearson correlation Th of the hidden activity at epoch 1 in the main network (dots; top). Correlation of synthetic gradients ry produced by CC-DNIs (bottom).
Next, to test whether the target reaching results generalise to more realistic tasks we explored a range of more advanced sensorimotor tasks building on the classical MNIST dataset. We first test the standard sequential MNIST (seqMIIST), in which the model gradually receives an MNIST image row by row before classifying the image a number 0 9 (Fig. 3 (Le al. 2015). As in the target reaching task, the loss function is only defined at the end of the sequence, making this a challenging temporal task.
With truncated BPTT with modest truncation sizes < 6, CC-DNI models learn more quickly and achieve better overall performance compared to standard LSTM (Fig. Bp.c). in line with the previous task. Interestingly, for this task we observe that the cerebellar architecture (i.c. high x ratio, and sparse input connections K) to be the best solution (Figs. S4 S5 As in the target reaching task,
M

---

Under review as a conference paper at ICLR 2021
sparse CC-DNI helps in the case of high correlations in the main network activity h. likely achieved by the decorrelation effect sCC-DNI has on the estimated gradients 9 (Figs. 31, S7)
To see how well the above results generalise to a regression task in which the model has to reach particular point (as in the target reaching task), we develop a coordinate seqMNIST (N-SCQMNIST) task. As before, the model receives the image sequentially, but must now output one out of ten different equally spaced 2D coordinates (Fig. 1). Note that this is harder task as models have to learn to transform complex input pattern towards specific 2D point (Fig. (p). Both DNI models outperformed the LSTM model. There is however notably less difference between sCC-DNI and CC-DNI between compared to the standard version of seqMTIST (see also the following tasks), which may suggests that the cerebellum is better suited for pattern separation tasks, in line with existing cerehellar pattern separation theories (Albus. 1971: Sanger al. 2020 Cayco-Gajic et al. 2017
Litwin- -itwin-Kumar et al. 2017).
b
<start large white tropical bard perched on polm tree <end>>
e
3.30
3.50
loss (BPW)
3.28
3.25
3.26
LSTM
3.24
<starts
large
CC-ONI
dest
3.22
SCC-ONI
RSST
epoch
10
Feature
BLBU
Rouge-L. I.
METEOR
CIDE:
SPICE
LSTM
0.2288
0.4790
0.2114
0.6025
0.1418
CC-DNI
0.2294
0,4800
0.2112
0.6940
0.1422
<atart
sCC-DNI
0.2300
0.4802
0.2112
0.0976
0.1410
10-1
a sitting on rock a bird perched en top of a tree branch
loss
10-2
0
white bird sitting
brandd
-10-2
a large white tropical bird perched on palm tree
aloss
0.2
3
12
15
i mam and woman new standing in a Intelen
truncation
a mam and a woman are standing in front of a table wish a bunch of bausuzse a man standing next womnnn
0.01
-0.01
people in lime by table with waters, grapes bauoune
10
15
20
25
caption length
Figure 4: Caption generation task. (a) A graphical description of the model. The image is first processed by a pretrained convolutional neural network (CNN). The resulting feature vector is then provided connected but distinct RNN which is trained to predict the next word given the previous words of a provided "gold standard"' caption to the image. The cerebellum module C is only applied to the RNN. (b) Top: Learning curves in bits per word (BPW) validation set for gradient truncation T = 4. Full learning curve shown as inset. Bottom: standard language metrics (BLEU, Rouge-L, METEOR, CIDEr, SPICE). (c) Two example images from the validation set with corresponding model captions and gold standard captions (black). (d) Top: Change in loss with respect to LSTM as a function of T. Middle:
Standard deviation doss over the average loss across all epochs as a function of T. Bottom: Change in loss (i.e. CC-DNIs LSTM) as a function of caption length, where the cross designates the mean (gold standard) caption length.
Next, we test an explicit drawing task in which we trained the models to draw either a template digit (Nd-S)) or a straight line (NN-S) given an image of a digit (Fig-3p) For these tasks, the loss function is computed at every point in time as L = (y/ )2, where y/s and ÿs denote the desired and predicted coordinate at timestep i, respectively. Due to the loss being computed at every timestep in both tasks, CC-DNI models are not important and show minimal improvement in performance (Figs. 3p. S4). To demonstrate that the temporal resolution of the loss is important for CC-DNI, we varied the loss sparsity, giving the model loss every 7 timesteps for ld- INNSST This setup resembles sensorimotor feedback which is typically periodic rather than continuous (Sanes) et al. 1990 (Synofzik et al. 2008). As expected, sparser losses (higher 7) improve learning (Fig. 3b)
7

---

Under review as a conference paper at ICLR 2021
3.3 COGNITIVE TASK: CAPTION GENERATION
Our framework does not only apply to sensorimotor tasks, but should generalise to virtually any task within the grasp of deep learning systems, or indeed the brain. To demonstrate this and inspired by cognitive tasks in which cerebellar patients have shown deficits (Gebhart et al. 2002) we test our models in a caption generation task. In this task the network needs generate a textual description for a given image. All models have two components: a pretrained convolutional neural network (CNN) to extract lower dimensional representation of the image, and an LSTM on top to generate text (Fig. F.. For simplicity and to draw comparison with previous tasks the DNI models contain one synthesiser at the RNN-level, but more could be added to help learning of the CNN.
We use standard dataset (ILSVRC-2012-CIS (Russakovsky et al., 2015) and the networks are trained to maximise the likelihood of each target word given an image (SM for more details). We find that CC-DNI models exhibit faster learning (Fig. 4b) for truncation sizes T S 10 (Fig. 41 and better generalisation (Fig. S13). All models produce reasonable captions for images unseen during training, but CC-DNI models tend to produce captions more semantically accurate than LSTMs (Figs, 4 ST2 consistent with cerebellar deficits (Guell et al., 2015). In addition, we observe that sCC-DNI are more robust initial conditions (Fig. 4i), consistent with the decorrelation property. Moreover, consistent with bridging of gradient truncations provided by DNIs, we observe that CC-DNI helps learning in the presence of long captions, whereas sCC-DNI helps mostly for the most common caption lengths (Fig. Ii,, suggesting that sCC-DNI better captures the data distribution.
3.4 MODEL PREDICTIONS
Our model makes numerous predictions for ex- perimental neuroscience, some of which can be compared with existing observations. First, we tested how important are the exact cerebel- lar expansion M/N ~ and sparsity parame- ters (K = 4) observed experimentally. We find that the parameters found experimentally pro- vide good combination to facilitate learning in the pattern recognition sequential MNIST task (Fig. Saa). This region of good hyperparameters is much wider in regression tasks such as the coordinate sMNIST (Fig. 5)) or other variants (Fig, (5).. This predicts that the cerebellum has evolved specifically facilitate pattern recognition tasks.
SeqMNIST
27%
Figure 5: Cerebellar
1/2
23%
sparsity (K) and
M/N
1-
expansion (M/N)
20%
parameters in se-
10-
QMNIST (top) and
e-eqMMIST
C-seqMNIST (bot- tom). Grey, orange and cyan boxes de- note LSTM, CC-DNI, sCC-DNI models respectively.
W1
122
1-
-45
2-
20
10
30
60*
Figure 6: CC-DNI ablation results. (a) (top) Learning curve for the target reaching task with multiple total cerehellar ablations at specific epochs (vertical line). (bottom) Summary of average error normalized to the control model, for both CC-DNI (orange) and sCC-DNI (cyan). denotes LSTM case. (b) Same as (a) but for SeqMNIST.
b
100-
MSB
25%
100
200
300
25
100
normalized MSE
opach
formllized error
epoch
1.25
2-
1.00
0"
10
50
100
150
200
250
0*
5
10
15
25
50
ablation opach
ablatian epoch
A common experimental paradigm to do cerebellar ablation experiments. We used our model to generate specific predictions in terms of behavioural outcome with total cerebellar ablations at different points during learning. We find that ablations generally impair learning (Fig. Gb-b), but that this impairment reduces over learning and once the main network no longer needs the cerebellar estimations its presence can impair learning (Fig. bb).
Next, we used our model to make predictions in terms of expected correlations between the neocortex and the cerebellum. Because of the complex temporal gradients in the RNN case here we focus on a
See Czarneck et al. (017)) for discussion on the regularisation effect of synthetic gradients.
8

---

Under review as a conference paper at ICLR 2021
simple feedforward network trained MNIST. In addition, use this example to demonstrate that when approximating activity directly instead of gradients, as we propose to happen in biology, the cerebellum output converges to some non-zero value (Fig. [ top). We find that correlations between the main network and the cerebellum increase over learning for neurons with final high correlations whereas correlations decrease for neurons with initial high correlations (Fig. 7 bottom), consistent with recent experimental observations (Wagner et al., (019)). Moreover, we also observe that the variance of total correlations decreases over learning (not shown).
CONCLUSIONS AND DISCUSSION
We introduced a deep learning model of cortico- certico-cerebellar function, in which the cerebellum acts as decoupling machine.
losslactivity (a.u)
Figure 7: Correlation 7 and activity analysis of feedforward CC- DNL. (top) Training loss (transparent) and mean (absolute) activ- ity (opaque). (bottom)
FF network
1.0
cereb. network
We demonstrate that this idea is directly re- lated to classical cerebellar models (Marr
0.5
1969 Albus 1971). but with the key predic- tion that the cerebellum decouples activity propagation across the brain, speeding up credit assignment across multiple brain ar- eas. Our model is also step towards solv- ing the temporal credit assignment problem in the brain, as the CC-DNI models reduce the need for strong BPTT (i.e. without trun- cations). Temporal credit assignment directly related to the ability of networks maintain input information (Bellec et al. 2019). Therefore, our results on truncated BPTT suggest that the cere- bellum becomes increasingly important as task difficulty increases due to the inability of cortical networks to maintain a information for longer than hundreds of milliseconds (Deverett et al. 2019).
0.0
corre ation
0.8
cereb. nuclei FF hid. careb. GCsPCs vs FF hid.
Evolution of the most correlated pairs (256) at beginning (transpar- ent) and end (opaque) of learning.
0.6
0.4
200 400
600 800
batch number
Our results are largely consistent with observed cognitive deficits in cerebellar patients, such as in language (Guell et al. 2015), which cannot be attributed directly to deficits in motor control (Stoodley and Schmahmann] 2009). Because of the explicit use of labels (or desired targets) here we have relied on supervised learming setting=] but the same framework can be easily applied in reinforcement and other unsupervised learning settings (Jaderberg et al., 2017). which do not require an explicit teacher. Indeed the cerebellar model we propose here is of particular relevance for reinforcement learning due to the prevalence of sparse and delayed rewards, consistent with recent observations of reward-related signals in the cerebellum (Sendhilnathan et al. 2020).
[
Our work makes numerous predictions and opens the exciting possibility of comparing CC-DNI
models with recent studies across range of tasks (King et al., 2019 Ashida al., 2019). We performed three separate experiments to highlight predictions made by the model: (i) we used
ablations to show how behavioural performance is shaped by cerebellar predictions, (ii) a study on
cerebellar expansion and sparsity that suggests that the cerebellum may have evolved to facilitate learning in specific tasks and (iii) study how correlations between main network and the cerebellum develop over learning (see Section 5). Moreover, the model also predicts which connections should project to GCs (source brain area activity) and the inferior olive (target brain area activity). In addition, as demonstrated by |Carneeki et al. (2017) the CC-DNI model also predicts that without a cerebellum the neocortical representations should become less distributed across different brain areas. Furthermore, shown by Jaderberg et al. (201)) these models enable network to be trained in fully decoupled fashion, which means that it can update asynchronously. Given that the brain is asynchronous this may be fundamental benefit of having a cerebellar system.
Finally, our framework can also inspire new deep learning DNI systems. For example, a key feature of cerebellar architectures are the many functionally separate modules (Apps et al. 20 Cerminaral and Apps| (201T); Suvrathan et al. (2016); Fig. Ic) to speed-up learning, generalising the ideas used by Jaderberg et al. (207))
Note, however, that learning of language models (as in caption generation) is often done in unsupervised (self-supervised) setting.
9

---

Under review as a conference paper at ICLR 2021
REFERENCES
David E Rumelhart, Geoffrey Hinton, and Ronald Williams, Learning representations by back- propagating errors. Nature, 323(6088):333-536 1986.
Jürgen Schmidhuber. Networks adjusting networks. In Proceedings of Distributed Adaptive Neural Information Processing" pages 197-208, 1990.
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Joint european conference machine learning and knowledge discovery in databases, pages 498 515, Springer, 2015.
Adam H Marblestone, Greg Wayne, and Konrad P Kording. Toward an integration of deep learning and neuroscience. Frontiers computational neuroscience, 10:94, 2016.
Max Jaderberg, Wojeiech Marian Czamecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu, Decoupled neural interfaces using synthetic gradients. In Pro- ceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1627-1635. JMLR. org, 2017.
David Marr. A theory of cerchellar cortex. The Journal of Physiology, 202()):437-470, jun 1969. ISSN 00223751. doi: 10.11 http:phyyysoo.119.00000000831 URL http::/ http::////oi..wle........
1113/jphysiol 1969. 015. sp008920
James S Albus. A theory of cerehellar function. Mathematical Biosciences, 10(1):25-61, 1971.
ISSN 0025-5564. doi: http:///aaaoo///...00/0004554411
URI. http://wwa
sciencedi com/ htt.a//sees............///.e/otr
Jennifer I. Raymond and Javier F Medina. Computational principles of supervised learning in the
cerehellum. Annual review neuroscience, 41:233-253, 2018.
Daniel M Wolpert, R Chris Miall, and Mitsuo Kawato. Internal models in the cerebellum. Trends in cognitive sciences, 2(9):338-347, 1998.
R. C. Miall, J. Weir, D. M. Wolpert, and Stein. Is the cerebellum smith predictor? Journal of Motor Behavior, 25(3):203-22) 1993. ISSN 19401027. doi: 10. 10.1000000222895.11 1993 99429000
Jeremy D. Schmahmann, Xavier Guell, Catherine J. Stoodley, and Mark A. Halko. The Theory and Neuroscience Cerchellar Cognition. Annual Review of Neumscience, 42(1):337-364, jul 2019. ISSN 0147-006X. doi: 10. 146/ammurcy (000m/mmuryymmmmmm070001-1---0-0 Camualeevin
URL httt.http:://// www
www./o//oo..... do 1 wttp://annuee----oo-000000001000 05010
Mark J. Wagner and Liqun Luo. Neocortex Cerebellum Circuits for Cognitive Processing. Trends in Neurosciences, 422554 2020. ISSN 1878 108X. doi: 10.101 11..1//.iins.209.11....... URI.
https://6o: / 1 0. 101 wtt.sii.inss.... tins. 2013 9. .002
James A. Brissenden and David C. Somers. Cortico cerebellar networks for visual attention and
working memory, Current Opinion in Psychology, 29:239 247, 2019. ISSN 2352250X. doi: copsyc. 2019.05
(0016f.copxyc..019..5.. URI. mtttpa://dol org 01 htt.://ooo.....c.......o.. 00.3
Xavier Guell, Franziska Hoche, and Jeremy D. Schmahmann. Metalinguistic Deficits in Patients with Cerebellar Dysfunction: Empirical Support for the Dysmetria of Thought Theory. Cerehellum, 14 (1):50-58, 2015. ISSN 14734230. doi: 10.
Xavier Guell, John D.E. Gabrieli, and Jeremy D. Schmahmann. Triple representation of language, working memory, social and emotion processing in the cerebellum: convergent evidence from task and sced-based resting-state IMRI analyses in single large cohort. Neurolmage, 172:437-449, may 2018, ISSN 10959572. doi: w..10166j.neuroimaa 20.280018..00
Ben Deverett, Mikhail Kislin, David W Tank, S Samuel, and H Wang. Cerebellar disruption impairs working memory during evidence accumulation. bioRxiv, page 521849, 2019.
S C Baker, R D Rogers, Adrian M Owen, C D Frith, Raymond Dolan, R S J Frackowiak, and Trevor W Robbins. Neural systems engaged by planning: a PET study of the Tower of London task. Neuropsychologia, 34(6):515-52 526, 1996.
Julie A Fiez, Steven E Petersen, Marshall K Cheney, and Marcus E Raichle. Impaired non-motor learning and error detection associated with cerchellar damage: A single case study. Brain, 115(1): 155-178, 1992.
10

---

Under review as a conference paper at ICLR 2021
Jörn Diedrichsen, Maedbh King, Carlos Heenandez-Castillo, Marty Sereno, and Richard B. Ivry. Universal Transform or Multiple Functionality? Understanding the Contribution of the Human Corebellum across Task Domains. Neuron, 102(5):918 928, 2019. ISSN 10974199, doi: 10.1016/j.
nouron.2019.04
Francois P Chabrol, Antonin Blot, and Thomas D Mrsic-Flogel. Cerebellar contribution to preparatory activity in motor neocortex. Neuron, 103(3):506-519, 2019.
Zhenyu Gao, Courtney Davis, Alysc M Thomas, Michacl N Economo, Amada M Abrego, Karel Svoboda, Chris I De Zeeuw, and Nuo Li. cortico-cerebellar loop motor planning, Nature, 563 (7729):133-116 2018.
Jerome Sanes, Bozhidar Dimitrov, and Mark Hallett. Motor learning in patients with cerchellar dysfunction. Brain, 113()):03--120 1990.
Peter A. Butcher, Richard B. Ivry, Sheng Han Kuo, David Rydz, John W. Krakauer, and Jordan A. Taylor. The cerebellum does more than sensory prediction crror-based learning in sensorimotor adaptation tasks. Journal of Neurophysiolooy, 112(3):1222-1666 2017. ISSN 15221598. doi: 10.1 10.1152/j..00451.2011
Abdulrahcem Nashef, Oren Cohen, Ran Harel, Zvi Israel, and Yifat Prut. Reversible Block of Cerchellar Outflow Reveals Cortical Circuitry for Motor Coordination. Cell Reports, 2769):2608- 2619.e4, 2019. ISSN 22111247 doi: 10. (0.1016f..eerrep201000 2019.04.. [RLhhttp:///do org/ 13. 10.10167] www.:p/..cerr...ppm 2019.
2019. 09.19.00..00
Terence D. Sanger, Okito Yamashita, and Mitsuo Kawato. Expansion coding and computation in the cerebellum: 50 years after the Marr-Albus codon theory. Journal of Physiology. 598(5):913-928,
mar 2020. ISSN 14697793. doi: 13/1113302278755
N Alex Cayco-Gajic, Claudia Clopath, and R Angus Silver. Sparse synaptic connectivity is required for decorrelation and pattern separation in feedforward networks. Nature communications, 8(1): 1-11, 2017.
Suzana Herculano-Houzel. The human brain in numbers: linearly scaled-up primate brain. Frontiers
Human Neuroscience, 3:31, 2009. ISSN 1662-516 5161. doi: 00.3389//euro.00.000 1..0.0....00000....00...0 wttggarttgeeeeeteeooooree 3389/ 3tt.
URL
https://www. www.:
C Eccles, M Ito, and Szentagothai. The Cerebellum as a Neuronal Machine. Springer Berlin Heidelberg, 2013. ISBN 9783662131473, URL htLhtts::// http:///oooosooooo.ooo www.ge..ooooooooooooooo co. wt..koookkkkk..s
id=OmLoCAAAQBAJ
N Schweighofer. K Doya, and F Lay. Unsupervised learning of granule cell sparse codes hances cerebellar adaptive control. Newroscience, 103(1):35 50, 2001. ISSN 0306-4522. doi: http:///w/....000000000000000000 8C1 Bccismne//rrttccc////////33533
com/
David S Broomhcad and David Lowe. Radial basis functions, multi-variable functional interpolation and adaptive networks. Technical report, Royal Signals and Radar Establishment Malvemn (United Kingdom), 1988.
Guy Billings, Eugenio Piasini, Andrea L Horincz, Zoltan Nusser, and Angus Silver. Network structure within cerchellar imput layer enables lossless sparse encoding. Neuron, 83(4):9609974, 2014.
Ashok Litwin-Kuman Kameron Decker Harris, Richard Axel, Haim Sompolinsky, and LF Abbott. Optimal degrees of synaptic connectivity. Neuron, 13(5):1153-116 2017.
Terence D Sanger, Okito Yamashita, and Mitsuo Kawato. Expansion coding and computation in the cerehellum: 50 years after the Marr-Albus codon theory. The Journal of Physiology, 2019.
M Ito. Neurophysiological aspects of the cerebellar motor control system. Internatiional journal of neu- vology, 7(2): 176, 1970. ISSN 0020-7446. URL http::///eorrppppccooo//aaaaaaae MED/ MED/5499516
Masao Ito. Control of mental activities by internal models in the cerebellum. Nature Reviews
Neuroscience, 344 apr 2008. ISSN 1471-003X. doi: 10. 10.100//mm333. URL www
nature. wwwprre.ccmrreeo http://wwwwww/e/eeuuueeeeee http:///wweeeeee////rrrrrtttt///
nature. com r wwt.iiceee//nn33...
11

---

Under review as a conference paper at ICLR 2021
Richard Apps, Richard Hawkes. Sho Aoki, Fredrik Bengtsson, Amanda M. Brown, Gang Chen, Timothy J. Ebner, Philippe Isope, Henrik Jörntell, Elizabeth Lackey, Charlotte Lawrenson, Bridget Lumb, Martijn Schonewille, Roy V. Sillitoe, Ludovic Spacth, Izumi Sugihara, Antoinc Valera, Jan Voogd, Douglas R. Wylie, and Tom J. H. Ruigrok. Cerebellar Modules and Their Role Operational Cerehellar Processing Units: A Consensus paper. The Cerebellum, 17(5)6683-684 2018. ISSN 1473-4222. doi: 1..1007/12311-018-09999
Sepp Hochreiter and Jürgen Schmidhuher. Long short-term memory. Neural computation, 9(8):
1785, 1997.
Rui Ponte Costa, Ioannis Alexandros Assael, Brendan Shillingford, Nando de Freitas, and Tim Vogels. Cortical microcireuits gated-rreurrent neural networks. In Advances in neural information processing systems, pages 272 272,283,22 2017.
Timothy P Lillicrap and Adam Santoro. Backpropagation through time and the brain. Current opinion in neurobiology, 55:82 89, 2019.
Jordan Guerguiev, Timothy P Lillierap, and Blake A Richards. Towards deep learning with segregated dendrites. Eliife, 6:c22901, 2017.
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcireuits approximate the backpropagation algorithm. In Advances in Neural Information Processing Systems, pages 8721-8732, 2018.
Blake A Richards and Timothy P Lillicrap. Dendritic solutions to the credit assignment problem. Current opinion neurobiology, 54:28-36 36,8866,2222 2019.
Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake Richards, and Richard Naud. Burst- dependent synaptic plasticity can coordinate learning in hierarchical circuits. bioRxiv. 2020. doi: 10.1 10..1100/0000003330000 30.0055501.0.001111111.11 URL |RLtttts:
org wwt.:/oooroooooeee//t..yy...
2020/03/31/20200 03. 015511
Nasir Ahmad, Marcel A van Gerven, and Luca Ambrogioni. GAIT-prop: A biologically plausible learning rule derived from backpropagation of error. arXiv preprint arXiv:2006.0 2644088080 2020.
Guillaume Bellec, Franz Scherr, Flias Hajek, Darjan Salaj. Robert Legenstein, and Wolfgang Maass. Biologically inspired alternatives backpropagation through time for learning in recurrent neural nets. arXiv preprint arXiv: /tt1::990..000. 0090999, 2019.
Jacques Kaiser, Hesham Mostafa, and Emre Neftci. Synaptic plasticity dynamics for deep continuous local learning. arXiv preprint 1811:811. 2018
John Porrill, Paul Dean, and Sean Anderson. Adaptive filters and internal models: Multilevel de- scription of cerebellar function. Neural Networks, 47:134 149, 2013. ISSN 0893-6080, doi: URL http::/www.scennceeee.
Quoc V Le, Navdeep Jaitly, and Geoffrey E Hinton. A simple way to initialize recurrent networks of rectified linear units. arXiv preprint ht0l::5540000000 2015.
Matthis Synofzik, Axel Lindner, and Peter Thier. The Cerebellum Updates Predictions about the Visual Consequences of One's Behavior. Current Biology, 18(1 18(11):814-818, 2008. ISSN 09609822.
doi: 10. w0w666.cub.2008.000 00.007..
Andrea I. Gebhart, Steven Petersen, and W Thomas Thach. Role of the posterolateral cerehellum in language. Annals of the New York Academy of Sciences, 978(1)3318-3)) 2002.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma. Zhiheng Huang. Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), 115 (3):211-2 252, 2015. doi: 10.10/ 120007051263--1-0811166
Wojciech Marian Czameecki Grzegorz Swirszez, Max Jaderberg. Simon Osindero, Oriol Vinyals, and Koray Kavukcuoglu. Understanding synthetic gradients and decoupled neural interfaces. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 904-912. J.L.R. org, 2017.
12

---

Under review as a conference paper at ICLR 2021
Mark J. Wagner, Tony Hyun Kim, Jonathan Kadmon, Nghia D. Nguyen, Surya Ganguli, Mark J. Schnitzer, and Liqun Luo. Shared Certe-Cerreblllm Dynamies the Execution and Learning of a Motor Task. Cell, 177(3)669 682.c24, 2019, ISSN 10974172. doi: 10.1016fj.cel122119.o www..clll......... 2019 9. 02. 019
Catherine Stoodley and Jeremy D. Schmahmann. Functional topography in human cercbellum: meta-analysis of neuroimaging studics. Neurolmage,
2009. ISSN 105381 19. doi: 10.
2008.00 08.0398 htL
http:
doi www.:ooo..oooo....ooo wttp/o..//101..... 1016 neuroimage
2008. 08.0.... 039
Naveen Sendhilnathan, Anna E. Ipata, and Michael E. Goldberg. Neural correlates of reinforcement learning in mid-lateral cerehellum. Neuron, 2020.
Maedbh King, Carlos R Hernandez- Castillo, Russell A Poldrack, Richard B Ivry, and Jöm Diedrichsen. Functional boundaries in the human cerebellum revealed by multi-domain task battery. Nature August 2019.
Neuroscience, 22( (2(8):1371111) 1371-1378,
Reiko Ashida, Nadia I. Cerminara, Richard Edwards, Richard Apps, and Jonathan C. W Brooks. Sensorimotor, language, and working memory representation within the human cerchellum. Human Brain Mapping, 40(16))773244777, July 2019.
Nadia I.. Cerminara and Richard Apps. Behavioural significance of cerehellar modules. Cerebellum, 10(3):844494, 2011. ISSN 14734222. doi: 10. 10. 1-010-0209-2.
Aparna Suvrathan, Hannah L. Payne, and Jennifer L. Raymond. Timing Rules for Synaptic Plasticity Matched to Behavioral Function. Newron, 9255)995--967, 2016. ISSN 10974199. doi: 10.1016/j. neuron.2016 10.022. http::cpss://..oro////////
neuron 2016. 10. 12.022
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1412::1412.6980, 2014.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pages 1026-1034, 2015.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 770-778, 2016.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. BLEU: method for automatic evaluation machine translation. Proceedings of 40th annual meeting on association for computational linguistics, pages 31 311-318. Association for Computational Linguistics, 2002.
Chin-Yew Lin. (ROUGE): Package for Automatic Evaluation of Summaries. In Text Sunmarization Branches Out, pages 74 81, Barcelona. Spain, jul 2004, Association for Computational Linguistics. htRL:ttp::/ www aclweb. org/ ho
Michael Denkowski and Alon Lavie. Meteor universal: Language specific translation evaluation for any target language. In Proceedingy of the ninth workshop on statistical machine translation, pages 376 380, 2014.
Ramakrishna Vedantam, C Lawrence Zitick, and Devi Parikh. Cider: Consensus-bass image description evaluation. In Proceedings the IEEE conference computer vision and pattern recognition, pages 4566-4575, 2015.
Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. Spice: Semantic propositional image caption evaluation. In European Conference on Computer Vision, pages 382-398 Springer, 2016.
Tom J.H. Ruigrok. Ins and outs cerebellar modules. Cerebellam, 10(3):464 474, 2011. ISSN 14734222. doi: 10.1007/s1231 -10-0014-.
Robert A Barton and Chris Venditti. Rapid evolution of the cerebellum humans and other great apes. Current Biology, 24(20)24410-22 2014.
Suzana Herculano-
Coordinated scaling cortical and cerebellar numbers neurons. Frontiers in Neuroanatomy, 4(MARCH):1 2010. ISSN 16625129, doi: 10.3389/fmana.0111 00012.
13

---

Under review as a conference paper at ICLR 2021
A EXPERIMENTAL DETAILS
In each of our tasks we use a long-short term memory network (LSTM; (Hochreiter and Schmidhuber 1997)) the main "cortical" network (Costa et al. 2017 and a simple feedforward network of one (sensorimotor tasks) or two hidden layers (caption generation) as the synthesiser "cerebellar" network. As in (Jaderberg et al. 2017) the cerebellar network first predicts gradients for both the memory cell and output state of the LSTM (i.e. for each neuron in the main network the cerebellar network receives two inputs), so that the "cerebellar' input and output size is two times the number of LSTM units, and this synthetic gradient 9 is then scaled by a factor of 0.1 before being used by the main model for stability purposes. The final 'readout' of the model is a (trained) linear sum of the LSTM output states. All networks are optimised using ADAM (Kingma and Ba 2014) (see learning rates below).
In each experiment all initial LSTM parameters are drawn from the uniform distribution U( VALSTM /OLSTM) ), where 7LSTM is the number of LSTM units. Other than the final layer, as in (Jaderberg al. 2017). the feedforward weights of the cerebellar network are initialised according to Zd( bk. by) where by, denotes the "kaiming bound" as computed in
Tiny /ilug where Tinp denotes the input size of the layer. As in (Jaderberg et al. 2017). the last layer (both weights and bias) of the cerebellar network is pero-mnitalised, so that the produced synthetic gradients at the start are zero.
(He et al. 2015) (slope = v5), and the biases are draw from L 1
A range of correlations (here quantified as Pearson or population correlation coefficient) for the LSTM activity h can be achieved naturally through different random initialisations (accessed through different random seeds), but can be also controlled by the scaling of initial parameters of the LSTM with bias factor b, where > and < 1 induces greater and smaller variability in parameter values respectively. The results demonstrating the effect of correlation (Figs g. Bit) are obtained across 10 random seeds, for multiple bias factor values b E 0,,
During learning, backpropogation through time (BPTT) takes place strictly within distinct truncations, and computed gradients are not propagated between them: truncated BPTT. As soon as the forward computation of a given truncation completed, assuming a corre- sponding (synthetic) loss gradient is available, the model parameters are updated; that is, at a resolution of the preset truncation size, there is online learning. We split the original input into truncations as follows. Given an input sequence of N timesteps T1; T2; T2: ZN and a truncation size T. we divide the sequence into T sized truncations with any re- mainder going to the last truncation. In other words, the sequence is now made up trun- cations of (I1,. FT), (I741,.... ..r)...., (I(m- 1)T++>......... FFTT) ), (IN-r,.... IN) T. Note that, along with the
where N = mT + r for positive integers m,r with 0 value T. how well the sequence is divided into truncations (i.e. values m, r) is an important
factor for learning.
Unless stated otherwise, reported error/loss values are on held-out validation set at the end of learning. All experiments were conducted using PyTorch.
A.I TARGET REACHING TASK
In the target reaching task, an LSTM network receives a discrete input cue which signals the network to move to particular target location in 2D space. In the results presented in the main text (Fig. 2} we set 6 distinet non-zero input-target pairs {xx,, 3)) where each input x; is a (one dimensional) integer E {+1, +2, 13}, and the targets [ lie equidistantly on a circle centred on the origin with radius 10. Once an input cue is received at timestep to. the model receives no new information (i.e. all future input is zero) and has 10 timesteps move the corresponding target location. The model is trained to minimise the mean squared error (MSE) between its final output and the cue-based target. In addition, if no input cue is provided (x = 0), the network is trained not move at all (y = 0). Note therefore that only the final model output 'matters' in the sense that it defines model performance.
The cortical network has one hidden layer of 10 LSTM units (i.c. 20 when including the memory cells); unless stated otherwise, the cerebellar network contains one hidden layer

---

Under review as a conference paper at ICLR 2021
of 80 hidden neurons. The initial learning rate is set to 0.001. Each epoch comprises one batch of 50 randomised examples. Model performance is averaged over 10 random seeds (with error bars), where each seed determines the initial weights of the network. Truncation values 7 E 32 4, 5, 7] are considered.
Furthermore, to see how the above conditions generalise we consider variants of the task with 1. different numbers of cue/target pairs (see Fig. [58p and b) and 2. higher dimensional imput (see Fig. [S8 and d). More specifically, we consider two cases where the model receives wwo-dimensional (Fig. (S and 28-dimensional (Fig. (SHi) imput. For the former the model architecture remains the same except now the model receives at time to the (two-dimensional) target values scaled by factor 0.1 as input. For the latter we have an LSTM with 30 hidden units and the input is now binary vector of size 28, In order to increase the task difficulty for these variants of higher dimensional input, we add to the input Gaussian noise with variance 0.1 each timestep.
A.2 SEQUENTIAL MNIST TASKS
For each Me based task the model receives the same temporal MNIST input, and the tasks are only differentiated by the model output. Given 28 x 28 MNIST image, at timestep the model receives the pixels from row i of the image, so that there are 28 total timesteps and an input size of 28. Only the final model output matters for the SeqMNIST and c-seqMNIST tasks that is, the loss is solely defined at the end of the sequence whereas the model output is assessed at every timestep for MNIST and Id-seqMMIIST.
In each case we have one hidden layer of 30 LSTM units in the main model and one hidden layer of 300 (unless stated otherwise) hidden units in the feedforward cerebellar network. Data was presented in batches of 50 with an initial learning rate of 0.0001.
Training and validation data was assigned a 4 1 split, containing 48000 and 12000 distinct image/number pairs respectively. The truncation values T E [2, 3, 4, 5, 7, 10] are considered. Model performance averaged (with error bars) over 3 random seeds for weight initialisation.
A.2. SEQMNIST
This is the standard form of seqMNIST, where at the end of the presentation of the image the model must classify in the image is vector probabilities of size 10 (one entry for each number), and the model was trained to maximise the likelihood of the correct number.
number between and The output of the model
A.2.2 C-SFOMNIST
In this variant each number 0-9 MNIST image is allocated an ay position on the edge of circle centred at O, with the position uniquely determined by the digit represented by the image (Fig. 3)). With the model output then a vector of size 2, the training loss is defined at the end by the mean squared error (MSE) between the final output of the model and the target coordinate; since the radius of the circle is length 10 (arbitrary unit), a static model, where the model output remains at O, would have a constant MSE of 100.
A.2.3 DD-SEQMNIST
Like c-SSTMNIST, in this variant the model outputs 2D coordinates, but now the model must learn predict an entire sequence of coordinates ( 1- The target sequence { can be considered drawing, and in this case resembles the shape of the number itself (Fig. 3 number 0 not shown). The model is then trained at each timestep to minimise the MSE between the model output at that time and corresponding target point, so that the loss at timestep defined as MSE(y, (t).
1.28
For each number, the corresponding target drawing lies in [0, 1]2, with the gap between each successive point roughly the same. To prevent the model being too harshly judged at timestep 1, all drawings begin in the top left corner (0, 1) (apart from the drawing of 1 which begins slightly beneath/to the right). MSE scores are reported 100 times their raw values to ease comparison with C-SEQMNIST&&--SEONNNNN
A.2.4 LD-SEQMNIST
This variant can be thought of as a mixture between c-seqMNIST and MNNSTT. Like dN-Se the model is constantly assessed and must produce desired shape, but now
2

---

Under review as a conference paper at ICLR 2021
the desired set of points form an equally spaced line where the start point is the origin and the end point is determined by the number-to-coording mapping of c-seq As with dd-se the loss is defined at each timestep by the difference between the model output and appropriate point on the line.
A.3 CAPTION GENERATION
The architecture for the caption generation task consists of a pretrained CNN coupled with an RNN (LSTM in this case). The synthesiser (cerebellar network) only communicates to the LSTM. The LSTM network has one layer of 256 LSTM units and the cerebellar network has two hidden layers of 1024 neurons.
The dynamics from image to caption is as follows. As part of image preprocessing and data augmentation, given image is randomly cropped to size 224 x 224, flipped horizontally with even chance, and appropriately normalised to be given to a pretrained resnet model (He) ct al. 2016). A feature vector X of size 256 is thus obtained and is passed to the LSTM at timestep O. The LSTM is subsequently presented the "gold standard" caption { ww; 1. word per timestep, each time learning predict the next word; i.e., at timestep the model learns P(wy) X, { w; }==)) The network simultaneously learns a word embedding so that each word w; is first transformed to feature vector of size 256 before being served input. With a preset vocabulary of 9956 distinct words, the final output of the model (P(w)) is probability vector of size 9956.
one
We find that though DNI could help without any explicit method for regularisation (not shown), all models training) on the input to the LSTM, where given input element is set to zero with 0.5 probability.
prone overfitting. For this reason, we apply dropout (during
Once training is complete the models can generate their own unique captions to previously unseen images (Figs. 4 ST2). Given image at timestep O., the model applies a 'greedy search' where the output of the model at timestep is the word with the highest probability, and the same word is then provided input the model at timestep + 1. In this way the model eventually outputs an entire sequence of words which forms a caption. In the (highly) rare case where the model generates sequence of > 20 words, consider only the first 20 words as its caption.
The coco training set ILSVRC-2012-CO image-caption pairs with 82783 unique images while the held-out validation set holds 202654 with 40504 unique images; note that each image therefore has 5 distinet gold standard captions. Training takes place in batches of 100 image/caption pairs, with learning rate of 0.001. Model performance averaged (with error bars) over 5 random seeds for weight initialisation.
(Russakovsky al., 2013) holds 414113 total
In order to judge the models beyond their learning curves in BPW, quantify their ability to generate captions using variety of language modelling metrics popular in the realm of language evaluation (image captioning, machine translation, etc). In particular, we
compare model-generated captions against the gold standard captions using the metrics BLEU, Rouge-L, METEOR, CIDEr, SPICE (Papineni al., 2002 Lin 2004 Denkowski and Lavie, 2014; Vedantam et al. 2015 Anderson et al.] 2016). We use BLEU_4 (i.c. consider only n- n-grams for n < 4) as the default BLEU score (see Fig ttoughh we also look at BLEU_1, BLEU_ 2, BLEU_3 (Fig S15
B FORWARD DNI: MNIST
We performed simple implementation of forward DNI - where now the cerebellar network is trained to predict the activity (not gradients) of the main network - on the MNIST database. With no temporal dimension, the main network is feedforward neural network which, given a 28 x 28 MNIST image, will tune its output (a vector of size 10, where each value corresponds
possible digit) as maximise the probability of the associated number (or equally minimise the cross-entropy loss Fig. 7 top).
In our case the feedforward network has two hidden layers, each of size 256; after each hidden layer a batch normalisation is applied as well as the non- non-linear ReLU function. We apply feedforward cerebellar network with one hidden layer of size 256 which receives
3

---

Under review as a conference paper at ICLR 2021
the original MNIST imput and is trained to predict the activity of the second hidden layer of the main network by; the synthesiser loss used to train the synthesiser parameters then becomes Le | C(=)]I- The cerebellar module then break the "forward lock" by allowing the main model to bypass its forward computations up until its second layer and instead use C(x) make its prediction.
As before, each network is trained using gradient descent with ADAM. Correlations reported are computed using the Pearson correlation. For ease of display reported losses, activity and correlations Fig. aare smoothed over batches using a Savizkk--oolay filter (window size 19, polynomial order 3). Since the output weights of the cerebellar network can be positive or negative, we do not distinguish between positive or anti correlation (i.e. show the absolute correlation value) between the hidden activity of the cerebellar network and its respective target hidden activity of the feedforward network.
COMPARISON WITH OTHER MODELS OF THE CEREBELLUM
section of the main text we argue that there is a strong similarity between the flow of information in DNI and classical internal models of the cerebellum. In Table we compare explicitly the input and output components of both types of models.
Forward Model
Backward DNI
Inverse Model Cerebellum h de funk (hg, d,)
Forward DNI
Controller
Neocortex
Main model
Synthesiser
Input
hm
hm
h Cº c²(h,)
Output
funk (hm)) Neocortex
(hm)) Main model
5L
Th.
Ouput destination
Controlled object
Main model
Table S1: Internal models versus DNI. The propertics of the forward model of the cerebellum can be set against those of backward DNI (blue); similarly, the properties of the inverse model of the cerebellum can be set against those of forward DNI (red). Notation is largely consistent with section 2 of the main text: h h, denotes the hidden activity of motor area and sensory area, respectively; C C denotes the computation of backward DNI and forward DNI, respectively; L denotes the loss function. In addition, the inverse model of the cerebellum traditionally also has access desired state d, (in particular, one can consider this special case of "context" provided to the synthesiser; ef (Jaderberg al., (2077). There are no explicit equations for the computational processes of the forward and inverse models, and both are thus represented by the unknown function funk.
C.O.1 INVERSE AND FORWARD DNI CEREBELLAR MODELS
The inverse model of the cerebellum motor control computes the motor predictions given target (or desired) response (Ito 1970). Feedback errors derived by comparing motor command from the motor cortex with the predicted motor command in the cerebellum. We propose that the inverse model is equivalent to forward DNI, as it aims predict the incoming activity at a given layer (e.g. motor command, has, or associative input, ha)) given the input activity observed at lower layer (e.g. sensory target response, hg; Fig. II. Consistent with the modular anatomy of the cerebellum (Apps et al., 2018 Cerminara and Apps 2011 Ruigrok (01T) multiple CC-DNI (backward and/or forward) modules can co-exist estimating different pathways components.
RECURRENT DNI
In section of the main text we formally describe the dynamics of DNI in a feedforward setting. Here we show how these dynamics easily generalise to the time dependent, recurrent setting. We now take the main model as a recurrent neural network (RNN), where the (recurrent) hidden layer is again connected to a distinct, feedforward synthesiser. By unrolling the network over several timesteps we can keep much of the notation as introduced in section 2, but now h, denotes the hidden (cortical) activity at timestep i (as opposed to layer i) with unique recurrent weight (= 0,Vi); furthermore, the loss L is now a sum of individual losses over the timesteps: L = E, L.-
4

---

Under review as a conference paper at ICLR 2021
Inverse cerebellum forward DNI
Inferior Olive
motor domain
Cognitive areas
Olive
non-motor
domain
Controlled
objects
error
error
RAT
SEAE
contitionn
Associative areas
A
Motar
hM
areas
M
Torper
its
his
9.A
hs?
Sensory
Cerebellum
Sensory areas
Comborba
Cerebellum
> unlocking
Figure S1: Equivalent to Fig. D for how cerebellar inverse models map onto forward DNI (forward- CC-DNI). (a) In the inverse model the cerebellum generates motor predictions (har) given desired target responses (h This allows the brain to break forward locks (dashed black arrows), for example to go directly from sensory areas to higher areas. The inferior olive plays similar role to (a.b) but here it allows the cerebellum to learn estimations of forward activities (e.g. C |/s harl)) (b) Example of forward CC-DNI model in a non-motor domain. The cerebellum computes an associative prediction hA given sensory input hg. Multiple backward and forward DNl-cerebellum modules can co-exist working in parallel. h and g variables represent forward and gradient information as in DNI (see main text).
DNI is strictly applied to RNNs which learn via truncated BPTT, where BPTT takes place in equally-spaced truncations time that any hidden activity will only be attributed to future losses within the truncation window (i.e.
0hett
> T), effectively ignoring
Sh.
a
Backward synthetic temporal gradient
b
Backward synthetic gradient eligibiliy traces and sparse error signals
L
L
Li+2
d
ò
L
L R₁
(Lu+,, Rets)
8
det3
el
C++11
Lt
Et
c
d
8643
8
16443
8,+3l
11
5,43l
8,43
C
1
heaz
heaz
Figure S2: Synthetic gradients in RNNs with eligibility traces. (a) An example of backpropagation through time with truncation every T=3 timesteps (dashed black arrows). A DNI can be used to bridge the gap between truncations allowing gradients to be sent further back in time (6, and 6,4 Adapted from Jaderberg al. (2017). (b) Schematic of biologically plausible RNN with eligibility traces that encode information needed to compute the gradients at a later point in time (green traces; cf. (Bellec et al. 2019)). The cerebellar-DN model C can be used to estimate gradients å derived from general losses reward signals to update synaptic weights at given point time. Note that teaching signals/rewards are often sparse. (c) Schematic of information used by synthesiser to learn in BPTT setting (6.43 represents the future gradients).
(d) Schematic information used by synthesiser learn to predict future losses ő from the current activity h. For biological simplicity assume that the synthesiser has access to the learning signal directly, but in practice its likely to function of T..
5

---

Under review as a conference paper at ICLR 2021
future losses and potentially disregarding important temporal correlations. The key idea behind recurrent DNI is to bridge these truncations using synthetic gradients (Fig. 1)2 For simplicity assume that timestep is the start of given truncation. By enabling DNI, (backward) synthesiser C is used provide an estimated gradient 8 for all future losses, outside of the truncation, with respect to the last hidden activity of the truncation h4TT; i.e.
Bh: The weight update is then analogous to the feedforward 84TT 8,-T= 2-=i+T+ Therr"
case in equation 1
++ JL,
ah
0 0 - a
+ 8
814 = C(h+TT)
(2)
00
00
It is not difficult to see that the synthesiser perfectly predicts the future gradient, 8,47 = ÓetT, then the weight will be perfectly updated according to the gradient of all future losses, 0 a Ef= aL. 0e Allowing the synthesiser access to this gold standard, 8,47, however, requires waiting an arbitrary amount of time and is therefore not practical for training. Instead, 8 is trained to replicate an itself approximate, bootstrapped version of 0Lا the future gradient, 8,47, where u = Lj=i+T+1 Bh,4r + 8:427 0h.42r Note that 0 is available one truncation's 's timespan.
8h,4r
In the brain approximation of BPTT can be implemented forward in time using mixture cligibility traces and sparse learning signals (e.g. reward) (Bellec et al. 2019); Fig. SSp)). The dynamics of this more "biologically plausible" learning regime described here; we simply mention that the cerebellum might be the source of such (future)
subtle and not
learning signals and predict that many of the results observed with BPTT will hold for the forward propagation of gradients.
E INPUT EXPANSION AND SPARSE CONNECTIVITY IN CEREBELLUM-DNI
We also study the effects of input expansion and sparse connectivity from the main model to the synthesiser in our 33CC-DDI model. Input expansion and sparse connectivity are two defining features of the cerebellum. Often human intelligence linked to its expanded neocortex. However, the cerebellar/neocortici cell ratio in humans 4/1 and the cerebellum has significantly expanded throughout evolution (Barton and Venditti (2014); Herculano- Houzel (2010). Here we aim to study the effect of this input expansion into the cerebellum by systematically varying the number of hidden units in the synthesiser relative to the number of units in the main network. We show this for both sensorimotor task: target reaching task (Fig. (33) and seq-MNIST tasks (Fig. SS4 and [S3). In addition, it is known that one granule cell receives input from four mossy fibres on average. Mossy fibres form the main inputs to the cerebellum. The largest source of mossy fibres is the neocortex which projects inputs the cerebellum via the pons. To quantify the effect of the sparse input connectivity we quantify next to the Pearson correlation also the population correlation coefficient for the target reaching task (Fig. S6) and the standard Seq-MNIST task (Fig. S7p.
F TARGET REACHING SUPPLEMENTARY DATA
Here we show that the target reaching task results extrapolate to variants the task with varied input-output combinations (Fig. S8), Furthermore, in line with Fig. 2b we include the learning curves across different truncation values for the three models (Fig.
G SEQMNIST SUPPLEMENTARY DATA
In line with Fig we include the learning curves across different truncation values for the standard MNIST task as well as its variants (Figs. S10
ST0 SIT.

---

Under review as a conference paper at ICLR 2021
1/2
MIN
MMN
10
20
MIN
M/V
25
30 60
K
Figure S3: (a) Learning curves for target reaching as in Fig. over different divergence ratios M/N, where M is the number of hidden 'granular' units cerebellar model, N the number of input units, and number of non-zero input connections K. LSTM performance (grey) shown along with fully (orange) and sparsely (cyan) connected CC-DNI models. Note that N is fixed here as 2 x 10 = 20 (10 LSTM units with gradients calculated for both cell and output states), hence * denotes full connectivity. The smaller x-axis in each subplot represents the epoch number and the smaller y-axis represents mean squared error. (b) Average performance over last five epochs (295-300) against divergence ratio M/N and input connection sparsity K for target reaching in Fig. 2 (c) Same as (a) but for complex input (input dimension =28) and seven 2-D target coordinates (30 LSTM units with gradients calculated for both cell and output states). (d) Same (b) for target reaching with complex input (input dimension 28) and seven 2-D target coordinates over last epochs (95-100).
IMAGE CAPTIONING SUPPLEMENTARY DATA
In line with Fig. include more examples of image-caption pairs (Fig. S12 well as the learning curves and metric scores across different truncation values (Figs. S14 ST5). Moreover, demonstrate that DNI models for this task often generalise unseen data better than regular LSTM models (Fig. [13).

---

Under review as a conference paper at ICLR 2021
SeqMNIST (error)
C-SeqMNIST (MSE)
20
20
20
20
20
20
20
20
20
20 5
20
20
50%
50%
70
70
20%
20%
40
40
1/1 50%
50%
70
70
20%
40
40
50%
50%
70
M/N
M/N
50%
20%
20%
50%
70
20%
20%
40
40
50%
50%
70
70
20%
20%
40
40
20
20
20
60
60
K
K
dd-seqMNIST (MSE)
Id-seqMNIST (MSE)
20
15
15
25
25
10
10
10
10
15
15
25
25
10
10
10
10
15
15
2/1
2/1
M/N
10
10
M/
10
10
15
25
5/1
5/1
25
25
10/1
10/1
10
15
15
25
25
20/1
20/1
10
10
10
10
20
20
20
20
20
20
20
20
20
20
10
60
10
30
60
Figure S4: Learning curves for all SeqMNIST based tasks over different cerebellum divergence ratios M/N, where M the number of hidden 'granular' units in cerebellar model, N the number of input units, and number non-zero input connections K. L.SMM performance (grey) shown as a reference along with fully (orange) and sparsely (cyan) connected cc-DNI models. Note that N is fixed here as 2 30 = 60 (30 L.STM units with gradients calculated for both cell and output states), hence * denotes full connectivity. The smaller x-axis in each subplot represents the epoch number and the y-axis represents performance over validation data (error for SeqMNIST, MSE for the others).
8

---

Under review as a conference paper at ICLR 2021
SeqMNIST (error)
C-seqMNIST (MSE)
0/1
28%
0.7
1/2
26%
1/2
1/1
1/1
24%
M/N
211
MMN
2/1
44
22%
42
5/1
5/1
40
10/1
10/1
38
20/1
18%
20/1
36
60'
60
dd-seqMNIST (MSE)
Id-seqMNIST (MSE)
000
9,4
0/1
6.8
12
12
1/1
1/1
M/N
M/N
221
2/1
5/1
8.8
5/1
10/1
8.6
10/1
6.0
20/2
20/1
5588
60
60
Figure S5: Related Fig. S4 Average performance over five epochs (21-25) against divergence ratio M/N and input connection sparsity K for each seqMNIST based task. denotes full connectiv- ity. The divergence ratio and connectivity of the default LSTM, CC-DNI and sCC-DNI models used for the segMNIST tasks (see Fig. are illustrated by the grey, orange and cyan squares respectively.

---

Under review as a conference paper at ICLR 2021
errer (MSE)
0.40
0.46
0.50
0.55
0.60
0.85
Ph)
0.8
0.4
0.2
20
40
80
90 100
epoch
Figure S6: Equivalent Fig. _ for population correlation popp (see equation ??). (top) Effect of correlation of hidden (cortical) activity f on performance for cc-DNI (orange) and sCC-DNI (cyan) on the target reaching task, where population correlation and performance are recorded during the first and last (300) epoch respectively. (bottom) Evolution of the population correlation of the synthetic gradient over the first hundred epochs.
0:32
30%
9
25%
0.10
0.15
0.20
0.25
0.30
0.35
0.40
0.45
0.50
(popth)
0.3
G
0.2
0.1
epech
Figure S7: Equivalent Fig. 3 for population correlation Tpop (see equation ??). (top) Effect of correlation of hidden (cortical) activity h on performance for cc-DNI (orange) and sCC-DNI (cyan) on the (standard) seqMNIST task, where population correlation and performance are recorded during the first and fifth epoch respectively. (bottom) Evolution of the population correlation of the synthetic gradient over the first five epochs.
10

---

Under review as a conference paper at ICLR 2021
LOTM
DOOM
VOCONI
LSTN
COOM
VODONI
Shankes
3onobe
Figure S8: (a, b) Target reaching task as in Fig. with varying number of targets. (a) Given one-dimensional input the network has to learn to drive its output towards one of the 3 targets over 10 timesteps. For this variant divergence ratio for CC-DNI M/N 4 and K for sCC-DNL. (b) Given one-dimensional input the network has to learn to drive its output towards one of the targets over 10 timesteps. For this variant divergence ratio for CC-DNI is M/N
and K for sCC- DNL. (c, d) Two variants the target reaching task with varying input dimension, two-dimensional (i.e. X x22) and and 28-dimensional (Fig. (8d) (i.e. = {#1,-.,, x28}) input respectively. (c) Given two-dimensional input the network has to learn drive its output towards of the 7 targets over 10 timesteps. For this variant divergence ratio for CC-DNI is M/N = 4 and = 4 for SCC-DNI. (d) Given 28-dimensional input the network has to learn to drive its output towards one of the targets over 10 timesteps. For this variant divergence ratio for CC-DNI is M/N = 4 and K for SCC-DNI.
T=7
100
arter (MSE)
0
0
100
200
300
100
200
300
100
200
300
100
200
300
100
200
800
epoch
Figure S9: Learning curves for the target reaching task as in Fig. P across different truncation values T' shown for the three models LSTM (grey), CC-DNI (orange) and sCC-DNI (cyan). Pararameters are the same as in Fig. 2p with divergence ratio M/N = 4 and sparse input connectivity K = 4 for SCC-DNI.
11

---

Under review as a conference paper at ICLR 2021
SeqMNIST
T
T-
T -
T 5
T-
T 10
60%
20%
20
10
10
10
20
10
20
10
C-SeqMNIST
T-3
aftor (MSE)
dd SeqMNIST
T
T
T 4
T 5
T
T 10
erner (MSE)
12
10
20
10
20
10
20
0
10
20
10
20
10
20
Id-seqMNIST
T-2
T
T-5 5
effor (MSE)
20
10
10
20
0
20
10
opach
Figure S10: Learning curves for all seqMNIST based tasks across different truncation values. Model colours as in the main text (LSTM grey; CC-DNI orange; sCC-DNI cyan), with the default parameters of 5/1 divergence (300 GCs) in the DNI models and 4 for sCC-DNI.
c:seqMNIST
dd-se id-seqqIST
Id MNIISTT
70
7.0
MITE (MSE)
0rT (MSE)
6.5
6.0
10
10
6
truncation
mruccation
truncation
Figure $11: Related to Fig. STO dd-seqMNIST, Id-Se tasks across truncation sizes for LSTM (grey), cc-DNI (orange), sCC- DNI (cyan). See Fig. Be for (standard) SeqMNIST.
Average performance last five epochs (21-25) for MNIST.
12

---

Under review as a conference paper at ICLR 2021
plaging enddy bell
aeady
perting raady
Shuomnag wabs diabee
grive neeple who sre on menil roont the pwople
trabbe court
tritree stace
people sranding at seow mver
frstave redtor cenet.
people standing tewelr court wakking tooms
yroung whall aen playing fiillin uatart
group of yourg people playtag tribes gne
ediling shase walk stualed bear
brcck percoed tap tranck
youcal bolding teddy bast ls Sondles
victin cokiing cate with betto
two moreee readlo no for feod talle
N
two aiceus progare sablo sals wanes Lobling
soic
Albal
scaey dineers woker tealde
parked
astting duskiicand
modiore aeed black dag tee the back of
voftonddeee unatee
wiite parked moner on
seall adtiing sean the book mindom al
white vstl partoo on eart was parkingg noter
parked owmer
atsting windon
and verees are eifing slepboenss down nead
totak and waite painte instongete
riling explose dart andd
partenl parking
yaske pmaple estting ou tuy shipinaa
parked west in Hiskt tres evopeds
patr of prople 18de on the beck steykeen
mreadd are partard sorr Tiayete altrweck
Figure S12: Example images from the validation set with corresponding model captions (LSTM (grey); CC-DNI (orange); sCC-DNI (cyan)) and gold standard captions (black). Here we show combination of examples of how the models describe the presented image. In some case or some models fail to give accurate description of the image. In other cases all models able to accurate caption describing the image, with each model displaying subtle differences in the generated captions.
0.015
= 3
T
0.010
validation
0.000
-0.005
00010
0.010
-0.005
0.000
0.005
0.010
0.015
A training
Figure S13: Generalisation of CC-DNI (orange) and SCC-DNI (cyan) for truncation values T from to 7. The change in loss computed with reference to the LSTM (i.e. (s)CC-DNI LSTM), Training loss is calculated after training (with dropout disabled) to enable fair comparison with final validation performance.
13

---

Under review as a conference paper at ICLR 2021
T T+4 T=5 T T=7
T 10
T 15
1008 (0990)
35
opoch
Figure S14: Validation learning curves bits per word (BPW) for LSTM (grey), CC-DNI (orange) and sCC-DNI (cyan) across different truncation sizes T (see Fig. 4h. BPW range restricted to enable comparison between truncation values; full curves are shown in inset. The surprising performance for T = 10 is likely due to how the sequence is divided into truncations. With strong majority of (gold standard) caption lengths between 11-15 (mean ~ 13, the sequence will often be divided into two uneven truncations, perhaps making BPTT difficult for the LSTM. However, in the case of two truncations synthetic gradient will only be required once (in between the two truncations) and is analogous to the "easier" job of a synthesiser predicting gradients for feedforward network, explaining the particular relative improvement seen for the CC-DNI models in this case.
14

---

Under review as a conference paper at ICLR 2021
0.67
0.66
B 2
0.40
-10
10a
Lleluu. 3
0.33
0.32
-10
10a
Bleuu 4
-10-
ROUGEE
-10
0.475
0.215
METEOR
0.210
-10
0.725
& CDDr
0.700
30t4
0.345
& SPICE
0.340
10
ratto
18
0.50
3.96
truncation
Figure S15: Evaluation of model-generated captions across truncation sizes for metrics (in order as shown) BLEU_1, BLEU__ BLEU_3, BLEU_4, Rouge-L, METEOR, CIDEr, SPICE. The caption length ratio Cm /ng/Cuv where C, the length the model-generated caption and Cgs the length of the corresponding gold standard caption of closest length, is also shown.
15


Reviews:
[{"review_id": "vZJ6DlckvCd", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors proposed that cerebellum in the brain computes synthetic gradients, as used in decoupled neural interfaces (DNI), to enable learning in neural circuits without waiting for gradients to propagate backwards. The authors incorporated several architectural properties of biological cerebellum into their cerebellar model. They showed that a LSTM trained with such synthetic gradients can learn a variety of tasks, from motor reaching to caption generation. The paper is clearly written, and the link between DNI and cerebellum is a novel idea. However, the authors made little attempt to actually compare their model with experimental findings from cerebellum (except for the cerebellar properties built into the network), limiting its scientific impact. Meanwhile, it is not clear whether the cerebellum-inspired DNI provides concrete advantages over DNI proposed in Jaderberg 2017.\n\nMajor comments:\n(1) The authors made little attempt to actually compare their model with experimental findings, or at least make concrete testable predictions.\n\nMain results (Fig. 2-4) are mostly showing that their core models, CC-DNI and sCC-DNI, can successfully reduce losses across a variety of tasks, and do so better than heavily truncated BPTT gradients.\n\nExcept for the architectural features incorporated into the model, and some loose arguments that cerebellum is involved in sensory, motor, and cognitive tasks, the link between the model and biological cerebellum appears somewhat weak.\n\nThe authors claimed that “our work makes numerous predictions and opens the exciting door to explicit comparison…” without actually spelling out any key prediction. One prediction of the model is that when a task is very well-learned, the gradients (both real and predicted) should be close to 0, and cerebellum output neurons in the deep cerebellar nuclei should be somehow silent. Another prediction is that cerebellum lesioning should only impact learning, but not performance of well-learned tasks. I’m not sure these predictions are supported by empirical evidences. \n\nI would feel much more comfortable supporting this manuscript if the authors provide more comparisons with experimental data, and make more concrete testable predictions.\n\n(2) Critical questions about how real gradients are computed and transmitted to inferior olive is not answered.\n\nFor the brain, the backward lock may not be the most acute issue when searching for approximated gradient descent in brains. DNI relies on computing the real gradients, and using it to train the generation of synthetic gradients. Both computations are challenging in the brain. The authors completely circumvent this problem by saying it is “outside of the scope of the current paper”. However, I think this issue is critical for considering the feasibility of the proposed mechanism. For example, how can cerebellum learn to predict the gradient of individual cortical neurons when there are many fold less Purkinje cells than cortical neurons? There are of course more granule cells than there are cortical neurons, but in the model, granule cells are not the ones representing the synthetic gradient, \\hat{g}_M, right?\n\nMinor\n(1) The references are at a number of places somewhere between inaccurate and incorrect. Here are a few that I noticed.\n\nIn the introduction, the authors wrote “These observations suggest that the cerebellum implements a universal function across the brain (Diedrichsen et al., 2019)”. However, if I’m not mistaken, the Diedrichsen review is arguing the exact opposite that cerebellum is not implementing a universal function.\n\nIn section 2.1.1, the authors wrote “Here we use LSTMs (Hochreiter and Schmidhuber, 1997) as a model of cortical networks, which have recently been mapped onto cortical microcircuit (Costa et al., 2017) ”. Costa 2017 provided a potential way to link cortical microcircuit to a LSTM-like structure. I don’t think it’s fair to say that LSTMs are \"mapped\" onto cortical microcircuits.\n\nIn section 2.1.2, the authors wrote “On the other hand Bellec et al. (2019) showed that temporal gradients as used in BPTT are equivalent to using eligibility traces that transmit gradient information forward in time”. The eligibility-trace-based algorithm proposed by Bellec 2019, namely e-prop, is not “equivalent” to back-prop. It is an approximation that works well empirically in the cases studied in that paper.\n\n(2) Fig. 2 panels are mislabeled.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "n0MTBxs_G7", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "**General**\n\nThe paper presents a very intriguing hypothesis, and I believe that its publication will benefit the community and stimulate fruitful discussion. The model seems to offer a compelling and fairly novel explanation of cerebellar deficits (including non-motor) with a broad significance across the neuroscience and deep learning communities. That said, I think that there are opportunities for improving clarity and filling in details that will be lost on readers who aren't strongly familiarity with Jaderberg et al.\n\nThe paper currently presents the model primarily in the feedforward setting, and the results in the recurrent setting. This becomes confusing, since the two settings can be associated with different locking problems in neuroscience (i.e. bio-plausible alternatives to backprop vs. learning from delayed rewards / bio-plausible BPTT). For instance, the abstract and main text intro (\"a given layer has to wait for all the layers above to finish computing its gradients\") seems to propose a solution for feedforward bio-plausible backprop. In the results, however, it focuses primarily on the ability to learn from delayed signals with BPTT models. I think it may help to focus earlier on the recurrent setting with references to the delayed feedback problem. \n\nIn Section 2, the cortical network uses the backward synthesiser to avoid needing to wait for the loss signal - however, this seems to merely shift the problem since now the synthesiser will need to wait for the loss signal to train. Reading Jaderberg et al. (and the SM), it's clear that the synthesiser is instead continually trained on bootstrapped estimates. I think that a reference to bootstrapping in the main text (and a reference to the analogy with bootstrapped value functions in RL, as in Jaderberg) would make the model much clearer.\n\nSince Jaderberg et al. have already shown that DNIs improve on truncated LSTMs, it seems like a more interesting comparison in the results would be DNI vs. CC-DNI.  If the authors are proposing that CC-DNI is a competitive deep learning approach than I would like to see something like the original DNI architecture included as a baseline. Otherwise, I would at least like to see a clearer description of the architectural differences between DNI (as previously implemented) and CC-DNI/sCC-DNI. \n\n**Details**\n\n- Do the predictions/consequences in Fig. 1, g, roughly correspond to dL/dh_i? If so it would help to make this explicit.\n- Figure 2(f)(g) are labelled in the caption as (e)(f).\n- Unlike Jaderberg et al., the paper combines update and backwards locking under the same label. A clarification, either in the SM or a footnote, would help.\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Cm6240NYdnU", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors consider the architecture of the cerebellum as the predictive component of a decoupled neural interface (DNI). Using this framework, they perform experiments training networks with BPTT on several temporal tasks. \n\nThe paper is exceptionally clear and the experimental investigations are well-done. \n\nHowever, it does not offer any new insight into either the cerebellum or DNI; rather it simply juxtaposes the details of two existing bodies of knowledge. The two statements that might most closely constitute new insight—that DNI is helpful on temporally challenging tasks and that sparsity-induced de-correlation can be helpful—were both established within their respective research domains. The logical induction that the authors make \"predicting that the cerebellum is particularly important for more temporally challenging tasks\" does not require the DNI to be established. That enforcing the architectural constraints of sparsity within a DNI might be helpful, although curious, does not constitute sufficiently extensive findings for a publication, and (although aided by) does not require connecting DNIs and the cerebellum.\n\nMuch like the platitude \"the brain is like a computer\" offers no new insight into either computers or brains, here, I do not believe that authors' investigations, although amusing to follow-along with, have added significant insight into either the cerebellum or DNIs. Therefore, I cannot offer strong support for acceptance.", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: Reviewers split on this paper with one arguing that it is an intriguing and significant paper for both neuroscience and deep learning, whereas others argued that it fails to answer some key questions and stops short of offering testable predictions or novel findings. In particular Reviewer 2 questioned the limited experimental predictions and their experimental backing, as well as the plausibility of gradient calculations.  Reviewer 4 raised more fundamental concerns about the significance of the paper's contributions. All reviewers appreciated the paper's clarity. Overall, though, Reviewers 2 and 4 raised enough significant concerns that I cannot recommend acceptance. 
Decision: Reject