Original: C ORTICO - CEREBELLAR NETWORKS AS DECOUPLED NEURAL INTERFACES
Revised: C ORTICO - CEREBELLAR NETWORKS AS DECOUPLED NEURAL INTERFACES

Original: Anonymous authors
Revised: Anonymous authors

Original: Paper under double-blind review
Revised: Paper under double-blind review

Original: A BSTRACT
Revised: A BSTRACT

Original: The brain solves the credit assignment problem remarkably well.
Revised: The brain solves the credit assignment problem remarkably well.

Original: For credit to be correctly assigned across multiple cortical areas a given area should, in principle, wait for others to ﬁnish their computation.
Revised: For credit to be correctly assigned across multiple cortical areas a given area should, in principle, wait for others to ﬁnish their computation.

Original: How the brain deals with this locking problem has remained unclear.
Revised: How the brain deals with this locking problem has remained unclear.

Original: Deep learning methods suffer from similar locking constraints both on the forward and backward phase.
Revised: Deep learning methods suffer from similar locking constraints both on the forward and backward phase.

Original: Recently, decoupled neural interfaces (DNI) were introduced as a solution to the forward and backward locking problems.
Revised: Recently, decoupled neural interfaces (DNI) were introduced as a solution to the forward and backward locking problems.

Original: Here we propose that a specialised brain region, the cerebellum, helps the cerebral cortex solve the locking problem closely matching the computations and architecture of DNI.
Revised: Here we propose that a specialised brain region, the cerebellum, helps the cerebral cortex solve the locking problem closely matching the computations and architecture of DNI.

Original: In particular, we propose that classical cerebellar forward and inverse models are equivalent to solving the backward and forward locking problems, respectively.
Revised: In particular, we propose that classical cerebellar forward and inverse models are equivalent to solving the backward and forward locking problems, respectively.

Revised: To demonstrate the potential of this framework we focus on modelling a given brain area as a recurrent neural network in which the cerebellum approximates temporal feedback signals as provided by BPTT.

Original: We tested the cortico-cerebellar-DNI (CC-DNI) model in a range of sensorimotor and cognitive tasks that have been shown to be cerebellardependent.
Revised: We tested the cortico-cerebellar-DNI (CC-DNI) model in a range of sensorimotor and cognitive tasks that have been shown to be cerebellar-dependent.

Original: First, we show that the CC-DNI unlocking mechanisms can facilitate learning in a simple target reaching task.
Revised: First, we show that the CC-DNI unlocking mechanisms can facilitate learning in a simple target reaching task.

Original: Next, by building on the sequential MNIST task we demonstrate that these results generalise to more complex sensorimotor tasks.
Revised: Next, by building on the sequential MNIST task we demonstrate that these results generalise to more complex sensorimotor tasks.

Original: Our cortico-cerebellar model readily applies to a wider range of modalities, to demonstrate this we tested the model in a cognitive task, caption generation.
Revised: Our cortico-cerebellar model readily applies to a wider range of modalities, to demonstrate this we tested the model in a cognitive task, caption generation.

Original: Models without the cerebellar-DNI component exhibit deﬁcits similar to what has been observed in cerebellar patients in both motor and cognitive tasks.
Revised: Models without the cerebellarDNI component exhibit deﬁcits similar to what has been observed in cerebellar patients in both motor and cognitive tasks.

Original: Finally, we introduce a CC-DNI model with highly sparse connectivity as observed in the cerebellum, which substantially reduces the number of parameters while improving learning through decorrelation.
Revised: Finally, we introduce a CC-DNI model with highly sparse connectivity as observed in the cerebellum, which substantially reduces the number of parameters while improving learning through decorrelation.

Original: Overall, our work offers a novel perspective on the cerebellum as a brain-wide decoupling machine for efﬁcient credit assignment and opens a new avenue of research between deep learning and neuroscience.
Revised: Overall, our work offers a novel perspective on the cerebellum as a brain-wide decoupling machine for efﬁcient credit assignment and opens a new avenue of research between deep learning and neuroscience.

Original:  I NTRODUCTION
Revised:  I NTRODUCTION

Original: Efﬁcient credit assignment in the brain is a critical part of learning.
Revised: Efﬁcient credit assignment in the brain is a critical part of learning.

Original: However, how the brain solves the credit assignment problem remains a mystery.
Revised: However, how the brain solves the credit assignment problem remains a mystery.

Original: One of the central issues of credit assignment across multiple stages of processing is the need to wait for previous stages to ﬁnish their computation before others can proceed (Rumelhart et al., 1986; Schmidhuber, 1990; Lee et al., 2015; Marblestone et al., 2016; Jaderberg et al., 2017).
Revised: One of the central issues of credit assignment across multiple stages of processing is the need to wait for previous stages to ﬁnish their computation before others can proceed (Rumelhart et al., 1986; Schmidhuber, 1990; Lee et al., 2015; Marblestone et al., 2016; Jaderberg et al., 2017).

Original: In deep artiﬁcial neural networks these constraints are explicit.
Revised: In deep artiﬁcial neural networks these constraints are explicit.

Original: During the forward phase a given layer has to wait for all its previous layers to ﬁnish before it can proceed, a constraint known as the forward lock .
Revised: During the forward phase a given layer has to wait for all its previous layers to ﬁnish before it can proceed, a constraint known as the forward lock .

Original: Similarly, during the backward phase a given layer has to wait for all the layers above to ﬁnish computing its gradients – backward lock .
Revised: Similarly, during the backward phase a given layer has to wait for all the layers above to ﬁnish computing its gradients – backward lock .

Original: Recently, a framework was introduced to decouple artiﬁcial neural networks – decoupled neural interfaces (DNI; (Jaderberg et al., 2017)) 1 , effectively breaking forward and/or backward locks.
Revised: Recently, a framework was introduced to decouple artiﬁcial neural networks – decoupled neural interfaces (DNI; (Jaderberg et al., 2017)) 1 , effectively breaking forward and/or backward locks.

Original: Here, we propose that a specialised brain area, the cerebellum, performs a similar role in the brain.
Revised: Here, we propose that a specialised brain area, the cerebellum, performs a similar role in the brain.

Original: In the classical view the cerebellum is key for ﬁne motor control and learning by constructing internal models of behaviour.
Revised: In the classical view the cerebellum is key for ﬁne motor control and learning by constructing internal models of behaviour.

Original: (Marr, 1969; Albus, 1971; Raymond and Medina, 2018;
Revised: (Marr, 1969; Albus, 1971; Raymond and Medina, 2018;

Original: Wolpert et al., 1998;
Revised: Wolpert et al., 1998;

Original: Miall et al., 1993).
Revised: Miall et al., 1993).

Original: More recently, however, the idea that the cerebellum is also involved in cognition has gained signiﬁcant traction (Schmahmann et al., 2019; Wagner and Luo, 2020; Brissenden and
Revised: More recently, however, the idea that the cerebellum is also involved in cognition has gained signiﬁcant traction (Schmahmann et al., 2019; Wagner and Luo, 2020; Brissenden and

Original: Somers, 2019).
Revised: Somers, 2019).

Original: An increasing body of behavioural, anatomical and imaging studies points to a role of the cerebellum in cognition in humans and non-human primates (Schmahmann et al., 2019;
Revised: An increasing body of behavioural, anatomical and imaging studies points to a role of the cerebellum in cognition in humans and non-human primates (Schmahmann et al., 2019;

Original: Brissenden and Somers, 2019; Guell et al., 2015; 2018).
Revised: Brissenden and Somers, 2019; Guell et al., 2015; 2018).

Original: Impairments in cerebellar patients occur across a range of tasks including language (Guell et al., 2015), working memory (Deverett et al., 2019), planning (Baker et al., 1996), and others (Fiez et al., 1992).
Revised: Impairments in cerebellar patients occur across a range of tasks including language (Guell et al., 2015), working memory (Deverett et al., 2019), planning (Baker et al., 1996), and others (Fiez et al., 1992).

Original: These observations suggest that the cerebellum implements a universal function across the brain (Diedrichsen et al., 2019).
Revised: These observations suggest that the cerebellum implements a universal function across the brain (Albus, 1971; Raymond and Medina, 2018; Diedrichsen et al., 2019; ? ).

Original: Moreover, experimental studies looking at cortico-cerebellar interactions have demonstrated that cerebellar output is crucial for maintaining neocortical representations in order to drive behaviour (Chabrol et al., 2019; Gao et al., 2018).
Revised: Moreover, experimental studies looking at corticocerebellar interactions have demonstrated that cerebellar output is crucial for maintaining neocortical representations in order to drive behaviour (Chabrol et al., 2019; Gao et al., 2018).

Original: However, to the best of our knowledge, no theoretical framework has considered what might be the function of such interactions between the cerebellum and cortical areas.
Revised: However, to the best of our knowledge, no theoretical framework has considered what might be the function of such interactions between the cerebellum and cortical areas.

Original: In an attempt to reduce the existing gap between experimental observations and existing computational approaches we introduce DNI as a cortico-cerebellar model – cortico-cerebellar DNI (CC-DNI).
Revised: In an attempt to reduce the existing gap between experimental observations and existing computational approaches we introduce DNI as a cortico-cerebellar model – cortico-cerebellar DNI (CC-DNI).

Original: Consistent with the cerebellar universal role we theorise that the cerebellum serves to break the locks inherent to both feedforward and feedback information processing in the brain, akin to DNI.
Revised: Consistent with the cerebellar universal role we theorise that the cerebellum serves to break the locks inherent to both feedforward and feedback information processing in the brain, akin to DNI.

Original: In particular, we posit that the two classical internal models of the cerebellum, forward and inverse models, are equivalent to DNI-mediated unlocking of feedback (gradients) and feedforward communication, respectively.
Revised: In particular, we posit that the two classical internal models of the cerebellum, forward and inverse models, are equivalent to DNI-mediated unlocking of feedback (gradients) and feedforward communication, respectively.

Original: Following this view the cerebellum not only provides motor or sensory estimates, but also any other modality encoded by a particular brain region.
Revised: Following this view the cerebellum not only provides motor or sensory estimates, but also any other modality encoded by a particular brain region.

Original: Inspired by neuroscientiﬁc studies, we test our model on sensorimotor tasks: (i) a target reaching task (Sanes et al., 1990; Butcher et al., 2017; Nashef et al., 2019) and (ii) a set of more complex temporal tasks based on the MNIST dataset, but also (iii) on a cognitive task – caption generation (Guell et al., 2015).
Revised: Inspired by neuroscientiﬁc studies, we test our model on sensorimotor tasks: (i) a target reaching task (Sanes et al., 1990; Butcher et al., 2017; Nashef et al., 2019) and (ii) a set of more complex temporal tasks based on the MNIST dataset, but also (iii) on a cognitive task – caption generation (Guell et al., 2015).

Original: Our results support the cortico-cerebellar DNI models we study and show that they generally speed up learning by unlocking the main network, qualitatively consistent with a wide range of behavioural observations (Guell et al., 2015; Sanes et al., 1990; Butcher et al., 2017; Nashef et al., 2019).
Revised: Our results support the cortico-cerebellar DNI models we study and show that they generally speed up learning by unlocking the main network, qualitatively consistent with a wide range of behavioural observations (Guell et al., 2015; Sanes et al., 1990; Butcher et al., 2017; Nashef et al., 2019).

Original: Two deﬁning features of the cerebellum are the large expansion at the granule cell input layer withbillion neurons (the most numerous cell in the brain) and the highly sparse connectivity (each granule cell receives ∼ 4 synapses)
Revised: Two deﬁning features of the cerebellum are the large expansion at the granule cell input layer withbillion neurons (the most numerous cell in the brain) and the highly sparse connectivity (each granule cell receives ∼ 4 synapses)

Original: (Sanger et al., 2020).
Revised: (Sanger et al., 2020).

Original: These observations have been long suggested to help speed-up learning in the cerebellum through decorrelation (Albus, 1971; Sanger et al., 2020;
Revised: These observations have been long suggested to help speed-up learning in the cerebellum through decorrelation (Albus, 1971; Sanger et al., 2020;

Original: Cayco-Gajic et al., 2017).
Revised: Cayco-Gajic et al., 2017).

Original: Building on these studies we introduce a new DNI model, sparse CC-DNI .
Revised: Building on these studies we introduce a new DNI model, sparse CC-DNI .

Original: Consistent with classical cerebellar models (Albus, 1971; Cayco-Gajic et al., 2017)
Revised: Consistent with classical cerebellar models (Albus, 1971; Cayco-Gajic et al., 2017)

Original: we show that input sparsity can improve learning in the presence of high correlations.
Revised: we show that input sparsity can improve learning in the presence of high correlations.

Original: We ﬁnish with a discussion on the implications and predictions of this new brain-wide model of the cerebellum.[Figure]
Revised: We ﬁnish with a discussion on the implications and predictions of this new brain-wide model of the cerebellum.

Original:  C EREBELLUM AS A DECOUPLING MACHINE
Revised:  C EREBELLUM AS A DECOUPLING MACHINE

Original: We ﬁrst describe DNIs following Jaderberg et al.
Revised: We ﬁrst describe DNIs following Jaderberg et al.

Original: and then establish the link to corticocerebellar networks.
Revised: and then establish the link to corticocerebellar networks.

Original: Assume that a feedforward neural network consists of N layers, with the i th layer ( 1 ≤ i ≤ N ) performing a “computational step”
Revised: Assume that a feedforward neural network consists of N layers, with the i th layer ( 1 ≤ i ≤ N ) performing a “computational step”

Original: f i with parameters θ i .
Revised: f i with parameters θ i .

Original: Given input x at layer 1, the output of the network at its ﬁnal layer is therefore given by f N ( f N − 1 ( . . .
Revised: Given input x at layer 1, the output of the network at its ﬁnal layer is therefore given by f N ( f N − 1 ( . . .

Original: f 2 ( f 1 ( x )) . . . )) .
Revised: f 2 ( f 1 ( x )) . . . )) .

Original: We use F ji to denote the composition of steps from layer i to layer j (inclusively).
Revised: We use F ji to denote the composition of steps from layer i to layer j (inclusively).

Original: Finally, let h i denote the (hidden) activity at layer i , so that h i
Revised: Finally, let h i denote the (hidden) activity at layer i , so that h i

Original: =
Revised: =

Original: f i ( h i − 1 ) with h 0 = x .
Revised: f i ( h i − 1 ) with h 0 = x .

Original: To illustrate the locking constraints of standard artiﬁcial neural networks used in deep learning suppose that a network is in the process of learning via backpropogation, with current input-target pair ( x, y targ ) .
Revised: To illustrate the locking constraints of standard artiﬁcial neural networks used in deep learning suppose that a network is in the process of learning via backpropogation, with current input-target pair ( x, y targ ) .

Original: To update the layer parameters θ i the gradient ∂L∂θ i is required, where L =
Revised: To update the layer parameters θ i the gradient ∂L∂θ i is required, where L =

Original: L ( y, y targ ) is the loss which compares the target value against the model output y =
Revised: L ( y, y targ ) is the loss which compares the target value against the model output y =

Original: F N 1 ( x ) under some loss function L ; we then apply gradient descent on the parameters, θ i ← θ i − α ∂L∂θ i , with learning rate α > 0 .
Revised: F N 1 ( x ) under some loss function L ; we then apply gradient descent on the parameters, θ i ← θ i − α ∂L∂θ i , with learning rate α > 0 .

Original: Suppose however that the network has only recently received the input x and is currently only at module i of the forward computation.
Revised: Suppose however that the network has only recently received the input x and is currently only at module i of the forward computation.

Original: In order to update the corresponding parameters of that layer, θ i , the layer must ﬁrst wait for all remaining layers to ﬁnish f j ( j > i ) for the loss to be computed.
Revised: In order to update the corresponding parameters of that layer, θ i , the layer must ﬁrst wait for all remaining layers to ﬁnish f j ( j > i ) for the loss to be computed.

Original: Only then the various gradients of the loss are backpropogated and ∂L∂θ i is ﬁnally available.
Revised: Only then the various gradients of the loss are backpropogated and ∂L∂θ i is ﬁnally available.

Original: These two characteristics of backpropogation make layer i “backward locked” to F ji +1 , enforcing a strong dependence of the layer’s learning to the speed of forward and backward propagation through the rest of the network.
Revised: These two characteristics of backpropogation make layer i “backward locked” to F ji +1 , enforcing a strong dependence of the layer’s learning to the speed of forward and backward propagation through [Figure] Approximating temporal feedback/gradients [Figure]

Revised: bootstrapingmodules

Original: The key idea behind backward DNI is to break the  backward lock by feeding the hidden layer i activity h
Revised: [Figure] the rest of the network. Similarly the the network is forward locked during the forward pass. DNI can be used to unlock both backward and forward locks Jaderberg et al.

Original:  i to a separate neural network, a backward synthesiser C Bi , that learns to produce a synthetic gradient  – an estimate of the real gradient expected by layer i ,
Revised: To illustrate the model here we focus on backward DNI , which goal is to break the backward lock by feeding the hidden layer i activity h i to a separate neural network, a backward synthesiser C Bi , that learns to produce a synthetic gradient ˆ g i – an estimate of the real gradient expected by layer i ,

Original: C i ( h i )
Revised: C i ( h i )

Original: =
Revised: =

Original: ˆ ∂L∂h i ≈ ∂L∂h
Revised: ˆ g i ≈ ∂L∂h

Original: i .
Revised: i .

Original: This synthetic gradient can then be used to update the weights of layer
Revised: This synthetic gradient can then be used to update the weights of layer

Original: i as soon as h i is available, following
Revised: i as soon as h i is available, following

Original: [Equation]
Revised: [Equation]

Original: More speciﬁcally, the parameters of C Bi ( h i ) are learned by comparing the estimated gradient with the true gradient, L  = ||∂L∂h
Revised: More speciﬁcally, the parameters of C Bi ( h i ) are learned by comparing the estimated gradient with a target gradient ¯ g i so as to minimise L C = ||

Original:  i − ˆ ∂L∂h
Revised: ¯ g i − ˆ g

Original: i || .
Revised: i || .

Original: Interestingly, using a simple one or two hidden layer feedforward network to model C i ( h i ) has been shown to converge to  the true  gradient for linear deep networks
Revised: Ideally we set the target gradient as the true gradient, ¯ g i = ∂L∂θ i , as can be implemented without difﬁculty in the case of feedforward networks. However, if we consider the temporal case (see sections 2.1.1 and 3) where we wish to estimate gradients of future losses many timesteps ahead, L = (cid:80) t L t , the true backpropagated gradient is computational expensive to obtain.

Original: Czarnecki et al.
Revised: Jaderberg et al.

Original: (2017), and to work well in practice for non-linear networks Jaderberg et al. In contrast to backward DNI, forward DNI  is used to break the “forward lock” within the network.
Revised: (2017) counter this potential problem by applying a bootstrapping principle in which a DNI module itself is used to guide learning; explicitly, the target gradient at timestep t is ¯ g t = (cid:80) Tτ>t ∂L τ h t + C T ( h T )

Original: In order to compute ∂L∂h i the activity h i must be calculated via the forward propagation of all preceding layers; until then any update of f i (and indeed, all subsequent layers) is locked to the forward pass of F i − 1. Forward DNI removes this restriction by providing a direct connection between the input x (or any other layer i − 1 ) and module i . This is achieved again with the use of a distinct network, a forward synthesiser C Fi , which attempts toreplicate the composition of steps between x and f i , so that C F ( x ) ≈ F i − 1( x ) .
Revised: ∂h T ∂h t , where T deﬁnes some limited horizon; note the interesting resemblance to the n -step return used in reinforcement learning algorithms.

Original: 2. C ORTICO - CEREBELLAR DNI MODELS
Revised: 2. C ORTICO - CEREBELLAR DNI MODELS

Original: Building directly on DNI we introduce a model of cortico-cerebellar computation (CC-DNI).
Revised: Building directly on DNI we introduce a model of cortico-cerebellar computation (CC-DNI).

Original: These models use a simple feedforward neural network, consistent with the mostly feedforward architecture of the cerebellum (Marr, 1969; Albus, 1971; Raymond and Medina, 2018).
Revised: These models use a simple feedforward neural network, consistent with the mostly feedforward architecture of the cerebellum (Marr, 1969; Albus, 1971; Raymond and Medina, 2018).

Original: The input layer of the cerebellum module C models the mossy ﬁber input onto granule cells (GCs; Fig. 1), which are the most numerous neurons in the brain ( > 50 billion in humans (Herculano-Houzel, 2009)).
Revised: The input layer of the cerebellum module C models the mossy ﬁber input onto granule cells (GCs; Fig. 1), which are the most numerous neurons in the brain ( > 50 billion in humans (Herculano-Houzel, 2009)).

Original: Consistent with this cerebellar dimensionality expansion, in our models we use M (cid:29) N , where M is the number of GCs and N the number of neurons in the main (cortical) network. (Fig. 1).
Revised: Consistent with this cerebellar dimensionality expansion, in our models we use M (cid:29) N , where M is the number of GCs and N the number of neurons in the main (cortical) network (Fig. 1).

Original: In particular, we use ratios MN ∼ 4 , consistent with experimental observations (Herculano-Houzel, 2009).
Revised: In particular, we use ratios MN ∼ 4 , consistent with experimental observations (Herculano-Houzel, 2009).

Revised: This is different to DNI, in which the synthesizer uses a single hidden layer with the same number of units as LSTM (for comparison in performance see ref to Fig. S2).

Original: In addition, the hidden layers and the output of C approximate the role of GCs and Purkinje cells, and the cerebellar output nuclei, respectively.
Revised: In addition, the hidden layers and the output of C approximate the role of GCs and Purkinje cells, and the cerebellar output nuclei, respectively.

Original: Cerebellar granule cells receive sparse input connections ( K ) with only around 3-7 synapses per GC (Herculano-Houzel, 2009; Eccles et al., 2013).
Revised: Cerebellar granule cells receive sparse input connections ( K ) with only around 3-7 synapses per GC (Herculano-Houzel, 2009; Eccles et al., 2013).

Original: These architectural constraints have led to sparse encoding and decorrelation theories of cerebellar function (Albus, 1971; Cayco-Gajic et al., 2017;
Revised: These architectural constraints have led to sparse encoding and decorrelation theories of cerebellar function (Albus, 1971; Cayco-Gajic et al., 2017;

Original: Schweighofer et al., 2001; Broomhead and Lowe, 1988; Billings et al., 2014; Litwin-Kumar et al., 2017; Sanger et al., 2019).
Revised: Schweighofer et al., 2001; Broomhead and Lowe, 1988; Billings et al., 2014; Litwin-Kumar et al., 2017; Sanger et al., 2019).

Original: Inspired on these features, we introduce a new model – sparse CC-DNI (sCC-DNI), for which we set a small number of incoming input connections K = 4 (Fig. 1).
Revised: Inspired on these features, we introduce a new model – sparse CC-DNI (sCC-DNI), for which we set a small number of incoming input connections K = 4 (Fig. 1).

Original: To measure decorrelation achieved by sCC-DNI we use the Pearson correlation and a population correlation metric r pop that has been shown to better capture cerebellar effects (Cayco-Gajic et al., 2017)
Revised: To measure decorrelation achieved by sCC-DNI we use the Pearson correlation and a population correlation metric r pop that has been shown to better capture cerebellar effects (Cayco-Gajic et al., 2017)

Original: [Equation]
Revised: [Equation]

Original: −
Revised: −

Original: [Equation]
Revised: [Equation]

Original:  where Z are the number of neurons being considered and λ i are the eigenvalues of the covariance matrix of the neuronal activity (e.g. h M ).
Revised: (cid:17) where Z are the number of neurons being considered and λ i are the eigenvalues of the covariance matrix of the neuronal activity (e.g. h M ).

Original: To manipulate the hidden correlations of the model, we adjust the variance of its input and recurrent weights by scaling each by a factor b with b (cid:54) =
Revised: To manipulate the hidden correlations of the model, we adjust the variance of its input and recurrent weights by scaling each by a factor b with b (cid:54) =

Original: 1 (see SM).
Revised: 1 (see SM).

Original: There is a strong similarity between CC-DNI models and the ﬂow of information in standard internal models of the cortico-cerebellar networks (Fig. 1, Table S1).
Revised: There is a strong similarity between CC-DNI models and the ﬂow of information in standard internal models of the cortico-cerebellar networks (Fig. 1, Table S1).

Original: Below we draw a parallel between classical cerebellar internal models and our CC-DNI models.
Revised: Below we draw a parallel between classical cerebellar internal models and our CC-DNI models.

Original: For simplicity, below and in our results we focus on the link between forward internal models and backward DNI (but see S1 for our interpretation of inverse internal model as forward DNI).
Revised: For simplicity, below and in our results we focus on the link between forward internal models and backward DNI (but see S1 for our interpretation of inverse internal model as forward DNI).

Original: 2.1. F ORWARD CEREBELLAR MODELS AS BACKWARD DNI
Revised: 2.1. F ORWARD CEREBELLAR MODELS AS BACKWARD DNI

Original: We propose that classical forward cerebellar models are equivalent to backward DNI.
Revised: We propose that classical forward cerebellar models are equivalent to backward DNI.

Original: In the forward model of sensorimotor control, the cerebellum receives an efferent copy of the motor command from the motor cortex (Miall et al., 1993; Ito, 1970), and sensory feedback from motor centres.
Revised: In the forward model of sensorimotor control, the cerebellum receives an efferent copy of the motor command from the motor cortex (Miall et al., 1993; Ito, 1970), and sensory feedback from motor centres.

Original: With these two inputs the forward model learns to predict the sensory consequences of motor commands (Fig. 1a).
Revised: With these two inputs the forward model learns to predict the sensory consequences of motor commands.

Original: Recently it has been argued that a similar predictive model can be applied to predict cerebral activity in brain regions such as the prefrontal cortex and the temporo-parietal cortex which are involved in planning of cognitive behaviour and decision making (Schmahmann et al., 2019; Wagner and Luo, 2020; Brissenden andSomers, 2019; Ito, 2008)
Revised: We argue that a similar predictive model can be applied to predict cerebral activity in brain regions such as the prefrontal cortex and the temporo-parietal cortex which are involved in planning of cognitive behaviour and decision making (Schmahmann et al., 2019; Wagner and Luo, 2020; Brissenden and

Original:  (Fig. 1b).
Revised: Somers, 2019; Ito, 2008) (Fig. 1a).

Original: Similarly to the forward model backward-DNI also receives an efferent copy, which is represented by the hidden activity from a given layer (or brain area)
Revised: Similarly to the forward model backward-DNI also receives an efferent copy, which is represented by the hidden activity from a given layer (or brain area)

Original: h i .
Revised: h i .

Original: From this input the synthesiser learns to predict the gradient with respect to the activity of layer
Revised: From this input the synthesiser learns to predict the gradient with respect to the activity of layer

Original: h
Revised: h

Original: i +1 (Fig.
Revised: i +1 (Fig.

Original: 1a,c).
Revised: 1a,c).

Original: In addition, we suggest that the cost function is computed by the inferior olive (e.g. L = ||
Revised: In addition, we suggest that the cost function is computed by the inferior olive (e.g. L = ||

Original: g M − ˆ g M || ; Fig.
Revised: g M − ˆ g M || ; Fig.

Original: 1a), which mediates learning in the cerebellum via climbing ﬁbres, consistent with existing cerebellar theoretical frameworks (Marr, 1969; Albus, 1971; Raymond and Medina, 2018).
Revised: 1a), which mediates learning in the cerebellum via climbing ﬁbres, consistent with existing cerebellar theoretical frameworks (Marr, 1969; Albus, 1971; Raymond and Medina, 2018).

Original: Finally, the prediction ˆ g M is sent back to the brain (e.g. as sensory prediction).
Revised: Finally, the prediction ˆ g M is sent back to the brain.

Original: In the brain such gradient signals are believed to be encoded through feedback activity between brain areas (Sacramento et al., 2018; Richards and Lillicrap, 2019).

Original: However, in our experiments we use gradients directly as that allows us to use existing deep learning libraries.

Original: Backward DNIs are of particular importance when using recurrent neural networks (RNNs), which learn to estimate future gradients from timestep t + 1 given current state at time t (Jaderberg et al., 2017)
Revised: Such backward DNIs are of particular importance when using recurrent neural networks (RNNs), which learn to estimate future gradients from timestep t + 1 given current state at time t (Jaderberg et al., 2017)

Original: (see SM for more details).
Revised: (Fig. 1b; see SM for more details).

Original: Our tasks use RNNs with  backpropagation through time (BPTT) as it allow us touse current deep learning libraries and to more clearly demonstrate the potential of the CC-DNI models.
Revised: Our tasks use RNNs with weak (i.e. with low truncations) backpropagation through time (BPTT) as it allow us to more clearly demonstrate the potential of the CC-DNI models.

Original: In the brain temporal gradient signals are likely computed forward in time, in which CC-DNI would play a similar role in predicting future gradients given the current state (Fig. S2; (Bellec et al., 2019)).
Revised: As is the case in DNIs the cerebellum relies on a bootstrapped cost function (Fig. 1c). We interpret such bootstrapping as being provided by the current or other cerebellar modules, consistent with the highly modular cerebellar architecture (Apps et al., 2018).

Original: Here we use LSTMs (Hochreiter and Schmidhuber, 1997) as a model of cortical networks, which have recently been mapped onto cortical microcircuit (Costa et al., 2017)
Revised: Here we use LSTMs (Hochreiter and Schmidhuber, 1997) as a model of cortical networks, which can be potentially linked to cortical microcircuit (Costa et al., 2017).

Original: . 2.1. E NCODING GRADIENTS IN THE BRAIN
Revised: 2.1. E NCODING GRADIENTS IN THE BRAIN

Original: Recent developments have introduced biologically plausible solutions to how the brain encodes gradients (Sacramento et al., 2018;
Revised: Recent developments have introduced biologically plausible solutions to how the brain encodes gradients (Lillicrap and Santoro, 2019;

Original: Richards and Lillicrap, 2019; Lillicrap and Santoro, 2019; Guerguiev et al., 2017;  Payeur et al., 2020; Ahmad et al., 2020).
Revised: Guerguiev et al., 2017; Sacramento et al., 2018; Richards and Lillicrap, 2019; Payeur et al., 2020; Ahmad et al., 2020).

Original: Regarding spatial backpropagation of gradients (i.e. across layers), Sacramento et al.
Revised: Regarding spatial backpropagation of gradients (i.e. across layers), Sacramento et al.

Original: (2018) demonstrated that biological networks do not need to explicitly send gradient information, but rather that this can be reconstructed locally in dendritic microcircuits.
Revised: (2018) demonstrated that biological networks do not need to explicitly send gradient information, but rather that this can be reconstructed locally in dendritic microcircuits.

Original: On the other hand Bellec et al.
Revised: On the other hand Bellec et al.

Original: (2019) showed that temporal gradients as used in BPTT are equivalent to using eligibility traces that transmitgradient information forward in time.
Revised: (2019) showed that temporal gradients as used in BPTT can be approximated by eligibility traces that transmit information forward in time.

Original: Both of these solutions can be incorporated into our framework, in which CC-DNI would predict feedback activity originating from upstream brain areas (as in Sacramento et al.
Revised: Both of these solutions can be incorporated into our framework, in which CC-DNI would predict feedback activity originating from upstream brain areas (as in Sacramento et al.

Original: and/or eligibility traces in RNNs (see S2 for a schematic; Bellec et al.
Revised: and/or eligibility traces in RNNs (see S2 for a schematic; Bellec et al.

Original: However, this is outside of the scope of the current paper and we leave this to future work (see more detailed discussion in SM).
Revised: However, this is outside of the scope of the current paper and we leave this to future work (see more detailed discussion in SM).

Original: 2. C OMPARISON WITH EXISTING CEREBELLAR COMPUTATIONAL MODELS
Revised: 2. C OMPARISON WITH EXISTING CEREBELLAR COMPUTATIONAL MODELS

Original: Over the past decades the regular circuitry of the cerebellum has driven theorists to model the cerebellar cortex in order to understand its function (Kaiser et al., 2018).
Revised: Over the past decades the regular circuitry of the cerebellum has driven theorists to model the cerebellar cortex in order to understand its function (Kaiser et al., 2018).

Original: The Marr-Albus models are particular popular models which view the local cerebellar circuitry as multi-layer perceptrons (Albus, 1971).
Revised: The Marr-Albus models are particular popular models which view the local cerebellar circuitry as multi-layer perceptrons (Albus, 1971).

Original: Another modelling framework that has been used is based on Kalman ﬁlters, which are also used to model the local circuitry to generate sensorimotor predictions (Porrill et al., 2013).
Revised: Another modelling framework that has been used is based on Kalman ﬁlters, which are also used to model the local circuitry to generate sensorimotor predictions (Porrill et al., 2013).

Original: These models have enabled signiﬁcant theoretical and experimental advances.
Revised: These models have enabled signiﬁcant theoretical and experimental advances.

Original: However, they do not consider potential interactions with the wider brain, in particular the neocortex, which are known to exist and appear to be critical for cerebellar function (Gao et al., 2018).
Revised: However, they do not consider potential interactions with the wider brain, in particular the neocortex, which are known to exist and appear to be critical for cerebellar function (Gao et al., 2018).

Original: Moreover, these models have so far only captured relatively simplistic (sensorimotor) tasks (e.g. eye-blink conditioning).
Revised: Moreover, these models have so far only captured relatively simplistic (sensorimotor) tasks (e.g. eye-blink conditioning).

Original: In contrast, our framework provides a solution to both of these issues, while being consistent with existing local models of the cerebellum.
Revised: In contrast, our framework provides a solution to both of these issues, while being consistent with existing local models of the cerebellum.

Original:  R ESULTS
Revised:  R ESULTS

Original: In this paper we focus our experiments on backward-CC-DNI models (cerebellar forward model;
Revised: In this paper we focus our experiments on backward-CC-DNI models (cerebellar forward model;

Original: Fig.
Revised: Fig.

Original: 1a,b) which we test on a range of task domains that have been shown to be cerebellar-dependent.
Revised: 1a,b) which we test on a range of task domains that have been shown to be cerebellar-dependent.

Original: Below we provide a summary of these different tasks and the respective results (see details in SM).
Revised: Below we provide a summary of these different tasks and the respective results (see details in SM)

Original: 3. S IMPLE SENSORIMOTOR TASK 
Revised: 3. S IMPLE SENSORIMOTOR TASK : TARGET REACHING e

Original: REACHING [Figure] by

Original: CC

Original: (orange) and

Original: DNI (cyan) models towards 7 targets using

Original: MSE.

Original: ( c ) Distance between model output and the targets.

Original: ( d ) MSE for multiple truncations T over the last 5 epochs.

Original: ( e ) MSE and correlation of hidden activity ( r h ) at epoch

Original: Inspired by classical sensorimotor studies in the cerebellum we ﬁrst test a simple target reaching task (Sanes et al., 1990;
Revised: Inspired by classical sensorimotor studies in the cerebellum we ﬁrst test a simple target reaching task (Sanes et al., 1990;

Original: Butcher et al., 2017; Nashef et al., 2019), in which given an input at time t 1 the network needs to reach a target (Fig.
Revised: Butcher et al., 2017; Nashef et al., 2019) in which given an input at time t 1 the network needs to reach a target (Fig.

Original: 2a and S8; see SM for more details).
Revised: 2a and S8; see SM for more details).

Original: In this task error signals are only available at the end of task, which must be associated with the initial input.
Revised: In this task error signals are only available at the end of task which must be associated with the initial input.

Original: In line with cerebellar experiments (Sanes et al., 1990;
Revised: In line with cerebellar experiments (Sanes et al., 1990;

Original: Butcher et al., 2017; Nashef et al., 2019), only the
Revised: Butcher et al., 2017; Nashef et al., 2019), only the

Original: CC-DNI models learn to reach the target with good accuracy, while LSTM still learns but much more slowly (Fig.
Revised: CC-DNI models learn to reach the target with good accuracy, while LSTM still learns but much more slowly (Fig.

Original: 2a,b).
Revised: 2a,b).

Original: Interestingly, sparse CC-DNI not only learns more quickly than CC-DNI (Fig. 2b,d, and S3), but also reaches the target more quickly (i.e. in fewer time steps) than the two other models (Fig. 2c).
Revised: Interestingly, sparse CC-DNI not only learns more quickly than CC-DNI (Fig. 2b,d, and S3), but also reaches the target more quickly (i.e. in fewer time steps) than the two other models (Fig. 2c).

Original: Consistent with cerebellar theories (Sanger et al., 2020; Cayco-Gajic et al., 2017; Billings et al., 2014) sCC-DNI helps when high correlations are present in the main network activity h (Fig. 2e), due to decorrelation of its estimated gradients ˆ g (Figs. 2f; see S6 for pop.
Revised: Consistent with cerebellar theories (Sanger et al., 2020; Cayco-Gajic et al., 2017; Billings et al., 2014) sCC-DNI helps when high correlations are present in the main network activity h (Fig. 2e), due to decorrelation of its estimated gradients ˆ g (Figs. 2f; see S6 for pop.

Original: corr.).
Revised: corr.).

Original: CC-DNI properties are particularly important when learning with low BPTT truncations (Fig. 2d), predicting that the cerebellum is particularly important for more temporally challenging tasks and consistent with DNI theory (Jaderberg et al., 2017).[Figure]
Revised: CC-DNI properties are particularly important when learning with low BPTT truncations (Fig. 2d), predicting that the cerebellum is particularly important for more temporally challenging tasks and consistent with DNI theory (Jaderberg et al., 2017).

Original:  A DVANCED SENSORIMOTOR TASK : SEQUENTIAL MNIST
Revised: 3. A DVANCED SENSORIMOTOR TASKS

Original: [Figure]
Revised: [Figure]

Original: rr
Revised: r (

Original: [Figure] 
Revised: [Figure] [Figure]

Revised: = = =

Original: [Figure]
Revised: [Figure]

Original: ld
Revised: r (

Original: [Figure]
Revised: [Figure]

Original: seqMNIST
Revised: )        10 truncation

Original: [Figure] 
Revised: [Figure] [Figure]

Revised: e rr o r ( % )

Original: [Figure]
Revised: [Figure]

Original:  Next, to test whether the target reaching results generalise to more realistic tasks we explored a range of more advanced sensorimotor tasks building on the classical MNIST dataset.3.2. S EQUENTIAL MNIST
Revised: Correlation of synthetic gradients r ˆ g produced by CC-DNIs ( Next, to test whether the target reaching results generalise to more realistic tasks we explored a range of more advanced sensorimotor tasks building on the classical MNIST dataset.

Original: We ﬁrst test the standard sequential MNIST (seqMNIST), in which the model gradually receives an MNIST image row by row before classifying the image as a number 0 − 9 (Fig. 3a)
Revised: We ﬁrst test the standard sequential MNIST (seqMNIST), in which the model gradually receives an MNIST image row by row before classifying the image as a number 0 − 9 (Fig. 3a)

Original: (Le et al., 2015).
Revised: (Le et al., 2015).

Original: As in the target reaching task, the loss function is only deﬁned at the end of the sequence, making this a challenging temporal task.
Revised: As in the target reaching task, the loss function is only deﬁned at the end of the sequence, making this a challenging temporal task.

Original: With truncated BPTT with modest truncation sizes T ≤ 6 , CC-DNI models learn more quickly and achieve better overall performance compared to a standard LSTM (Fig.
Revised: With truncated BPTT with modest truncation sizes T ≤ 6 , CC-DNI models learn more quickly and achieve better overall performance compared to a standard LSTM (Fig.

Original: 3b,c), in line with the previous task.
Revised: 3b,c), in line with the previous task.

Original: Interestingly, for this task we observe that the cerebellar architecture (i.e. high MN ratio, and sparse input connections K ) to be the best solution (Figs.
Revised: Interestingly, for this task we observe that the cerebellar architecture (i.e. high MN ratio, and sparse input connections K ) to be the best solution (Figs.

Original: S4, S5).
Revised: S4, S5).

Original: As in the target reaching task, sparse CC-DNI helps in the case of high correlations in the main network activity h , likely achieved by the decorrelation effect sCC-DNI has on the estimated gradients ˆ g (Figs. 3d, S7).
Revised: As in the target reaching task, sparse CC-DNI helps in the case of high correlations in the main network activity h , likely achieved by the decorrelation effect sCC-DNI has on the estimated gradients ˆ g (Figs. 3d, S7).

Original: 3.2. C OORDINATE SEQUENTIAL MNIST

Original: To see how well the above results generalise to a regression task in which the model has to reach a particular point (as in the target reaching task), we develop a coordinate seqMNIST (c-seqMNIST) task.
Revised: To see how well the above results generalise to a regression task in which the model has to reach a particular point (as in the target reaching task), we develop a coordinate seqMNIST (c-seqMNIST) task.

Original: As before, the model receives the image sequentially, but must now output one out of ten different equally spaced 2D coordinates (Fig. 3a).
Revised: As before, the model receives the image sequentially, but must now output one out of ten different equally spaced 2D coordinates (Fig. 3a).

Original: Note that this is a harder task as models have to learn to transform a complex input pattern towards a speciﬁc 2D point (Fig. 3b).
Revised: Note that this is a harder task as models have to learn to transform a complex input pattern towards a speciﬁc 2D point (Fig. 3b).

Original: Both DNI models outperformed the LSTM model.
Revised: Both DNI models outperformed the LSTM model.

Original: There is however notably less difference between sCC-DNI and
Revised: There is however notably less difference between sCC-DNI and

Original: CC-DNI between compared to the standard version of seqMNIST (see also the following tasks), which may suggests that the cerebellum is better suited for pattern separation tasks, in line with existing cerebellar pattern separation theories (Albus, 1971; Sanger et al., 2020; Cayco-Gajic et al., 2017; Litwin-Kumar et al., 2017).
Revised: CC-DNI between compared to the standard version of seqMNIST (see also the following tasks), which may suggests that the cerebellum is better suited for pattern separation tasks, in line with existing cerebellar pattern separation theories (Albus, 1971; Sanger et al., 2020; Cayco-Gajic et al., 2017; Litwin-Kumar et al., 2017).

Original: 3.2. D RAWING SEQUENTIAL MNIST
Revised: [Figure]

Original: Next, we test an explicit drawing task in which we trained the models to draw either a template digit (dd-seqMNIST) or a straight line (ld-seqMNIST) given an image of a digit (Fig. 3a).
Revised: Next, we test an explicit drawing task in which we trained the models to draw either a template digit (dd-seqMNIST) or a straight line (ld-seqMNIST) given an image of a digit (Fig. 3a).

Original: For these tasks, the loss function is computed at every point in time as L =
Revised: For these tasks, the loss function is computed at every point in time as L =

Original: ( y i − ˆ y i ) 2 ,
Revised: ( y i − ˆ y i ) 2 ,

Original: where y i and ˆ y
Revised: where y i and ˆ y

Original: i denote the desired and predicted coordinate at timestep i , respectively.
Revised: i denote the desired and predicted coordinate at timestep i , respectively.

Original: Due to the loss being computed at every timestep in both tasks, CC-DNI models are not as important and show a minimal improvement in performance (Figs.
Revised: Due to the loss being computed at every timestep in both tasks, CC-DNI models are not as important and show a minimal improvement in performance (Figs.

Original: 3b, S4).
Revised: 3b, S4).

Original: To demonstrate that the temporal resolution of the loss is important for
Revised: To demonstrate that the temporal resolution of the loss is important for

Original: CC-DNI, we varied the loss sparsity, giving the model a loss every γ timesteps for ld-seqMNIST.
Revised: CC-DNI, we varied the loss sparsity, giving the model a loss every γ timesteps for ld-seqMNIST.

Original: This setup resembles sensorimotor feedback which is typically periodic rather than continuous (Sanes et al., 1990; Synofzik et al., 2008).
Revised: This setup resembles sensorimotor feedback which is typically periodic rather than continuous (Sanes et al., 1990; Synofzik et al., 2008).

Original: As expected,for sparser losses (i.e. higher γ ) both DNI models improves learning (Fig. 3b).
Revised: As expected, sparser losses (higher γ ) improve learning (Fig. 3b).

Original: 3. C OGNITIVE TASK : C APTION GENERATION
Revised: 3. C OGNITIVE TASK : C APTION GENERATION

Original: Our framework does not only apply to sensorimotor tasks, but should generalise to virtually any task within the grasp of deep learning systems, or indeed the brain.
Revised: Our framework does not only apply to sensorimotor tasks, but should generalise to virtually any task within the grasp of deep learning systems, or indeed the brain.

Original: To demonstrate this and inspired by cognitive tasks in which cerebellar patients have shown deﬁcits (Gebhart et al., 2002)
Revised: To demonstrate this and inspired by cognitive tasks in which cerebellar patients have shown deﬁcits (Gebhart et al., 2002)

Original: we test our models in a caption generation task.
Revised: we test our models in a caption generation task.

Original: In this task the network needs to generate a textual description for a given image.
Revised: In this task the network needs to generate a textual description for a given image.

Original: All models have two components: a pretrained convolutional neural network (CNN) to extract a lower dimensional representation of the image, and an LSTM on top to generate text (Fig. 4a).
Revised: All models have two components: a pretrained convolutional neural network (CNN) to extract a lower dimensional representation of the image, and an LSTM on top to generate text (Fig. 4a).

Original: For simplicity and to draw comparison with previous tasks the DNI models contain one synthesiser at the RNN-level, but more could be added to help learning of the CNN.
Revised: For simplicity and to draw comparison with previous tasks the DNI models contain one synthesiser at the RNN-level, but more could be added to help learning of the CNN.

Original: We use a standard dataset (ILSVRC-2012-CLS (Russakovsky et al., 2015)) and the networks are trained to maximise the likelihood of each target word given an image (SM for more details).
Revised: We use a standard dataset (ILSVRC-2012-CLS (Russakovsky et al., 2015)) and the networks are trained to maximise the likelihood of each target word given an image (SM for more details).

Original: We ﬁnd that CC-DNI models exhibit faster learning (Fig. 4b) for truncation sizes T ≤ 10 (Fig. 4d) and better generalisation 2 (Fig. S13).
Revised: We ﬁnd that CC-DNI models exhibit faster learning (Fig. 4b) for truncation sizes T ≤ 10 (Fig. 4d) and better generalisation 2 (Fig. S13).

Original: All models produce reasonable captions for images unseen during training, but CC-DNI models tend to produce captions more semantically accurate than LSTMs (Figs.
Revised: All models produce reasonable captions for images unseen during training, but CC-DNI models tend to produce captions more semantically accurate than LSTMs (Figs.

Original: 4c, S12), consistent with cerebellar deﬁcits (Guell et al., 2015).
Revised: 4c, S12), consistent with cerebellar deﬁcits (Guell et al., 2015).

Original: In addition, we observe that sCC-DNI are more robust to initial conditions (Fig. 4d), consistent with the decorrelation property.
Revised: In addition, we observe that sCC-DNI are more robust to initial conditions (Fig. 4d), consistent with the decorrelation property.

Original: Moreover, consistent with bridging of gradient truncations provided by DNIs, we observe that CC-DNI helps learning in the presence of long captions, whereas sCC-DNI helps mostly for the most common caption lengths (Fig. 4d), suggesting that sCC-DNI better captures the data distribution. 
Revised: Moreover, consistent with bridging of gradient truncations provided by DNIs, we observe that CC-DNI helps learning in the presence of long captions, whereas sCC-DNI helps mostly for the most common caption lengths (Fig. 4d), suggesting that sCC-DNI better captures the data distribution. 3. M ODEL PREDICTIONS [Figure]

Revised: Our model makes numerous predictions for experimental neuroscience, some of which can be compared with existing observations.

Original: C ONCLUSIONS
Revised: First, we tested how important are the exact cerebellar expansion M/N ∼ 5 and sparsity parameters ( K = 4 ) observed experimentally. We ﬁnd that the parameters found experimentally provide a good combination to facilitate learning in the pattern recognition sequential MNIST task (Fig. 5a). This region of good hyperparameters is much wider in regression tasks such as the coordinate sMNIST (Fig. 5b) or other variants (Fig. S4).

Revised: This predicts that the cerebellum has evolved to speciﬁcally facilitate pattern recognition tasks

Revised: [Figure]

Revised: A common experimental paradigm is to do cerebellar ablation experiments.

Revised: We used our model to generate speciﬁc predictions in terms of behavioural outcome with total cerebellar ablations at different points during learning.

Revised: We ﬁnd that ablations generally impair learning (Fig.

Revised: 6a-b), but that this impairment reduces over learning and once the main network no longer needs the cerebellar estimations its presence can impair learning (Fig.

Revised: 6a-b).

Revised: Next, we used our model to make predictions in terms of expected correlations between the neocortex and the cerebellum.

Revised: Because of the complex temporal gradients in the RNN case here we focus on a simple feedforward network trained on MNIST.

Revised: In addition, we use this example to demonstrate that when approximating activity directly instead of gradients, as we propose to happen in biology, the cerebellum output converges to some non-zero value (Fig. 7, top).

Revised: We ﬁnd that correlations between the main network and the cerebellum increase over learning for neurons with ﬁnal high correlations whereas correlations decrease for neurons with initial high correlations (Fig. 7, bottom), consistent with recent experimental observations (Wagner et al., 2019).

Revised: Moreover, we also observe that the variance of total correlations decreases over learning (not shown).

Revised:  C ONCLUSIONS AND DISCUSSION

Original: Our work introduces a deep learning model of cortico-cerebellar function,  whichsuggests that the cerebellum functions as a decoupling machine.
Revised: most correlated We introduced a deep learning model of cortico-cerebellar function, in which the cerebellum acts as a decoupling machine.

Original: We demonstrate that this idea is directly related to classical cerebellarforward and inverse models (Marr, 1969; Albus, 1971; Raymond and Medina, 2018), but with the key prediction that the cerebellum decouples activity propagation across the brain, speeding up credit assignment across multiple brain areas. 
Revised: We demonstrate that this idea is directly related to classical cerebellar models (Marr, 1969; Albus, 1971), but with the key prediction that the cerebellum decouples activity propagation across the brain, speeding up credit assignment across multiple brain areas. Our model is also a step towards solving the temporal credit assignment problem in the brain, as the CC-DNI models reduce the need for strong BPTT (i.e. without truncations).

Original: Temporal credit assignment is directly related to the ability of networks to maintain input information (Bellec et al., 2019).
Revised: Temporal credit assignment is directly related to the ability of networks to maintain input information (Bellec et al., 2019).

Original: Therefore, our results on truncated BPTT suggest that the cerebellum becomes increasingly important as task difﬁculty increases due to the inability of cortical networks to maintain a information for longer than hundreds of millisecondsuntil an error signal arrives, consistent with recent observations (Deverett et al., 2019).[Figure]
Revised: Therefore, our results on truncated BPTT suggest that the cerebellum becomes increasingly important as task difﬁculty increases due to the inability of cortical networks to maintain a information for longer than hundreds of milliseconds (Deverett et al., 2019).

Original: Our results are largely consistent with observed cognitive deﬁcits in cerebellar patients, such as in language (Guell et al., 2015), which cannot be attributed directly to deﬁcits in motor control (Stoodley and Schmahmann, 2009) and can in principle be used to develop behavioural strategies optimised to reduce deﬁcits of cerebellar patients (Donchin and Timmann, 2019).
Revised: Our results are largely consistent with observed cognitive deﬁcits in cerebellar patients, such as in language (Guell et al., 2015), which cannot be attributed directly to deﬁcits in motor control (Stoodley and Schmahmann, 2009).

Original: Because of the explicit use of labels (or desired targets) here we have relied on supervised learning settings 3 , but the same framework can be easily applied in reinforcement and other unsupervised learning settings (Jaderberg et al., 2017), which do not require an explicit teacher.
Revised: Because of the explicit use of labels (or desired targets) here we have relied on supervised learning settings 3 , but the same framework can be easily applied in reinforcement and other unsupervised learning settings (Jaderberg et al., 2017), which do not require an explicit teacher.

Original: Indeed the cerebellar model we propose here is of particular relevance for reinforcement learning due to the prevalence of sparse and delayed rewards, consistent with recent observations of reward-related signals in the cerebellum (Sendhilnathan et al., 2020).
Revised: Indeed the cerebellar model we propose here is of particular relevance for reinforcement learning due to the prevalence of sparse and delayed rewards, consistent with recent observations of reward-related signals in the cerebellum (Sendhilnathan et al., 2020).

Original: Finally, our work makes numerous predictions and opens the exciting door to explicit comparison between the representations learned in the proposed cortico-cerebellar model and those recently observed across a range ofmotor and cognitive tasks (King et al., 2019; Ashida et al., 2019). 
Revised: Our work makes numerous predictions and opens the exciting possibility of comparing CC-DNI models with recent studies across a range of tasks (King et al., 2019; Ashida et al., 2019). We performed three separate experiments to highlight predictions made by the model: (i) we used ablations to show how behavioural performance is shaped by cerebellar predictions, (ii) a study on cerebellar expansion and sparsity that suggests that the cerebellum may have evolved to facilitate learning in speciﬁc tasks and (iii) study how correlations between main network and the cerebellum develop over learning (see Section 5). Moreover, the model also predicts which connections should project to GCs (source brain area activity) and the inferior olive (target brain area activity). In addition, as demonstrated by Czarnecki et al.

Original: On thedeep learning side, with sparse CC-DNI we have only just scratched the surface of new DNI models  that could be developed inspired by cortico-cerebellar networks.
Revised: (2017) the CC-DNI model also predicts that without a cerebellum the neocortical representations should become less distributed across different brain areas. Furthermore, as shown by Jaderberg et al. (2017) these models enable a network to be trained in a fully decoupled fashion, which means that it can update asynchronously. Given that the brain is asynchronous this may be a fundamental beneﬁt of having a cerebellar system.

Original: Overall, we put forward a novel cerebellar theory that is consistent with experimental observations while generating several experimental predictions and a window of interactions with deep learning research.
Revised: Finally, our framework can also inspire new deep learning DNI systems. For example, a key feature of cerebellar architectures are the many functionally separate modules (Apps et al. ; Cerminara and Apps (2011); Suvrathan et al. (2016); Fig. 1c) to speed-up learning, generalising the ideas used by Jaderberg et al.

Original: A E XPERIMENTAL DETAILS
Revised: A E XPERIMENTAL DETAILS

Original: In each of our tasks we use a long-short term memory network (LSTM; (Hochreiter and Schmidhuber, 1997)) as the main “cortical” network (Costa et al., 2017), and a simple feedforward network of one (sensorimotor tasks) or two hidden layers (caption generation) as the synthesiser “cerebellar” network.
Revised: In each of our tasks we use a long-short term memory network (LSTM; (Hochreiter and Schmidhuber, 1997)) as the main “cortical” network (Costa et al., 2017), and a simple feedforward network of one (sensorimotor tasks) or two hidden layers (caption generation) as the synthesiser “cerebellar” network.

Original: As in (Jaderberg et al., 2017) the cerebellar network ﬁrst predicts gradients for both the memory cell and output state of the LSTM (i.e. for each neuron in the main network the cerebellar network receives two inputs), so that the ‘cerebellar’ input and output size is two times the number of LSTM units, and this synthetic gradient ˆ g is then scaled by a factor of 0.1 before being used by the main model for stability purposes.
Revised: As in (Jaderberg et al., 2017) the cerebellar network ﬁrst predicts gradients for both the memory cell and output state of the LSTM (i.e. for each neuron in the main network the cerebellar network receives two inputs), so that the ‘cerebellar’ input and output size is two times the number of LSTM units, and this synthetic gradient ˆ g is then scaled by a factor of 0.1 before being used by the main model for stability purposes.

Original: The ﬁnal ‘readout’ of the model is a (trained) linear sum of the LSTM output states.
Revised: The ﬁnal ‘readout’ of the model is a (trained) linear sum of the LSTM output states.

Original: All networks are optimised using ADAM (Kingma and Ba, 2014)
Revised: All networks are optimised using ADAM (Kingma and Ba, 2014)

Original: (see learning rates below).
Revised: (see learning rates below).

Original: In each experiment all initial LSTM parameters are drawn from the uniform distribution
Revised: In each experiment all initial LSTM parameters are drawn from the uniform distribution

Original: U ( 1 √ n LSTM , 1 √ n LSTM ) , where n LSTM is the number of LSTM units.
Revised: U ( 1 √ n LSTM , 1 √ n LSTM ) , where n LSTM is the number of LSTM units.

Original: Other than the ﬁnal layer, as in (Jaderberg et al., 2017), the feedforward weights of the cerebellar network are initialised according to U ( − b k , b k ) where b k denotes the “kaiming bound” as computed in (He et al., 2015)
Revised: Other than the ﬁnal layer, as in (Jaderberg et al., 2017), the feedforward weights of the cerebellar network are initialised according to U ( − b k , b k ) where b k denotes the “kaiming bound” as computed in (He et al., 2015)

Original: (slope a = √ 5 ), and the biases are draw from U ( 1 √ n inp , 1 √ n inp ) , where n inp denotes the input size of the layer.
Revised: (slope a = √ 5 ), and the biases are draw from U ( 1 √ n inp , 1 √ n inp ) , where n inp denotes the input size of the layer.

Original: As in (Jaderberg et al., 2017), the last layer (both weights and bias) of the cerebellar network is zero-initialised, so that the produced synthetic gradients at the start are zero.
Revised: As in (Jaderberg et al., 2017), the last layer (both weights and bias) of the cerebellar network is zero-initialised, so that the produced synthetic gradients at the start are zero.

Original: A range of correlations (here quantiﬁed as Pearson or population correlation coefﬁcient) for the LSTM activity h can be achieved naturally through different random initialisations (accessed through different random seeds), but can be also controlled by the scaling of initial parameters of the LSTM with a bias factor b , where b > 1 and b < 1 induces greater and smaller variability in parameter values respectively.
Revised: A range of correlations (here quantiﬁed as Pearson or population correlation coefﬁcient) for the LSTM activity h can be achieved naturally through different random initialisations (accessed through different random seeds), but can be also controlled by the scaling of initial parameters of the LSTM with a bias factor b , where b > 1 and b < 1 induces greater and smaller variability in parameter values respectively.

Original: The results demonstrating the effect of correlation (Figs 2f,g, 3d) are obtained across 10 random seeds, for multiple bias factor values b ∈ [0 . 01 , 0 . 1 , 0 . 5 , 1 , 2] .
Revised: The results demonstrating the effect of correlation (Figs 2f,g, 3d) are obtained across 10 random seeds, for multiple bias factor values b ∈ [0 . 01 , 0 . 1 , 0 . 5 , 1 , 2] .

Original: During learning, backpropogation through time (BPTT) takes place strictly within distinct truncations, and computed gradients are not propagated between them: truncated BPTT.
Revised: During learning, backpropogation through time (BPTT) takes place strictly within distinct truncations, and computed gradients are not propagated between them: truncated BPTT.

Original: As soon as the forward computation of a given truncation is completed, assuming a corresponding (synthetic) loss gradient is available, the model parameters are updated; that is, at a resolution of the preset truncation size, there is online learning .
Revised: As soon as the forward computation of a given truncation is completed, assuming a corresponding (synthetic) loss gradient is available, the model parameters are updated; that is, at a resolution of the preset truncation size, there is online learning .

Original: We split the original input into truncations as follows.
Revised: We split the original input into truncations as follows.

Original: Given an input sequence of N timesteps x 1 , x 2 , . . .
Revised: Given an input sequence of N timesteps x 1 , x 2 , . . .

Original: , x N and a truncation size T , we divide the sequence into T sized truncations with any remainder going to the last truncation.
Revised: , x N and a truncation size T , we divide the sequence into T sized truncations with any remainder going to the last truncation.

Original: In other words, the sequence is now made up truncations of ( x 1 , . . .
Revised: In other words, the sequence is now made up truncations of ( x 1 , . . .

Original: , x T ) , ( x T +1 , . . .
Revised: , x T ) , ( x T +1 , . . .

Original: , x 2 T ) , . . .
Revised: , x 2 T ) , . . .

Original: , ( x ( m − 1) T +1 , . . .
Revised: , ( x ( m − 1) T +1 , . . .

Original: , x mT ) , ( x N − r , . . .
Revised: , x mT ) , ( x N − r , . . .

Original: , x N ) , where 
Revised: , x N ) , where N = mT

Original: N = mT + r for positive integers m, r with 0 ≤ r < T .
Revised: + r for positive integers m, r with 0 ≤ r < T .

Original: Note that, along with the value T , how well the sequence is divided into truncations (i.e. values m, r ) is an important factor for learning.
Revised: Note that, along with the value T , how well the sequence is divided into truncations (i.e. values m, r ) is an important factor for learning.

Original: Unless stated otherwise, reported error/loss values are on a held-out validation set at the end of learning.
Revised: Unless stated otherwise, reported error/loss values are on a held-out validation set at the end of learning.

Original: All experiments were conducted using PyTorch.
Revised: All experiments were conducted using PyTorch.

Original: A. T ARGET REACHING TASK
Revised: A. T ARGET REACHING TASK

Original: In the target reaching task, an LSTM network receives a discrete input cue which signals the network to move to a particular target location in 2D space.
Revised: In the target reaching task, an LSTM network receives a discrete input cue which signals the network to move to a particular target location in 2D space.

Original: In the results presented in the main text (Fig. 2) we set 6 distinct non-zero input-target pairs { ( x i , y i ) } 6 i =1 , where each input x i is a (one dimensional) integer ∈ {± 1 , ± 2 , ± 3 } , and the targets { y i } 6 i =1 lie equidistantly on a circle centred on the origin with radius 10.
Revised: In the results presented in the main text (Fig. 2) we set 6 distinct non-zero input-target pairs { ( x i , y i ) } 6 i =1 , where each input x i is a (one dimensional) integer ∈ {± 1 , ± 2 , ± 3 } , and the targets { y i } 6 i =1 lie equidistantly on a circle centred on the origin with radius 10.

Original: Once an input cue is received at timestep t 0 , the model receives no new information (i.e. all future input is zero) and has 10 timesteps to move to the corresponding target location.
Revised: Once an input cue is received at timestep t 0 , the model receives no new information (i.e. all future input is zero) and has 10 timesteps to move to the corresponding target location.

Original: The model is trained to minimise the mean squared error (MSE) between its ﬁnal output and the cue-based target.
Revised: The model is trained to minimise the mean squared error (MSE) between its ﬁnal output and the cue-based target.

Original: In addition, if no input cue is provided ( x = 0 ), the network is trained to not move at all ( y = 0 ).
Revised: In addition, if no input cue is provided ( x = 0 ), the network is trained to not move at all ( y = 0 ).

Original: Note therefore that only the ﬁnal model output ‘matters’ in the sense that it deﬁnes model performance.
Revised: Note therefore that only the ﬁnal model output ‘matters’ in the sense that it deﬁnes model performance.

Original: The cortical network has one hidden layer of 10 LSTM units (i.e. 20 when including the memory cells); unless stated otherwise, the cerebellar network contains one hidden layer of 80 hidden neurons.
Revised: The cortical network has one hidden layer of 10 LSTM units (i.e. 20 when including the memory cells); unless stated otherwise, the cerebellar network contains one hidden layer of 80 hidden neurons.

Original: The initial learning rate is set to 0.001.
Revised: The initial learning rate is set to 0.001.

Original: Each epoch comprises one batch of 50 randomised examples.
Revised: Each epoch comprises one batch of 50 randomised examples.

Original: Model performance is averaged over 10 random seeds (with error bars), where each seed determines the initial weights of the network.
Revised: Model performance is averaged over 10 random seeds (with error bars), where each seed determines the initial weights of the network.

Original: Truncation values T ∈ [2 , 3 , 4 , 5 , 7] are considered.
Revised: Truncation values T ∈ [2 , 3 , 4 , 5 , 7] are considered.

Original: Furthermore, to see how the above conditions generalise we consider variants of the task with 1.
Revised: Furthermore, to see how the above conditions generalise we consider variants of the task with 1.

Original: different numbers of cue/target pairs (see Fig.
Revised: different numbers of cue/target pairs (see Fig.

Original: S8a and b) and 2.
Revised: S8a and b) and 2.

Original: higher dimensional input (see Fig.
Revised: higher dimensional input (see Fig.

Original: S8c and d).
Revised: S8c and d).

Original: More speciﬁcally, we consider two cases where the model receives two-dimensional (Fig.
Revised: More speciﬁcally, we consider two cases where the model receives two-dimensional (Fig.

Original: S8c) and 28-dimensional (Fig.
Revised: S8c) and 28-dimensional (Fig.

Original: S8d) input.
Revised: S8d) input.

Original: For the former the model architecture remains the same except now the model receives at time t 

Revised: For the former the model architecture remains the same except now the model receives at time t

Original: the (two-dimensional) target values scaled by a factor 0.1 as input.
Revised: the (two-dimensional) target values scaled by a factor 0.1 as input.

Original: For the latter we have an LSTM with 30 hidden units and the input x is now a binary vector of size 28.
Revised: For the latter we have an LSTM with 30 hidden units and the input x is now a binary vector of size 28.

Original: In order to increase the task difﬁculty for these variants of higher dimensional input, we add to the input Gaussian noise with variance 0.1 at each timestep.
Revised: In order to increase the task difﬁculty for these variants of higher dimensional input, we add to the input Gaussian noise with variance 0.1 at each timestep.

Original: A. S EQUENTIAL MNIST TASKS
Revised: A. S EQUENTIAL MNIST TASKS

Original: For each seqMNIST based task the model receives the same temporal MNIST input, and the tasks are only differentiated by the model output.
Revised: For each seqMNIST based task the model receives the same temporal MNIST input, and the tasks are only differentiated by the model output.

Original: Given a 28 × 28 MNIST image, at timestep i the model receives the pixels from row i of the image, so that there are 28 total timesteps and an input size of 28.
Revised: Given a 28 × 28 MNIST image, at timestep i the model receives the pixels from row i of the image, so that there are 28 total timesteps and an input size of 28.

Original: Only the ﬁnal model output matters for the seqMNIST and c-seqMNIST tasks - that is, the loss is solely deﬁned at the end of the sequence - whereas the model output is assessed at every timestep for dd-seqMNIST and ld-seqMNIST.
Revised: Only the ﬁnal model output matters for the seqMNIST and c-seqMNIST tasks - that is, the loss is solely deﬁned at the end of the sequence - whereas the model output is assessed at every timestep for dd-seqMNIST and ld-seqMNIST.

Original: In each case we have one hidden layer of 30 LSTM units in the main model and one hidden layer of 300 (unless stated otherwise) hidden units in the feedforward cerebellar network.
Revised: In each case we have one hidden layer of 30 LSTM units in the main model and one hidden layer of 300 (unless stated otherwise) hidden units in the feedforward cerebellar network.

Original: Data was presented in batches of 50 with an initial learning rate of 0.0001.
Revised: Data was presented in batches of 50 with an initial learning rate of 0.0001.

Original: Training and validation data was assigned a 4 : 1 split, containing 48000 and 12000 distinct image/number pairs respectively.
Revised: Training and validation data was assigned a 4 : 1 split, containing 48000 and 12000 distinct image/number pairs respectively.

Original: The truncation values T ∈ [2 , 3 , 4 , 5 , 7 , 10] are considered.
Revised: The truncation values T ∈ [2 , 3 , 4 , 5 , 7 , 10] are considered.

Original: Model performance averaged (with error bars) over 3 random seeds for weight initialisation.
Revised: Model performance averaged (with error bars) over 3 random seeds for weight initialisation.

Original: [Equation]
Revised: A.2. S EQ MNIST

Original: This is the standard form of seqMNIST, where at the end of the presentation of the image the model must classify in the image as a number between 0 and 9.
Revised: This is the standard form of seqMNIST, where at the end of the presentation of the image the model must classify in the image as a number between 0 and 9.

Original: The output of the model is a vector probabilities of size 10 (one entry for each number), and the model was trained to maximise the likelihood of the correct number.
Revised: The output of the model is a vector probabilities of size 10 (one entry for each number), and the model was trained to maximise the likelihood of the correct number.

Original: A.2. C - SEQ MNIST
Revised: A.2. C - SEQ MNIST

Original: In this variant each number 0-9 MNIST image is allocated an xy position on the edge of a circle centred at 0, with the position uniquely determined by the digit represented by the image (Fig. 3a).
Revised: In this variant each number 0-9 MNIST image is allocated an xy position on the edge of a circle centred at 0, with the position uniquely determined by the digit represented by the image (Fig. 3a).

Original: With the model output then a vector of size 2, the training loss is deﬁned at the end by the mean squared error (MSE) between the ﬁnal output of the model and the target coordinate; since the radius of the circle is length 10 (arbitrary unit), a static model, where the model output remains at 0, would have a constant MSE of 100.
Revised: With the model output then a vector of size 2, the training loss is deﬁned at the end by the mean squared error (MSE) between the ﬁnal output of the model and the target coordinate; since the radius of the circle is length 10 (arbitrary unit), a static model, where the model output remains at 0, would have a constant MSE of 100.

Original: A.2. DD - SEQ MNIST
Revised: A.2. DD - SEQ MNIST

Original: Like c-seqMNIST, in this variant the model outputs 2D coordinates, but now the model must learn to predict an entire sequence of coordinates { ˆ y i } 28 i =1 .
Revised: Like c-seqMNIST, in this variant the model outputs 2D coordinates, but now the model must learn to predict an entire sequence of coordinates { ˆ y i } 28 i =1 .

Original: The target sequence { y i } 28 i =1 can be considered a drawing, and in this case resembles the shape of the number itself (Fig.
Revised: The target sequence { y i } 28 i =1 can be considered a drawing, and in this case resembles the shape of the number itself (Fig.

Original: 3c; number 0 not shown).
Revised: 3c; number 0 not shown).

Original: The model is then trained at each timestep to minimise the MSE between the model output at that time and corresponding target point, so that the loss at timestep t deﬁned as MSE( y t , ˆ y t ) .
Revised: The model is then trained at each timestep to minimise the MSE between the model output at that time and corresponding target point, so that the loss at timestep t deﬁned as MSE( y t , ˆ y t ) .

Original: For each number, the corresponding target drawing lies in [0 , 1] 2 , with the gap between each successive point roughly the same.
Revised: For each number, the corresponding target drawing lies in [0 , 1] 2 , with the gap between each successive point roughly the same.

Original: To prevent the model being too harshly judged at timestep 1, all drawings begin in the top left corner (0 , 1) (apart from the drawing of1 which begins slightly beneath/to the right).
Revised: To prevent the model being too harshly judged at timestep 1, all drawings begin in the top left corner (0 , 1) (apart from the drawing of

Original: MSE scores are reported as 100 times their raw values to ease comparison with c-SEQMNIST/dd-SEQMNIST.

Original: A.2. LD - SEQ MNIST
Revised: A.2. LD - SEQ MNIST

Original: This variant can be thought of as a mixture between c-seqMNIST and dd-seqMNIST.
Revised: This variant can be thought of as a mixture between c-seqMNIST and dd-seqMNIST.

Original: Like ddseqMNIST the model is constantly assessed and must produce a desired shape, but now the desired set of points form an equally spaced line where the start point is the origin and the end point is determined by the number-to-coordinate mapping of c-seqMNIST.
Revised: Like dd-seqMNIST the model is constantly assessed and must produce a desired shape, but now the desired set of points form an equally spaced line where the start point is the origin and the end point is determined by the number-to-coordinate mapping of c-seqMNIST.

Original: As with dd-seqMNIST, the loss is deﬁned at each timestep by the difference between the model output and appropriate point on the line.
Revised: As with dd-seqMNIST, the loss is deﬁned at each timestep by the difference between the model output and appropriate point on the line.

Original: A. C APTION G ENERATION
Revised: A. C APTION G ENERATION

Original: The architecture for the caption generation task consists of a pretrained CNN coupled with an RNN (LSTM in this case).
Revised: The architecture for the caption generation task consists of a pretrained CNN coupled with an RNN (LSTM in this case).

Original: The synthesiser (cerebellar network) only communicates to the LSTM.
Revised: The synthesiser (cerebellar network) only communicates to the LSTM.

Original: The

Original:  LSTM network has one layer of 256 LSTM units and the cerebellar network has two hidden layers of 1024 neurons.
Revised: The LSTM network has one layer of 256 LSTM units and the cerebellar network has two hidden layers of 1024 neurons.

Original: The dynamics from image to caption is as follows.
Revised: The dynamics from image to caption is as follows.

Original: As part of image preprocessing and data augmentation, a given image is randomly cropped to size 224 × 224 , ﬂipped horizontally with even chance, and appropriately normalised to be given to a pretrained resnet model
Revised: As part of image preprocessing and data augmentation, a given image is randomly cropped to size 224 × 224 , ﬂipped horizontally with even chance, and appropriately normalised to be given to a pretrained resnet model

Original: (He et al., 2016).
Revised: (He et al., 2016).

Original: A feature vector X of size 256 is thus obtained and is passed to the LSTM at timestep 0.
Revised: A feature vector X of size 256 is thus obtained and is passed to the LSTM at timestep 0.

Original: The LSTM is subsequently presented the “gold standard” caption { w i } ni =1 one word per timestep, each time learning to predict the next word; i.e., at timestep t the model learns P ( w t | X, { w i } t − 1 i =1 ) .
Revised: The LSTM is subsequently presented the “gold standard” caption { w i } ni =1 one word per timestep, each time learning to predict the next word; i.e., at timestep t the model learns P ( w t | X, { w i } t − 1 i =1 ) .

Original: The network simultaneously learns a word embedding so that each word w i is ﬁrst transformed to a feature vector of size 256 before being served as input.
Revised: The network simultaneously learns a word embedding so that each word w i is ﬁrst transformed to a feature vector of size 256 before being served as input.

Original: With a preset vocabulary of 9956 distinct words, the ﬁnal output of the model ( P ( w i ) ) is a probability vector of size 9956.
Revised: With a preset vocabulary of 9956 distinct words, the ﬁnal output of the model ( P ( w i ) ) is a probability vector of size 9956.

Original: We ﬁnd that though DNI could help without any explicit method for regularisation (not shown), all models are prone to overﬁtting.
Revised: We ﬁnd that though DNI could help without any explicit method for regularisation (not shown), all models are prone to overﬁtting.

Original: For this reason, we apply dropout (during training) on the input to the LSTM, where a given input element is set to zero with p = 0 . 5 probability.
Revised: For this reason, we apply dropout (during training) on the input to the LSTM, where a given input element is set to zero with p = 0 .probability.

Original: Once training is complete the models can generate their own unique captions to previously unseen images (Figs. 4, S12).
Revised: Once training is complete the models can generate their own unique captions to previously unseen images (Figs. 4, S12).

Original: Given an image at timestep 0„ the model applies a ‘greedy search’ where the output of the model at timestep i is the word with the highest probability, and the same word is then provided as input to the model at timestep i + 1 .
Revised: Given an image at timestep 0„ the model applies a ‘greedy search’ where the output of the model at timestep i is the word with the highest probability, and the same word is then provided as input to the model at timestep i + 1 .

Original: In this way the model eventually outputs an entire sequence of words which forms a caption.
Revised: In this way the model eventually outputs an entire sequence of words which forms a caption.

Original: In the (highly) rare case where the model generates a sequence of > 20 words, we consider only the ﬁrst 20 words as its caption.
Revised: In the (highly) rare case where the model generates a sequence of > 20 words, we consider only the ﬁrst 20 words as its caption.

Original: The coco training set ILSVRC-2012-CLS (Russakovsky et al., 2015) holds 414113 total imagecaption pairs with 82783 unique images while the held-out validation set holds 202654 withunique images; note that each image therefore has ∼ 5 distinct gold standard captions.
Revised: The coco training set ILSVRC-2012-CLS (Russakovsky et al., 2015) holds 414113 total image-caption pairs with 82783 unique images while the held-out validation set holds 202654 with 40504 unique images; note that each image therefore has ∼ 5 distinct gold standard captions.

Original: Training takes place in batches of 100 image/caption pairs, with learning rate of 0.001.
Revised: Training takes place in batches of 100 image/caption pairs, with learning rate of 0.001.

Original: Model performance averaged (with error bars) over 5 random seeds for weight initialisation.
Revised: Model performance averaged (with error bars) over 5 random seeds for weight initialisation.

Original: In order to judge the models beyond their learning curves in BPW, we quantify their ability to generate captions using a variety of language modelling metrics popular in the realm of language evaluation (image captioning, machine translation, etc).
Revised: In order to judge the models beyond their learning curves in BPW, we quantify their ability to generate captions using a variety of language modelling metrics popular in the realm of language evaluation (image captioning, machine translation, etc).

Original: In particular, we compare model-generated captions against the gold standard captions using the metrics BLEU, Rouge-L, METEOR, CIDEr, SPICE (Papineni et al., 2002; Lin, 2004; Denkowski and Lavie, 2014; Vedantam et al., 2015; Anderson et al., 2016).
Revised: In particular, we compare model-generated captions against the gold standard captions using the metrics BLEU, Rouge-L, METEOR, CIDEr, SPICE (Papineni et al., 2002; Lin, 2004; Denkowski and Lavie, 2014; Vedantam et al., 2015; Anderson et al., 2016).

Original: We use BLEU_4 (i.e. consider only n -grams for n ≤ 4 ) as the default BLEU score (see Fig4), though we also look at BLEU_1, BLEU_2, BLEU_3 (Fig S15. 
Revised: We use BLEU_4 (i.e. consider only n -grams for n ≤ 4 ) as the default BLEU score (see Fig 4), though we also look at BLEU_1, BLEU_2, BLEU_3 (Fig S15. B F ORWARD DNI: MNIST

Revised: We performed a simple implementation of forward DNI - where now the cerebellar network is trained to predict the activity (not gradients) of the main network - on the MNIST database.

Original: B C OMPARISON WITH OTHER MODELS OF THE CEREBELLUM
Revised: With no temporal dimension, the main network is a feedforward neural network which, given a 28 × 28 MNIST image, will tune its output (a vector of size 10, where each value corresponds to a possible digit) so as to maximise the probability of the associated number (or equally minimise the cross-entropy loss - Fig. 7, top). In our case the feedforward network has two hidden layers, each of size 256; after each hidden layer a batch normalisation is applied as well as the non-linear ReLU function. We apply a feedforward cerebellar network with one hidden layer of size 256 which receives the original MNIST input x and is trained to predict the activity of the second hidden layer of the main network h 2 ; the synthesiser loss used to train the synthesiser parameters then becomes L C :

Original: In section 2 of the main text we argue that there is a strong similarity between the ﬂow of information in DNI and classical internal models of the cerebellum.
Revised: = || h 2 − C ( x ) || . The cerebellar module can then break the “forward lock” by allowing the main model to bypass its forward computations up until its second layer and instead use C ( x ) to make its prediction. As before, each network is trained using gradient descent with ADAM.

Original: In Table S1 we compare explicitly the input and output components of both types of models. B.0. I NVERSE AND FORWARD DNI CEREBELLAR MODELS
Revised: Correlations reported are computed using the Pearson correlation. For ease of display reported losses, activity and correlations in Fig.

Original: The inverse model of the cerebellum in motor control computes the motor predictions given a target (or desired) response (Ito, 1970).
Revised: 7 are smoothed over batches using a Savitzky-Golay ﬁlter (window size 19, polynomial order 3).

Original: Feedback errors are derived by comparing a motor command from the motor cortex with the predicted motor command in the cerebellum. We propose that the inverse model is equivalent to forward DNI, as it aims to predict the incoming activity at a given layer (e.g. motor command, h M , or associative input, h A ) given the input activity observed at a lower layer (e.g. sensory target response, h S ; Fig. 1).
Revised: Since the output weights of the cerebellar network can be positive or negative, we do not distinguish between positive or anti correlation (i.e. show the absolute correlation value) between the hidden activity of the cerebellar network and its respective target hidden activity of the feedforward network.

Original: Consistent with the modular anatomy of the cerebellum (Apps Forward Model
Revised: C C OMPARISON WITH OTHER MODELS OF THE CEREBELLUM

Original: Backward DNI Inverse Model Forward DNI Controller
Revised: In section 2 of the main text we argue that there is a strong similarity between the ﬂow of information in DNI and classical internal models of the cerebellum.

Original: Neocortex Main model
Revised: In Table S1 we compare explicitly the input and output components of both types of models.

Original: Cerebellum Synthesiser Input h
Revised: [Table]

Original:  m h m h s , d s h s
Revised: h m h m h s , d s h s

Original: Output f unk ( h m )
Revised: [Table] f unk ( h m )

Original: C B ( h m )
Revised: C B ( h m )

Original: =
Revised: =

Original: ˆ ∂L∂ h m f unk ( h s , d s )
Revised: ˆ ∂L∂ h m f unk ( h s , d s )

Original: C F ( h s )
Revised: C F ( h s )

Original: Ouput destination
Revised: [Table]

Original: Neocortex Main model Controlled object
Revised: activity of a motor area and sensory area, respectively;

Original: Main model 
Revised: C B , C F denotes the computation of backward DNI and forward DNI, respectively; L denotes the loss function. In addition, the inverse model of the cerebellum traditionally also has access to a desired state d s (in particular, one can consider this a special case of “context” provided to the synthesiser; cf (Jaderberg et al., 2017)). There are no explicit equations for the computational processes of the forward and inverse models, and both are thus represented by the unknown function f unk .

Original: [Figure]
Revised: C.

Original: inverse models map onto forward DNI forward
Revised: . I NVERSE AND FORWARD DNI CEREBELLAR MODELS

Original: In inverse model  the cerebellum generates motor predictions( ˆ M given desired black arrows example
Revised: The inverse model of the cerebellum in motor control computes the motor predictions given a target (or desired) response (Ito, 1970).

Original: Example of forward CC-DNI model in a non-motor domain. The cerebellum computes an associative prediction h
Revised: Feedback errors are derived by comparing a motor command from the motor cortex with the predicted motor command in the cerebellum.

Original:  A  given  a  sensory input h S . Multiple backward and forward DNI-cerebellum modules can co-exist working in parallel. h and g variables represent forward and gradient information as in DNI (see main text).
Revised: We propose that the inverse model is equivalent to forward DNI, as it aims to predict the incoming activity at a given layer (e.g. motor command, h M , or associative input, h A ) given the input activity observed at a lower layer (e.g. sensory target response, h S ; Fig. 1).

Original:  et al., 2018; Cerminara and Apps, 2011; Ruigrok, 2011) multiple CC-DNI (backward and/or forward) modules can co-exist estimating different pathways components.
Revised: Consistent with the modular anatomy of the cerebellum (Apps et al., 2018; Cerminara and Apps, 2011; Ruigrok, 2011) multiple CC-DNI (backward and/or forward) modules can co-exist estimating different pathways components.

Original: C R ECURRENT DNI
Revised: D R ECURRENT DNI

Original: In section 2 of the main text we formally describe the dynamics of DNI in a feedforward setting.
Revised: In section 2 of the main text we formally describe the dynamics of DNI in a feedforward setting.

Original: Here we show how these dynamics easily generalise to the time dependent, recurrent setting.
Revised: Here we show how these dynamics easily generalise to the time dependent, recurrent setting.

Original: We now take the main model as a recurrent neural network (RNN), where the (recurrent) hidden layer is again connected to a distinct, feedforward synthesiser.
Revised: We now take the main model as a recurrent neural network (RNN), where the (recurrent) hidden layer is again connected to a distinct, feedforward synthesiser.

Original: By unrolling the network over several timesteps we can keep much of the notation as introduced in section 2, but now h i denotes the hidden (cortical) activity at timestep
Revised: By unrolling the network over several timesteps we can keep much of the notation as introduced in section 2, but now h i denotes the hidden (cortical) activity at timestep

Original: i (as opposed to layer i ) with a unique recurrent weight θ ( = θ i ∀ i ); furthermore, the loss L is now a sum of individual losses over the timesteps: L = (cid:80) i
Revised: i (as opposed to layer i ) with a unique recurrent weight θ ( = θ i ∀ i ); furthermore, the loss L is now a sum of individual losses over the timesteps: L = (cid:80) i

Original: L i .
Revised: L i .

Revised: [Figure]

Original: DNI is strictly applied to RNNs which learn via truncated BPTT, where BPTT takes place in equallyspaced truncations of time so that any hidden activity will only be attributed to future losses within the truncation window T (i.e. ∂L i + t 
Revised: cerebellum [Figure] a T a r g e t r e s p o n s e

Revised: Cerebellum Motor modules

Revised: Forward unlocking

Revised: GC

Revised: PC

Revised: Inferior Olive - + error

Revised: Associativeareas

Revised: Cognitiveareas input

Revised: Sensoryareas

Revised: Associativepred.

Revised: T a r g e t r e s p o n s e

Revised: Cerebellum Associative modules

Revised: [Figure]

Revised: Sensory modules

Revised: CerebellumCerebellum

Revised: Cognitive modules

Revised: CerebellumCerebellum

Revised: GC

Revised: [Figure]

Revised: DNI is strictly applied to RNNs which learn via truncated BPTT, where BPTT takes place in equally-spaced truncations of time so that any hidden activity will only be attributed to future losses within the truncation window T (i.e. ∂L i + t

Original: ∂h i = 

Revised: ∂h i =

Revised: ∀ t > T ), effectively ignoring a b ...

Original: ∀ t > T ), effectively ignoring future losses and potentially disregarding important temporal correlations.
Revised: [Figure] [Figure] c d Figure future losses and potentially disregarding important temporal correlations.

Original: The key idea behind recurrent DNI is to bridge these truncations using synthetic gradients (Fig. S2a).
Revised: The key idea behind recurrent DNI is to bridge these truncations using synthetic gradients (Fig. S2a).

Original: For simplicity assume that timestep i is the start of a given truncation.
Revised: For simplicity assume that timestep i is the start of a given truncation.

Original: By enabling DNI, a (backward) synthesiser C is used to provide an estimated gradient ˆ δ i + T for all future losses, outside of the truncation , with respect to the last hidden activity of the truncation
Revised: By enabling DNI, a (backward) synthesiser C is used to provide an estimated gradient ˆ δ i + T for all future losses, outside of the truncation , with respect to the last hidden activity of the truncation

Original: h i + T ; i.e. ˆ δ i + T
Revised: h i + T ; i.e. ˆ δ i + T

Original: ≈
Revised: ≈

Original: δ
Revised: δ

Original: i + T = (cid:80) ∞
Revised: i + T = (cid:80) ∞

Original: j = i + T
Revised: j = i + T

Original: +1 ∂L
Revised: +1 ∂L

Original: j ∂h i + T .
Revised: j ∂h i + T .

Original: The weight update is then analogous to the feedforward case in equation
Revised: The weight update is then analogous to the feedforward case in equation

Original: [Equation]
Revised: [Equation]

Original: It is not difﬁcult to see that if the synthesiser perfectly predicts the future gradient, ˆ δ i + T
Revised: It is not difﬁcult to see that if the synthesiser perfectly predicts the future gradient, ˆ δ i + T

Original: = δ i + T , then the weight will be perfectly updated according to the gradient of all future losses, θ ← θ − α (cid:80) ∞
Revised: = δ i + T , then the weight will be perfectly updated according to the gradient of all future losses, θ ← θ − α (cid:80) ∞

Original: j = i ∂L
Revised: j = i ∂L

Original: i ∂θ .
Revised: i ∂θ .

Original: Allowing the synthesiser access to this gold standard, δ i + T , however, requires waiting an arbitrary amount of time and is therefore not practical for training.
Revised: Allowing the synthesiser access to this gold standard, δ i + T , however, requires waiting an arbitrary amount of time and is therefore not practical for training.

Original: Instead, ˆ δ i + T is trained to replicate an itself approximate, bootstrapped version of the future gradient, ¯ δ i + T , where ¯ δ i + T =
Revised: Instead, ˆ δ i + T is trained to replicate an itself approximate, bootstrapped version of the future gradient, ¯ δ i + T , where ¯ δ i + T =

Original: (cid:80) i +

Revised: (cid:80) i +

Original: T j =
Revised: T j =

Original: i + T
Revised: i + T

Original: +1 ∂L
Revised: +1 ∂L

Original: j ∂h i + T
Revised: j ∂h i + T

Original: + ˆ
Revised: + ˆ

Original: δ
Revised: δ

Original: i
Revised: i

Original: +

Revised: +

Original: T
Revised: T

Original: ∂h
Revised: ∂h

Original: i +

Revised: i +

Original: T
Revised: T

Original: ∂h i + T .
Revised: ∂h i + T .

Original: Note that ¯ δ i + T is available in one truncation’s timespan.
Revised: Note that ¯ δ i + T is available in one truncation’s timespan.

Original: In the brain an approximation of BPTT can be implemented forward in time using a mixture of eligibility traces and sparse learning signals (e.g. reward) ((Bellec et al., 2019); Fig.
Revised: In the brain an approximation of BPTT can be implemented forward in time using a mixture of eligibility traces and sparse learning signals (e.g. reward) ((Bellec et al., 2019); Fig.

Original: S2b).
Revised: S2b).

Original: The dynamics of this more “biologically plausible” learning regime are subtle and are not described here; we simply mention that the cerebellum might be the source of such (future) learning signals and predict that many of the results observed with BPTT will hold for the forward propagation of gradients.[Equation] [Figure]
Revised: The dynamics of this more “biologically plausible” learning regime are subtle and are not described here; we simply mention that the cerebellum might be the source of such (future) learning signals and predict that many of the results observed with BPTT will hold for the forward propagation of gradients.

Original: D I NPUT EXPANSION AND SPARSE CONNECTIVITY IN CEREBELLUM -DNI
Revised: E I NPUT EXPANSION AND SPARSE CONNECTIVITY IN CEREBELLUM -DNI

Original: We also study the effects of input expansion and sparse connectivity from the main model to the synthesiser in our (s)CC-DNI model.
Revised: We also study the effects of input expansion and sparse connectivity from the main model to the synthesiser in our (s)CC-DNI model.

Original: Input expansion and sparse connectivity are two deﬁning features of the cerebellum.
Revised: Input expansion and sparse connectivity are two deﬁning features of the cerebellum.

Original: Often human intelligence is linked to its expanded neocortex.
Revised: Often human intelligence is linked to its expanded neocortex.

Original: However, the cerebellar/neocortical cell ratio in humans 4/1 and the cerebellum has signiﬁcantly expanded throughout evolution (Barton and Venditti (2014); Herculano-Houzel (2010)).
Revised: However, the cerebellar/neocortical cell ratio in humans 4/1 and the cerebellum has signiﬁcantly expanded throughout evolution (Barton and Venditti (2014); HerculanoHouzel (2010)).

Original: Here we aim to study the effect of this input expansion into the cerebellum by systematically varying the number of hidden units in the synthesiser relative to the number of units in the main network.
Revised: Here we aim to study the effect of this input expansion into the cerebellum by systematically varying the number of hidden units in the synthesiser relative to the number of units in the main network.

Original: We show this for both sensorimotor task: target reaching task (Fig.
Revised: We show this for both sensorimotor task: target reaching task (Fig.

Original: S3) and seq-MNIST tasks (Fig.
Revised: S3) and seq-MNIST tasks (Fig.

Original: S4 and S5) .
Revised: S4 and S5) .

Original: In addition, it is known that one granule cell receives input from four mossy ﬁbres on average.
Revised: In addition, it is known that one granule cell receives input from four mossy ﬁbres on average.

Original: Mossy ﬁbres form the main inputs to the cerebellum.
Revised: Mossy ﬁbres form the main inputs to the cerebellum.

Original: The largest source of mossy ﬁbres is the neocortex which projects inputs to the cerebellum via the pons.
Revised: The largest source of mossy ﬁbres is the neocortex which projects inputs to the cerebellum via the pons.

Original: To quantify the effect of the sparse input connectivity we quantify next to the Pearson correlation also the population correlation coefﬁcient for the target reaching task (Fig. S6) and the standard seq-MNIST task (Fig. S7).
Revised: To quantify the effect of the sparse input connectivity we quantify next to the Pearson correlation also the population correlation coefﬁcient for the target reaching task (Fig. S6) and the standard seq-MNIST task (Fig. S7).

Original: E T ARGET REACHING SUPPLEMENTARY DATA
Revised: F T ARGET REACHING SUPPLEMENTARY DATA

Original: Here we show that the target reaching task results extrapolate to variants of the task with varied input-output combinations (Fig. S8).
Revised: Here we show that the target reaching task results extrapolate to variants of the task with varied input-output combinations (Fig. S8).

Original: Furthermore, in line with Fig.
Revised: Furthermore, in line with Fig.

Original: 2b we include the learning curves across different truncation values for the three models (Fig. S9).
Revised: 2b we include the learning curves across different truncation values for the three models (Fig. S9).

Original: F SEQ MNIST SUPPLEMENTARY DATA
Revised: G SEQ MNIST SUPPLEMENTARY DATA

Original: In line with Fig.3c, we include the learning curves across different truncation values T for the standard seqMNIST task as well as its variants (Figs.
Revised: In line with Fig.3c, we include the learning curves across different truncation values T for the standard seqMNIST task as well as its variants (Figs.

Original: S10, S11).
Revised: S10, S11).

Original: G I MAGE CAPTIONING SUPPLEMENTARY DATA
Revised: H I MAGE CAPTIONING SUPPLEMENTARY DATA

Original: In line with Fig.4, we include more examples of image-caption pairs (Fig. S12) as well as the learning curves and metric scores across different truncation values (Figs.
Revised: In line with Fig. we include more examples of image-caption pairs (Fig. S12) as well as the learning curves and metric scores across different truncation values (Figs.

Original: S14, S15).
Revised: S14, S15).

Original: Moreover, we demonstrate that DNI models for this task often generalise to unseen data better than regular LSTM models (Fig. S13).
Revised: Moreover, we demonstrate that DNI models for this task often generalise to unseen data better than regular LSTM models (Fig. S13).

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Figure]
Revised: [Figure]

Original: r p o p ( g )
Revised: r p o p ( g )

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: [Table]
Revised: [Table]

Original: 4 and sparse input connectivity K
Revised: 4 and sparse input connectivity K

Original: [Table]
Revised: [Table]

Original: [Table]
Revised: [Table]

Original: 20% 40% 60% e rr o r ( % )
Revised: 20% 40% 60% e rr o r ( % )

Original: T =
Revised: T =

Original: T = T =
Revised: T = T =

Original: T = T =
Revised: T = T =

Original: T =  20   e rr o r
Revised: T =  20   e rr o r

Original: M S E )
Revised: M S E )

Original: T =
Revised: T =

Original: T = T =
Revised: T = T =

Original: T = T =
Revised: T = T =

Original: T =        
Revised: T =

Original: e rr o r
Revised: e rr o r

Original: M S E )
Revised: M S E )

Original: T =
Revised: T =

Original: T = T =
Revised: T = T =

Original: T = T =
Revised: T = T =

Original: T =        
Revised: T =

Original: e rr o r
Revised: e rr o r

Original: M S E )
Revised: M S E )

Original: T =
Revised: T =

Original: T = T =
Revised: T = T =

Original: T = T =
Revised: T = T =

Original: T =
Revised: T =

Original: seqMNIST c-seqMNIST dd-seqMNIST ld-seqMNIST
Revised: seqMNIST c-seqMNIST dd-seqMNIST ld-seqMNIST

Original: [Figure]
Revised: [Figure]

Original: [Figure] [Figure]
Revised: [Figure] [Figure]

Original: l o ss ( B P
Revised: l o ss ( B P

Original: W )
Revised: W )

Original: T =0 53.
Revised: T =0 53.

Original: T =0 53.0 3. T =0 53.0 3.
Revised: T =0 53.0 3. T =0 53.0 3.

Original: 10 epoch
Revised: 10 epoch

Original: T =0 53.
Revised: T =0 53.

Original: T =0 53.0 3.
Revised: T =0 53.0 3.

Original: T =0 53.03.54.
Revised: T =0 53.03.54.

Original: T =0 

Revised: T =0

Original: [Figure] [Figure]
Revised: [Figure] [Figure]