Cortico-cerebellar networks as decoupled neural interfaces

1 INTRODUCTION . Efficient credit assignment in the brain is a critical part of learning . However , how the brain solves the credit assignment problem remains a mystery . One of the central issues of credit assignment across multiple stages of processing is the need to wait for previous stages to finish their computation before others can proceed ( Rumelhart et al. , 1986 ; Schmidhuber , 1990 ; Lee et al. , 2015 ; Marblestone et al. , 2016 ; Jaderberg et al. , 2017 ) . In deep artificial neural networks these constraints are explicit . During the forward phase a given layer has to wait for all its previous layers to finish before it can proceed , a constraint known as the forward lock . Similarly , during the backward phase a given layer has to wait for all the layers above to finish computing its gradients – backward lock . Recently , a framework was introduced to decouple artificial neural networks – decoupled neural interfaces ( DNI ; ( Jaderberg et al. , 2017 ) ) 1 , effectively breaking forward and/or backward locks . Here , we propose that a specialised brain area , the cerebellum , performs a similar role in the brain . In the classical view the cerebellum is key for fine motor control and learning by constructing internal models of behaviour . ( Marr , 1969 ; Albus , 1971 ; Raymond and Medina , 2018 ; Wolpert et al. , 1998 ; 1DNIs are related to earlier work on using network critics to train neural networks ( Schmidhuber , 1990 ) . Miall et al. , 1993 ) . More recently , however , the idea that the cerebellum is also involved in cognition has gained significant traction ( Schmahmann et al. , 2019 ; Wagner and Luo , 2020 ; Brissenden and Somers , 2019 ) . An increasing body of behavioural , anatomical and imaging studies points to a role of the cerebellum in cognition in humans and non-human primates ( Schmahmann et al. , 2019 ; Brissenden and Somers , 2019 ; Guell et al. , 2015 ; 2018 ) . Impairments in cerebellar patients occur across a range of tasks including language ( Guell et al. , 2015 ) , working memory ( Deverett et al. , 2019 ) , planning ( Baker et al. , 1996 ) , and others ( Fiez et al. , 1992 ) . These observations suggest that the cerebellum implements a universal function across the brain ( Marr , 1969 ; Albus , 1971 ; Raymond and Medina , 2018 ; Diedrichsen et al. , 2019 ) . Moreover , experimental studies looking at corticocerebellar interactions have demonstrated that cerebellar output is crucial for maintaining neocortical representations in order to drive behaviour ( Chabrol et al. , 2019 ; Gao et al. , 2018 ) . However , to the best of our knowledge , no theoretical framework has considered what might be the function of such interactions between the cerebellum and cortical areas . In an attempt to reduce the existing gap between experimental observations and existing computational approaches we introduce DNI as a cortico-cerebellar model – cortico-cerebellar DNI ( CC-DNI ) . Consistent with the cerebellar universal role we theorise that the cerebellum serves to break the locks inherent to both feedforward and feedback information processing in the brain , akin to DNI . In particular , we posit that the two classical internal models of the cerebellum , forward and inverse models , are equivalent to DNI-mediated unlocking of feedback ( gradients ) and feedforward communication , respectively . Following this view the cerebellum not only provides motor or sensory estimates , but also any other modality encoded by a particular brain region . Inspired by neuroscientific studies , we test our model on sensorimotor tasks : ( i ) a target reaching task ( Sanes et al. , 1990 ; Butcher et al. , 2017 ; Nashef et al. , 2019 ) and ( ii ) a set of more complex temporal tasks based on the MNIST dataset , but also ( iii ) on a cognitive task – caption generation ( Guell et al. , 2015 ) . Our results support the cortico-cerebellar DNI models we study and show that they generally speed up learning by unlocking the main network , qualitatively consistent with a wide range of behavioural observations ( Guell et al. , 2015 ; Sanes et al. , 1990 ; Butcher et al. , 2017 ; Nashef et al. , 2019 ) . Two defining features of the cerebellum are the large expansion at the granule cell input layer with 50 billion neurons ( the most numerous cell in the brain ) and the highly sparse connectivity ( each granule cell receives ∼ 4 synapses ) ( Sanger et al. , 2020 ) . These observations have been long suggested to help speed-up learning in the cerebellum through decorrelation ( Albus , 1971 ; Sanger et al. , 2020 ; Cayco-Gajic et al. , 2017 ) . Building on these studies we introduce a new DNI model , sparse CC-DNI . Consistent with classical cerebellar models ( Albus , 1971 ; Cayco-Gajic et al. , 2017 ) we show that input sparsity can improve learning in the presence of high correlations . We finish with a discussion on the implications and predictions of this new brain-wide model of the cerebellum . 2 CEREBELLUM AS A DECOUPLING MACHINE . We first describe DNIs following Jaderberg et al . ( 2017 ) and then establish the link to corticocerebellar networks . Assume that a feedforward neural network consists of N layers , with the ith layer ( 1 ≤ i ≤ N ) performing a “ computational step ” fi with parameters θi . Given input x at layer 1 , the output of the network at its final layer is therefore given by fN ( fN−1 ( . . . f2 ( f1 ( x ) ) . . . ) ) . We use Fji to denote the composition of steps from layer i to layer j ( inclusively ) . Finally , let hi denote the ( hidden ) activity at layer i , so that hi = fi ( hi−1 ) with h0 = x . To illustrate the locking constraints of standard artificial neural networks used in deep learning suppose that a network is in the process of learning via backpropogation , with current input-target pair ( x , ytarg ) . To update the layer parameters θi the gradient ∂L∂θi is required , where L = L ( y , ytarg ) is the loss which compares the target value against the model output y = FN1 ( x ) under some loss function L ; we then apply gradient descent on the parameters , θi ← θi − α ∂L∂θi , with learning rate α > 0 . Suppose however that the network has only recently received the input x and is currently only at module i of the forward computation . In order to update the corresponding parameters of that layer , θi , the layer must first wait for all remaining layers to finish fj ( j > i ) for the loss to be computed . Only then the various gradients of the loss are backpropogated and ∂L∂θi is finally available . These two characteristics of backpropogation make layer i “ backward locked ” to F ji+1 , enforcing a strong dependence of the layer ’ s learning to the speed of forward and backward propagation through the rest of the network . Similarly the the network is forward locked during the forward pass . DNI can be used to unlock both backward and forward locks Jaderberg et al . ( 2017 ) . To illustrate the model here we focus on backward DNI , which goal is to break the backward lock by feeding the hidden layer i activity hi to a separate neural network , a backward synthesiser CBi , that learns to produce a synthetic gradient ĝi – an estimate of the real gradient expected by layer i , Ci ( hi ) = ĝi ≈ ∂L∂hi . This synthetic gradient can then be used to update the weights of layer i as soon as hi is available , following θi ← θi − αC ĝi ∂hi ∂θi ; ĝi = Ci ( hi ) ( 1 ) More specifically , the parameters of CBi ( hi ) are learned by comparing the estimated gradient with a target gradient ḡi so as to minimise LC = ||ḡi − ĝi|| . Ideally we set the target gradient as the true gradient , ḡi = ∂L∂θi , as can be implemented without difficulty in the case of feedforward networks . However , if we consider the temporal case ( see sections 2.1.1 and 3 ) where we wish to estimate gradients of future losses many timesteps ahead , L = ∑ t Lt , the true backpropagated gradient is computational expensive to obtain . Jaderberg et al . ( 2017 ) counter this potential problem by applying a bootstrapping principle in which a DNI module itself is used to guide learning ; explicitly , the target gradient at timestep t is ḡt = ∑T τ > t ∂Lτ ht + CT ( hT ) ∂hT∂ht , where T defines some limited horizon ; note the interesting resemblance to the n-step return used in reinforcement learning algorithms . 2.1 CORTICO-CEREBELLAR DNI MODELS . Building directly on DNI we introduce a model of cortico-cerebellar computation ( CC-DNI ) . These models use a simple feedforward neural network , consistent with the mostly feedforward architecture of the cerebellum ( Marr , 1969 ; Albus , 1971 ; Raymond and Medina , 2018 ) . The input layer of the cerebellum module C models the mossy fiber input onto granule cells ( GCs ; Fig . 1 ) , which are the most numerous neurons in the brain ( > 50 billion in humans ( Herculano-Houzel , 2009 ) ) . Consistent with this cerebellar dimensionality expansion , in our models we useM N , whereM is the number of GCs and N the number of neurons in the main ( cortical ) network ( Fig . 1 ) . In particular , we use ratios MN ∼ 4 , consistent with experimental observations ( Herculano-Houzel , 2009 ) . This is different to DNI , in which the synthesizer uses a single hidden layer with the same number of units as LSTM ( for comparison in performance see ref to Fig . S2 ) . In addition , the hidden layers and the output of C approximate the role of GCs and Purkinje cells , and the cerebellar output nuclei , respectively . Cerebellar granule cells receive sparse input connections ( K ) with only around 3-7 synapses per GC ( Herculano-Houzel , 2009 ; Eccles et al. , 2013 ) . These architectural constraints have led to sparse encoding and decorrelation theories of cerebellar function ( Albus , 1971 ; Cayco-Gajic et al. , 2017 ; Schweighofer et al. , 2001 ; Broomhead and Lowe , 1988 ; Billings et al. , 2014 ; Litwin-Kumar et al. , 2017 ; Sanger et al. , 2019 ) . Inspired on these features , we introduce a new model – sparse CC-DNI ( sCC-DNI ) , for which we set a small number of incoming input connections K = 4 ( Fig . 1 ) . To measure decorrelation achieved by sCC-DNI we use the Pearson correlation and a population correlation metric rpop that has been shown to better capture cerebellar effects ( Cayco-Gajic et al. , 2017 ) rpop = ZZ−1 ( max { √ λi } i∑ i √ λi − 1Z ) where Z are the number of neurons being considered and λi are the eigenvalues of the covariance matrix of the neuronal activity ( e.g . hM ) . To manipulate the hidden correlations of the model , we adjust the variance of its input and recurrent weights by scaling each by a factor b with b 6= 1 ( see SM ) . There is a strong similarity between CC-DNI models and the flow of information in standard internal models of the cortico-cerebellar networks ( Fig . 1 , Table S1 ) . Below we draw a parallel between classical cerebellar internal models and our CC-DNI models . For simplicity , below and in our results we focus on the link between forward internal models and backward DNI ( but see S1 for our interpretation of inverse internal model as forward DNI ) .

Summary: The authors proposed that cerebellum in the brain computes synthetic gradients, as used in decoupled neural interfaces (DNI), to enable learning in neural circuits without waiting for gradients to propagate backwards. The authors incorporated several architectural properties of biological cerebellum into their cerebellar model. They showed that a LSTM trained with such synthetic gradients can learn a variety of tasks, from motor reaching to caption generation. The paper is clearly written, and the link between DNI and cerebellum is a novel idea. However, the authors made little attempt to actually compare their model with experimental findings from cerebellum (except for the cerebellar properties built into the network), limiting its scientific impact. Meanwhile, it is not clear whether the cerebellum-inspired DNI provides concrete advantages over DNI proposed in Jaderberg 2017.