Local plasticity rules can learn deep representations using self-supervised contrastive predictions

1 Introduction . Synaptic connection strengths in the brain are thought to change according to ‘ Hebbian ’ plasticity rules [ Hebb , 1949 ] . Such rules are local and depend only on the recent state of the pre- and postsynaptic neurons [ Sjöström et al. , 2001 , Caporale and Dan , 2008 , Markram et al. , 2011 ] , potentially modulated by a third factor related to reward , attention or other high-level signals [ Kuśmierz et al. , 2017 , Gerstner et al. , 2018 ] . Therefore , one appealing hypothesis is that representation learning in sensory cortices emerges from local and unsupervised plasticity rules . Following a common definition in the field [ Fukushima , 1988 , Riesenhuber and Poggio , 1999 , LeCun , 2012 , Lillicrap et al. , 2020 ] , a hierarchical representation ( i ) builds higher-level features out of lower-level ones , and ( ii ) provides more useful features in higher layers . Now there seems to be a substantial gap between the rich hierarchical representations observed in the cortex and the representations emerging from local plasticity rules implementing principal/independent component analysis [ Oja , 1982 , Hyvärinen and Oja , 1998 ] , sparse coding [ Olshausen and Field , 1997 , Rozell et al. , 2008 ] or slow-feature analysis [ Földiák , 1991 , Wiskott and Sejnowski , 2002 , Sprekeler et al. , 2007 ] . Hebbian rules seem to struggle especially when ‘ stacked ’ , i.e . when asked to learn deep , hierarchical representations . ⇤shared last author 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) This performance gap is puzzling because there are learning rules , relying on back-propagation ( BP ) , that can build hierarchical representations similar to those found in visual cortex [ Yamins et al. , 2014 , Zhuang et al. , 2021 ] . Although some progress towards biologically plausible implementations of back-propagation has been made [ Lillicrap et al. , 2016 , Guerguiev et al. , 2017 , Sacramento et al. , 2018 , Payeur et al. , 2021 ] , most models rely either on a neuron-specific error signal that needs to be transmitted by a separate error network [ Crick , 1989 , Amit , 2019 , Kunin et al. , 2020 ] , or timemultiplexing feedforward and error signals [ Lillicrap et al. , 2020 , Payeur et al. , 2021 ] . Algorithms like contrastive divergence [ Hinton , 2002 ] , contrastive Hebbian learning [ Xie and Seung , 2003 ] or equilibrium propagation [ Scellier and Bengio , 2017 ] use local activity exclusively to calculate updates , but they require to wait for convergence to an equilibrium which is not appropriate for online learning from quickly varying inputs . The present paper demonstrates that deep representations can emerge from a local , biologically plausible and unsupervised learning rule , by integrating two important insights from neuroscience : First , we focus on self-supervised learning from temporal data – as opposed to supervised learning from labelled examples – because this comes closest to natural data , perceived by real biological agents , and because the temporal structure of natural stimuli is a rich source of information . In particular , we exploit the self-awareness of typical , self-generated changes of gaze direction ( ‘ saccades ’ ) to distinguish input from a moving object during fixation from input arriving after a saccade towards a new object . In our plasticity rule , a global factor modulates plasticity , depending on the presence or absence of such saccades . Although we do not model the precise circuit that computes this global factor , we see it related to global , saccade-specific signals from motor areas , combined with surprise or prediction error , as in other models of synaptic plasticity [ Angela and Dayan , 2005 , Nassar et al. , 2012 , Heilbron and Meyniel , 2019 , Liakoni , 2021 ] . Second , we notice that electrical signals stemming from segregated apical dendrites can modulate synaptic plasticity in biological neurons [ Körding and König , 2001 , Major et al. , 2013 ] , enabling context-dependent plasticity . Algorithmically , our approach takes inspiration from deep self-supervised learning algorithms that seek to contrast , cluster or predict stimuli in the context of BP [ Van den Oord et al. , 2018 , Caron et al. , 2018 , Zhuang et al. , 2019 , Löwe et al. , 2019 ] . Interestingly , Löwe et al . [ 2019 ] demonstrated that such methods even work if end-to-end BP is partially interrupted . We build upon this body of work and suggest the Contrastive , Local And Predictive Plasticity ( CLAPP ) model which avoids BP completely , yet still builds hierarchical representations.2 2 Main goals and related work . In this paper , we propose a local plasticity rule that learns deep representations . To describe our model of synaptic plasticity , we represent a cortical area by the layer l of a deep neural network . The neural activity of this layer at time t is represented by the vector zt , l = ⇢ ( at , l ) , where ⇢ is a non-linearity and at , l = W lzt , l 1 is the vector of the respective summed inputs to the neurons through their basal dendrites W l ( the bias is absorbed into W l ) . To simplify notation , we write the pre-synaptic input as xt , l = zt , l 1 and we only specify the layer index l when it is necessary . Our plasticity rule exploits the fact that the temporal structure of natural inputs affects representation learning [ Li and DiCarlo , 2008 ] . Specifically , we consider a scenario where an agent first perceives a moving object at time t ( e.g . a flying eagle in Figure 1 a ) , and then spontaneously decides to change gaze direction towards another moving object at time t + t ( e.g . saccade towards the elephant in Figure 1 a ) . We further assume that the visual pathway is ‘ self-aware ’ of saccades due to saccade-specific modulation of processing [ Ross et al. , 2001 ] . In line with classical models of synaptic plasticity , we assume that weight changes follow biologically plausible , Hebbian , learning rules [ Hebb , 1949 , Markram et al. , 2011 ] which are local in space and time : updates W tji of a synapse , connecting neurons i and j , can only depend on the current activity of the pre-synaptic and post-synaptic neurons at time t , or slightly earlier at time t t , and one or several widely broadcasted modulating factors [ Urbanczik and Senn , 2009 , Gerstner et al. , 2018 ] . Furthermore , we allow the activity of another neuron k to influence the weight update Wji , as long as there is an explicit connection W predjk from k to j . The idea is to overcome the representational limitations of classical Hebbian learning by including dendritic inputs , which are thought to predict 2Our code is available at https : //github.com/EPFL-LCN/pub-illing2021-neurips the future somatic activity [ Körding and König , 2001 , Urbanczik and Senn , 2014 ] and take part in the plasticity of the post-synaptic neuron [ Larkum et al. , 1999 , Dudman et al. , 2007 , Major et al. , 2013 ] . Hence we assume that each neuron j in a layer l may receive dendritic inputs ( W predct , l ) j coming either from the layer above ( ct , l = zt , l+1 ) or from lateral connections in the same layer ( ct , l = zt , l ) . For algorithmic reasons , that we detail in section 3 , we assume that the dendritic input ( W predct , l ) j influences the weight updates Wji of the post-synaptic neuron j , but not its activity ztj . This assumption is justified by neuroscientific findings that the inputs to basal and apical dendrites affect the neural activity and plasticity in different ways [ Larkum et al. , 1999 , Dudman et al. , 2007 , Major et al. , 2013 , Urbanczik and Senn , 2014 ] . In general , we do not rule out influence of dendritic activity on somatic activity in later processing phases , but see this beyond the scope of the current work . Given these insights from neuroscience , we gather the essential factors that influence synaptic plasticity in the following learning rule prototype : Wji / modulators| { z } broadcast factors · ( W predct1 ) j| { z } dendritic prediction · postt2j · pre t2 i| { z } local-activity . ( 1 ) The modulating broadcast factors are the same for large groups of neurons , for example all neurons in the same area , or even all neurons in the whole network . postt2j and pre t2 i are functions of the preand post- synaptic activities . At this point , we do not specify the exact timing between t1 and t2 , as this will be determined by our algorithm in section 3 . Related work Many recent models of synaptic plasticity fit an apparently similar learning rule prototype [ Lillicrap et al. , 2016 , Nøkland , 2016 , Roelfsema and Holtmaat , 2018 , Nøkland and Eidnes , 2019 , Lillicrap et al. , 2020 , Pozzi et al. , 2020 ] if we interpret the top-down signals emerging from the BP algorithm as the dendritic signal . However , top-down error signals in BP are not directly related to the activity ct of the neurons in the main network during processing of sensory input . Rather , they require a separate linear network mirroring the initial network and feeding back error signals ( see Figure 2 a and Lillicrap et al . [ 2020 ] ) , or involved time-multiplexing of feedforward and error signals in the main network [ Lillicrap et al. , 2020 , Payeur et al. , 2021 ] . Our model is fundamentally different , because in our case , the dendritic signal onto neuron j is strictly ( W predct ) j which is a weighted sum of the main network activity and there is no need of a ( linear ) feedback network transmitting exact error values across many layers . Moreover , we show in simulations in section 4 , that the dendritic signal does not have to come from a layer above but that the prediction fed to layer l may come from the same layer . This shows that our learning rule works even in the complete absence of downward signaling from l + 1 to l. This last point is a significant difference to other methods that also calculate updates using only activities of the main network , but require tuned top-down connections to propagate signals downwards in the network hierarchy [ Kunin et al. , 2020 ] , such as methods in the difference target propagation family [ Lee et al. , 2015 , Bartunov et al. , 2018 , Golkar et al. , 2020 ] , contrastive divergence [ Hinton , 2002 ] and equilibrium propagation [ Scellier and Bengio , 2017 ] . Furthermore , the latter two require convergence to an equilibrium state for each input [ Laborieux et al. , 2021 ] . Our model does not require this convergence because it uses the recurrent dendritic signal ( W predct ) j only for synaptic plasticity and not for inference . Most previous learning rules which include global modulating factors interpret it as a reward prediction error [ Schultz et al. , 1997 , Gerstner et al. , 2018 , Pozzi et al. , 2020 ] . In this paper , we address self-supervised learning and view global modulating factors as broadcasting signals , modeling the self-awareness that something has changed in the stimulus ( e.g . because of a saccade ) . Hence , the main function of the broadcast factor in our model is to identify contrastive inputs , which avoids a common pitfall for self-supervised learning models : ‘ trivial ’ or ‘ collapsed ’ solutions , where the model produces a constant output , which is easily predictable , but useless for downstream tasks . In vision , we use a broadcast factor to model the strong , saccade-specific activity patterns identified throughout the visual pathway [ Kowler et al. , 1995 , Leopold and Logothetis , 1998 , Ross et al. , 2001 , McFarland et al. , 2015 ] . In other sensory pathways , like audition , this broadcast factor may model attention signals arising when changing focus on a new input source [ Fritz et al. , 2007 ] , cross-modal input indicating a change in head or gaze direction , or signal/speaker-identity inferred from blind source separation , which can be done on low-level representation with biologically plausible learning rules [ Hyvärinen and Oja , 1997 , Ziehe and Müller , 1998 , Molgedey and Schuster , 1994 ] . Our learning rule further requires this global factor to predict the absence or presence of a gaze change , hence conveying a change prediction error rather than classical reward prediction error . Here , we do not model the precise circuitry computing this factor in the brain , however , we speculate that a population of neurons could express such a scalar factor e.g . through burst-driven multiplexing of activity , see Payeur et al . [ 2021 ] and Appendix C. Our theory takes inspiration from the substantial progress seen in unsupervised machine learning in recent years and specifically from contrastive predictive coding ( CPC ) [ Van den Oord et al. , 2018 ] . CPC trains a network ( called encoder ) to make predictions of its own responses to future inputs , while keeping this prediction as different as possible to its responses to fake inputs ( contrasting ) . A key feature of CPC is that predicting and contrasting happens in latent space , i.e . on the output representation of the encoder network . This avoids modeling a generative model for perfect reconstruction of the input and all its details ( e.g . green , spiky ) . Instead the model is forced to focus on extracting high-level information ( e.g . cactus ) . In our notation , CPC evaluates a prediction W predct such that a score function u⌧t = z⌧ > W predct becomes larger for the true future ⌧ = t + t ( referred to as positive sample ) than for any other vector zt 0 taken at arbitrary time points t0 elsewhere in the entire training set ( referred to as negative samples in CPC ) . This means , that the prediction should align with the future activity zt+ t but not with the negative samples . Van den Oord et al . [ 2018 ] formalizes this as a softmax cross-entropy classification , which leads to the traditional CPC loss : LtCPC = log exput+ ttP ⌧2T expu ⌧ t , ( 2 ) where T = t t+ t , t 0 1 . . . t 0 N comprises the positive sample and N negative samples . The learned model parameters are the elements of the matrix W pred , as well as the weights of the encoder network . The loss function LtCPC is then minimized by stochastic gradient descent on these parameters using BP . Amongst numerous recent variants of contrastive learning [ He et al. , 2019 , Chen et al. , 2020 , Xiong et al. , 2020 ] , we focus here on CPC [ Van den Oord et al. , 2018 ] , for which a more local variant , Greedy InfoMax , was recently proposed by Löwe et al . [ 2019 ] . Greedy InfoMax ( GIM ) [ Löwe et al. , 2019 ] is a variant of CPC which makes a step towards local , BP-free learning : the main idea is to split the encoder network into a few gradient-isolated modules = W retro , kj ( atj ) xti+ t ( W retro z ) j W retro to avoid back-propagation between these modules . As the authors mention in their conclusion , “ the biological plausibility of GIM is limited by the use of negative samples and within-module backpropagation ” . This within-module back-propagation still requires a separate feedback network to propagate prediction errors ( Figure 2 a ) , but can be avoided in the most extreme version of GIM , where each gradient-isolated module contains a single layer ( layer-wise GIM ) . However , the gradients of layer-wise GIM , derived from Equation 2 , still can not be interpreted as synaptic plasticity rules because the gradient computation requires ( 1 ) the transmission of information other than the network activity ( see Figure 2 b ) , and ( 2 ) perfect memory to replay the negative samples zt 0 , as mentioned in the above quote ( see Appendix A for details ) . Overall it is not clear how this weight update of layer-wise GIM could be implemented with realistic neuronal circuits . Our CLAPP rule solves the above mentioned implausibilities and allows a truly local implementation in space and time .

Summary: This paper proposes a "Contrastive, Local And Predictive Plasticity" learning rule (CLAPP) that could be implemented by deep neural networks of the brain. The authors claim that it results in deep hierarchical representations akin to those recorded in the brain / from deep image classifiers trained with Backprop, while avoiding the need for a backward pass.   CLAPP relies on a global modulation signal which is related to eye movements: for a relatively static eye, the visual input and its encoding are expected to change little ("positive" samples); whereas every saccade leads to a large change in the image / code ("negative" examples). Another modulating factor is delivered via apical dendrites, which control the learning, but play no role in the feedforward signal transmission (see below for more detail).  Finally, the authors show the CLAPP training on the 6-layer VGG network for images (STL-10 dataset), as well as speech and video. They provide arguments for why they believe their network develops "deep representations" using visualisation (patches that best activate a few  neurons and t-SNE across layers) and probes (linear classifiers trained for each layer separately).   The authors state that the error bars and a proper statistical evaluation are missing due to computational and time constraints.   **More detail:** The predictive signal computed by the apical dendrites has no impact on the neural firing rate, but it has to be somehow read out across the entire layer  to form, together with the neural outputs, the global prediction of the saccade $u=\sum_j \text{neuron output}_j \times \text{dendritic prediction}_j.$ It is then compared to the presence of saccade signal and used to train: - the feedforward weights of the given layer, - the predictive weights (requiring the neuron's action potentials to affect the apical dendrite synapses).  The same signal (gathered from the layer's dendrites), affects the learning of the "prediction source" layer, which applies similar rules, but reversed in time: Instead of updating based on past input and current output, the synapses are updated based on the current input and past output. These result in symmetrical rules for updates of:   - the feedforward weights of the "prediction source" layer, - the retrograde weights which feed back the layer L's output to the "prediction layer".