Local plasticity rules can learn deep representations using self-supervised contrastive predictions

1 Introduction . Synaptic connection strengths in the brain are thought to change according to ‘ Hebbian ’ plasticity rules [ Hebb , 1949 ] . Such rules are local and depend only on the recent state of the pre- and postsynaptic neurons [ Sjöström et al. , 2001 , Caporale and Dan , 2008 , Markram et al. , 2011 ] , potentially modulated by a third factor related to reward , attention or other high-level signals [ Kuśmierz et al. , 2017 , Gerstner et al. , 2018 ] . Therefore , one appealing hypothesis is that representation learning in sensory cortices emerges from local and unsupervised plasticity rules . Following a common definition in the field [ Fukushima , 1988 , Riesenhuber and Poggio , 1999 , LeCun , 2012 , Lillicrap et al. , 2020 ] , a hierarchical representation ( i ) builds higher-level features out of lower-level ones , and ( ii ) provides more useful features in higher layers . Now there seems to be a substantial gap between the rich hierarchical representations observed in the cortex and the representations emerging from local plasticity rules implementing principal/independent component analysis [ Oja , 1982 , Hyvärinen and Oja , 1998 ] , sparse coding [ Olshausen and Field , 1997 , Rozell et al. , 2008 ] or slow-feature analysis [ Földiák , 1991 , Wiskott and Sejnowski , 2002 , Sprekeler et al. , 2007 ] . Hebbian rules seem to struggle especially when ‘ stacked ’ , i.e . when asked to learn deep , hierarchical representations . ⇤shared last author 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) This performance gap is puzzling because there are learning rules , relying on back-propagation ( BP ) , that can build hierarchical representations similar to those found in visual cortex [ Yamins et al. , 2014 , Zhuang et al. , 2021 ] . Although some progress towards biologically plausible implementations of back-propagation has been made [ Lillicrap et al. , 2016 , Guerguiev et al. , 2017 , Sacramento et al. , 2018 , Payeur et al. , 2021 ] , most models rely either on a neuron-specific error signal that needs to be transmitted by a separate error network [ Crick , 1989 , Amit , 2019 , Kunin et al. , 2020 ] , or timemultiplexing feedforward and error signals [ Lillicrap et al. , 2020 , Payeur et al. , 2021 ] . Algorithms like contrastive divergence [ Hinton , 2002 ] , contrastive Hebbian learning [ Xie and Seung , 2003 ] or equilibrium propagation [ Scellier and Bengio , 2017 ] use local activity exclusively to calculate updates , but they require to wait for convergence to an equilibrium which is not appropriate for online learning from quickly varying inputs . The present paper demonstrates that deep representations can emerge from a local , biologically plausible and unsupervised learning rule , by integrating two important insights from neuroscience : First , we focus on self-supervised learning from temporal data – as opposed to supervised learning from labelled examples – because this comes closest to natural data , perceived by real biological agents , and because the temporal structure of natural stimuli is a rich source of information . In particular , we exploit the self-awareness of typical , self-generated changes of gaze direction ( ‘ saccades ’ ) to distinguish input from a moving object during fixation from input arriving after a saccade towards a new object . In our plasticity rule , a global factor modulates plasticity , depending on the presence or absence of such saccades . Although we do not model the precise circuit that computes this global factor , we see it related to global , saccade-specific signals from motor areas , combined with surprise or prediction error , as in other models of synaptic plasticity [ Angela and Dayan , 2005 , Nassar et al. , 2012 , Heilbron and Meyniel , 2019 , Liakoni , 2021 ] . Second , we notice that electrical signals stemming from segregated apical dendrites can modulate synaptic plasticity in biological neurons [ Körding and König , 2001 , Major et al. , 2013 ] , enabling context-dependent plasticity . Algorithmically , our approach takes inspiration from deep self-supervised learning algorithms that seek to contrast , cluster or predict stimuli in the context of BP [ Van den Oord et al. , 2018 , Caron et al. , 2018 , Zhuang et al. , 2019 , Löwe et al. , 2019 ] . Interestingly , Löwe et al . [ 2019 ] demonstrated that such methods even work if end-to-end BP is partially interrupted . We build upon this body of work and suggest the Contrastive , Local And Predictive Plasticity ( CLAPP ) model which avoids BP completely , yet still builds hierarchical representations.2 2 Main goals and related work . In this paper , we propose a local plasticity rule that learns deep representations . To describe our model of synaptic plasticity , we represent a cortical area by the layer l of a deep neural network . The neural activity of this layer at time t is represented by the vector zt , l = ⇢ ( at , l ) , where ⇢ is a non-linearity and at , l = W lzt , l 1 is the vector of the respective summed inputs to the neurons through their basal dendrites W l ( the bias is absorbed into W l ) . To simplify notation , we write the pre-synaptic input as xt , l = zt , l 1 and we only specify the layer index l when it is necessary . Our plasticity rule exploits the fact that the temporal structure of natural inputs affects representation learning [ Li and DiCarlo , 2008 ] . Specifically , we consider a scenario where an agent first perceives a moving object at time t ( e.g . a flying eagle in Figure 1 a ) , and then spontaneously decides to change gaze direction towards another moving object at time t + t ( e.g . saccade towards the elephant in Figure 1 a ) . We further assume that the visual pathway is ‘ self-aware ’ of saccades due to saccade-specific modulation of processing [ Ross et al. , 2001 ] . In line with classical models of synaptic plasticity , we assume that weight changes follow biologically plausible , Hebbian , learning rules [ Hebb , 1949 , Markram et al. , 2011 ] which are local in space and time : updates W tji of a synapse , connecting neurons i and j , can only depend on the current activity of the pre-synaptic and post-synaptic neurons at time t , or slightly earlier at time t t , and one or several widely broadcasted modulating factors [ Urbanczik and Senn , 2009 , Gerstner et al. , 2018 ] . Furthermore , we allow the activity of another neuron k to influence the weight update Wji , as long as there is an explicit connection W predjk from k to j . The idea is to overcome the representational limitations of classical Hebbian learning by including dendritic inputs , which are thought to predict 2Our code is available at https : //github.com/EPFL-LCN/pub-illing2021-neurips the future somatic activity [ Körding and König , 2001 , Urbanczik and Senn , 2014 ] and take part in the plasticity of the post-synaptic neuron [ Larkum et al. , 1999 , Dudman et al. , 2007 , Major et al. , 2013 ] . Hence we assume that each neuron j in a layer l may receive dendritic inputs ( W predct , l ) j coming either from the layer above ( ct , l = zt , l+1 ) or from lateral connections in the same layer ( ct , l = zt , l ) . For algorithmic reasons , that we detail in section 3 , we assume that the dendritic input ( W predct , l ) j influences the weight updates Wji of the post-synaptic neuron j , but not its activity ztj . This assumption is justified by neuroscientific findings that the inputs to basal and apical dendrites affect the neural activity and plasticity in different ways [ Larkum et al. , 1999 , Dudman et al. , 2007 , Major et al. , 2013 , Urbanczik and Senn , 2014 ] . In general , we do not rule out influence of dendritic activity on somatic activity in later processing phases , but see this beyond the scope of the current work . Given these insights from neuroscience , we gather the essential factors that influence synaptic plasticity in the following learning rule prototype : Wji / modulators| { z } broadcast factors · ( W predct1 ) j| { z } dendritic prediction · postt2j · pre t2 i| { z } local-activity . ( 1 ) The modulating broadcast factors are the same for large groups of neurons , for example all neurons in the same area , or even all neurons in the whole network . postt2j and pre t2 i are functions of the preand post- synaptic activities . At this point , we do not specify the exact timing between t1 and t2 , as this will be determined by our algorithm in section 3 . Related work Many recent models of synaptic plasticity fit an apparently similar learning rule prototype [ Lillicrap et al. , 2016 , Nøkland , 2016 , Roelfsema and Holtmaat , 2018 , Nøkland and Eidnes , 2019 , Lillicrap et al. , 2020 , Pozzi et al. , 2020 ] if we interpret the top-down signals emerging from the BP algorithm as the dendritic signal . However , top-down error signals in BP are not directly related to the activity ct of the neurons in the main network during processing of sensory input . Rather , they require a separate linear network mirroring the initial network and feeding back error signals ( see Figure 2 a and Lillicrap et al . [ 2020 ] ) , or involved time-multiplexing of feedforward and error signals in the main network [ Lillicrap et al. , 2020 , Payeur et al. , 2021 ] . Our model is fundamentally different , because in our case , the dendritic signal onto neuron j is strictly ( W predct ) j which is a weighted sum of the main network activity and there is no need of a ( linear ) feedback network transmitting exact error values across many layers . Moreover , we show in simulations in section 4 , that the dendritic signal does not have to come from a layer above but that the prediction fed to layer l may come from the same layer . This shows that our learning rule works even in the complete absence of downward signaling from l + 1 to l. This last point is a significant difference to other methods that also calculate updates using only activities of the main network , but require tuned top-down connections to propagate signals downwards in the network hierarchy [ Kunin et al. , 2020 ] , such as methods in the difference target propagation family [ Lee et al. , 2015 , Bartunov et al. , 2018 , Golkar et al. , 2020 ] , contrastive divergence [ Hinton , 2002 ] and equilibrium propagation [ Scellier and Bengio , 2017 ] . Furthermore , the latter two require convergence to an equilibrium state for each input [ Laborieux et al. , 2021 ] . Our model does not require this convergence because it uses the recurrent dendritic signal ( W predct ) j only for synaptic plasticity and not for inference . Most previous learning rules which include global modulating factors interpret it as a reward prediction error [ Schultz et al. , 1997 , Gerstner et al. , 2018 , Pozzi et al. , 2020 ] . In this paper , we address self-supervised learning and view global modulating factors as broadcasting signals , modeling the self-awareness that something has changed in the stimulus ( e.g . because of a saccade ) . Hence , the main function of the broadcast factor in our model is to identify contrastive inputs , which avoids a common pitfall for self-supervised learning models : ‘ trivial ’ or ‘ collapsed ’ solutions , where the model produces a constant output , which is easily predictable , but useless for downstream tasks . In vision , we use a broadcast factor to model the strong , saccade-specific activity patterns identified throughout the visual pathway [ Kowler et al. , 1995 , Leopold and Logothetis , 1998 , Ross et al. , 2001 , McFarland et al. , 2015 ] . In other sensory pathways , like audition , this broadcast factor may model attention signals arising when changing focus on a new input source [ Fritz et al. , 2007 ] , cross-modal input indicating a change in head or gaze direction , or signal/speaker-identity inferred from blind source separation , which can be done on low-level representation with biologically plausible learning rules [ Hyvärinen and Oja , 1997 , Ziehe and Müller , 1998 , Molgedey and Schuster , 1994 ] . Our learning rule further requires this global factor to predict the absence or presence of a gaze change , hence conveying a change prediction error rather than classical reward prediction error . Here , we do not model the precise circuitry computing this factor in the brain , however , we speculate that a population of neurons could express such a scalar factor e.g . through burst-driven multiplexing of activity , see Payeur et al . [ 2021 ] and Appendix C. Our theory takes inspiration from the substantial progress seen in unsupervised machine learning in recent years and specifically from contrastive predictive coding ( CPC ) [ Van den Oord et al. , 2018 ] . CPC trains a network ( called encoder ) to make predictions of its own responses to future inputs , while keeping this prediction as different as possible to its responses to fake inputs ( contrasting ) . A key feature of CPC is that predicting and contrasting happens in latent space , i.e . on the output representation of the encoder network . This avoids modeling a generative model for perfect reconstruction of the input and all its details ( e.g . green , spiky ) . Instead the model is forced to focus on extracting high-level information ( e.g . cactus ) . In our notation , CPC evaluates a prediction W predct such that a score function u⌧t = z⌧ > W predct becomes larger for the true future ⌧ = t + t ( referred to as positive sample ) than for any other vector zt 0 taken at arbitrary time points t0 elsewhere in the entire training set ( referred to as negative samples in CPC ) . This means , that the prediction should align with the future activity zt+ t but not with the negative samples . Van den Oord et al . [ 2018 ] formalizes this as a softmax cross-entropy classification , which leads to the traditional CPC loss : LtCPC = log exput+ ttP ⌧2T expu ⌧ t , ( 2 ) where T = t t+ t , t 0 1 . . . t 0 N comprises the positive sample and N negative samples . The learned model parameters are the elements of the matrix W pred , as well as the weights of the encoder network . The loss function LtCPC is then minimized by stochastic gradient descent on these parameters using BP . Amongst numerous recent variants of contrastive learning [ He et al. , 2019 , Chen et al. , 2020 , Xiong et al. , 2020 ] , we focus here on CPC [ Van den Oord et al. , 2018 ] , for which a more local variant , Greedy InfoMax , was recently proposed by Löwe et al . [ 2019 ] . Greedy InfoMax ( GIM ) [ Löwe et al. , 2019 ] is a variant of CPC which makes a step towards local , BP-free learning : the main idea is to split the encoder network into a few gradient-isolated modules = W retro , kj ( atj ) xti+ t ( W retro z ) j W retro to avoid back-propagation between these modules . As the authors mention in their conclusion , “ the biological plausibility of GIM is limited by the use of negative samples and within-module backpropagation ” . This within-module back-propagation still requires a separate feedback network to propagate prediction errors ( Figure 2 a ) , but can be avoided in the most extreme version of GIM , where each gradient-isolated module contains a single layer ( layer-wise GIM ) . However , the gradients of layer-wise GIM , derived from Equation 2 , still can not be interpreted as synaptic plasticity rules because the gradient computation requires ( 1 ) the transmission of information other than the network activity ( see Figure 2 b ) , and ( 2 ) perfect memory to replay the negative samples zt 0 , as mentioned in the above quote ( see Appendix A for details ) . Overall it is not clear how this weight update of layer-wise GIM could be implemented with realistic neuronal circuits . Our CLAPP rule solves the above mentioned implausibilities and allows a truly local implementation in space and time .

Summary: The authors propose a biologically plausible implementation of contrastive self supervised learning. They solve the problem of remembering negative samples by relying on saccades, taking the post-saccade incoming signal to be a negative sample. They perform a number of comparisons showing the viability of their approach.  -------------------------------------- Post-rebuttal update: Due to the increased quality of the work stemming from added clarifications and a fuller discussion of biological plausibility, I have increased my score from 5 to 7.