Local plasticity rules can learn deep representations using self-supervised contrastive predictions
Authors: Bernd Illing
Jean Robin Ventura
Guillaume Bellec
Wulfram Gerstner
Venue: NeurIPS 2021 Poster (2021)
Keywords: Synaptic plasticity
Hebbian learning
deep learning
self-supervised learning
contrastive predictive coding

Abstract: Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes a simple layer-specific loss function and does not need to back-propagate error signals within or between layers. Instead, weight updates follow a local, Hebbian, learning rule that only depends on pre- and post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups of neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (i.e. rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video. 

Local plasticity rules can learn deep representations using self-supervised contrastive predictions
Bernd Illing
Jean Ventura
Guillaume Bellee"
Wulfram Gerstner"
(firstname. 1astemmeeeaaaameeppiieei
Department of Computer Science Department of Life Sciences École Polytechnique Fédérale Lausanne
1015 Switzerland
Abstract
Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes simple layer- specific loss function and does not need back-propagate error signals within or between layers. Instead, weight updates follow local, Hebbian, learning rule that only depends on pre- and post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (i.e. rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video.
Introduction
Synaptic connection strengths in the brain are thought to change according to 'Hebbian' plasticity rules (Hebb] [949]. Such rules are local and depend only the recent state of the pre- and post-
synaptic neurons (jööström et al. 2001 Caporale and Dan 2008 Markram et al., 201T]. potentially modulated by third factor related reward, attention other high-level signals [Kušmierz al.) Gerstner et al. 2018). Therefore, one appealing hypothesis is that representation learning in sensory cortices emerges from local and unsupervised plasticity rules.
2017
Following a common definition in the field [Fukushima 1988 Riesenhuber and Poggio [999]
LeCun 2012 Lillicrap et al. 2020). hierarchical representation (i) builds higher-level features out of lower-level ones, and (ii) provides more useful features in higher layers. Now there seems to be substantial gap between the rich hierarchical representations observed in the cortex and the representations emerging from local plasticity rules implementing principal/independent component analysis
Oja 1982 Hyvärinen and Ojal 1998), sparse coding [Olshausen and Field 1997 Rozell et al. 2008 or Low-ffattr analysis [Pöldak 1991] Wiskott and Sejnowski] 2002, Sprekeler et al. struggle especially when 'stacked', i.e. when asked to learn deep,
2007 Hebbian rules
hicrarchical representations.
sshared last author
35th Conference on Neural Information Processing Systems (NcurIPS 2021)

---

This performance gap is puzzling because there are learning rules, relying on back proppapoagttiion (BP). that can build hierarchical representations similar to those found in visual cortex Yamins et al. 2014 Zhuang et al. 2021]. Although some progress towards biologically plausible implementations of back-propagation has been made [Lillierap et al., 2016 Guerguiev al.
2017 Sacramento et al.
2018, Payeur et al., 202T], most models rely either on a neuron-specific error signal that needs to
be transmitted by a separate error network [Crick] 1989, Amit 2019, Kunin et al., 2020], or time-
multiplexing feedforward and error signals Lillicrap et al. 2020 Payeur et al. 2021]] Algorithms
like contrastive divergence Hinton 2002], contrastive Hebbian learning [Xie and Seung, 2003]
or equilibrium propagation (Scellier and Bengio 2017] use local activity exclusively to calculate
updates, but they require to wait for convergence to an equilibrium which is not appropriate for online
learning from quickly varying inputs.
The present paper demonstrates that deep representations can emerge from a local, biologically plausible and unsupervised learning rule, by integrating two important insights from neuroscience: First, we focus on self-supervised learning from temporal data - as opposed to supervised learning from labelled examples because this comes closest to natural data, perceived by real biological agents, and because the temporal structure of natural stimuli is a rich source of information. In particular, we exploit the self-awareness of typical, self-generated changes of gaze direction ('sac- cades') to distinguish input from a moving object during fixation from imput arriving after a saccade towards new object. In our plasticity rule, a global factor modulates plasticity, depending on the presence or absence of such saccades. Although we do not model the precise circuit that computes this global factor, we see it related to global, saccade-specins seeecaddsppcccccccccceeeee signals from motor areas, combined with surprise or prediction error, as in other models of synaptic plasticity [Angela and Dayan 2005
Nassar al. 2012 Heilbron and Meyniel 2019 Liakoni 202T]. Second, we notice that electrical signals stemming from segregated apical dendrites can modulate synaptic plasticity in biological neurons (Körding and König 2001 Major et al., 2013 enabling context-dependent plasticity.
Algorithmically, our approach takes inspiration from deep self-supervised learning algorithms that seek to contrast, cluster or predict stimuli in the context of BP [Van den Oord et al., 2018 Caron et al. 2018 Zhuang et al., 2019 Löwe et al. 2019 Interestingly, Löwe et al. [2019] demonstrated that such methods even work if end-to-end BP is partially interrupted. We build upon this body of work and suggest the Contrastive, Local And Predictive Plasticity (CLAPP) model which avoids BP completely, yet still builds hierarchical representations F
2 Main goals and related work
In this paper, we propose a local plasticity rule that learns deep representations. To describe our model of synaptic plasticity, we represent cortical area by the layer of deep neural network. The neural activity of this layer at time t is represented by the vector nnd = p(a""),, where p is a non-linearity and a = W' =1!1=1 is the vector of the respective summed inputs to the neurons through their basal dendrites W (the bias is absorbed into W'). To simplify notation, we write the pre-synaptic input as xdd = - and we only specify the layer index / when it is necessary.
Our plasticity rule exploits the fact that the temporal structure of natural inputs affects representation learning [Li and DiCarlo 2008). Specifically, we consider a scenario where an agent first perceives moving object at time ł (e.g. flying eagle in FFigure a), and then spontaneously decides to change gaze direction towards another moving object at time t + òt (e.g. saccade towards the elephant in Figure a). We further assume that the visual pathway is 'self-aware' of saccades due to saccade-specific modulation of processing [Ross et al., 2001].
In line with classical models of synaptic plasticity, we assumee that weight changes follow biologically plausible, Hebbian, learning rules [Hebb] 1949 Markram et al., 2011] which are local in space and time: updates A AW; of a synapse, connecting neurons i and j. can only depend the current activity of the pre-synaptic and post-synaptic neurons at time f, or slightly earlier at time & - St, and one or several widely broadcasted modulating factors [Urbanczik and Senn 2009 Gerstner et al. 2018
j4
Furthermore, we allow the activity of another neuron k to influence the weight update AW,., j.. as long as there is an explicit connection Wxpred from ls to j. The idea is to overcome the representational limitations of classical Hebbian learning by including dendritic inputs, which are thought to predict
Our code is available at http://hhhhh..ooooo.e.oo.//aa///

---

CLAPP biologically
b Neuronal circuity
CLAPP algorithmically
crea
prodict
CLAPPIA loss
prodict
W
prolici
Spicc deddrie
Chamge
3803
Eine
Enn
Eine
ppredictive'
update
gaze change
Gerditie
I+ Bt
1 8r
'cnntrasivve'
update
Frem arca
context
Julure
commrstiive somples
Figure I: Contrastive, local and predictive plasticity (CLAPP). a Perceiving a moving object (e.g. an eagle) times and I 51 leads to neural responses in the visual cortex. After gaze change ("saccade"), ), different object (elephant) is seen. (zoom) At each time step, pyramidal neurons integrate input activity the basal dendrites (matrix W of feedforward weights) and pass on their response to downstream areas (WI+1). At any point in time, neurons predict future neural responses through recurrent connections Wrrred These imputs target the apical dendrites and modulate ongoing synaptic plasticity through 'predictive' updates. Information about a saccade is transmitted by broadcast signal triggered at the moment of saccade initiation, which leads to 'contrastive' updates. As no external supervision or reward signals are provided, learning is self-supervised and local in time and space ('Hebbian'). с Algorithmically, an encoder network (Enc) produces 'context' representation c' at time t. Given c', CLAPP tries to predict the encoding of the future input 2 *++5 In case of a gaze change between and L + 8t, CLAPP seeks to keep the prediction as different as possible from the encoding of the upcoming contrastive sample.
the future somatic activity Körding and König 200T plasticity of the post-synaptic neuron [Larkum et al. 1999 Dudman et al.
Urbanczik and Senn
2014 and take part the
2007 Major et al. 2013
Hence we assume that each neuron j in a layer / may receive dendritic inputs (W Wprea ch*)); )5 coming
either from the layer above (c 2(1+1) or from lateral connections the same layer (c 2..)..
For algorithmic reasons, that we detail in section 3 we assume that the dendritic input (Wered cl+)) influences the weight updates AW,; the post-synaptic neuron j. but not its activity z/. This assumption is justified by neuroscientific findings that the inputs to basal and apical dendrites affect the neural activity and plasticity different ways [Larrum et al. 1999 Dudman et al. 2007 Major et al., 2013 Urbanczik and Senn 2014). In general, we do not rule out influence of dendritic activity on somatic activity in later processing phases, but see this beyond the scope of the current work.
Given these insights from neuroscience, we gather the essential factors that influence synaptic plasticity in the following learning rule prototype:
AW j
α
modulators
WPed
post prr
breadenst factors
dendritic prodiction
local-activity
The modulating broadcast factors are the same for large groups of neurons, for example all neurons in the same area, or even all neurons in the whole network. post and pre, are functions of the pre- and post- synaptic activities. At this point, we do not specify the exact timing between & and 12, as this will be determined by our algorithm in Section 3
Related work
Many recent models of synaptic plasticity fit an apparently similar learning rule
prototype [illierap et al. 2016 Nokland 2016 Roelisema and Holmaat 2018 Nokland and Eidnes
2019
Lillicrap et al.) 2020 Pozzi et al.) 2020 if we interpret the top-down signals emerging from the BP algorithm as the dendritic signal. However, top-down error signals BP are not directly related to the activity c' of the neurons in the main network during processing of sensory input. Rather, they require a separate linear network mirroring the initial network and feeding back error signals (see Figure 2 and Lillicrap et al. (2020) or involved mule-nuliipeexing of feedforward and error signals the main network [Lillicrap et al. 2020 Payeur et al. 2021). Our model fundamentally different, because in case, the dendritic signal onto neuron j strictly (Wereed c²); which is weighted
3

---

sum of the main network activity and there is no need of a (linear) feedback network transmitting exact error values across many layers.
Moreover, we show in simulations in incciionnn 4 that the dendritic signal does not have to come from layer above but that the prediction fed to layer may come from the same layer. This shows that our learning rule works even in the complete absence of downward signaling from + 1 to L. This last point is a significant difference other methods that also calculate updates using only activities of the main network, but require tuned top-down connections to propagate signals downwards in the network hierarchy (Kunin et al., [020], such as methods in the difference target propagation family [Lee et al. 2015] [Bartunov et al. 2018 Golkar et al., 2020), contrastive divergence (Hinton 2002] and equilibrium propagation [Scellier and Bengio 2017 Furthermore, the latter two require convergence to an equilibrium state for each imput [Laboriexx et al. 2021]. Our model does not require this convergence because uses the recurrent dendritic signal W pred preddc)) c²), only for synaptic
plasticity and not for inference.
Most previous learning rules which include global modulating factors interpret it as a reward predic- tion error [Schultz et al., 1997. Gerstner et al. 2018 Pozzi et al. 2020]. In this paper, we address self-supervised learning and view global modulatinn lating lactors as broadcasting signals, modeling the self-awareness that something has changed in the stimulus (e.g. because of a saccade). Hence, the main function of the broadcast factor in our model is to identify contrastive inputs, which avoids a common pitfall for self-supereiied learning models: 'trivial' or 'collapsed' solutions, where the model produces constant output, which is easily predictable, but useless for downstream tasks. In vision, we use a broadcast factor to model the strong, saccade-specifte activity patterns identified throughout the visual pathway (Kowler et al. 1995 Leopold and Logothetis) 1998 Ross et al., 2001 McFarland et al.) 2015]- In other sensory pathways, like audition, this broadcast factor may model attention signals arising when changing focus on a new input source (Fritz et al. 2007], cooal input indicating a change in head gaze direction, or signal/speaker-identiis
inferred from blind
source separation, which can be done on low-level representation with biologically plausible learning
rules [Hyvärinen and Oja, 1997 Ziehe and Müller 1998 Molgedey and Schuster 1994]. Our learning rule further requires this global factor to predict the absence or presence of a gaze change. hence conveying change prediction error rather than classical reward prediction error. Here, we do not model the precise circuitry computing this factor in the brain, however, we speculate that population of neurons could express such a scalar factor e.g. through burst-driven multiplexing of activity, see Payeur et al. [002]] and Appendixx C
Our theory takes inspiration from the substantial progress seen in unsupervised machine learning in recent years and specifically from contrastive predictive coding (CPC) [Van den Oord et al. 2018 CPC trains network (called encoder) to make predictions of its own responses to future inputs, while keeping this prediction as different as possible to its responses to fake inputs (contrasting). A key feature of CPC is that predicting and contrasting happens in latent space, i.c. the output representation of the encoder network. This avoids modeling a generative model for perfect recon- struction of the input and all its details (e.g. green, spiky). Instead the model is forced to focus on extracting high-level information (e.g. cactus). In our notation, CPC evaluates a prediction Wredd cf such that a score function ut = =T c' becomes larger for the true future 7 = + ôt (referred to as positive sample) than for any other vector 2 taken at arbitrary time points ' elsewhere in the entire training set (referred to as negative samples in CPC). This means, that the prediction should align with the future activity z zt-át but not with the negative samples. Van den Oord et al. (2018) formalizes this softmax cross-entropy classification, which leads to the traditional CPC loss:
++tt
C'pe = -log
exp u
ETET exp uf
where T T= ( .""...'' ..y) comprises the positive sample and N negative samples. The learned ... model parameters are the elements of the matrix Wrree, as well as the weights of the encoder network.
The loss function E'pe is then minimized by stochastic gradient descent on these parameters using BP. Amongst numerous recent variants of contrastive learning [He et al. 2019 Chen et al. 2020 Xiong et al. 2020), we focus here on CPC [Van den Oord et al. 2018), for which a more local variant, Greedy InfoMax, was recently proposed by Löwe al. [2019]
Greedy InfoMax (GIM) Löwe et al. 2019 a variant of CPC which makes a step towards local, BP-free learning: the main idea is to split the encoder network into a few gradient-isolate modules

---

FF-liee
b layer-wise GIM
CLAPP
Wrreed
p(a)
8
CPC loss network
W
x
8
X
A By pp(a))))) x2
AW, = (V.lbrell pl(a) oa
A AW = (Wrred " "- p(a)-af
+
Activity of main network
Error signal of error network
A Woree % of c
Figure 2: Comparison of weight updates a Networks trained with poockk-rppggtion (BP) or Feedback Alignment (FA)-like methods require separate error networks (red) for computing weight updates. Layer-wise GIM, with one layer per gradient-isolated module, does not transmit error signals across layers (i.e. modules) but requires (1) the transmission of information other than the network activity (red) and (2) perfect replay of negative samples. Thus, the resulting update computation needs separate loss network and cannot be interpreted local learning rule. Contrastive Local and Predictive Plasticity (CLAPP) calculates updates using locally and temporally available information: pre- and post-synaptic activity and predictive recurrent input onto the apical dendrite W c Global broadcasting factors "/t modulate plasticity depending on the presence or absence of saccade.
to avoid back prekappppagatoon between these modules. As the authors mention in their conclusion, "the biological plausibility of GIM is limited by the use of negative samples and within-modute back- propagation". This within-module back-propagation still requires separate feedback network to propagate prediction errors (Figure a). but can be avoided the most extreme version of GIM, where each gradient-isolated module contains a single layer (layer-wise GIM). However, the gradients of layer-wise GIM, derived from Equaation 2] still cannot be interpreted as synaptic plasticity rules because the gradient computation requires (I) the transmission of information other than the network activity (see Figure b). and (2) perfect memory to replay the negative samples z" as mentioned in the above quote (see (eppendii A for details). Overall it is not clear how this weight update of layer-wise GIM could be implemented with realistic neuronal circuits. Our CLAPP rule solves the above mentioned implausibilities and allows a truly local implementation in space and time.
Derivation of the CLAPP rule: contrastive, local and predictive plasticity
We now suggest a simpler contrastive learning algorithm which solves the issues encountered with layer-wise GIM and for which a gradient descent update is naturally compatible with the learning rule prototype from Equation I The most essential difference compared to CPC GIM is, that do not require the network to simultancously access the true future activity x -++8t and recall (or imagine) the network activity z" seen at some other time. Rather, we consider the naturalistic time-flow illustrated in IFigure Ia, where agent fixates moving animal for a while and then changes gaze spontaneously. In this way, the prediction Wrred e is expected to be meaningful during fixation, but inappropriate right after saccade. In our simulations, with subsequent frames from the same sample (e.g. different views of an eagle), and then abruptly changing to frames from another sample (c.g. different views of an elephant).
model this by feeding the network
We note that the future activity zt-ót and the context cf are always taken from the main feedforward encoder network. We focus on the case where the context stems from the same layer the future activity (c = = however, the model allows for the more general case, where the context stems from another layer (e.g. the layer above cdd = =11++)..
Derivation of the CLAPP rule from self-supervised learning principle Rather than using a global loss function for multi-class classification to separate the true future from multiple negative samples, as in CPC, we consider here a binary classification problem at every layer I: we interpret the
score function u² |+5,,, = W ctd the layer's 'guess' whether the agent performed fixation or a saccade. In Appendix C. we discuss how u4 ++t. could be (approximately) computed in real neuronal circuits. In short, every neuron i has access to its 'own' dendritic prediction stil 2
!+ stul =,, W. S of somatic activity [Urbanzzik and Senn 2014]. and the product 2;
5

---

can be seen as a coincidence detector of dendritic and somatic activity, communicated by specific burst signals Larkum et al. 1999]. These burst signals allow time-multiplexed communication Paayeur et al. 2021 of the products 2 2466,, snd of many neurons, which can then be summed by an "
interneuron representing u, (+66,.1
As mentioned in section 2 information about the presence absence of saccade between two time points is available in the visual processing stream and is modeled here by the variable y =-1 and y' +1, respectively. We interpret y' as the label of binary classification problem, characterized by the Hinge loss, and define the CLAPP loss at layer I as:
1LL CLAPP -max (0. -y-
y un a with
y' =+=+1 for fixation y¹ =
for saccade
We now derive the gradients of [Equation 3 with respect to the feedforward weights and show that gradient descent on this loss function compatible with the learning rule prototype suggested in [qqaatio Note that CLAPP optimises [quaation 3 for each layer / independently, without any gradient flow between layers. That being said, the following derivation is the same for every layer 1, which is why we omit the layer index / from here on.
Since chose formalize the binary classification with Hinge loss, the gradient vanishes when the classification is already correct: high score uf > during fixation (y' +1), low score -i < -1 after saccade (y' 1). Otherwise, it is Vu,+ot during fixation Vu,+ after saccade. In the 'predicted layer' 2, i.e. the target of the prediction, let W denote the feedforward weight from neuron i in the previous layer (with activity ar{) neuron j, with summed input at and activity zf. Similarly, in the 'predicting layer' c, i.e. the source of the prediction, let W denote the feedforward weight between the neuron / in the previous layer (with activity 2f and neuron k, with summed input dg of and activity c/. Therefore, .c as an upper index refers to the context layer, whereas c as a full-size letter refers to the respective neuronal activity. We then find the gradients with respect to these weights as:
OCCLAPP = +(Wpred ('(attr af450
(4)
OW
OCCLAPP = +(WPred T 2(+01) o' p((a c.t) re
(5)
aWe km
where the sign is negative during fixation and positive after a saccade. To change these equations into online weight updates, we consider the gradient descent update delayed by öt, such that AWJ;
SCCLAPD pr where is the learning rate. Let us define a modulating factor 7/t = y² : H where aW.
y² = +1 a network wide broadcast signal (self-awareness) indicating a saccade (-1) or a fixation (+1) and H' E {0. n/} is a layer-wide broadcast signal indicating whether the saccade or fixation was correctly classified as such. In this way, Equation becomes a weight update which follows strictly the ideal learning rule prototype from Equation
AW
(Worod ct 8t); pf (at))a)
breadcast factors
dendritic prediction
local activity
For the updates of the connections onto the neuron ct. which emits the prediction rather than receiving it, our theory in Equutiion F]equiress the opposite temporal order and the transmission of the information in the opposite direction: from zt back to c'. Since connections the brain are
unidirectional Lillicrap et al. 2016 we introduce another matrix Wretro which replaces W/rred in the final weight update. Given the inverse temporal order, we interpret Wrerro as a retrodiction rather than prediction. In Appendixx C we show that using Wreetro minimises loss function of the same form as Equation and empirically performs as well as using Wrree T The resulting weight update satisfies the learning rule prototype from Equation as it can be written:
AWe,t km
Wereroo gt
(a (( ) s,t-
brondeust Factirs
dendritic retrudiction
local activity

---

a
layer
lnyer
layer
neuron
bird
stip
doer
trick
airplane
horse
neuron
monkey
b
CLAPP CLAPP layer
CLAPP layer
Random init.
hinds
planes
'vehicles
"tury animals'
Figure 3: Hierarchical representations learned by CLAPP. Red boxes in STL-10 images indicate patches that best activate specific neuron (rows) in a network trained with CLAPP. Layer extracts simple features like gratings uniform patches, higher layers extract richer features like parts of objects. b 2-dimensional t-SNE projection of neuronal activities at different layers unveils increasing representational structure in higher layers (every dot represents input image). Note that CLAPP has not seen any class labels during training.
In the (standard) case, where context and predicted activity are from the same layer (c = 21,,, W and W are the same weights and the updates Equutiio [annatiooooonn Jree added up linearly.
The prediction and retrodiction weights, Wrred and Wzetro, respectively, are also plastic. By deriving the gradients of CLLAPD with respect Wpred, find an even simpler Hebbian learning rule for these weights:
AWPred -W
St
pre and post
where neuron l in the predicting layer e is pre-synaptic (post-synaptic) and neuron j in the predicted (retrodiction weights Wreetro). jk
layer is post-synaptic (pre-synaptic) for the prediction weights Wirred
Note that the update rules for Wiieed and Weetro reciprocal, method that leads mirrored connections, given small enough initialisation (Burbank 2015
Amit 2019
Pozzi et al.
2020
We emphasize that all information needed calculate the above CLAPP updates (Equations spatially and temporally available, either as neuronal activity at time t, or as traces of recent activity Eagaatiotnngutonnnnnnn
8t) [Gerstner et 2018 In order implement Equation 6. the dendritic prediction has to be retained during öt. However, we argue that dendritic activity can outlast (50-100 ms) somatic neuronal activity (2-10 ms) [Major al. 2013). which makes predictive input from several time steps in the past (t 8t) available at time f.
Generalizations While the above derivation considers fully-connected feedforward networks, apply analogous learning rules to convolutional neural networks (CNN) and recurrent neural networks (RNN). Analyzing the biological plausibility of the standard spatial weight sharing and spatial MeanPooling operations in CNNs is beyond the scope of the current work. Furthermore, we discuss Appendix C that MaxPooling can be interpreted as a simple model of lateral inhibition and that gradient flow through such layers is compatibility with the learning rule prototype in Equation
To obtain local learning rules for RNNs, we combine CLAPP with the e-prop theory [Bellec et al., 2020), which provides a biologically plausible alternative to BP through time: gradients can be propagated forward in time through the intrinsic neural dynamics of neuron using eligibility traces. The propagation of gradients across recurrently connected units is forbidden and disabled. This yields biologically plausible updates in GRU units, explained in Appendix C
4 Empirical results
Building hierarchical representations We first demonstrate numerically, that CLAPP yields deep hierarchical representations, despite using local plasticity rule compatible with Equation We
7

---

report here the results for c¹ = = i.c. the dendritic prediction in Equation is generated from lateral connections and the representations qualitatively similar results with eld 2(1+1 (i.e. the dendritic prediction is generated from one layer above), suggesting that top-down signaling neither necessary for, nor incompatible with, our algorithm (also see Appendix Cp.
the same layer. We note, however, that we obtained
We first consider the STL-10 image dataset [Coates et al.) these static images, we follow Héeaff et al. [011 and Löwe et al. [2019]: each image is split into 16 16 patches and the patches are viewed one after the other in a vertical order (one time step is one patch). Other hyper- parameters and data-augmennttion are taken from Löwe et al. 2019 see Appendix B We then train a 6-layer VGG-like (Simonyan and Zisserman 2015] encoder (VGG-6) using the CLAPP rule (Equations 6 Sb. Training is performed on the unlabelled part of the STL-10 dataset for 300 epochs. We use GPUs (NVIDIA Tesla V100-SXM2 32 GB) for data-paralled training, resulting in a simulation time of around 4 days per run.
2011]. To simulate a time dimension in
In order to study how neuronal selectivity changes over layers, we select neurons randomly and show image patches which best activate these neurons the most (rows in Figure 3aa). As expected for visual hierarchy, first-layer neurons (first column in Figure 3 a) are selective horizontal or vertical gratings, or homogeneous colors. In the third layer of the network (second column), neurons start to be selective to more semantic features like grass, or parts of vehicles. Neurons in the last layer (third column) are selective to specific object parts (e.g. wheel touching the road). The same analysis for a random, untrained encoder does not reveal a clear hierarchy across layers, see Appendix C
To get a qualitative idea of the learned representation manifold, we use the non-linear dimension reduction technique t-SNE [Van der Maaten and Hinton 2008 visualise the encodings of the (labeled) STL-10 test set in Figure 3 3/b. We see that the representation in the first layer is mostly unrelated the underlying class. In the third and sixth layers' representation, a coherent clustering emerges, yielding an almost perfect separation between furry animals and vehicles. This clustered representation remarkable since the network has never seen class labels, and was never instructed to separate classes, during CLAPP training The representation of the same architecture, but without training (Random init.), shows that a convolutional architecture alone does not yield semantic features.
To produce a more quantitative measurement of the quality of learned representations, we follow the methodology of Van den Oord al. (018 and Löwe et al. 2019]: : we freeze the trained encoder weights and train a linear classifier to recognize the class labels from each individual layer (Figure 4p. As expected for deep representation, the classification accuracy increases monotonically with the layer number and only saturates at layers 5 and 6. The accuracies obtained with layer-wise GIM are almost indistinguishable from those obtained with CLAPP. It only at the last two layers, that layer-wise GIM performs slightly better than CLAPP: yet GIM has multiple biologically implausible features that are removed by CLAPP. As further benchmark, we also plot the accuracies obtained with an encoder trained with greedy supervised training. This method trains each layer independently using a supervised classifier each layer, without BP between layers, which results in an almost local update (see) Löwe et al. [2019) and [ppendix B). We find that accuracy is overall lower and saturates already at layer 4. On this dataset, with many more unlabelled than labelled images, greedy supervised accuracy is almost 10% below the accuracy obtained with CLAPP. Again, we see that a convolutional architecture alone does not yield hierarchical representations, as performance decreases at higher layers for fixed random encoder.
Comparing CPC and CLAPP Since CLAPP can seen as a simplification of CPC (or GIM) we study four algorithmic differences between CPC and CLAPP individually. They are: (1) Gradients in CLAPP (layer-wise GIM) cannot flow from a layer to the next one, as opposed to BP in CPC, (2) CLAPP performs a binary comparison (fixation saccade) with the Hinge loss, whereas CPC does multi-class classification with the cross entropy loss, (3) CLAPP processes a single input at a time, whereas CPC uses many positive and negative samples synchronously, and (4) we introduced Wretro to avoid the weight transport problem in Wrrred
We first study features (1) and (2) but relax constraints (3) and (4). That means, in this paragraph, we allow a fixation and N = 16 synchronous saceades and set Wretto = T. We refer to Hinge Loss CPC as the algorithm minimizing the CLAPP loss (Euaation 3) but using end-to-end BP. CLAPP-s (for synchronous) applies the Hinge Loss to every layer, but with gradients blocked between layer. We find that the difference between the CPC loss and Hinge Loss CPC is less than 1 1%, see
8

---

Figure 4: CLAPP stacks well: representations after stacking up to 5 layers increase perfor- mance of a linear classifier STL-10,, despite the local learning rule (blue and orange lines), while performance decreases for convolutional network with weights fixed at random initial- isation (dotted). Greedy supervised training (see Axeendipd B) also stacks, but already sat- urates layer 4 and shows overall lower per- formance. Direct linear classification on image pixels (black star) and CPC performance after 6 layers (gray star) serve as upper and lower performance bounds, respectively.
CLAPP
Greedy supervised
CPC Diect class.
CPPSS
Random init.
Layer-wise GIM
80
test accuracy [%)
70
60
50
40
30
20
1
3
classify from layer #
Table
In contrast, additional blocking of gradients between layers causes a performance drop of almost 5% for both loss functions. We investigate how gradient blocking influences performance with series of simulations, splitting the 6 layers of the network into two or three gradient isolated modules, exploring the transition from Hinge Loss CPC to CLAPP-s. Performance drops monotonously but not catastrophically, the number of gradient blocks increases TTblle T).
CLAPP's temporal locality allows the interpretation that an agent alternates between fixations and saccades, rather than perfect recall and synchronous processing of negative samples, required by CPC. To study the effect of temporal locality, we apply features (2) and (3) and relax the constraints (1) and (4). The algorithm combining temporal locality and the CLAPP loss function is referred to as time-local Hinge Loss CPC. We find that the temporal locality constraint decreases accuracy by 1.2% compared to Hinge Loss CPC. The last feature introduced for biological plausibility is using the matrix Weero and we observe almost no difference in classification accuracy with this alternative (the accuracy decreases by 0.1%). Conversely, omitting the update in [Equation 7 entirely, i.c. setting the retrodiction Wretto = O, compromises accuracy by 2% compared to vanilla Hinge Loss CPC.
When combining all features (1) to (4), we find that the fully local CLAPP learning rule leads to an accuracy of 73.6% at layer 5. We conclude from the analysis above, that the feature with the biggest impact on performance is (1): blocking the gradients between each layer. However, despite the performance drop caused by blocking the gradients, CLAPP still stacks well and leverages the depth of the network FFiuue 4). All other features (2) (4), introduced to derive weight update compatible with our prototype Equation T,, only caused a minor performance loss.
Applying CLAPP to speech and video We now demonstrate that CLAPP is applicable to other modalities like the LibriSpeech dataset of spoken speech [Panayotov et al. 2015) and the UCF- 101 dataset containing short videos of human actions [Soomro et al.) 2012 When applying CLAPP to auditory signals, we do not explicitly model the contrasting mechanism (saccades in the vision task; see discussion in Inlsection22 2 for the auditory pathway) and hence consider the application of CLAPP as benchmark application, rather than a neuroscientifically exact study. To increase computational efficiency, we study CLAPP-s similar results for CLAPP, given enough time to converge. We use the same performance criteria as for the image experiments and summarize results in Table I for details see Appendix B
speech and video. Based on the image experiments, we expect
For the audio example, we use the same architecture as Van den Oord et al. 2018 and Löwe et al. (019]: multiple temporal Id-convolution layers and one recurrent GRU layer on top. As in the feedforward case, CLAPP still optimises the objective of Eqquation 3] For the Id-convolution layers, the context c' is computed for the image task, for the last layer, cf is the output of the recurrent GRU layer. We compare the performance of the algorithms on phoneme classification (41 classes) using labels provided by Van den Oord et al. (2088). In this setting, layer-wise training lowers performance by only 0.4% for waser-wise GIM, and by 1.1% for CLAPP-s. Implemented as such, CLAPP-s still relies on BP through time (BPTT) train the GRU layer. Using CLAPP-s with biologically plausible e-prop [Bellec et al. 2020], instead of non-local BPTT, reduces performance by only 3.1 %, whereas omitting the GRU layer compromises performance by 9.3 %, see Appendix C.
Applying CLAPP to videos especially interesting because their temporal sequence of images perfectly fits the scenario of Figure Da. In this setting, we take inspiration from Han et al. 2019)]

---

Table CLAPP performs best among methods that are local in space and time. Linear classification
test accuracy [%] on STL-10, phone classification on LibriSpeech. and video human action recognition
on UCF-101 using features from the encoder trained with different methods. STL-10, performance
degrades gracefully with the number of gradient-isolated modules in the VGG-6 encoder (at fixed number of encoder layers). Greedy supervised training uses BP auxiliary classifier networks
('almost' local in space). For LibriSpeech, BP through time is used (can be avoided, see see Appendix Cj.
Values with * are taken from
Löwe al. 2019 For simulation details, see Appendix B
local in space?
Method
STL-10
LibriSpeech
UCF-101
time?
Chance performance
10.0
2.4
0.99
Random init.
21.8
27.7*
30.5
MFCC
39.7*
Greedy supervised
(V)
66.3
73.4*
Supervised
X
/
73.2
77.7*
51.5
CPC
X
X
81.1
64.3
35.7
Layer-wise GIM
75.6
63.9
41.2
Hinge Loss CPC (ours)
80.3
62.8
36.1
CLAPP-s (2 modules of 3 layers)
77.6
CLAPP-s (3 modules of 2 layers)
X
77.4
CLAPP-s (ours)
75.0
61.7
41.6
time-local Hinge Loss CPC (ours)
X
79.1
CLAPP (ours)
73.6
and use a VGG-like stack of 2D and 3D convolutions to process video frames over time. On this task (101 classes), we found layer-wise GIM and CLAPP-s to achieve higher downstream classification accuracy than their end-to-end counterparts CPC and Hinge Loss CPC (see Table D. in line with the findings on STL-10 10 in Löwe et al. [019]. On the other hand, we found that CLAPP-s requires more negative samples (i.e. more simultaneous comparisons of positive and negative samples) on videos than on STL- 10 and LibriSpeech. Under the constraint of temporal locality in fully local CLAPP, this leads to prohibitively long convergence times in the current setup. However, since CLAPP linearly combines updates stemming from multiple negative and positive samples, we eventually expect the same final performance, if we run the online CLAPP algorithm for sufficiently long time.
5 Discussion
We introduced CLAPP, self-supervised and biologically plausible learning rule that yields deep hierarchical representations neural networks. CLAPP integrates neuroscientific evidence on the dendritic morphology of neurons and takes the temporal structure of natural data into account. Algorithmically, CLAPP minimises a layer-wise contrastive predictive loss function and stacks well on different task domains like images, speech and video despite the locality in space and time.
While the performance loss due to layer-wise training a limitation of the current model, the stacking property is preserved and preliminary results suggest improved versions that stack even better (e.g. using adaptive encoding patch sizes). Note that CLAPP models self-supervised learning of cortical hierarchies and does not provide a general credit assignment method, such as BP. However, the representation learned with CLAPP could serve as an initialisation for transfer learning, where the encoder is fine-tuned later with standard BP. Alternatively, fine-tuning could even start already during CLAPP training. CLAPP in its current form is data- and compute-intensin however, it runs on unlabelled data with quasi infinite supply, and is eligible for neuromorphie hardware, which could decrease energy consumption dramatically [Wunderlich et al. 2019
Classical predictive coding models alter neural activity at inference time, e.g. by cancelling predicted future activity [Rao and Ballard 1999 Keller and Mrsic-Flogel] 2018). Here, we suggest different, perhaps complementary, role of predictive coding in synaptic plasticity, where dendritic activity predicts future neural activity, but directly enters the learning rule (Körding and König 2001 Urbanczik and Senn 2014). CLAPP currently does not model certain features of biological neurons, e.g. spiking activity or long range feedhack, and requires neurons to transmit signals with precise value and timing. We plan to address these topics in future work.
10

---

Acknowledgments and Disclosure of Funding
This research was supported by the Swiss National Science Foundation (no. 200020 (1120_184615) and the Intel Neuromorphic Research Lab. Many thanks to Sindy Löwe, Julie Grollier, Maxence Ernoult, Franz Scherr, Johanni Brea and Martin Barry for helpful discussions. Special thanks Sindy Löwe for publishing the GIM code.
References
Y. Amit. Deep learning with asymmetric connections and hebbian updates. Frontiers in computational neuroscience, 13:18, 2019.
J. Y. Angela and P. Dayan. Uncertainty, neuromodulation, and attention. Neuron, 46(4):681-692 2005.
Bartunov, A. Santoro, B. A. Richards, L. Marris, Hinton, and Lillicrap. Assessing the scalability of biologically-motivaee deep learning algorithms and architectures. arXiv preprint arXiv:1807.04587, 2018.
G. Bellec, F. Scherr, Subramoney, Hajek, Salaj. R. Legenstein, and W. Maass. A solution to the learning dilemma for recurrent networks. Nature communications, 11, 2020.
K. S. Burbank. Mirrored stdp implements autoencoder learning in a network of spiking neurons. PLoS computational biology, 1 1004566, 2015.
N. Caporale and Dan. Spike timing dependent plasticity: hebbian learning rule. Annu. Rev. Neurosci,, 31:25 46, 2008.
M. Caron, P. Bojanowski, A. Joulin, and M. Douze. Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision (ECCV), pages 132-149, 2018.
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. Simple Framework for Contrastive Learning of Visual Representations. Int. Conf. Mach. Learn., 2020. URL https://github. com/ google-research/ste simmlr.
A. Coates, A. Ng, and H. Lee. An analysis of single-layer networks in unsupervised feature learning. In Proceedings the fourteenth international conference artificial intelligence and stafistics, pages 215-223, 2011.
E. Crick. The recent excitement about neural networks. Nature, 3376600) 1989.
W. Dabney, Z. Kurth-Nelson, N. Uchida, C. K. Starkweather, D. Hassabis, R. Munos, and M. Botvinick. A distributional code for value in dopamine-based reinforcement learning. Nature, (77(7792)667--677) 2020.
T. Dudman, D. Tsay, and S. A. Siegelbaum. Role for Synaptic Inputs at Distal Dendrites: Instructive Signals for Hippocampal Long-Temn Plasticity. Neuron, 56(5):866-87) -799, dec 2007. ISSN 08966273. doi: 10. (0.1006/j.neul w0t.m..000..000 10.02...
Földiák. Learning Invariance from Transformation Sequences. Neural Comput., 3(2): 194-200, 1991. ISSN 0899-7667, doi: 10. 16.1ec2/ http://ecoo.99113..... 1991.3.2.194.
URL http: /ttt:://ww //ww
mit mitpressjournalls wwwg//o///ioeeo ww11//eccc.... 1991.3.2. 2.
B. Fritz, M. Elhilali, S. V. David, and S. A. Shamma. Auditory attention focusing the searchlight on sound. Curr. Opin. Neurobiol, 17(4):437-455, aug 2007. ISSN 09594388. doi: 10. 10116/j.conb 2007.07.011.
K. Fukushima. Neocognitron: A hicrarchical neural network capable of visual pattern recognition. Neural Networks, 12):119-130, 1988. ISSN 08936080, doi: 11.10166099--00008889900000101
W. Gerstner, M. Lehmann, V. Liakoni, D. Corneil, and J. Brea. Eligibility traces and plasticity on behavioral time scales: experimental support of neohebbian three-factor learning rules. Frontiers in neural circuits, 12:53, 2018.

---

S. Golkar, D. Lipshutz, Y. Bahroun, A. M. Sengupta, and D. B. Chklovskii. A biologically plausible neural network for local supervision in cortical microcircuits. arXiv preprint arXiv:2011.10311, 2020.
Guerguiev, T. P. Lillicrap, and B. Richards. Towards deep learning with segregated dendrites. Elife, 6:e22901, 2017.
T. Han, W. Xie, and A. Zisserman. Video representation learning by dense predictive coding. arXiv preprint, 2019.
K. He, H. Fan, Y. Wu, S. Xie, and R. Girshick. Momentum Contrast for Unsupervised Visual Representation Learning. In Conf. Comput. Vis. Pattern Recognit. pages 9726-9735 IEEE Computer Society, nov 2019. URL http:/ /ttt:://axxi...o pexiv.orge abs/ 1911 05722
D. O. Hebb. The Organization of Behavior. 1949. ISBN 0805843000.
M. Heilbron and F. Meyniel. Confidence resets reveal hierarchical adaptive learning in humans, PLoS computational biology, 15(4)ee000997, 2019.
O. J. Hénaff, A. Razavi, C. Doersch, S. M. A. Eslami, and A. van den Oord. Data-Efficient Image Recognition with Contrastive Predictive Coding. arXiv Prepr, 2019.
G. Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771-1800, 2002.
A. Hyvärinen and Oja. fast fixed-point algorithm for independent component analysis. Neural computation, 9(7): 1483-199, 1997.
A. Hyvärinen and E. Oja. Independent component analysis by general nonlinear Hebbian-like learning rules. Signal Processing, 64:301-313, 1998.
G. B. Keller and T. D. Mloic-Flogel. Predictive Processing: A Canonical Cortical Computation. Neuron, 100(2):424-436 2018. ISSN 10974199. doi: 10.1016/j.euron2218... https://doi. org/ 10. 10. 1016/j neuron.
URL
K. Körding and P. König. Neurons with Two Sites of Synaptic Integration Learn Invariant Representations. Neural Comput., 13:2823-283 28499, 2001.
E. Kowler, E. Anderson, Dosher, and E. Blaser. The role attention the programming of saccades. Vision Res., 35(13):1897-1916 1995. ISSN 00426989. doi: 10.1016/0042-6989994 00279-U, URL
ncbi nlm, nih, gov/7660596/
D. Kunin, A. Nayebi, J. Sagastuy-Brena, S. Ganguli, J. M. Bloom, and D. L. K. Yamins. Two Routes to Scalable Credit Assignment without Weight Symmetry. In ICMI,, 2020. http
//arxiv.. org/ abs/2003. 01513
E. Kuśmierz, T. Isomura, and Toyoizumi. Learning with three factors: modulating Hebbian plasticity with errors. Curr. Opin. Neurobiol, 46: 17-1177 oct 2017. ISSN 18736882. doi: 10.1016//.conb.201
wtwp://.coob.017008.222. 7.00000000 URL htl.:ttp::/ http:tt/txxd.oo.oorott.. doi org.rr.gg/.......... 10. 1016/j conb. 2017 00..0.8.02
A. Laborieux, M. Emoult, B. Scellier, Y. Bengio. J. Grollier, and D. Querlioz. Scaling equilibrium propagation deep convnets by drastically reducing its gradient estimator bias. Frontiers in neuroscience, 15:129, 2021.
M. E. Larkum, J. J. Zhu, and B. Sakmann. A new cellular mechanism for coupling inputs arriving different cortical layers. Nature, 398(6725) 341, 1999. ISSN 00280836... doi: 10.1038/1 8686.
Y. LeCun. Learning invariant feature hicrarchies. In European conference computer vision, pages 496-505. Springer, 2012.
D.-H. Lee, S. Zhang, A. Fischer, and Y. Bengio. Difference target propagation. In Joint european conference on machine learning and knowledge discovery in databases, pages 498-515. Springer, 2015.
12

---

D. A. Leopold and N. K. Logothetis. Microsaccades differentially modulate neural activity in the striate and extrastriate visual cortex. Experimental Brain Research, 123(3):341-345, 1998.
N. Li and J. J. DiCarlo. Unsupervised natural experience rapidly alters invariant object representation in visual cortex. science, (5165555: 2008.
V. Liakoni. Surprise-based model estimation reinforcement learning: algorithms and brain signatures. Technical report, EPFL, 2021.
V. Liakoni, A. Modirshanechi, W. Gerstner, and J. Brea. Learning in volatile environments with the bayes factor surprise. Neural Computation, 33(2):269-344 2021.
T. P. Lillicrap, D. Cownden, D. B. Tweed, and C. J. Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature communications, 7(1):1-10, 2016.
T. P. Lillicrap, A. Santoro, L. Marris, C. J. Akerman, and G. Hinton. Backpropagation and the brain. Nature Reviews Neuroscience, 21(6):335-346) 2020.
Löwe, O'Connor, and B. S. Veeling. Putting An End End-to-End: Gradient- Learning of Representations Advances in neural information processing systems, 2019.
G. Major, M. E. Larkum, and J. Schiller. Active properties of neocortical pyramidal neuron den- drites. Annual review of neuroscience, 36:1 36:1-24, jul 2013. ISSN 1545-4126. doi: 10. 146/ annurev mnnurev-neuro-tee ceuro-062111-15000
URL WRLLhttp:// http: //www www ncbi mlm. mih pwt.goppbbbbme//841177777
H. Markram, W. Gerstner, and P. Sjöström. history of ppike-timing-eependern plasticity, Frontiers in synaptic neuroscience, 3:4, 2011.
J. M. McFarland, A. G. Bondy, R. C. Saunders, B. G, Cumming, and D. A. Butts. Saccadic modulation of stimulus processing in primary visual cortex. Nature communications, 6(1):1-14, 2015.
L. Molgedey and H. G. Schuster. Separation of mixture of independent signals using time delayed correlations. Physical review letters, 72(23):3634, 1994.
M. R. Nassar, K. M. Rumsey, R. C. Wilson, Parikh, B. Heasly, and J. Gold. Rational regulation of lcarning dynamics by pupil-linked arousal systems. Nature neuroscience, 1577):100--0466 2012.
A. Nøkland. Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems, pages 1037-1045. 2016.
A. Nøkland and L. Eidnes. Training Neural Networks with Local Error Signals. International Conference Machine Leurning, 2019. doi: S0Gi:199010066666
E. Oja. A simplified neuron model principal component analyzer. J. Math. Biol., 1:267-273. 1982.
B. A. Olshausen and D. Field. Sparse coding with an overcomplete basis set: A strategy employed by VI? Vision Res., 37(23):3311-22 1-3325, 1997. ISSN 00426989, doi: (0.1016/50042-69881177 00169-7.
D. Ostwald, B. Spitzer, M. Guggenmos, T. Schmidt, S. J. Kiebel, and F. Blankenburg. Evidence for neural encoding of bayesian surprise in human somatosensation. Neurolmage, 6771):177-188, 2012.
V. Panayotov, G. Chen, Povey, and Khudanpur. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE International Conference Acoustics, Speech and Signal Processing (ICASSP), pages 5206-5210. IEEE, 2015.
A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, and A. Lerer. Automatic differentiation in pytorch. 2017.
A. Payeur, Guerguiev, Zenke, B. A. Richards, and Naud. Burst-dependent synaptic plasticity can coordinate learning in hicrarchical circuits. Nature neuroscience, pages 1-10, 2021.
13

---

I. Pozzi, S. M. Bohté, and P. R. Roelfsema. Attention- Brain Propagation: How the brain can implement reward-basse error backpropagation. In NewrIPS, 2020.
R. P. Rao and D. H. Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-fiele effects. Nat. Neurosci., 2(1), 1999. URL http://neurosci nature. com
M. Riesenhuber and T. Poggio. Hierarchical models of object recognition in cortex. Nat. Neurosci., (11):1019 25, 1999. ISSN 1097-6256. doi: 10.1038/14819. URL http::tttpttt..// ncbi .nlm. nih
gov/pubmed/1052633
P. R. Roelfsema and A. Holtmaat. Control of synaptic plasticity in deep cortical networks. Nat. Rev. Neurosci., 19, 2018. doi: 10.1038/mm.2018.66 URL www. nature. com/ nrn
J. Ross, M. C. Morrone, M. E. Goldberg, and D. C. Burr. Changes in visual perception at the time of saccades. Trends neurosciences, 24(2):1 13-121, 2001.
C. J. Rozell, D. H. Johnson, R. G. Baraniuk, and B. A. Olshausen. Sparse coding via thresholding and local competition in neural circuits. Neural computation, 20(10):2526-25) 2563, 2008
Sacramento, R. Costa, Y. Bengio, and W. Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. In Advances in neural information processing systems, pages 8721-8732 2018.
B. Scellier and Bengio. Equilibrium propagation: Bridging the gap between energy- models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
W. Schultz, P. Dayan, and P. Montague. neural substrate of prediction and reward. Science, 275 (5306):1593-159,, 1997.
P. Schwartenbeck, T. FitzGerald, R. Dolan, and K. Friston. Exploration, novelty, surprise, and free energy minimization. Frontiers in psychology, 4:710, 2013.
K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. In ICLR, 2015. URL http:://wwwrrbbttttt robots. . ac..ackk/
Sjöström, G. G. Turrigiano, and S. B. Nelson. Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron, (15/6):11491111 2001.
K. Soomro, R. Zamir, and M. Shah. Ucf101: dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv: 2012.
H. Sprekeler, C. Michaelis, and L. Wiskott. Slowness: An Objective for Plasticity? PLoS Comput. Biol., 362) 2007. ISSN 1553-734X. doi: 10. 10.133//jurrnal.pe 0030112. URL http://dx plos www/o0.g//.....o 1371/ pcbi 0030112
R. Urbanezik and W. Senn. Reinforcement learning in populations of spiking neurons. Nature neuroscience, 12(3):250-22 252-22,,, 2009.
R. Urbanczik and W. Senn. Learning by the dendritic prediction of somatic spiking. Neuron, 81(3); 521-528, 2014.
A. Van den Oord, Y. Li, and O. Vinyals. Representation Learning with Contrastive Predictive Coding. arXiv Prepr., 2018.
L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
L. Wiskott and T. Sejnowski. Slow Feature Analysis Unsupervised Learning of Invariances. Neural Comput., 770:715-770, 2002.
T. Wunderlich, A. Kungl, E. Müller, A. Hartel, Y. Stradmann, S. A. Aamir, A. Grübl, A. Heimbrecht, K. Schreiber, D. Stöckel, et Demonstrating advantages of neuromorphic computation: a pilot study. Frontiers in neuroscience, 13:260, 2019.
14

---

X. Xie and H. S. Seung. Equivalence of backpropagationn and contrastive hebbian learning in layered network. Neural computation, 15(2):41-454, 2003.
Y. Xiong, M. Ren, and R. Urtasun. LoCo: Local Contrastive Representation Learning. Advances in neural information processing systems, 2020.
D. L. Yamins, H. Hong, C. F. Cadieu, E. A. Solomon, D. Seibert, and DiCarlo. Performance- optimized hierarchical models predict neural responses in higher visual cortex. Proceedings of the National Academy of Sciences, (11(23):8619-866) 2014.
C. Zhuang, A. L. Zhai, and Yamins. Local aggregation for unsupervised learning of visual embeddings. In Proceedings of the IEEE International Conference on Computer Vision, pages 6002-6012, 2019.
C. Zhuang. Yan, A. Nayebi, M. Schrimpf, M. C. Frank, J. J. DiCarlo, and Yamins. Unsuper- vised neural network models of the ventral visual stream. Proceedings of the National Academy of Sciences, 118(3), 2021.
A. Ziehe and K. Müller. Tdsep an efficient algorithm for blind separation using time structure. In International Conference on Artificial Neural Networks, pages 675-680. Springer, 1998.
15


Reviews:
[{"review_id": "woJAXINvE0F", "reviewer": "Reviewer_5YhK", "summary": "The authors propose a biologically plausible implementation of contrastive self supervised learning. They solve the problem of remembering negative samples by relying on saccades, taking the post-saccade incoming signal to be a negative sample. They perform a number of comparisons showing the viability of their approach.\n\n--------------------------------------\nPost-rebuttal update:\nDue to the increased quality of the work stemming from added clarifications and a fuller discussion of biological plausibility, I have increased my score from 5 to 7.", "questions": "", "limitations": "", "rating": 7, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The paper is novel, interesting and well written. In particular I really like the idea of using saccades to solve the problem of memorizing negative samples. In general I think the paper is strong but to be acceptable for submission a number of claims need to be either toned down or further substantiated, see below.\n\n### Strenghts \n(in no particular order).\n* This is a solid work in a relevant field. In particular, the authors astutely take advantage of signals already available in the visual cortex to circumvent the negative sample problem.\n* The experiments are more or less thorough. I would have liked to see more results and error bars but when experiments take 4 days on 4 V100s, there is a limit to what can feasibly be done. \n* The ablation study is also more or less convincing of the points the authors want to make.\n\n### Weaknesses \n(in the order of appearance in the paper.)\n* The authors criticize prior work for the need to wait for convergence (line 34). However, the only reason their model avoids this is a simplifying assumption mentioned in line 77, the assumption that lateral and feedback connections do not affect the activity. In other words, if the authors relax this simplifying assumption (for more realism for example), their dynamics would again be recurrent and they would find themselves in the same situation as prior work.\n* Some references for recent work are missing. For instance, there are biologically plausible models that do not have the issues mentioned in lines 101-102 (e.g. https://arxiv.org/abs/2011.15031, although the author's proposed work is more general than this since 2011.15031 only considers shallow networks.)\n* Applying the proposed algorithm to auditory pathways seems a stretch (lines 108-109). In this case, as opposed to the auditory pathway where the negative sample can be deduced based on saccades, the presence of a negative sample has to be deduced. The authors assume that this is a solved problem, however, I think this is very non-trivial. Especially since the goal of the proposed circuit itself is to be able to do things like distinguishing different input sources. This can be alleviated by using cross-modality inputs, e.g. visual cues to distinguish sources, but in this case, competing algorithms would also be take advantage of the cross-modality input.\n* (line 164) It should be mentioned that the loss in Eq. 3 is only for one layer and each layer has its own loss. That is, there is an implicit stop gradient preventing the weights of the previous layers to accumulate gradients.  This is made more confusing by not having any layer indices, so it looks like there is a single loss objective and a single deicision made regarding whether the saccade/fixation was correctly classified. I think in general the paper would be more clear if the layer indices were present as well.\n* Computation of $H^t$ in line 180. The computation of whether or not the saccade was correctly predicted is done according to the definition on line 160. How is this computation actually performed? Suppressing the time sub and superscripts and including the neuron index instead we have $u = \\sum_i z_i W_{ij} c_j$. Now  $z_i$ and  $W_{ij} c_j$ are both local to the $i$'th neuron and the product can be computed. But how is this summed? Is there an interneuron that connects to every neuron and sums these? If so, the product  $z_i W_{ij} c_j$ needs to be the output of the neuron. But this is not the case. Altenratively, each neuron can broadcast its own product through some modulatory compound and this can be averaged to give the overall sentiment. This would be plausible but not biologically realistic (also is not what is actually implemented). Unless I misunderstood what is happening, I think this is the biggest weakness of the paper and needs to be fully addressed.\n* Line 188: 'We show in appendix C that ... '. This sounds like there is something akin to an analytic demonstration if not a proof, therefore is sounds a lot stronger than what was actually claimed at the end of appendix C which is that empirically it seems fine. \n* Line 208, 'we allow gradient flow through max-pool' This statement should be expanded (perhaps in the appendix). I cannot tell based on the paper if this is cause for worry or just a trivial assumption.\n* Line 219, 'comparable results were obtained', please provide the resulting figures in the appendix and the details of the implementation in the code. It seems like it would not be a totally trivial change in implementation. It would be good for the authors to clarify these in case anyone would want to reproduce their results in the future.\n* Lines 230-234, emergence of a hierarchy. This is a weak part of the empirical demonstrations. The authors show that the neurons of later layers respond to more complex shapes. Is this not expected of a conv-net? Afterall the receptive field of the first layer is just 3x3, it cannot possibly see anything more complex. Similarly, deeper layers would have wider receptive fields. I think the definition of hierarchical representation - that is deeper representations are built out of the smaller representations - is in fact built into a layered-conv net and therefore the hierarchical claims are all trivial. If the authors meant a useful or interpretable hierarchical representation, then this was addressed elsewhere in the paper.\n* Lines 236-246 Straight line in t-SNE. I am baffled by this. t-SNE is a non-linear mapping and has many bells and whistles that can change the final shape of the resulting representation. As such I'm not sure if this straight line business means anything. It seems like the 3rd layer can do the vehicles/animals discrimination with a slightly bent line so they are similar to me. The following paragraph using a linear decoder for classification is a much better way of seeing performance. \n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "6"}, "extra_text": {}}, {"review_id": "naBjlq22a5J", "reviewer": "Reviewer_5VGo", "summary": "This paper proposes a \"Contrastive, Local And Predictive Plasticity\" learning rule (CLAPP) that could be implemented by deep neural networks of the brain. The authors claim that it results in deep hierarchical representations akin to those recorded in the brain / from deep image classifiers trained with Backprop, while avoiding the need for a backward pass. \n\nCLAPP relies on a global modulation signal which is related to eye movements: for a relatively static eye, the visual input and its encoding are expected to change little (\"positive\" samples); whereas every saccade leads to a large change in the image / code (\"negative\" examples). Another modulating factor is delivered via apical dendrites, which control the learning, but play no role in the feedforward signal transmission (see below for more detail).\n\nFinally, the authors show the CLAPP training on the 6-layer VGG network for images (STL-10 dataset), as well as speech and video. They provide arguments for why they believe their network develops \"deep representations\" using visualisation (patches that best activate a few  neurons and t-SNE across layers) and probes (linear classifiers trained for each layer separately). \n\nThe authors state that the error bars and a proper statistical evaluation are missing due to computational and time constraints.\n\n\n**More detail:**\nThe predictive signal computed by the apical dendrites has no impact on the neural firing rate, but it has to be somehow read out across the entire layer  to form, together with the neural outputs, the global prediction of the saccade $u=\\sum_j \\text{neuron output}_j \\times \\text{dendritic prediction}_j.$\nIt is then compared to the presence of saccade signal and used to train:\n- the feedforward weights of the given layer,\n- the predictive weights (requiring the neuron's action potentials to affect the apical dendrite synapses).\n\nThe same signal (gathered from the layer's dendrites), affects the learning of the \"prediction source\" layer, which applies similar rules, but reversed in time: Instead of updating based on past input and current output, the synapses are updated based on the current input and past output. These result in symmetrical rules for updates of:  \n- the feedforward weights of the \"prediction source\" layer,\n- the retrograde weights which feed back the layer L's output to the \"prediction layer\".", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "- This work is original and it clearly states the difference to other approaches of developing biologically plausible learning algorithms.\n\n- There are no new theoretical results, equations are correct and easy to check with basic linear algebra tools.\nAny claims about deep hierarchical representations are very hard to evaluate. \n\n    This reviewer is not convinced by the argumentation referring to the tSNE representation (tSNE is an algorithm that fits local neighbourhood, thus a line has no meaning in this image; another run of tSNE or a slight change of parameters could have brought the \"birds\" class closer to the \"furry animals\"; there are plenty of samples on that side of the line anyway). The linear decodability is a more useful metric, but it's hard to tell how much of the benefit is due to the increasing receptive field sizes; in the end, the local learning rule (greedy supervised) shows a similar pattern across depth.\n\n- The submission is clearly written and well organised. (But note LL 130--131.)\n\n- This submission is an interesting voice in the ongoing discussion of what are the learning mechanisms used by the neural networks of the brain.\n\n\n\n \n\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors state their model is more biologically plausible than the existing proposals, but they do not discuss the new set of assumptions that is required for their model to work. Most importantly, the \"broad modulation signal\" affecting learning of a given layer needs to be computed based on the predictive dendritic and output signals of the entire layer. The output of the zWc computation needs to be compared to the  \"saccade\" signal and broadcast back to all neurons (in both the feedforward and the prediction layers), to affect learning in their apical and basal dendrites. In the current implementation, this entire computation happens instantaneously. ", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "mWCpNGcGm5N", "reviewer": "Reviewer_pEVz", "summary": "The manuscript proposes a new biologically-inspired learning mechanism for learning deep representations in feedforward circuits, from continuous input streams which in include an active component in which inputs change as a result of known actions (for instance 'saccades'). Exploiting the temporal dependencies in the input, learning can proceed based on local temporal prediction without the need of top-down learning signals (as used in backprop). Abrupt changes in input statistics due to saccades are further exploited for contrastive learning. Because the switch between fixation and saccades are known, one can define a supervised learning objective for the overall optimization. This procedure yields performance that is roughly on par with less biological state of the art methods (GIM), and the paper includes extensive numerical evaluations of its properties and comparisons to other methods.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Overall: The emphasis on exploiting temporal input correlations for learning is especially welcomed, for expanding our understanding of biological learning; i like how the combination of prediction and contrastive learning can be incorporated in a single system by using natural input statistics as the main argument. Mechanistically, it's to a large degree the usual suspects (separate dendritic compartment for learning signals, gating the base synapses plasticity, paired with global modulation, etc) but with a new interpretation which is different enough to be able to maybe generate distinguishable experimental predictions. \n\nDetailed comments:\n- is there a role for top-down signals that affect lower level representations in these scheme? As i read it, the basic setup is surely feedforward (i get that the learning signals can be top-down, i am asking about feedback signals affecting dynamics -- which admittedly is beyond basic training of a feedforward deep net with an unsupervised objective).\n- more generally, is there a way to combine the learning scheme with task specific goals/supervision signals.\n- how does the loss work for RNNs exactly? what statistics is it trying to learn in this case? The appendix c seems to provide info on the mechanics but the initial conceptualization is still unclear.\n- not sure where the index c in eq 7 comes from\n- the planes vs birds example is neat but somewhat cherry picked the structure of other classes looks a lot messier\n\nOriginality: This puts together a range of ideas in NN learning in a single learning system; the mechanistic form of the learning dynamics is very similar to previous biologically inspired approximations of backprop (learning signals in dendritic compartment gating plasticity at primary) although differing in some of the details;  the addition of the saccades that directly affect the temporal input correlation structure is novel to the bio-inspired learning literature, to my knowledge, although substantial previous work on contrastive learning exists in the ML literature. The algorithm also seems new in the technical details, e.g. the loss objective formulation.\n\nSignificance: What signals drive hierarchical learning in the brain is still a mystery and a focus of intense theoretical research, especially in recent years. The CLAPP procedure adds a potentially novel piece to the puzzle. This may be interesting both for neuroscientists and for machine learning practitioners interested in online/ local learning.\n\nClarity: The paper is generally well written and easy to read. ", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "No concerns.", "review_text": "", "extra_scores": {"ethical_concerns": "No concerns.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "3"}, "extra_text": {}}, {"review_id": "YdeVJCkvL-U", "reviewer": "Reviewer_hGsk", "summary": "This paper proposes a new Hebbian-like learning algorithm for neural networks that is able to produce hierarchical representations. This new self-supervised learning rule, which is based on the ideas of Contrastive Predictive Coding (CPC), obeys several key biological constraints (such as locality) while attaining similar benchmarks on several standard ML-classification tasks as previous, unconstrained rules.", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "The gap between biologically plausible learning algorithms and implausible once such as backpropagation has recently attracted a lot of interest. The authors' approach here is quite original, as they build their learning rule around CPC, and, quite interestingly, assume that the brain self-generates contrastive samples by e.g. performing saccades. That's a great idea! (I hadnt seen that voiced before.) The paper is well written and the exposition is quite scholarly, in that the authors clearly delineate how their model compares to other methods in the field, and how the different aspects can be traced back to other studies (locality of rule, the avoidance of BP, the presence of modulating factors, etc.). The final simulations are convincing, and show that the rule does generate hierarchical and meaningful representations.\nI only have a few suggestions/questions:\n\n(1) Why does CLAPP require many more negative samples on video than on STL-10? That wasnt clear to me.\n\n(2) The distinction between the predicted layer z and predicting layer c gets confusing at times. On the one hand, Figure 2c and L75-77 seem to indicate that the feedforward network of z’s is the only network needed (Note specially L85-94). On the other hand, L170-174 and the remaining equations seem to indicate that z and c are separate neurons with separate feedforward connections. The clarity of this distinction could be improved both in text and figures.\n\n(3) L36-37: “The present paper demonstrates that deep representations can emerge from a local, biologically plausible and unsupervised learning rule”. Despite having some components which are biologically plausible, the author introduce a shortcut which seems to violate biology: the learning rule requires a synaptic connection $W^{pred}$ that does not affect the neuron's firing but only synaptic plasticity. In other words, how could a signal that arrives at an apical dendrite influence synaptic plasticity at a basal dendrite while not directly influencing the activity of the neuron itself? Please clarify or word more carefully.\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "The authors have acknowledged that the current learning rule in its current form is data- and compute-intensive. However, the locality in space and time allows it to be used in neuromorphic hardware, potentially reducing energy consumption significantly.", "review_text": "", "extra_scores": {"ethical_concerns": "none.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "10h (together with a postdoc in the lab)"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a biologically plausible learning algorithm that implements unsupervised (contrastive) learning. The algorithm can learn hierarchical representations. A particular innovation is the use of saccades to generate contrastive samples. Reviewers all agree that the contribution is novel and significant, the manuscript is well-written, and the computational experiments are convincing.
Decision: Accept (Poster)