Local plasticity rules can learn deep representations using self-supervised contrastive predictions
Year: 2021

Abstract:
Learning in the brain is poorly understood and learning rules that respect biological constraints, yet yield deep hierarchical representations, are still unknown. Here, we propose a learning rule that takes inspiration from neuroscience and recent advances in self-supervised deep learning. Learning minimizes a simple layerspecific loss function and does not need to back-propagate error signals within or between layers. Instead, weight updates follow a local, Hebbian, learning rule that only depends on preand post-synaptic neuronal activity, predictive dendritic input and widely broadcasted modulation factors which are identical for large groups of neurons. The learning rule applies contrastive predictive learning to a causal, biological setting using saccades (i.e. rapid shifts in gaze direction). We find that networks trained with this self-supervised and local rule build deep hierarchical representations of images, speech and video.

heading: 1 Introduction
text: Synaptic connection strengths in the brain are thought to change according to ‘Hebbian’ plasticity rules [Hebb, 1949]. Such rules are local and depend only on the recent state of the pre- and postsynaptic neurons [Sjöström et al., 2001, Caporale and Dan, 2008, Markram et al., 2011], potentially modulated by a third factor related to reward, attention or other high-level signals [Kuśmierz et al., 2017, Gerstner et al., 2018]. Therefore, one appealing hypothesis is that representation learning in sensory cortices emerges from local and unsupervised plasticity rules.
Following a common definition in the field [Fukushima, 1988, Riesenhuber and Poggio, 1999, LeCun, 2012, Lillicrap et al., 2020], a hierarchical representation (i) builds higher-level features out of lower-level ones, and (ii) provides more useful features in higher layers. Now there seems to be a substantial gap between the rich hierarchical representations observed in the cortex and the representations emerging from local plasticity rules implementing principal/independent component analysis [Oja, 1982, Hyvärinen and Oja, 1998], sparse coding [Olshausen and Field, 1997, Rozell et al., 2008] or slow-feature analysis [Földiák, 1991, Wiskott and Sejnowski, 2002, Sprekeler et al., 2007]. Hebbian rules seem to struggle especially when ‘stacked’, i.e. when asked to learn deep, hierarchical representations.
⇤shared last author
35th Conference on Neural Information Processing Systems (NeurIPS 2021)
This performance gap is puzzling because there are learning rules, relying on back-propagation (BP), that can build hierarchical representations similar to those found in visual cortex [Yamins et al., 2014, Zhuang et al., 2021]. Although some progress towards biologically plausible implementations of back-propagation has been made [Lillicrap et al., 2016, Guerguiev et al., 2017, Sacramento et al., 2018, Payeur et al., 2021], most models rely either on a neuron-specific error signal that needs to be transmitted by a separate error network [Crick, 1989, Amit, 2019, Kunin et al., 2020], or timemultiplexing feedforward and error signals [Lillicrap et al., 2020, Payeur et al., 2021]. Algorithms like contrastive divergence [Hinton, 2002], contrastive Hebbian learning [Xie and Seung, 2003] or equilibrium propagation [Scellier and Bengio, 2017] use local activity exclusively to calculate updates, but they require to wait for convergence to an equilibrium which is not appropriate for online learning from quickly varying inputs.
The present paper demonstrates that deep representations can emerge from a local, biologically plausible and unsupervised learning rule, by integrating two important insights from neuroscience: First, we focus on self-supervised learning from temporal data – as opposed to supervised learning from labelled examples – because this comes closest to natural data, perceived by real biological agents, and because the temporal structure of natural stimuli is a rich source of information. In particular, we exploit the self-awareness of typical, self-generated changes of gaze direction (‘saccades’) to distinguish input from a moving object during fixation from input arriving after a saccade towards a new object. In our plasticity rule, a global factor modulates plasticity, depending on the presence or absence of such saccades. Although we do not model the precise circuit that computes this global factor, we see it related to global, saccade-specific signals from motor areas, combined with surprise or prediction error, as in other models of synaptic plasticity [Angela and Dayan, 2005, Nassar et al., 2012, Heilbron and Meyniel, 2019, Liakoni, 2021]. Second, we notice that electrical signals stemming from segregated apical dendrites can modulate synaptic plasticity in biological neurons [Körding and König, 2001, Major et al., 2013], enabling context-dependent plasticity.
Algorithmically, our approach takes inspiration from deep self-supervised learning algorithms that seek to contrast, cluster or predict stimuli in the context of BP [Van den Oord et al., 2018, Caron et al., 2018, Zhuang et al., 2019, Löwe et al., 2019]. Interestingly, Löwe et al. [2019] demonstrated that such methods even work if end-to-end BP is partially interrupted. We build upon this body of work and suggest the Contrastive, Local And Predictive Plasticity (CLAPP) model which avoids BP completely, yet still builds hierarchical representations.2

heading: 2 Main goals and related work
text: In this paper, we propose a local plasticity rule that learns deep representations. To describe our model of synaptic plasticity, we represent a cortical area by the layer l of a deep neural network. The neural activity of this layer at time t is represented by the vector zt,l = ⇢(at,l), where ⇢ is a non-linearity and at,l = W lzt,l 1 is the vector of the respective summed inputs to the neurons through their basal dendrites W l (the bias is absorbed into W l). To simplify notation, we write the pre-synaptic input as xt,l = zt,l 1 and we only specify the layer index l when it is necessary.
Our plasticity rule exploits the fact that the temporal structure of natural inputs affects representation learning [Li and DiCarlo, 2008]. Specifically, we consider a scenario where an agent first perceives a moving object at time t (e.g. a flying eagle in Figure 1 a), and then spontaneously decides to change gaze direction towards another moving object at time t + t (e.g. saccade towards the elephant in Figure 1 a). We further assume that the visual pathway is ‘self-aware’ of saccades due to saccade-specific modulation of processing [Ross et al., 2001].
In line with classical models of synaptic plasticity, we assume that weight changes follow biologically plausible, Hebbian, learning rules [Hebb, 1949, Markram et al., 2011] which are local in space and time: updates W tji of a synapse, connecting neurons i and j, can only depend on the current activity of the pre-synaptic and post-synaptic neurons at time t, or slightly earlier at time t t, and one or several widely broadcasted modulating factors [Urbanczik and Senn, 2009, Gerstner et al., 2018].
Furthermore, we allow the activity of another neuron k to influence the weight update Wji, as long as there is an explicit connection W predjk from k to j. The idea is to overcome the representational limitations of classical Hebbian learning by including dendritic inputs, which are thought to predict
2Our code is available at https://github.com/EPFL-LCN/pub-illing2021-neurips
the future somatic activity [Körding and König, 2001, Urbanczik and Senn, 2014] and take part in the plasticity of the post-synaptic neuron [Larkum et al., 1999, Dudman et al., 2007, Major et al., 2013]. Hence we assume that each neuron j in a layer l may receive dendritic inputs (W predct,l)j coming either from the layer above (ct,l = zt,l+1) or from lateral connections in the same layer (ct,l = zt,l).
For algorithmic reasons, that we detail in section 3, we assume that the dendritic input (W predct,l)j influences the weight updates Wji of the post-synaptic neuron j, but not its activity ztj . This assumption is justified by neuroscientific findings that the inputs to basal and apical dendrites affect the neural activity and plasticity in different ways [Larkum et al., 1999, Dudman et al., 2007, Major et al., 2013, Urbanczik and Senn, 2014]. In general, we do not rule out influence of dendritic activity on somatic activity in later processing phases, but see this beyond the scope of the current work.
Given these insights from neuroscience, we gather the essential factors that influence synaptic plasticity in the following learning rule prototype:
Wji / modulators| {z } broadcast factors · (W predct1)j| {z } dendritic prediction · postt2j · pre t2 i| {z }
local-activity
. (1)
The modulating broadcast factors are the same for large groups of neurons, for example all neurons in the same area, or even all neurons in the whole network. postt2j and pre t2 i are functions of the preand post- synaptic activities. At this point, we do not specify the exact timing between t1 and t2, as this will be determined by our algorithm in section 3.
Related work Many recent models of synaptic plasticity fit an apparently similar learning rule prototype [Lillicrap et al., 2016, Nøkland, 2016, Roelfsema and Holtmaat, 2018, Nøkland and Eidnes, 2019, Lillicrap et al., 2020, Pozzi et al., 2020] if we interpret the top-down signals emerging from the BP algorithm as the dendritic signal. However, top-down error signals in BP are not directly related to the activity ct of the neurons in the main network during processing of sensory input. Rather, they require a separate linear network mirroring the initial network and feeding back error signals (see Figure 2 a and Lillicrap et al. [2020]), or involved time-multiplexing of feedforward and error signals in the main network [Lillicrap et al., 2020, Payeur et al., 2021]. Our model is fundamentally different, because in our case, the dendritic signal onto neuron j is strictly (W predct)j which is a weighted
sum of the main network activity and there is no need of a (linear) feedback network transmitting exact error values across many layers.
Moreover, we show in simulations in section 4, that the dendritic signal does not have to come from a layer above but that the prediction fed to layer l may come from the same layer. This shows that our learning rule works even in the complete absence of downward signaling from l + 1 to l. This last point is a significant difference to other methods that also calculate updates using only activities of the main network, but require tuned top-down connections to propagate signals downwards in the network hierarchy [Kunin et al., 2020], such as methods in the difference target propagation family [Lee et al., 2015, Bartunov et al., 2018, Golkar et al., 2020], contrastive divergence [Hinton, 2002] and equilibrium propagation [Scellier and Bengio, 2017]. Furthermore, the latter two require convergence to an equilibrium state for each input [Laborieux et al., 2021]. Our model does not require this convergence because it uses the recurrent dendritic signal (W predct)j only for synaptic plasticity and not for inference.
Most previous learning rules which include global modulating factors interpret it as a reward prediction error [Schultz et al., 1997, Gerstner et al., 2018, Pozzi et al., 2020]. In this paper, we address self-supervised learning and view global modulating factors as broadcasting signals, modeling the self-awareness that something has changed in the stimulus (e.g. because of a saccade). Hence, the main function of the broadcast factor in our model is to identify contrastive inputs, which avoids a common pitfall for self-supervised learning models: ‘trivial’ or ‘collapsed’ solutions, where the model produces a constant output, which is easily predictable, but useless for downstream tasks. In vision, we use a broadcast factor to model the strong, saccade-specific activity patterns identified throughout the visual pathway [Kowler et al., 1995, Leopold and Logothetis, 1998, Ross et al., 2001, McFarland et al., 2015]. In other sensory pathways, like audition, this broadcast factor may model attention signals arising when changing focus on a new input source [Fritz et al., 2007], cross-modal input indicating a change in head or gaze direction, or signal/speaker-identity inferred from blind source separation, which can be done on low-level representation with biologically plausible learning rules [Hyvärinen and Oja, 1997, Ziehe and Müller, 1998, Molgedey and Schuster, 1994]. Our learning rule further requires this global factor to predict the absence or presence of a gaze change, hence conveying a change prediction error rather than classical reward prediction error. Here, we do not model the precise circuitry computing this factor in the brain, however, we speculate that a population of neurons could express such a scalar factor e.g. through burst-driven multiplexing of activity, see Payeur et al. [2021] and Appendix C.
Our theory takes inspiration from the substantial progress seen in unsupervised machine learning in recent years and specifically from contrastive predictive coding (CPC) [Van den Oord et al., 2018]. CPC trains a network (called encoder) to make predictions of its own responses to future inputs, while keeping this prediction as different as possible to its responses to fake inputs (contrasting). A key feature of CPC is that predicting and contrasting happens in latent space, i.e. on the output representation of the encoder network. This avoids modeling a generative model for perfect reconstruction of the input and all its details (e.g. green, spiky). Instead the model is forced to focus on extracting high-level information (e.g. cactus). In our notation, CPC evaluates a prediction W predct such that a score function u⌧t = z⌧
>W predct becomes larger for the true future ⌧ = t + t (referred to as positive sample) than for any other vector zt 0 taken at arbitrary time points t0 elsewhere in the entire training set (referred to as negative samples in CPC). This means, that the prediction should align with the future activity zt+ t but not with the negative samples. Van den Oord et al. [2018] formalizes this as a softmax cross-entropy classification, which leads to the traditional CPC loss:
LtCPC = log exput+ ttP ⌧2T expu ⌧ t , (2)
where T = t t+ t , t 0 1 . . . t 0 N comprises the positive sample and N negative samples. The learned model parameters are the elements of the matrix W pred, as well as the weights of the encoder network. The loss function LtCPC is then minimized by stochastic gradient descent on these parameters using BP. Amongst numerous recent variants of contrastive learning [He et al., 2019, Chen et al., 2020, Xiong et al., 2020], we focus here on CPC [Van den Oord et al., 2018], for which a more local variant, Greedy InfoMax, was recently proposed by Löwe et al. [2019].
Greedy InfoMax (GIM) [Löwe et al., 2019] is a variant of CPC which makes a step towards local, BP-free learning: the main idea is to split the encoder network into a few gradient-isolated modules
= W retro, kj
(atj ) xti+ t (W retro z )j
W retro
to avoid back-propagation between these modules. As the authors mention in their conclusion, “the biological plausibility of GIM is limited by the use of negative samples and within-module backpropagation”. This within-module back-propagation still requires a separate feedback network to propagate prediction errors (Figure 2 a), but can be avoided in the most extreme version of GIM, where each gradient-isolated module contains a single layer (layer-wise GIM). However, the gradients of layer-wise GIM, derived from Equation 2, still cannot be interpreted as synaptic plasticity rules because the gradient computation requires (1) the transmission of information other than the network activity (see Figure 2 b), and (2) perfect memory to replay the negative samples zt 0 , as mentioned in the above quote (see Appendix A for details). Overall it is not clear how this weight update of layer-wise GIM could be implemented with realistic neuronal circuits. Our CLAPP rule solves the above mentioned implausibilities and allows a truly local implementation in space and time.

heading: 3 Derivation of the CLAPP rule: contrastive, local and predictive plasticity
text: We now suggest a simpler contrastive learning algorithm which solves the issues encountered with layer-wise GIM and for which a gradient descent update is naturally compatible with the learning rule prototype from Equation 1. The most essential difference compared to CPC or GIM is, that we do not require the network to simultaneously access the true future activity zt+ t and recall (or imagine) the network activity zt 0 seen at some other time. Rather, we consider the naturalistic time-flow illustrated in Figure 1 a, where an agent fixates on a moving animal for a while and then changes gaze spontaneously. In this way, the prediction W predct is expected to be meaningful during fixation, but inappropriate right after a saccade. In our simulations, we model this by feeding the network with subsequent frames from the same sample (e.g. different views of an eagle), and then abruptly changing to frames from another sample (e.g. different views of an elephant).
We note that the future activity zt+ t and the context ct are always taken from the main feedforward encoder network. We focus on the case where the context stems from the same layer as the future activity (ct,l = zt,l), however, the model allows for the more general case, where the context stems from another layer (e.g. the layer above ct,l = zt,l+1).
Derivation of the CLAPP rule from a self-supervised learning principle Rather than using a global loss function for multi-class classification to separate the true future from multiple negative samples, as in CPC, we consider here a binary classification problem at every layer l: we interpret the score function ut+ t,lt = zt+ t,l > W pred,lct,l as the layer’s ‘guess’ whether the agent performed a fixation or a saccade. In Appendix C, we discuss how ut+ t,lt could be (approximately) computed in real neuronal circuits. In short, every neuron i has access to its ‘own’ dendritic prediction ẑ
t,l i = P j W pred,l ij c t,l j of somatic activity [Urbanczik and Senn, 2014], and the product z t+ t,l i ẑ t,l i
can be seen as a coincidence detector of dendritic and somatic activity, communicated by specific burst signals [Larkum et al., 1999]. These burst signals allow time-multiplexed communication [Payeur et al., 2021] of the products zt+ t,li ẑ t,l i of many neurons, which can then be summed by an interneuron representing ut+ t,lt .
As mentioned in section 2, information about the presence or absence of a saccade between two time points is available in the visual processing stream and is modeled here by the variable yt = 1 and y
t = +1, respectively. We interpret yt as the label of a binary classification problem, characterized by the Hinge loss, and define the CLAPP loss at layer l as:
Lt,lCLAPP = max ⇣ 0, 1 yt · ut+ t,lt ⌘ with ⇢ y t = +1 for fixation y t = 1 for saccade (3)
We now derive the gradients of Equation 3 with respect to the feedforward weights and show that gradient descent on this loss function is compatible with the learning rule prototype suggested in Equation 1. Note that CLAPP optimises Equation 3 for each layer l independently, without any gradient flow between layers. That being said, the following derivation is the same for every layer l, which is why we omit the layer index l from here on.
Since we chose to formalize the binary classification with a Hinge loss, the gradient vanishes when the classification is already correct: high score ut+ tt > 1 during fixation (yt = +1), or a low score u
t+ t t < 1 after a saccade (yt = 1). Otherwise, it is rut+ tt during a fixation or rut+ tt after a saccade. In the ‘predicted layer’ z, i.e. the target of the prediction, let Wji denote the feedforward weight from neuron i in the previous layer (with activity xti) to neuron j, with summed input atj and activity ztj . Similarly, in the ‘predicting layer’ c, i.e. the source of the prediction, let W ckl denote the feedforward weight between the neuron l in the previous layer (with activity xc,tl ) and neuron k, with summed input ac,tk and activity c t k. Therefore, ·c as an upper index refers to the context layer, whereas c as a full-size letter refers to the respective neuronal activity. We then find the gradients with respect to these weights as:
@LtCLAPP @Wji = ±(W predct)j ⇢0(at+ tj ) x t+ t i (4)
@LtCLAPP @W
c km
= ±(W pred>zt+ t)k ⇢0(ac,tk ) x c,t m , (5)
where the sign is negative during fixation and positive after a saccade. To change these equations into online weight updates, we consider the gradient descent update delayed by t, such that W tji =
⌘ @L t t CLAP P @Wji
, where ⌘ is the learning rate. Let us define a modulating factor t = yt · Ht, where y
t = ±1 is a network-wide broadcast signal (self-awareness) indicating a saccade ( 1) or a fixation (+1) and Ht 2 {0, ⌘} is a layer-wide broadcast signal indicating whether the saccade or fixation was correctly classified as such. In this way, Equation 4 becomes a weight update which follows strictly the ideal learning rule prototype from Equation 1:
W tji = t|{z} broadcast factors · (W predct t)j| {z } dendritic prediction · ⇢0(atj)xti| {z } local activity . (6)
For the updates of the connections onto the neuron ctk, which emits the prediction rather than receiving it, our theory in Equation 5 requires the opposite temporal order and the transmission of the information in the opposite direction: from zt back to ct. Since connections in the brain are unidirectional [Lillicrap et al., 2016], we introduce another matrix W retro which replaces W pred> in the final weight update. Given the inverse temporal order, we interpret W retrozt as a retrodiction rather than a prediction. In Appendix C, we show that using W retro minimises a loss function of the same form as Equation 3, and empirically performs as well as using W pred>. The resulting weight update satisfies the learning rule prototype from Equation 1, as it can be written:
W c,tkm = t|{z} broadcast factors · (W retrozt)k| {z } dendritic retrodiction · ⇢0(ac,t tk )x c,t t m| {z } local activity . (7)
In the (standard) case, where context and predicted activity are from the same layer (ct,l = zt,l), W and W c are the same weights and the updates Equation 6 and Equation 7 are added up linearly.
The prediction and retrodiction weights, W pred and W retro, respectively, are also plastic. By deriving the gradients of LtCLAPP with respect to W pred, we find an even simpler Hebbian learning rule for these weights:
W predjk = W retro kj = t|{z}
broadcast factors · ztj · ct tk| {z } pre and post , (8)
where neuron k in the predicting layer c is pre-synaptic (post-synaptic) and neuron j in the predicted layer z is post-synaptic (pre-synaptic) for the prediction weights W predjk (retrodiction weights W retro kj ). Note that the update rules for W predjk and W retro kj are reciprocal, a method that leads to mirrored connections, given small enough initialisation [Burbank, 2015, Amit, 2019, Pozzi et al., 2020].
We emphasize that all information needed to calculate the above CLAPP updates (Equations 6 – 8) is spatially and temporally available, either as neuronal activity at time t, or as traces of recent activity (t t) [Gerstner et al., 2018]. In order to implement Equation 6, the dendritic prediction has to be retained during t. However, we argue that dendritic activity can outlast (50-100 ms) somatic neuronal activity (2-10 ms) [Major et al., 2013], which makes predictive input from several time steps in the past (t t) available at time t.
Generalizations While the above derivation considers fully-connected feedforward networks, we apply analogous learning rules to convolutional neural networks (CNN) and recurrent neural networks (RNN). Analyzing the biological plausibility of the standard spatial weight sharing and spatial MeanPooling operations in CNNs is beyond the scope of the current work. Furthermore, we discuss in Appendix C, that MaxPooling can be interpreted as a simple model of lateral inhibition and that gradient flow through such layers is compatibility with the learning rule prototype in Equation 1.
To obtain local learning rules even for RNNs, we combine CLAPP with the e-prop theory [Bellec et al., 2020], which provides a biologically plausible alternative to BP through time: gradients can be propagated forward in time through the intrinsic neural dynamics of a neuron using eligibility traces. The propagation of gradients across recurrently connected units is forbidden and disabled. This yields biologically plausible updates in GRU units, as explained in Appendix C.

heading: 4 Empirical results
text: Building hierarchical representations We first demonstrate numerically, that CLAPP yields deep hierarchical representations, despite using a local plasticity rule compatible with Equation 1. We
report here the results for ct,l = zt,l, i.e. the dendritic prediction in Equation 1 is generated from lateral connections and the representations in the same layer. We note, however, that we obtained qualitatively similar results with ct,l = zt,l+1 (i.e. the dendritic prediction is generated from one layer above), suggesting that top-down signaling is neither necessary for, nor incompatible with, our algorithm (also see Appendix C).
We first consider the STL-10 image dataset [Coates et al., 2011]. To simulate a time dimension in these static images, we follow Hénaff et al. [2019] and Löwe et al. [2019]: each image is split into 16 ⇥ 16 patches and the patches are viewed one after the other in a vertical order (one time step is one patch). Other hyper-parameters and data-augmentation are taken from Löwe et al. [2019], see Appendix B. We then train a 6-layer VGG-like [Simonyan and Zisserman, 2015] encoder (VGG-6) using the CLAPP rule (Equations 6 – 8). Training is performed on the unlabelled part of the STL-10 dataset for 300 epochs. We use 4 GPUs (NVIDIA Tesla V100-SXM2 32 GB) for data-parallel training, resulting in a simulation time of around 4 days per run.
In order to study how neuronal selectivity changes over layers, we select neurons randomly and show image patches which best activate these neurons the most (rows in Figure 3 a). As expected for a visual hierarchy, first-layer neurons (first column in Figure 3 a) are selective to horizontal or vertical gratings, or homogeneous colors. In the third layer of the network (second column), neurons start to be selective to more semantic features like grass, or parts of vehicles. Neurons in the last layer (third column) are selective to specific object parts (e.g. a wheel touching the road). The same analysis for a random, untrained encoder does not reveal a clear hierarchy across layers, see Appendix C.
To get a qualitative idea of the learned representation manifold, we use the non-linear dimension reduction technique t-SNE [Van der Maaten and Hinton, 2008] to visualise the encodings of the (labeled) STL-10 test set in Figure 3 b. We see that the representation in the first layer is mostly unrelated to the underlying class. In the third and sixth layers’ representation, a coherent clustering emerges, yielding an almost perfect separation between furry animals and vehicles. This clustered representation is remarkable since the network has never seen class labels, and was never instructed to separate classes, during CLAPP training The representation of the same architecture, but without training (Random init.), shows that a convolutional architecture alone does not yield semantic features.
To produce a more quantitative measurement of the quality of learned representations, we follow the methodology of Van den Oord et al. [2018] and Löwe et al. [2019]: we freeze the trained encoder weights and train a linear classifier to recognize the class labels from each individual layer (Figure 4). As expected for a deep representation, the classification accuracy increases monotonically with the layer number and only saturates at layers 5 and 6. The accuracies obtained with layer-wise GIM are almost indistinguishable from those obtained with CLAPP. It is only at the last two layers, that layer-wise GIM performs slightly better than CLAPP; yet GIM has multiple biologically implausible features that are removed by CLAPP. As a further benchmark, we also plot the accuracies obtained with an encoder trained with greedy supervised training. This method trains each layer independently using a supervised classifier at each layer, without BP between layers, which results in an almost local update (see Löwe et al. [2019] and Appendix B). We find that accuracy is overall lower and saturates already at layer 4. On this dataset, with many more unlabelled than labelled images, greedy supervised accuracy is almost 10% below the accuracy obtained with CLAPP. Again, we see that a convolutional architecture alone does not yield hierarchical representations, as performance decreases at higher layers for a fixed random encoder.
Comparing CPC and CLAPP Since CLAPP can be seen as a simplification of CPC (or GIM) we study four algorithmic differences between CPC and CLAPP individually. They are: (1) Gradients in CLAPP (layer-wise GIM) cannot flow from a layer to the next one, as opposed to BP in CPC, (2) CLAPP performs a binary comparison (fixation vs. saccade) with the Hinge loss, whereas CPC does multi-class classification with the cross entropy loss, (3) CLAPP processes a single input at a time, whereas CPC uses many positive and negative samples synchronously, and (4) we introduced W retro to avoid the weight transport problem in W pred.
We first study features (1) and (2) but relax constraints (3) and (4). That means, in this paragraph, we allow a fixation and N = 16 synchronous saccades and set W retro = W pred,>. We refer to Hinge Loss CPC as the algorithm minimizing the CLAPP loss (Equation 3) but using end-to-end BP. CLAPP-s (for synchronous) applies the Hinge Loss to every layer, but with gradients blocked between layer. We find that the difference between the CPC loss and Hinge Loss CPC is less than 1%, see
Table 1. In contrast, additional blocking of gradients between layers causes a performance drop of almost 5% for both loss functions. We investigate how gradient blocking influences performance with a series of simulations, splitting the 6 layers of the network into two or three gradient isolated modules, exploring the transition from Hinge Loss CPC to CLAPP-s. Performance drops monotonously but not catastrophically, as the number of gradient blocks increases (Table 1).
CLAPP’s temporal locality allows the interpretation that an agent alternates between fixations and saccades, rather than perfect recall and synchronous processing of negative samples, as required by CPC. To study the effect of temporal locality, we apply features (2) and (3) and relax the constraints (1) and (4). The algorithm combining temporal locality and the CLAPP loss function is referred to as time-local Hinge Loss CPC. We find that the temporal locality constraint decreases accuracy by 1.2% compared to Hinge Loss CPC. The last feature introduced for biological plausibility is using the matrix W retro and we observe almost no difference in classification accuracy with this alternative (the accuracy decreases by 0.1%). Conversely, omitting the update in Equation 7 entirely, i.e. setting the retrodiction W retro = 0, compromises accuracy by 2% compared to vanilla Hinge Loss CPC.
When combining all features (1) to (4), we find that the fully local CLAPP learning rule leads to an accuracy of 73.6% at layer 5. We conclude from the analysis above, that the feature with the biggest impact on performance is (1): blocking the gradients between each layer. However, despite the performance drop caused by blocking the gradients, CLAPP still stacks well and leverages the depth of the network (Figure 4). All other features (2) - (4), introduced to derive a weight update compatible with our prototype (Equation 1), only caused a minor performance loss.
Applying CLAPP to speech and video We now demonstrate that CLAPP is applicable to other modalities like the LibriSpeech dataset of spoken speech [Panayotov et al., 2015] and the UCF-101 dataset containing short videos of human actions [Soomro et al., 2012]. When applying CLAPP to auditory signals, we do not explicitly model the contrasting mechanism (saccades in the vision task; see discussion in section 2 for the auditory pathway) and hence consider the application of CLAPP as benchmark application, rather than a neuroscientifically exact study. To increase computational efficiency, we study CLAPP-s on speech and video. Based on the image experiments, we expect similar results for CLAPP, given enough time to converge. We use the same performance criteria as for the image experiments and summarize our results in Table 1, for details see Appendix B.
For the audio example, we use the same architecture as Van den Oord et al. [2018] and Löwe et al. [2019]: multiple temporal 1d-convolution layers and one recurrent GRU layer on top. As in the feedforward case, CLAPP still optimises the objective of Equation 3. For the 1d-convolution layers, the context ct is computed as for the image task, for the last layer, ct is the output of the recurrent GRU layer. We compare the performance of the algorithms on phoneme classification (41 classes) using labels provided by Van den Oord et al. [2018]. In this setting, layer-wise training lowers performance by only 0.4% for layer-wise GIM, and by 1.1% for CLAPP-s. Implemented as such, CLAPP-s still relies on BP through time (BPTT) to train the GRU layer. Using CLAPP-s with biologically plausible e-prop [Bellec et al., 2020], instead of non-local BPTT, reduces performance by only 3.1 %, whereas omitting the GRU layer compromises performance by 9.3 %, see Appendix C.
Applying CLAPP to videos is especially interesting because their temporal sequence of images perfectly fits the scenario of Figure 1 a. In this setting, we take inspiration from Han et al. [2019],
and use a VGG-like stack of 2D and 3D convolutions to process video frames over time. On this task (101 classes), we found layer-wise GIM and CLAPP-s to achieve higher downstream classification accuracy than their end-to-end counterparts CPC and Hinge Loss CPC (see Table 1), in line with the findings on STL-10 in Löwe et al. [2019]. On the other hand, we found that CLAPP-s requires more negative samples (i.e. more simultaneous comparisons of positive and negative samples) on videos than on STL-10 and LibriSpeech. Under the constraint of temporal locality in fully local CLAPP, this leads to prohibitively long convergence times in the current setup. However, since CLAPP linearly combines updates stemming from multiple negative and positive samples, we eventually expect the same final performance, if we run the online CLAPP algorithm for a sufficiently long time.

heading: 5 Discussion
text: We introduced CLAPP, a self-supervised and biologically plausible learning rule that yields deep hierarchical representations in neural networks. CLAPP integrates neuroscientific evidence on the dendritic morphology of neurons and takes the temporal structure of natural data into account. Algorithmically, CLAPP minimises a layer-wise contrastive predictive loss function and stacks well on different task domains like images, speech and video – despite the locality in space and time.
While the performance loss due to layer-wise training is a limitation of the current model, the stacking property is preserved and preliminary results suggest improved versions that stack even better (e.g. using adaptive encoding patch sizes). Note that CLAPP models self-supervised learning of cortical hierarchies and does not provide a general credit assignment method, such as BP. However, the representation learned with CLAPP could serve as an initialisation for transfer learning, where the encoder is fine-tuned later with standard BP. Alternatively, fine-tuning could even start already during CLAPP training. CLAPP in its current form is data- and compute-intensive, however, it runs on unlabelled data with quasi infinite supply, and is eligible for neuromorphic hardware, which could decrease energy consumption dramatically [Wunderlich et al., 2019].
Classical predictive coding models alter neural activity at inference time, e.g. by cancelling predicted future activity [Rao and Ballard, 1999, Keller and Mrsic-Flogel, 2018]. Here, we suggest a different, perhaps complementary, role of predictive coding in synaptic plasticity, where dendritic activity predicts future neural activity, but directly enters the learning rule [Körding and König, 2001, Urbanczik and Senn, 2014]. CLAPP currently does not model certain features of biological neurons, e.g. spiking activity or long range feedback, and requires neurons to transmit signals with precise value and timing. We plan to address these topics in future work.
Acknowledgments and Disclosure of Funding
This research was supported by the Swiss National Science Foundation (no. 200020_184615) and the Intel Neuromorphic Research Lab. Many thanks to Sindy Löwe, Julie Grollier, Maxence Ernoult, Franz Scherr, Johanni Brea and Martin Barry for helpful discussions. Special thanks to Sindy Löwe for publishing the GIM code.

Review summaries:
The authors propose a biologically plausible implementation of contrastive self supervised learning. They solve the problem of remembering negative samples by relying on saccades, taking the post-saccade incoming signal to be a negative sample. They perform a number of comparisons showing the viability of their approach.  -------------------------------------- Post-rebuttal update: Due to the increased quality of the work stemming from added clarifications and a fuller discussion of biological plausibility, I have increased my score from 5 to 7.

The manuscript proposes a new biologically-inspired learning mechanism for learning deep representations in feedforward circuits, from continuous input streams which in include an active component in which inputs change as a result of known actions (for instance 'saccades'). Exploiting the temporal dependencies in the input, learning can proceed based on local temporal prediction without the need of top-down learning signals (as used in backprop). Abrupt changes in input statistics due to saccades are further exploited for contrastive learning. Because the switch between fixation and saccades are known, one can define a supervised learning objective for the overall optimization. This procedure yields performance that is roughly on par with less biological state of the art methods (GIM), and the paper includes extensive numerical evaluations of its properties and comparisons to other methods.

This paper proposes a "Contrastive, Local And Predictive Plasticity" learning rule (CLAPP) that could be implemented by deep neural networks of the brain. The authors claim that it results in deep hierarchical representations akin to those recorded in the brain / from deep image classifiers trained with Backprop, while avoiding the need for a backward pass.   CLAPP relies on a global modulation signal which is related to eye movements: for a relatively static eye, the visual input and its encoding are expected to change little ("positive" samples); whereas every saccade leads to a large change in the image / code ("negative" examples). Another modulating factor is delivered via apical dendrites, which control the learning, but play no role in the feedforward signal transmission (see below for more detail).  Finally, the authors show the CLAPP training on the 6-layer VGG network for images (STL-10 dataset), as well as speech and video. They provide arguments for why they believe their network develops "deep representations" using visualisation (patches that best activate a few  neurons and t-SNE across layers) and probes (linear classifiers trained for each layer separately).   The authors state that the error bars and a proper statistical evaluation are missing due to computational and time constraints.   **More detail:** The predictive signal computed by the apical dendrites has no impact on the neural firing rate, but it has to be somehow read out across the entire layer  to form, together with the neural outputs, the global prediction of the saccade $u=\sum_j \text{neuron output}_j \times \text{dendritic prediction}_j.$ It is then compared to the presence of saccade signal and used to train: - the feedforward weights of the given layer, - the predictive weights (requiring the neuron's action potentials to affect the apical dendrite synapses).  The same signal (gathered from the layer's dendrites), affects the learning of the "prediction source" layer, which applies similar rules, but reversed in time: Instead of updating based on past input and current output, the synapses are updated based on the current input and past output. These result in symmetrical rules for updates of:   - the feedforward weights of the "prediction source" layer, - the retrograde weights which feed back the layer L's output to the "prediction layer".