Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary: The manuscript proposes a new biologically-inspired learning mechanism for learning deep representations in feedforward circuits, from continuous input streams which in include an active component in which inputs change as a result of known actions (for instance 'saccades'). Exploiting the temporal dependencies in the input, learning can proceed based on local temporal prediction without the need of top-down learning signals (as used in backprop). Abrupt changes in input statistics due to saccades are further exploited for contrastive learning. Because the switch between fixation and saccades are known, one can define a supervised learning objective for the overall optimization. This procedure yields performance that is roughly on par with less biological state of the art methods (GIM), and the paper includes extensive numerical evaluations of its properties and comparisons to other methods.
main_review: Overall: The emphasis on exploiting temporal input correlations for learning is especially welcomed, for expanding our understanding of biological learning; i like how the combination of prediction and contrastive learning can be incorporated in a single system by using natural input statistics as the main argument. Mechanistically, it's to a large degree the usual suspects (separate dendritic compartment for learning signals, gating the base synapses plasticity, paired with global modulation, etc) but with a new interpretation which is different enough to be able to maybe generate distinguishable experimental predictions. 

Detailed comments:
- is there a role for top-down signals that affect lower level representations in these scheme? As i read it, the basic setup is surely feedforward (i get that the learning signals can be top-down, i am asking about feedback signals affecting dynamics -- which admittedly is beyond basic training of a feedforward deep net with an unsupervised objective).
- more generally, is there a way to combine the learning scheme with task specific goals/supervision signals.
- how does the loss work for RNNs exactly? what statistics is it trying to learn in this case? The appendix c seems to provide info on the mechanics but the initial conceptualization is still unclear.
- not sure where the index c in eq 7 comes from
- the planes vs birds example is neat but somewhat cherry picked the structure of other classes looks a lot messier

Originality: This puts together a range of ideas in NN learning in a single learning system; the mechanistic form of the learning dynamics is very similar to previous biologically inspired approximations of backprop (learning signals in dendritic compartment gating plasticity at primary) although differing in some of the details;  the addition of the saccades that directly affect the temporal input correlation structure is novel to the bio-inspired learning literature, to my knowledge, although substantial previous work on contrastive learning exists in the ML literature. The algorithm also seems new in the technical details, e.g. the loss objective formulation.

Significance: What signals drive hierarchical learning in the brain is still a mystery and a focus of intense theoretical research, especially in recent years. The CLAPP procedure adds a potentially novel piece to the puzzle. This may be interesting both for neuroscientists and for machine learning practitioners interested in online/ local learning.

Clarity: The paper is generally well written and easy to read. 
limitations_and_societal_impact: No concerns.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a "Contrastive, Local And Predictive Plasticity" learning rule (CLAPP) that could be implemented by deep neural networks of the brain. The authors claim that it results in deep hierarchical representations akin to those recorded in the brain / from deep image classifiers trained with Backprop, while avoiding the need for a backward pass. 

CLAPP relies on a global modulation signal which is related to eye movements: for a relatively static eye, the visual input and its encoding are expected to change little ("positive" samples); whereas every saccade leads to a large change in the image / code ("negative" examples). Another modulating factor is delivered via apical dendrites, which control the learning, but play no role in the feedforward signal transmission (see below for more detail).

Finally, the authors show the CLAPP training on the 6-layer VGG network for images (STL-10 dataset), as well as speech and video. They provide arguments for why they believe their network develops "deep representations" using visualisation (patches that best activate a few  neurons and t-SNE across layers) and probes (linear classifiers trained for each layer separately). 

The authors state that the error bars and a proper statistical evaluation are missing due to computational and time constraints.


**More detail:**
The predictive signal computed by the apical dendrites has no impact on the neural firing rate, but it has to be somehow read out across the entire layer  to form, together with the neural outputs, the global prediction of the saccade $u=\sum_j \text{neuron output}_j \times \text{dendritic prediction}_j.$
It is then compared to the presence of saccade signal and used to train:
- the feedforward weights of the given layer,
- the predictive weights (requiring the neuron's action potentials to affect the apical dendrite synapses).

The same signal (gathered from the layer's dendrites), affects the learning of the "prediction source" layer, which applies similar rules, but reversed in time: Instead of updating based on past input and current output, the synapses are updated based on the current input and past output. These result in symmetrical rules for updates of:  
- the feedforward weights of the "prediction source" layer,
- the retrograde weights which feed back the layer L's output to the "prediction layer".
main_review: - This work is original and it clearly states the difference to other approaches of developing biologically plausible learning algorithms.

- There are no new theoretical results, equations are correct and easy to check with basic linear algebra tools.
Any claims about deep hierarchical representations are very hard to evaluate. 

    This reviewer is not convinced by the argumentation referring to the tSNE representation (tSNE is an algorithm that fits local neighbourhood, thus a line has no meaning in this image; another run of tSNE or a slight change of parameters could have brought the "birds" class closer to the "furry animals"; there are plenty of samples on that side of the line anyway). The linear decodability is a more useful metric, but it's hard to tell how much of the benefit is due to the increasing receptive field sizes; in the end, the local learning rule (greedy supervised) shows a similar pattern across depth.

- The submission is clearly written and well organised. (But note LL 130--131.)

- This submission is an interesting voice in the ongoing discussion of what are the learning mechanisms used by the neural networks of the brain.



 



limitations_and_societal_impact: The authors state their model is more biologically plausible than the existing proposals, but they do not discuss the new set of assumptions that is required for their model to work. Most importantly, the "broad modulation signal" affecting learning of a given layer needs to be computed based on the predictive dendritic and output signals of the entire layer. The output of the zWc computation needs to be compared to the  "saccade" signal and broadcast back to all neurons (in both the feedforward and the prediction layers), to affect learning in their apical and basal dendrites. In the current implementation, this entire computation happens instantaneously.
review_title: null

final_score: rating: 8: Top 50% of accepted NeurIPS papers, clear accept
review_content: summary: This paper proposes a new Hebbian-like learning algorithm for neural networks that is able to produce hierarchical representations. This new self-supervised learning rule, which is based on the ideas of Contrastive Predictive Coding (CPC), obeys several key biological constraints (such as locality) while attaining similar benchmarks on several standard ML-classification tasks as previous, unconstrained rules.
main_review: The gap between biologically plausible learning algorithms and implausible once such as backpropagation has recently attracted a lot of interest. The authors' approach here is quite original, as they build their learning rule around CPC, and, quite interestingly, assume that the brain self-generates contrastive samples by e.g. performing saccades. That's a great idea! (I hadnt seen that voiced before.) The paper is well written and the exposition is quite scholarly, in that the authors clearly delineate how their model compares to other methods in the field, and how the different aspects can be traced back to other studies (locality of rule, the avoidance of BP, the presence of modulating factors, etc.). The final simulations are convincing, and show that the rule does generate hierarchical and meaningful representations.
I only have a few suggestions/questions:

(1) Why does CLAPP require many more negative samples on video than on STL-10? That wasnt clear to me.

(2) The distinction between the predicted layer z and predicting layer c gets confusing at times. On the one hand, Figure 2c and L75-77 seem to indicate that the feedforward network of z’s is the only network needed (Note specially L85-94). On the other hand, L170-174 and the remaining equations seem to indicate that z and c are separate neurons with separate feedforward connections. The clarity of this distinction could be improved both in text and figures.

(3) L36-37: “The present paper demonstrates that deep representations can emerge from a local, biologically plausible and unsupervised learning rule”. Despite having some components which are biologically plausible, the author introduce a shortcut which seems to violate biology: the learning rule requires a synaptic connection $W^{pred}$ that does not affect the neuron's firing but only synaptic plasticity. In other words, how could a signal that arrives at an apical dendrite influence synaptic plasticity at a basal dendrite while not directly influencing the activity of the neuron itself? Please clarify or word more carefully.


limitations_and_societal_impact: The authors have acknowledged that the current learning rule in its current form is data- and compute-intensive. However, the locality in space and time allows it to be used in neuromorphic hardware, potentially reducing energy consumption significantly.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The authors propose a biologically plausible implementation of contrastive self supervised learning. They solve the problem of remembering negative samples by relying on saccades, taking the post-saccade incoming signal to be a negative sample. They perform a number of comparisons showing the viability of their approach.

--------------------------------------
Post-rebuttal update:
Due to the increased quality of the work stemming from added clarifications and a fuller discussion of biological plausibility, I have increased my score from 5 to 7.
main_review: The paper is novel, interesting and well written. In particular I really like the idea of using saccades to solve the problem of memorizing negative samples. In general I think the paper is strong but to be acceptable for submission a number of claims need to be either toned down or further substantiated, see below.

### Strenghts 
(in no particular order).
* This is a solid work in a relevant field. In particular, the authors astutely take advantage of signals already available in the visual cortex to circumvent the negative sample problem.
* The experiments are more or less thorough. I would have liked to see more results and error bars but when experiments take 4 days on 4 V100s, there is a limit to what can feasibly be done. 
* The ablation study is also more or less convincing of the points the authors want to make.

### Weaknesses 
(in the order of appearance in the paper.)
* The authors criticize prior work for the need to wait for convergence (line 34). However, the only reason their model avoids this is a simplifying assumption mentioned in line 77, the assumption that lateral and feedback connections do not affect the activity. In other words, if the authors relax this simplifying assumption (for more realism for example), their dynamics would again be recurrent and they would find themselves in the same situation as prior work.
* Some references for recent work are missing. For instance, there are biologically plausible models that do not have the issues mentioned in lines 101-102 (e.g. https://arxiv.org/abs/2011.15031, although the author's proposed work is more general than this since 2011.15031 only considers shallow networks.)
* Applying the proposed algorithm to auditory pathways seems a stretch (lines 108-109). In this case, as opposed to the auditory pathway where the negative sample can be deduced based on saccades, the presence of a negative sample has to be deduced. The authors assume that this is a solved problem, however, I think this is very non-trivial. Especially since the goal of the proposed circuit itself is to be able to do things like distinguishing different input sources. This can be alleviated by using cross-modality inputs, e.g. visual cues to distinguish sources, but in this case, competing algorithms would also be take advantage of the cross-modality input.
* (line 164) It should be mentioned that the loss in Eq. 3 is only for one layer and each layer has its own loss. That is, there is an implicit stop gradient preventing the weights of the previous layers to accumulate gradients.  This is made more confusing by not having any layer indices, so it looks like there is a single loss objective and a single deicision made regarding whether the saccade/fixation was correctly classified. I think in general the paper would be more clear if the layer indices were present as well.
* Computation of $H^t$ in line 180. The computation of whether or not the saccade was correctly predicted is done according to the definition on line 160. How is this computation actually performed? Suppressing the time sub and superscripts and including the neuron index instead we have $u = \sum_i z_i W_{ij} c_j$. Now  $z_i$ and  $W_{ij} c_j$ are both local to the $i$'th neuron and the product can be computed. But how is this summed? Is there an interneuron that connects to every neuron and sums these? If so, the product  $z_i W_{ij} c_j$ needs to be the output of the neuron. But this is not the case. Altenratively, each neuron can broadcast its own product through some modulatory compound and this can be averaged to give the overall sentiment. This would be plausible but not biologically realistic (also is not what is actually implemented). Unless I misunderstood what is happening, I think this is the biggest weakness of the paper and needs to be fully addressed.
* Line 188: 'We show in appendix C that ... '. This sounds like there is something akin to an analytic demonstration if not a proof, therefore is sounds a lot stronger than what was actually claimed at the end of appendix C which is that empirically it seems fine. 
* Line 208, 'we allow gradient flow through max-pool' This statement should be expanded (perhaps in the appendix). I cannot tell based on the paper if this is cause for worry or just a trivial assumption.
* Line 219, 'comparable results were obtained', please provide the resulting figures in the appendix and the details of the implementation in the code. It seems like it would not be a totally trivial change in implementation. It would be good for the authors to clarify these in case anyone would want to reproduce their results in the future.
* Lines 230-234, emergence of a hierarchy. This is a weak part of the empirical demonstrations. The authors show that the neurons of later layers respond to more complex shapes. Is this not expected of a conv-net? Afterall the receptive field of the first layer is just 3x3, it cannot possibly see anything more complex. Similarly, deeper layers would have wider receptive fields. I think the definition of hierarchical representation - that is deeper representations are built out of the smaller representations - is in fact built into a layered-conv net and therefore the hierarchical claims are all trivial. If the authors meant a useful or interpretable hierarchical representation, then this was addressed elsewhere in the paper.
* Lines 236-246 Straight line in t-SNE. I am baffled by this. t-SNE is a non-linear mapping and has many bells and whistles that can change the final shape of the resulting representation. As such I'm not sure if this straight line business means anything. It seems like the 3rd layer can do the vehicles/animals discrimination with a slightly bent line so they are similar to me. The following paragraph using a linear decoder for classification is a much better way of seeing performance. 


limitations_and_societal_impact: Yes
review_title: null

Meta-review: This paper proposes a biologically plausible learning algorithm that implements unsupervised (contrastive) learning. The algorithm can learn hierarchical representations. A particular innovation is the use of saccades to generate contrastive samples. Reviewers all agree that the contribution is novel and significant, the manuscript is well-written, and the computational experiments are convincing.
Decision: Accept (Poster)