Venue: ICLR 2023 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: Authors build upon the mitigation of back-propagation and aim at reaching higher accuracies. They first highlight these limits of BP. They then make very strong claims about soft Hebb, an algorithm capable of notably learn in a self-supervised manner or without feedback signals. They evaluate their algorithms or standard benchmarks, which yield satisfactory results, yet with rather shallow networks.


strength_and_weaknesses: A strength of the proposed paper is that it presents well the limits of BP and the challenges ahead. The proposal to use a softmax normalisation (equation 1) is well-founded and I would have wished to more clearly see the link with Bayesian inference. Results are quite good and it is worth highlighting that this paper is among the few which explore the learned networks to understand why it would behave better (or not).
A major weakness of the paper is that the claims are too strong with respect to the results obtained. for instance, the studied networks are very shallow, or the kernels shown in figure 2 are not very interpretable. A comparison with stacked sparse auto-encoders or deep predictive coding would have been constructive. Moreover, the model, as described in section 3, seems to involve many different heuristics and parameters. If width scaling is studied (Figure 4), it is not clear how the other parameters are crucial to your results. Finally, the claim "Soft Hebb learns hierarchical representations" is perhaps premature, as this would require more layers (rather than distinguishing animals versus non-animals and their sub-categories).





clarity,_quality,_novelty_and_reproducibility: The paper is in general clear and well written. However, the claims are often exaggerated compared to the obtained results, and the paper would benefit from tuning them slightly down. Also, there are sometimes some inadequacies or inappropriate mixtures in the text. Page 1, why speak of neuromorphic hardware in general terms in a paragraph about weight transport? Page 3, the Binzegger paper is more about local cortical microcircuits rather than on competition by lateral connections which would justify the WTA. In table 2, is SimCLR really reaching 76.5% on Image Net?


summary_of_the_review: As a summary, the proposed algorithm is well described and analyzed. It however makes many strong claims which are not reached with deep-learning when it is "very "deep (see end of page 9). This week point could be well alleviated by comparing to other networks with a similar number of parameters.
review_title: null

final_score: rating: 8: accept, good paper
review_content: summary_of_the_paper: The paper proposes a new method for training deep (multi-layered, convolutional) networks with Hebbian-like learning, rather than backpropagation. This is important, because backpropagation is  both computationally demanding and biologically implausible. 

An additional motivation comes from neuroscience. There are many Hebbian-based models that reproduce the type of receptive fields (RFs) observed in primary visual cortex (mostly oriented edge detectors). However, to my knowledge, when applied to multi-layer networks, no Hebbian system has been able to reproduce the emergence  of increasingly complex, selective features in successive layers that is observed in visual cortex - at least not without grossly implausible hacks. This is frustrating, since such an emergence is the intuitively expected result (higher layers composing lower-layer features into increasingly complex detectors). Thus it would be interesting if the authors could demonstrate that their model does learn increasingly complex features in higher layers, using only Hebbian learning with plausible modifications.

The method produces strong performance on multiple benchmarks, and increasing performance with additional layers (something that is surprisingly difficult to achieve with Hebbian learning).
strength_and_weaknesses: - Strengths: The problem is important, the method seems novel and interesting, and the results are strong.

- Weaknesses: The main concern is that one of the paper's strong claims (increasingly complex RFs in higher layers) is not clearly substantiated. 

The supposed better semantic separation in layer 4 than layer 1 in Figure 5a-b is tenuous, at best. Similarly, Figure 5D is highly ambiguous: most of the responses shown in Figure 5D seem compatible with one or two blobs of the right colors at the right locations, i.e. very similar to typical first-layer RFs.

The claim of increasingly complex RFs in higher areas (which would be very interesting if confirmed) should be better substantiated (see below), or toned down.

There are more minor problems with the explanations and description, which should be easily fixed (see below).
clarity,_quality,_novelty_and_reproducibility: Most important:

- As stated above, the claim that more complex RFs emerge in higher layers needs more substantiation (or some toning  down). I suggest the authors should give a better view of what the higher area RFs look like. One way to do this, for a given layer, is simply to aggregate the RFs of the lower-level layers at their correct relative positions weighted by their incoming weight (neglecting  spatial pooling). When applied  to other models, this often shows larger Gabor-like features,  composed by the precise arrangement of lower-level Gabor features of similar orientation and adequate phase/position  (sparse Hebbian learning really likes Gabors). IF the current method shows something more complex, that would be interesting.

- I may be wrong , but Figure 2A suggests that you did not preprocess the images with a whitening / decorrelating /center-surround filter, as is commonly done in this type of research. Did you try it? It can be done very easily (with a simple difference-of-Gaussians filter) and might actually help performance  - and make  your RFs more realistic.

- The actual implementation of convolutional plasticity is not explained clearly. We read "The plasticity of one such neuron over all patches is itself an operation that involves convolutions.", but I'm not sure what that means. Is the image just divided into patches, which must replicate  a lot of data since the patches are fully overlapping (IIUC)? How efficient is the process, compared to using  surrogate losses and just running an automatic differentiator (as in Miconi 2021, IIRC)? There needs to be a much more detailed  explanation (perhaps in the  Appendix) and preferably some code!

Minor:

- The method is claimed to use SoftMax rather than WTA, but 1) this would nnot in itself be original and 2) it's not fully correct, because their method actually does restrict positive Hebbian updates to the single most activated cell, i.e. the actual Hebbian update is by WTA. The true novelty of the method lies in the use of *negated* SoftMax outputs to perform anti-Hebbian learning on the non-maximally-activated neurons. This clever technique is  novel    (to my knowledge) and seems to perform a decorrelating function that is often performed with much more complex mechanisms (e.g. anti-Hebbian lateral connection  in Foldiak 1990). Thus it should be mentioned  immediately in the description of SoftHebb in p. 4, next to Equation 2! 


- Eq 2 looks like an Instar rule except for the u_k term... What is the intuitive justification for multiplying by u_k ? 

- In p. 5: please provide a brief one-sentence explanation of Triangle activations, so the reader gets an idea of what it involves (chiefly the fact that it is a population-wide thresholding scheme, rather than an individual activation principle like the RePU mentioned just before).

- Apparently the weights start from large norm and are allowed to decay to  norm 1. This is contrary to common  practice, which starts from small weights and lets them increase. Please provide a short one-sentence justification for this decision. Also, what is it in the equations that ensures the weight norm will go down towards 1, rather than up to a higher equilibrium value?

- Appendix  A.1.2  suggests that there is "no learning" when weight-norm is < 1. Briefly, why is that the case? 

- Miconi 2021 *did* obtain higher performance in higher layers, which should be made clear - however this required ad hoc methods (massive pruning of connectivity) and resulted in unrealistic RFs, so it does not detract from the novelty and impact of the present submission. 

- The color scheme in Figure 3 makes it very hard to distinguish the various colors (especially the two dark greens), please use something more legible.

- In section "A New Learning Regime": What is the "anti-Hebbian" term with hard WTA? In SoftHebb, it is applied to the not-most-activated neurons, but with hard WTA that can't be the case? Please explain it briefly.

-  Figure  4: are the bars just identical to the darkest curve? If so, please remove them.


summary_of_the_review: == Updated review == 

I thank the authors for their clarifications. I have updated my recommendations accordingly.

== Original review ==


The paper is interesting and potentially important. However one of the claims (increasingly complex and elaborate RFs in higher  layers) needs more substantiation, and the paper requires a few clarifications. I am willing to increase my score if these concerns are addressed.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The paper proposes an application of SoftHebb to deep convolutional neural networks. SoftHebb has been proposed before (supposedly by the same authors?), it is a soft winner-take-all version of the Hebbian learning rule implemented using the softmax function. The novelty of the paper lies in making SoftHebb scale to multiple (convolutional) layers. The proposed method compares favorably to other unsupervised counterparts, and exhibits nice properties such as the emergence of hierarchical structures. The method is also posed as biologically plausible, which may be the most important point of the paper.
strength_and_weaknesses: Strengths:
* A biologically plausible alternative to back propagation is shown to work on multi-layer network structures.
* Experiments include complicated tasks such as STL-10 and CIFAR-10, the proposed method compares favorably to existing approaches.
* The related work section is detailed and beautifully written, it gives an excellent perspective on prior attempts at merging neuroscience insights with deep learning.

Weaknesses:
* The authors themselves state this as a limitation: SoftHebb is applied to computer vision tasks only, in an exclusively convolutional setting. Moreover, random weights (prior to SoftHebb training) already seem to have expressiveness in the WTA setting. How much of the learning can be attributed to SoftHebb, and how much is due to the convolutional feature extraction?
* Comparisons lack statistical backing. What are the standard deviations? How consistently does SoftHebb outperform other techniques?
* All results reported are for classification tasks, i.e. a standard BP is used to fine-tune the weights of the SoftHebb network. Is there any way to evaluate the performance of this unsupervised technique without BP? Perhaps clustering, e.g. unsupervised image segmentation?
* The authors can be easily guessed based on the prior work cited.
clarity,_quality,_novelty_and_reproducibility: Clarity: The paper is well-written, with minor typos. Related work is well-described. Many references to the appendices are made throughout the paper, which does not help with the reading, since the main body of the paper should be self-contained. However, the important ideas are communicated clearly.

Quality: This is a thorough work with interesting well-designed experiments.

Novelty: To a large extent, the work is based on SoftHebb, proposed in an earlier paper. This is the first successful multi-layer implementation, however it seems heavily reliant on the CNN architecture - is it the convolutions that perform the heavy lifting?

Reproducibility: The authors list experimentation parameters in the appendices, I am assuming the code may also be made available should the paper be accepted.
summary_of_the_review: Overall, I think this is a very interesting paper with a lot of potential for impact. Its main limitation is that it has only been tested in the vision domain. However, in the vision domain, deep SoftHebb performed the best on STL-10, which is an unsupervised learning computer vision benchmark. Since convolutions are easily transferable to 1D data, perhaps the method can also be tested on time series data in future?

Please see below for suggested corrections and questions to the authors.

Page 4: “…model and algorithm that combined achieve good accuracy…” -> …model and algorithm is that combined they achieve good accuracy…

Table 1 is placed in the paper a few pages before its first mention in the text. Figures are also referenced for the first time out of order. I think it does not help the narrative.

“Batch normalization was used, with its standard initial parameters, which we did not train.” - what are the initial parameters, exactly? Such information is crucial for reproducibility.

Page 8: “prior work Table2)” - missing bracket before “Table”.

What if SoftHebb was allowed to experience the data for longer than just one epoch? Why was the decision made to stop at one epoch? 

Page 9: “it being fully unsupervised, It is also founded…” - second “it” should not be capitalized.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: The authors of the given manuscript extend a bio-plausible backpropagation alternative, “SoftHebb,” to apply to training multiple layers. In biological plausibility, this algorithm bows to the constraints of no weight transport, local plasticity, and no time-locked updates. In addition, the learning is unsupervised. In this restrictive setting, multilayer SoftHebb outperforms competing bio-plausible methods. 

strength_and_weaknesses: Strengths:
(+) Writing is essentially error-free.
(+) Clear and informative figures and tables.
(+) Neuro-inspired mechanism (e.g., lateral inhibition).
(+) An insightful discussion of limitations, in the end, and also in the main text.

Weaknesses:
(-) Not accompanied by code for reproducing the experiments.
(-) Novelty is somewhat limited.

Minor:
- The motivation for biological learning could have been more robust in the introduction to motivate the reader. Later in the discussion, neuromorphic chips are mentioned as a potential application. Why not in the introduction?
- Multiple letters for variables such as “lr” and “lrp” may be confusing.
clarity,_quality,_novelty_and_reproducibility: - High-quality writing, figures, and tables make understanding easy.
- The novelty is to extend a previous method, Softhebb, to work for multiple-layer architectures by combining anti-Hebbian plasticity, specific activation functions, and convolutions. Not entirely proposing a new approach or algorithm, but combining existing work in a novel way. There clearly is a novelty but it is limited.
- The method is clearly described and referenced, but a code repository would be essential for optimal reproducibility and for allowing other people to work with it.
summary_of_the_review: The proposed algorithm operates in a challenging setting and performs well. The paper’s writing and overall presentation are impeccable. The only drawback is that the novelty is somewhat limited. For better reproducibility, the authors should provide code for the reported experiments. Given the code, I would consider increasing my rating.
review_title: null

Meta-review: metareview:_summary,_strengths_and_weaknesses: This paper proposes how to extend successfully SoftHebb for deep multi-layered neural networks, maintaing the advantages of backpropagation-free training (no weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory). All reviewers agree that the paper is very well written, results are convincing, and overall, the paper presents a positive contribution to the literature.
justification_for_why_not_higher_score: Quotting one of the reviewers, *to a large extent, the work is based on SoftHebb, proposed in an earlier paper. This is the first successful multi-layer implementation, however it seems heavily reliant on the CNN architecture - is it the convolutions that perform the heavy lifting?* The response of the authors confirming the point explains why the submission does not receive a higher score.
justification_for_why_not_lower_score: Good extension of an existing algorithm to the case of multi-layered networks.
Decision: Accept: notable-top-25%