Hebbian Deep Learning Without Feedback
Authors: Adrien Journé
Hector Garcia Rodriguez
Qinghai Guo
Timoleon Moraitis
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Hebbian
winner-take-all
cortical circuits
unsupervised
online
biologically plausible
neuromorphic

Abstract: Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approximations significantly decrease accuracy in benchmarks, suggesting that an entirely different approach may be more fruitful. Here, grounded on recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals. As a result, it achieves efficiency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory or other feedback signals – which were necessary in other approaches. Its increased efficiency and biological compatibility do not trade off accuracy compared to state-of-the-art bio-plausible learning, but rather improve it. With up to five hidden layers and an added linear classifier, accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with a radically different approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning. Code is available at https://github.com/NeuromorphicComputing/SoftHebb.

Published as a conference paper at ICLR 2023
HEBBIAN DEEP LEARNING WITHOUT FEEDBACK
Adrien Journé', Hector Garcia Rodriguez', Qinghai Guo?, Timoleon Moraitis
(adrien. ; hector waw.oaaaaaaaa.a.rr.a.aaa. guoqinghai, timoleon. moraitis mor.atisseehuaeiiccom chu
ABSTRACT
Recent approximations to backpropagation (BP) have mitigated many of BP's computational inefficiencies and incompatibilities with biology, but important limitations still remain. Moreover, the approximations signifi- cantly decrease accuracy in benchmarks, suggesting that an entirely different approach may be more fruitful. Here, grounded recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals. As result, it achieves efficiency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iter- alive equilibria, and (sell-) supervisory other feedback signals which were necessary in other approaches. Its increased efficiency and biological compatibility do not trade off accuracy compared to state-of-the- art bio- plausible learning, but rather improve it. With up to five hidden layers and an added linear classifier, accuracies MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%. In conclusion, SoftHebb shows with radically different approach from BP that Deep Learning over few layers may be plausible in the brain and in- creases the accuracy of bio-plausible machine learning. Code is available https://github. www.aa......//aaaaaaaeeeeeeeeeee http:///oooooorrcoooooottiiiiiia tofbbbbbb...
INTRODUCTION: BACKPROPAGATION AND ITS LIMITATIONS
The core algorithm in deep learning (DL) is backpropagation (BP), which operates by first defining an error or loss function between the neural network's output and the desired output. Despite its enormous practical utility (Sejnowski, 2020). BP requires operations that make training a highly expensive process computationallyy setting limits to its applicability in scenarios. In addition, the operations are largely incompatible with biological learning, demanding alternatives. In the following we describe these limitations and their significance for DL, neuromorphic computing hardware, and neuroscience. Notably, after the proprint of this papor, Hinton (2022) presented an algorithm with similar considcrations.
Weight transport. Backpropagating errors involves the transpose matrix of the forward connection weights. This is not possible in biology, as synapses are unidirectional. Synaptic conductances cannot. be transported to separate backward synapses either. That the weight, transport problem (Grossberg. 1987; Crick, 1989), and it also prevents learning on energy-efficient hardware substrates (Crafton et al., 2019) such as neuromorphic computing hardware, which is one of the most researched approaches to overcoming the efficiency and scaling bottlenecks of the von Neumann architocture that underlies today's computer chips (Indiveri, 2021). A key element of the neuromorphic strategy is to place computation within the same devices that also store memories, akin the biological synapses, which perform compulations but also store weights (Sebastian el al., 2020; Sarwal el al., 2022a;b). However, in BP, the transposition of stored memories necessary for weight. transport, and this implies significant circuitry and energy expenses, by preventing the full in-memory implementation of neuromorphic technology (Crafton et al., 2019).
Non-local plasticity. BP cannot update each weight based only the immediate activa- tions the two nenrons that the weight connects, ie. the pre- and post-synaptic neurons, as
Huawei Zurich Research Center, Switzerland Huuawei ACS Lab, Shenzhen, China Corresponding author

---

Published as a conference paper at ICLR 2023
Table Accuracies on CIFAR-10, and qualities of biological plausibility and computational efficiency, for various algorithms. SoftHebb's type of algorithm has all four efficient and plausible qualities. Sobb also achieves the highest accuracy, except for backprop, and its unsupervised learning only involved single epoch. Note, our literalure search was not limited by number of layers, but its results are.
Qualities
Accuracy
Layers
A Algorithm
Reference
99.4
152
Backprop (cross-cntropy)
Kolesnikov ct al. 2020
84.0
Backprop (cross-entropy)
Ours
71.8
Feedback Alignment
Frenkel et al. 2021
60
Predictive Coding
Millidge al. 2020
13.4
Equilibrium Propagation (2-phase)
Laborieux al. 2021
78.5
EP (2-phase, random sign)
Laborieux et al. 2021
Weight- Seigghttransports free
79.9
Burstprup
Payeur et al. 2021
61.0
BustCCN
Greudy el
70.5
Direct Feedback Alignment
Frenkel et al. 2021
71.5
DEA (untrained convs)
Frenkel et al. 2021
Local plasticity
65.6
Diroct Random Target Projection
Freukel et 2021
69.0
DRTP (untrained convs)
Frenkel et al. 2021
Update-tnlbcae
73.1
Single Sparse DFA
Crafton et al. 2019
53.5
Latent Predictive Learning
Halvagal and Zenke 2022
73.7
Self Organising Maps
Stuhr Brauer 2019
Unstpervised
72.2
Hard WTA
Grinberg et al. 2019
64.6
Hard WTA
Miconi 2021
80.3
SoftHebb epoch)
Ours
requires the error signal, which is computed at a different point in time and elsewhere in the network, i.c. at the output. That makes BP non-local in space and time, which is critical discrepancy from the locality that generally believed to govern biological synaptic plasticity (Baldi et al., 2017). This non-locality implies further computational inefficien- cies. Specifically, forward-passing variables must. be memorized, which increases memory requirements (Löwe et al., 2019). Moreover, additional backward signals must be computed and propagated, which increases operations and electrical currents. It is noteworthy that these aspects are not limiting only future neuromorphic technologies, but even the hardware foundation of today's DL, i.c. graphical processing units (GPUs), which have their own constraints in memory and FLOPS.
Update locking. The error credited by BP to synapse can only be computed after the information has propagated forward and then backward through the entire network. The weight updates are therefore time-locked to these delays (Czarnecki et al., 2017; Jaderberg et al., 2017; Frenkel et al., 2021). This slows down learning, so that training examples must be provided least slowly as the time to process the propagation through the two directions. Besides this important practical limitation of BP for DL, also does not appear plausible that multiple distant neurons in the brain coordinate their processing and learning operations with such precision in time, nor that the brain can only learn from slow successions of training examples.
Global loss function. BP is commonly applied in the supervised selting, where humans provide descriptive labels of training examples. This is a costly process, thus supervised BP cannot exploit most of the available data, which is unlabelled. In addition, it does not explain how humans or animals can learn without supervisors. As a result, significant research effort has been dedicated to techniques for learning without labels, with increasing success recently, especially from self-supervised learning (SSL) (Chen et al., 2020; Mitrovic et al., 2020; Lee et al., 2021; Tomasev et al., 2022; Scherr et al., 2022). In SSL, BP can also use certain supervisory signals generated by the model itself as a global error. Therefore, while BP does not require labels per se, it does require top-down supervision in the form of a global loss function. The drawback this that learning then becomes specialized to the particular task that is explicitly defined by the minimization of the loss function, as
2

---

Published as a conference paper at ICLR 2023
opposed to the learning of generic features. Practically, this is expressed in DL overfitting, sensitivity to adversarial attacks (Madry et al., 2017; Moraitis et al., 2021), and limited transferability generalizability of the learned features to other tasks datasets (Lee et al., 2021). Moreover, a global optimization scheme such as BP cannot be considered plausible model of all learning in the brain, because learning in MI. tasks does emerge from highly biological plasticity rules without global (self- Jspperviioe, e.g. unsupervised and Hebbian-like (Diehl and Cook, 2015; Moraitis et al., 2020; 2021; Rodriguez et al., 2022).
2 ALTERNATIVES TO BACKPROPAGATION AND THEIR LIMITATIONS
Unsupervised Hebbian learning in cortex. Hebbian-like are those plasticity rules that depend only on the activity of pro- and post-synaptic ncurons. If such plasticity is combined with between-neuron competition that suppresses weakly activated neurons in layer, e.g. as an argmax, it can lead to learning of useful features in the absence of any sspperviion (Sanger, 1989 Linsker, 1992). This is radically different approach from BP. It is not governed by a global optimization process, but rather emerges from local synaptic plasticity as a purely bottom-up self-organization. Without supervision, feedbacks, or targets, competitive Hebbian learning circumvents all five limitations of BP, as it does not require any back-propagating signals. Namely, it is free of weight transport, non-localities, locking problems, and global losses. These are significant practical advantages. Moreover, these properties make Hebbian learning much more plausible biologically. Besides, there is abundant evidence for Hebbian-like plasticity in the brain, in forms based on the spiking type of biological neuron activations (Sjöström et al., 2001; Markram et al., 2011; Feldman, 2012). Even the competitive network conneetivity that leads to useful learning through such plasticity is strongly supported by biological observations. Specifically, competition between neurons emerges from lateral inhibition and this is found throughout the cortical sheet of the mammalian brain (Douglas el al., 1989; Douglas and Martin, 2004; Binzegger et al., 2004; 2009), and are commonly described as winner-take-all (WTA). Furthermore, such competitive learning has been deeply studied computationally and theoretically for long time under various forms (Von der Malsburg, 1973; Nessler al., 2013; Dichl and Cook, 2015; Krotov and Hopfield, 2019; Moraitis et al., 2020). All these aspects would be important advantages if such learning could underlic DL. However, unsupervised Hebbian learning has only been effective in shallow networks. To be precise, adding layers has failed to show significant improvements standard beuchmarks (Amato et al., 2019; Lagani et al., 2021), except to certain degree in Miconi (2021) (Fig. 1). Arguably, this has been the case because emergent bottom-up learning from plasticity is hard to reconcile with top-down learning from a loss function, and the latter approach has been the backbone of DT. Those competitive Hebbian approaches that were normative, i.c. that derived a plasticity rule from an optimization principle, were based on principles that do not appear compatible with successful DL techniques, either because of dramatically different network types (Nessler et al., 2009; 2013) or loss functions (Pehlevan and Chklovskii, 2015).
CIFAR degth ccoputtio
50
Sodtlebb (orzs)
Accuncy (8)
Miconi, 2021
Loguai et ni, 2021
Amato er al, 2019
56
Japprs
Figure 1: First successful multilayer results. Soft- Hebb's CIFAR-10 accuracy increases with depth (hid- den layers), compared with prior work.
SobtH Recent work, however, has advanced the theory behind Hebbian WTA learning in terms that are more compatible with DL. Specifically, Moraitis et al. (2021) used a simple softmax to implement a soft WTA (Equation (1)), which lends a Bayesian interpretation to the network and its learning (see also Nessler et al. (2009; 2013)). Moraitis al. (2021) also derived a Hebbian-like plasticity rule (Equation (2)) that minimizes the Kullback-Leibler divergence of the model's probability distribution from the input's, and cross-entropy from the labels under certain assumptions, without having access to those labels. In one-layer networks, SobtHebb showed increased learning speed and significantly higher robustness to noise and adversarial attacks. Its theoretical results seem important towards deeper networks, but they were not sufficient for a practical demonstration. Here (Section 3), we provide a Hebb--abee setup that does achieve certain depth in Hebbian learning.
3

---

Published as a conference paper at ICLR 2023
Approximations of backpropagation. Alternatives that not radically different from BP but rather approximate it, while mitigating some of its issues, have attracted heavy research interest with increasing success. For example, self-supervised learning enables BP to learn without labels (Hadsell et al., 2006), with instances such as SimCI.R (Chen et al., 2020) and follow-ups (Mitrovic et al., 2020; Grill et al., 2020; He et al., 2020; Caron et al., 2020; Tomasev et al., 2022) reaching rather high performance. A different approach, Feedback Alignment (FF), (Lillicrap et al., 2016; Frenkel et al., 2021), solves the weight- transport problem. Direct Feedback Alignment (Nokland, 2016; Frenkel et al., 2021) as an extension of FA, also avoids non-local computations for weight updates. The spatial non- locality of backpropagation has also been addressed by other algorithuss such as predictive coding (Hadsell et al., 2006; Chen el al.; 2020), equilibrium propagation (Scellier and Bengio, 2017: Laborieux et al., 2021), burstprop (Payeur et al., 2021). The work of Löwe et. al. (2019), and CLAPP by Illing et al. (2021) are self-supervised algorithms that avoid not only reliance on labels but also spatially non-local learning, However, they require contrasting of examples known to be of different type. A very recent self-supervised approach avoids this requirement by adding Hebbian term to learning (Halvagal and Zenke, 2022). This makes it a rather plausible algorithm for the brain, but it still relies on comparisons between distinetly different views of each individual imput. Moreover, its performance is significantly worse on CIFAR-10 and STL-10 than this paragraph's previous references. In fact, all aforementioned approaches cause significant drop in accuracy compared to BP, and most cases are not applicable to datasets as complex as ImageNet (Bartunov et al., 2018). An algorithm that avoids the non- locality of end-to-end BP and also achieves quite high accuracy in CIFAR-10 was proposed by Nøkland and Eidnes (2019). However, it relies on an auxiliary trained network added to each layer, which increases complexity, and supervision, which requires labels. Other works have achieved learning with local plasticity while avoiding weight transport and the update-locking problem (Crafton et al., 2019; Frenkel et al., 2021). However, they suffer from large performance drops and rely on supervision. Clearly, significant progress compared to standard BP has been made through multiple approaches, but important limitations remain (Table 1). These limitations would be surmounted if a competitive Hebbian algorithm performed well in multilayer networks and difficult tasks. Here, we explore the potential of SoftHebb (Moraitis et al., 2021), i.e. the recent algorithm that seems fitting to this goal. Our results in fact demonstrate multilayer learning setup, reaching relatively high performance for bio-plausible algorithms on difficult tasks, with high efficiency and tight biological constraints. In the next section we describe this setup.
3 OVERVIEW OF MULTILAYERR SOFTHEBB
The key elements of the multilayer SoftHebb model and algorithm that achieve good accuracy without compromising its efficiency and biological-plausibbie SoftHebb (Moraitis et al., 2021) in a layer of K neurons realizes soft WTA competition through softmax, parametrized by base b or equivalently a temperature T:
are the following.
buk yk ER
(1)
6t
21 E E
where up is the k-th neuron's 's total weighted input, and is its output after accounting for competition from neurons 1. The second key aspect in the SoftHebb algorithm is the plasticity rule of synaptic weights and neuronal biases. Biases represent prior probabilities in the probabilistic model realized by the network. In our experiments here we omitted them, presuming a fixed uniform prior. The plasticity defined for synaptic weight wis from presynaptic neuron i with activation x; to a neuron k is
(SoftHebb) Au ik
17 yk (z+
Uy Wak)
(2)
Notably, all variables are temporally and spatially local to the synapse. The rule provably optimizes the model to perform Bayesian inforence of the hidden causes of the data (Moraitis et al., 2021) (see also Section 2). Soft anti-Hebbian plasticity. Previous Hebbian-like algorithms have found anti- Hebbian terms in the plasticity to be helpful in single-layer competitive learning (Krotov and
4

---

Published as a conference paper at ICLR 2023
Hopfield, 2019; Grinberg et al., 2019). However, such plasticity matched the assumptions of a hard WTA, as opposed to SoftHebb's s distributed activation, and involved additional hyperparameters. Here we introduce new, simple form of anti- plasticity for soft WTA networks, that simply negates SoftHebb's weight update (Equation (2)) in all neurons except the maximally activated Convolutions. Towards a multilayer architecture, and to represent imput information of each layer in more distributed manner, we used localized representation through convolutional kernels. The plasticity rule is readily transferable to such an architecture. Convolution can be viewed as a data augmentation, where the inputs are no longer the original images but are rather cropped into smaller patches that are presented to fully connected Soft.Hbb nelwork. Convolution with weight sharing between patches efficient for parallel computing platforms like GPUs, but in its literal sense it is not biologically plausible. However, this does not fundamentally affect the plausibility convolutions, because the weights between neurons with different localized receptive fields can become matching through biologically plausible rules (Pogodin et al., 2021).
Alternative activations for forward propagation. In addition to the softmax involved in the plasticity rule, different activation functions can be considered for propagation to each subsequent layer. In biology, this dual lype of activation may be implemented by multiplexing overlapping temporal or rate codes of spiking neurons, which have been studied and modelled extensively (Naud al., 2008; Kayser al., 2009; Akam and Kullmann, 2014: Herzfeld al., 2015; Moraitis et al., 2018; Payeur et al., 2021). We settled on combination of rectified polynomial unit (RePU) (Krotov and Hopfield, 2016; 2019), with Triangle (A Apendix A.3.1), which applies lateral inhibition by subtracting the layer's mcan activity. These perform well (Coates et al., 2011; Miconi, 2021), and offer tunable parametrization.
adaptive learning rate. We introduce per-neuron adaptive learning rate scheme that stabilizes to zero as neuron weight vectors converge to a sphere of radius 1, and is initially big when the weight vectors' norms large compared to 1: ? = ( 17) where q is a power hyperparameter. This per-neuron adaptation based on the weights remains local operation and is reminiscent. of another important adaptive learning rate scheme that is individualized per synapse, has biological and theoretical foundations and speeds up learning (Aitchison, 2020; Aitchison et al., 2021). Ours is arguably simpler, and its relevance is that it increases robustness to hyperparameters and initializations, and, combinod with the Baycsian nature of SoftHebb (Section 2), it spoeds up lcarning so that mere single learning epoch suffices (Section 4).
Width scaling. Each new layer halves the image resolution in each dimension by a pooling
operation, while the layer width, i.e. the nunber of convolutional neurons, is multiplied by "width factor" hyperparameter. Our reported benchmarks used factor of
(A) Layer
(B) Layer 2
(C) Layer 3
(D) Layer 4
Figure? 2: Example SoftHebb receptive fields, learned from STL-10. More in Appendix B.6.
The most crucial elements for deep representation learning are the soft competition and the corresponding Hebbian plasticity rule that underpin SoftHebb (Figures 3 and B.2), the similarly soft auti-Heebbian plasticity that introduced (Fig. B.2), the convolutional neurons, and the width scaling architecture that involves a depth-wise diminishing output resolution (Fig. 4). The adaptive learning rate significantly speeds up the training (Fig. B.3B), such that we only use a single unsupervised learning epoch for the deep network. The specific activation function and its tunable parametrization are less crucial but do improve
5

---

Published as a conference paper at ICLR 2023
performance (Appendix B). We arrived at this novel setup grounded on Moraitis et al. (2021) and through a literature- and cuuition-guiden search of possible additions.
RESULTS
Summary of experimental protocol. The first layer used 96 convolutional neurons to match related works (Fig. 1), except our ImageNet experiments that used 48 units. The width of the subsequent layers was determined by the width factor (see previous section). Unsupervised Hebbian learning received only one presentation of the training set, i.e. epoch. Each layer was fully trained and frozen before the next one, a common approach known as greedy layer-wise training in such local learning schemes (Bengio et al., 2006: Tavanaci and Maida, 2016; Löwe et al., 2019). Batch normalization (Ioffe and Szegody, 2015) was used, with its standard initial parameters (7 = 1, 8 0), which we did not train. Subsequently, a linear classifier head was trained with cross-entropy loss, using dropout regularization, with mini-batches of 64 examples, and trained for 50, 50, 100, and 200 supervised epochs for MNIST, CIFAR-10, STL-10, and ImageNet accordingly. We used an NVIDIA Tesla V100 32GB GPU. All details to the experimental methods provided in Appendix A, and control experiments, including the hyperparameters' impact in Appendix B.
Fully connected basclines. The work of Moraitis et al. (2021) presented SoftHebb mainly through theorctical analysis. Experiments showed interesting generative and Bayesian properties of these networks, such as high learning speed and adversarial ro- bustness. Reported accuracies focused on fully connected single-hiddden-llyeer nel works showing that it was well applicable to MNIST and Fashion-MNIST datasels reaching ac- curacies of (96.94 + 0.15)% and (75.14 + 0.17)%, respectively, using 2000 hidden neurons. Starting from that work, we found that when moving to more com- plex datasets such as CIFAR-10 or STL-10, SoftHebb performance was not competitive. Specif- ically, the same shallow fully- connected network's accuracies reached (43.9 + 0.18)% and (36.9 + 0.19)% accordingly. Compared to BP's (55.7 + 0.13)% and (50.0 + 0.16)% %, this suggested that hingle-hidden-layee networks were insufficient for extraction of mcan- ingful features and scparation of input classes. We then stacked two such fully connected Heb- bian WTA layers. This network actually performed worse, reach- ing (32.9 + 0.22)% and (31.5 + 0.20)% on these tasks. Convo- lutional basclines. Recent search has applied Hebbian plas- ticity convolutional hard-W'TA neural networks (CNN) (Miconi, 2021: Lagani et al., 2021; Amato et al., 2019). However, it has not achieved significant. if any, im- provement through the addition of layers (Fig. 1, green curves). In our control experiments, we found that these networks with the plasticity rules from the literature do not learn helpful features, the fixed random weights performed better than the learned ones, also in agreement with results from Miconi (2021). Indeed, we find that the features learned by such hard- WTA networks are simple Gabor-like filters in the first layer (Fig. B.5A) and in deeper ones (see also Miconi (2021)).
Hard Wilk
Btcckppop
(5)
(A) CIFAR-10
(B) CIFAR-100
61
M4
3
3lyso
4om
(C) STL-10
(D) ImageNette
Figure 3: Depth-wiss wise performance for various training setups and for untrained random weights, in 4 datasets. Number of hidden layers is indicated.
A new learning regime. One way to learn more complex features is by adding anti-

---

Published as a conference paper at ICLR 2023
Hebbian term to the plasticity of WTA networks (Krotov and Hopfield, 2019; Grinberg et al., 2019). Notably, this method was previously tested only with Bard-WTA networks and their associated plasticity rules and not with the recent Sebt model and plasticity. In those cases, anti-Hebbian plasticity was applied to the k-th most active neuron. Here, we introduced new; soft type of anti-Hebbian plasticity (see Section 3). We studied its effect first by proxy of the number of "R1" features (Moraitis et al., 2021), i.e. the weight vectors that lie on a unit sphere, according to their norm. Simple Gabor-like filters emerge without anti-Hebbian terms in the plasticity (Fig. B.5A) and are R1. Even with anti-Hebbian plasticity, in hard WTA networks, the learned features are R1 (Krotov and Hopfield, 2019). In the case of SoltHebb with our soft type of mebi-Hebiian plasticity, standard, L.e. non-R1, features emerge (Fig. B.1 & B.5B). By measuring the accuracy of the SobtHebb network while varying the temperature, we discovered a regime around T = (Fig. B.1) where R1 and non-Rl features co-exist, and accuracy is highest. This regime only emerges with SoftHebb. For example, on CIFAR- 10 with a single convolutional layer and an added linear classifier, SoftHebb accuracy (71.10- = 0.06)% significantly outperformed hard WTA (62.69 + 0.47)% and random weights (63.93 + 0.22)%, almost reaching the accuracy of BP (72.42 + 0.24)% on the same two-layer network trained end-to-end.
observed that less
Convergence in a single epoch. Adaptive learning rate. By studying convergence through R1 features and by experimenting with learning rates, we conceived an adaptive learning rate that adapts to the norm of each neuron's 's weight vector (Section 3). We found that it also speeds up learning compared to more conventional learning schedules (Fig, B.3B) with respect to the number of training iterations. The extent of the speed-up is such that we only needed to present the training dataset once, before evaluating the network's performance. All results we report on SobtHebb are in fact after just one epoch of unsupervised learning. The speed-up is in agreement with observations of Moraitis et al. (2021) that attributed the speed to SoftHebb's 's Bayesian nature. Moreover, we found that with our adaptive learning rate, convergence is robust to the initial conditions, to which such unsupervised learning schemes are usually highly sensitive (Fig. B.3A & B.3B).
The architecture's impact. The multilayer architecture uses a pooling operation of stride 2, which halves each dimension of the resolution after cach layer. We stop adding layers when the output resolution becomes at most 4 x 4. The layer where this occurs depends on the original input's 's resolution. Thus, the multilayer network has three hidden convolutional layers for MNIST or CIFAR-10. four layers for STL-10, and five layers for ImageNet at a resolution setting of 160 x 160 px. We used four times more neurons each layer than in the previous layer. This architecture its own, with random initial weights, shows inconsistent increases in classification accuracy, up to a variable depth (Fig. 3). The performance increases caused by the addition of random layers seem surprising but are consistent with the literature where random weights often outperform biologically- plausible learning rules (Miconi, 2021; Frenkel et al., 2021). Indeed, by using the more common hard-WTA approach train such a network, performance not only deteriorated compared to random weights but also failed to increase with added depth (Fig, 3).
Width scaling
Acconna (9))
A
3
66 layer
2 Jayan
3 layyc
Figure 4: CIFAR-10 layer-wise performance of SoftHebb, for differ- ent width factors. Soft- Hebb enables depth- scaling when the width of deep layers scales suf- ficiently. Factors (Ix, 2x, 4x) indicate layer- wisc increase in the num- ber of neurons.
Classification accuracy (see Tables 1 & 2). We report that with SoftHebb and the techniques we have described, we achieve Deep Learning with up to 5 hidden layers. For example, layer-wise accuracy increases on CIFAR-10 are visible in Fig. 1. Learning does occur, and the layer-wise accuracy improvement is not merely due to the architecture choice. That is testified, first, by the fact that the weights do change and the receptive fields that emerge are meaningful (Fig. 5). Second, and more concretely, for the end-point task of classification, accuracy improves significantly compared to the untrained random weights, and this is truc in all datascts (Fig. 3). SoftHebb achioves test accuracics of (99.35 = 0.03)%, (80.31 + 0.14)%, (76.23 + 0.19)%, 27.3 27.3% and (80.98 + 0.43)% on MNIST, CIFAR-10. STL-10, the full ImageNet, and ImageNetle. We also evaluated the same networks trained in a
7

---

Published as a conference paper at ICLR 2023
fully label-supervised manner with end-to-end BP on all datasets. Due to BP's resource demands, we could not compare with BP directly on ImageNet but we did apply BP to the ImageNette subset. The resulting accuracies not too distant from SoftHebb's (Fig. 3). The BP-trained network reaches (99.45 + 0.02)%, (83.97 + 0.07)%, (74.51 + 0.36) % and (85.30 + 0.45)% MNTST, CIFAR-10, STT.-10, and ImageNette respectively. Notably, this performance is very competitive (see Tables 1 & 2).
planes
Layer 1
Layer 4
bird
car
cat
deer
track airplane
dog
vehicles'
"furry animals!
monkey
Layer 1
Layer 4
neczon
neuron
Botzon $4
Figure 5: Indications for hierarchical representations learned by SoftHebb on STL-10. A, B: UMAP projection of the test set after passing through 1 or 4 Sofb layers. The learned 4-layer embedding is visibly more organized by input class and classes are better separated. C, D: Images and patches (red bounding boxes) that best activate 5 random neurons from each layer. The 1st layer's neurons (C) have become receptive to small patches with simple features (note the very small red bounding boxes), and are unspecific to the pictured object. The 4th layer's ncurons (D) are recoptive to patches with larger and complex features, and specific object types. Plastic synapses produce this apparent hierarchy in the absence of non-local signals, label-related information, other top-down supervision, or self-supervision
Evidence for hicrarchical representations. One central question concerning Sobt was its capacity to build hicrarchical representation of fcatures, i.c. features that increase in semantic abstraction with inercasing depth, and also become more useful for downstream tasks such as object classificatton. One source of evidence for hierarchy is the layer-wise increase in accuracy that we observe, which shows that the added layers are increasingly useful Fig. 1. By also using method from Illing et al. (2021), we found further evidence (Fig. 5). In the UMAP projection (MeInnes et al., 2018), the classes of inputs appear better separated after four layers than after one (Fig. 5A, B). Moreover found the patches in the dataset that maximally activate hidden neurons. These too appear increasingly complex and abstract (Fig. 5C, D). Further, we visualized the neurons" receptive field, by numerically optimizing the input to maximize hidden activations, through projected gradient descent, similar Le et al. (2012) (see Appendix A for methods). Again, depth appears to increase the complexity and potential usefulness of the learned features (Fig. 2 and Appendix).
SoftHebb as unsupervised multilayer learning. In STL-10, SoftHebb (76.23 0.19)% outperforms fully supervised BP (74.51 + 0.36)% (Fig. 3C). The very few labelled examples in the training set do not suffice to learn good representations through only-supervised BP. By fine-tuning Soft.Hbb with end-to-end BP on the few labelled STL-10 examples, SobtHebb's accuracy further improves to (78.5 + 0.35)%. This shows that the objective that is implicitly optimized by SoftHebb''s learning is compatible with the explicit objective of cross-cntropy, as was more formally thcorized and predicted in Moraitis et al. (2021). Comparisons with other important algorithms for learning without labels show that SoftHebb in 4- or baadden-layer-ddee networks outperforms the partly biologically plaussibe prior work (Table 2). Of course, BP in more complex models significantly perporos Sobb.
8

---

Published as a conference paper at ICLR 2023
Table STL-10 & ImageNet top-1 accuracy (%) of un- or self-supervised (blue frame) & partly bio-plausible networks (green frame). Bold indicates the best-performing biologically- plausible row, i.. SoftHebb. SoftHcbb's unsupervisod learning only involved 1 epoch.
Training algorithm
Model
STL-10
ImageNet
Reference
TEC (1000/100 epochs)
Re
91.6
66.3
Scheer et al. 2022
ShmCLR (1000 epochs)
ResNet-50
91.5
76,5
Scherr et al. 2022; Chen et al. 2020
mm- or self- supervised seerviiedd
SimCLR (100 epochs)
ResNet-18
86.3
30.0
Chen et al. 2020 (our repr.)
Greedy InfoMax
ResNet-50
81.9
n.a.
Löwe et al. 2019
None (Random chance)
None
10.0
0.1
Chance
None (Random weights)
SoftHlebb
68.2
14.0
Ours
biologically plausible
Hebbian
Hard WTA
54.8
u.a.
Our
Softllebb (1 epoch)
Softllebb
76.2
27.3
Ours
CLAPP
VGG-6
73.6
n.a.
Ming el al. 2021
LPI.
VGG 11
61.9
n.a.
Halvagal and Zenke 2022
K-means
K-means
74.1
Dundar et al. 2015
Fredback Aligument
5-layer CNN
6.9
Bartunov et al. 2018
Direct Feedback Aligument
AlexNet
6.2
Crafton et al. 2019
Single Sparse DFA
AlexNet
n.a.
2.8
Crafton al. 2019
DISCUSSION
Hebb's accuracy and applicability in difficult tasks challenges several other biologically- constrained DL algorithms. Arguably it also a highly biologically plausible and computa- tionally efficient method, based on it being free of time-locking of weight updates, it being fully unsupervised, experimental observations in cortical circuits, such as Hebbian plasticity and WTA structure. Importantly, such Hebbian WTA networks enable non-von Neumann neuromorphic learning chips (Qiao et al., 2015; Kreiser et al., 2017; Sebastian et al., 2020; Indiveri, 2021; Sarwat et al., 2022a). That is an extremely efficient emerging computing technology, and Hebb makes high performance with such hardware more likely. The algorithm is applicable in tasks such as MNIST, CIFAR-10, STL-10 and even ImageNet where other algorithms with similar goals were either not applicable or have underperformed Soft (Fig. 1, Table 1, Table 2, & Bartumov el al. (2018)). This is despite the fact that most alternatives only address subsets of SoftHebb's goals of efficiency and plausibility (Section & Table 1). Löwe et al. (2019) and Burstprop (Payeur et al., 2021) results on STL-10 and ImageNet are not included in Table 2, because the ResNet-50 of Löwe et al. (2019) used standard BP through modules of at least 15 layers, and because Paycur et al. (2021) did not report ImageNet top-1 accuracy. Sobb did outperform Burstprop and its successor BurstCCN (Greedy et al., 2022) on CIFAR-10 (Table 1). Beyond neural networks, K-means has also been applied to CIFAR-10 (Coates et al., 2011), however, without successful stacking of K-means "layers".
non-local plasticity, and also founded on physiological
From the perspective of neuroscience, our results suggest that Deep Learning up to few layers may be plausible in the brain not only with approximations of BP (Payeur et al., 2021; Illing et al,, 2021; Greedy et al., 2022), but also with radically different approaches. Nevertheless, to maximize applicability, biological details such as spiking neurons were avoided in our simulations. In ML context, our work has important limitations that should be noted. For example, we have tested Sobt only in computer vision tasks. In addition, it is unclear how to apply SobbHebb to generic deep network architectures, because thus far we have only used specifically width-scaled convolutional networks. Furthermore, our deepest SoftHebb network has only 6 layers in the case of ImageNet, deeper than most bio- plausible approaches (see Table 1). but limited, As a consequence, SoftHebb cannot compete with the true state of the art in MI. (sce e.g. ResNet-50 SimCLR result in Table 2). Such networks have been termed "very deep" (Simonyan and Zisserman, 2014) and "extremely deep" (He et al., 2016). This distinguishes from the more general term "Deep Learning" that was originally introduced for networks as shallow ours (Hinton et al., 2006) has continued to be used so (sce c.g. Frenkel et al. (2021) and Table 1), and its hallmark is the hierarchical representation that appears to emerge in our study. We propose that Soft is worth practical exploitation of its present advantages, research around its limitations, and search for its possible physiological signatures in the brain.
9

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENTS
This work was partially supported by the Science and Technology Innovation 2030 Major Project (Brain Science and Brain-Like Intelligence Technology) under Grant 20222D0208700.
The authors would like to thank Lukas Cavigelli, Renzo Andri, Edouard Carré, and the rest of Huawei's Von Neumann Lab, for offering compute resources. TM would like thank Yansong Chua, Alexander Simak, and Dmitry Toichkin for the discussions.
REFERENCES
Aitchison, L. (2020). Bayesian filtering unifies adaptive and non von-adaptive neural network optimization methods. Advances Neural Information Processing Systems, 33:18173- 18182.
Aitchison, L., Jegminat, J., Menendez, J. A., Pfister, J.-P., Pouget, A., and Latham, P. E. (2021). Synaptic plasticity as bayesian inference. Nature neuroscience, 24(4):565-3) 571.
Akam, T. and Kullmann, M. (2014). Oscillatory multiplexing of population codes for selective communication in the mammalian brain. Nature Reviews Neuroscience, 13(2):111-122.
Amato, G., Carrara, F., Falchi, F., Gennaro, C., and Lagani, G. (2019). Hebbian learning meets deep convolutional neural networks. In International Conference on Image Analysis and Processing, pages 324 334. Springer.
Baldi, P., Sadowski, P., and Lu, Z. (2017). Learning in the machine: The symmetries of the deep learning channel. Neural Networks, 95:110-133.
Bartunov, S., Santoro, A., Richards, B., Marris, L.,, Hinton, G. E., and Lillicrap, T. (2018). As- sessing the scalability of biologically-motivaeed deep learning algorithms and architectures. Advances in neural information processing systems, 31.
Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H. (2006). Greedy layer-wise training of deep networks. Advonces in newral information processing systems, 19.
Binzegger, T., Douglas, R. J., and Martin, K. A. (2004). A quantitative map of the circuit of cat primary visual cortex. Journal of Neuroscience, 24(3)):8441-BAM
Binzegger, T., Douglas, R. J., and Martin, K. A. (2009). Topology and dynamics of the canonical circuit of cat vl. Newral Networks, 22(8):1071 1078.
Caron, M., Misra, L., Mairal, J., Goyal, P., Bojanowski, P., and Joulin, A. (2020). Unsuper- vised learning of visual fcatures by contrasting cluster assignments. Advances in Neural Information Processing Systems, 33:9912-9924.
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. (2020). A simple framework for contrastive learning of visual representations. In International conference machine learning, pages 1597 1597-1607. PMLR.
Coates, A. Ng, A., and Lee, H. (2011). An analysis of single-layer networks in unsupervised feature learning. In Proceedings of the fourleenth international conference artificial intelligence and statistics, pages 215 223. JMLR Workshop and Conference Proceedings.
Crafton, B., West, M., Basnet, P., Vogel, E., and Raychowdhury, A. (2019). Local learning in rram neural networks with sparse direct feedback alignment. In 2019 IEEE/ACM International Symposium on Low Power Electronics and Design (ISLPED), pages 1-6. IEEE.
Crick, F. (1989). The recent excitement about neural networks. Nature, (6203)))) 132.
Czarnecki, W. M., Šwirszez, G., Jaderberg, M., Osindero, S., Vinyals, O., and Kavukeuoglu, K. (2017). Understanding synthetic gradients and decoupled neural interfaces. In International Conference Machine Learning, pages 904-912. PMT.R.
10

---

Published as a conference paper at ICLR 2023
Diehl, P. U. and Cook, M. (2015). Unsupervised learning of digit recognition using spike- plasticity. Frontiers in computational neuroscience, 9:99.
Douglas, R. J. and Martin, K. A. (2004). Neuronal circuits of the ncocortex. Annu. Rev. Neurosci., 27:419-451.
Douglas, R. J., Martin, K. A., and Whitteridge, D. (1989). A canonical microcircuit for ncocortex. Neural computation, 1(4):480-488)
Dundar, A., Jin, J., and Culurciello, E. (2015). Convolutional clustering for unsupervised learning. Xrv preprint urXic:1511.06241.
Erhan, D., Bengio, Y., Courville, A., and Vincent, P. (2009). Visualizing higher-layer features of a deep network. University of Montreal, 1341(3):1.
Feldman, D. E. (2012). The spike-timing dependence of plasticity, Neuron, 75(4):556 571.
Frenkel, C., Lefebvre, M., and Bol, D. (2021). Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks. Frontiers in neuroscience, page 20.
Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572
Greody, W., Zhu, H. W., Pemberton, J., Mellor, J., and Costa, R. P. (2022). Singlo-phase deep learning in cortico-cortical networks. arXiv preprint arx:::2206. 11769.
Grill, J.-B., Strub, F., Altché, F., Tallec, C.,, Richemond, P., Buchatskaya, F.,, Doersch, C., Avila Pires, B., Guo, Z., Gheshlaghi Azar, M., et al. (2020). Bootstrap your own latent-a new approach to self-supervised learning. Advances in neural information processing systems, 33:21271 21284.
Grinberg, L., Hopfield, J., and Krotov, D. (2019). Local unsupervised learning for image analysis. arXiv preprint ar Kie:1908.008
Grossberg, S. (1987). Competitive learning: From interactive activation to adaptive resonance. Cognilive science, 11(1):23 ::23):6360 63.
Hadsell, R., Chopra, S., and LeCun, Y. (2006). Dimensionality reduction by learning an invariant mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Paltern Recognition (CVPR'00) 06), volume 2, pages 1735 1742. IEEE.
Halvagal, M. S. and Zenke, F. (2022). The combination of hebbian and predictive plasticity learns invariant object representations in deep sensory networks. bioRziv.
He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. (2020). Momentum contrast for unsu- pervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages
He, K., Zhang, X., Ren, S., and Sun, (2016). Deep residual learning for image recognition. In Proceedings of the IEFE conference compuler vision and pallern recogniliinn, pages 770-778.
Herzfeld, D. J., Kojima, Y., Soetedjo, R., and Shadmehr, R. (2015). Encoding of action by the purkinje cells of the cerebellun. Nature, 526(3333):439 442.
Hinton, G. (2022). The forward algorithm: Some preliminary investigations. arXiv preprint or Xrii:22112113314
Hinton, G. E., Osindero, S., and Teh, Y.-., (2006). fast learning algorithm for deep belief nets. Neural computation, 18(7):1527-1555 1554.
Illing, B., Ventura, J., Bellec, G., and Gerstner, W. (2021). Local plasticity rules can learn deep representations using self-supervised contrastive predictions. Advances in Neural Information Processing Systems, 34.
11

---

Published as a conference paper at ICLR 2023
Indiveri, G. (2021). Introducing" neuromorphic computing and engineering". Neuromorphic Computing and Engineering.
Ioffe. S. and Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 188-456.77 PMLR.
Jaderberg, M., Czarnecki, W. M., Osindero, S., Vinyals, O., Graves, A., Silver, D., and Kavukcuoglu, K. (2017). Decoupled neural interfaces using synthetic gradients. In International conference on machine learning, pages 1627-1635. PMLR.
Kayser, C., Montemurro, M. A., Logothetis, N. K., and Panzeri, S. (2009). Spike-phase coding boosts and stabilizes information carried by spatial and temporal spike patterns. Newron, (4):597-6088
Kolesuikov, A., Beyer, L., Zhai, X., Puigcerver, J., Yung, J., Gelly, S., and Houlsby, N. (2020). Big transfer (bit): General visual representation learning. European conference computer vision, pages 491-507. Springer.
Kreiser, R., Moraitis, T., Sandamirskaya, Y., and Indiveri, G. (2017). On-chip unsupervised learning in winner-take-all networks of spiking neurons. In BioCAS, pages 1 4.
Krotov, D. and Hopfield, J. (2016). Dense associative memory for pattern recognition. Advances in neural information processing systems, 29:1172 1180.
Krotov, D. and Hopfield, J. J. (2019). Unsupervised learning by competing hidden units. Proceedings of the National Academy of Sciences, 116(16):7733-7731.
Laborieux, A., Ernoult, M., Scellier, B., Bengio, Y., Grollier, J., and Querlioz, D. (2021). Scaling cquilibrium propagation to deep convnets by drastically reducing its gradient estimator bias. Frontiers in newroscience, 15:129.
Lagani, G., Falchi, F., Gennaro, C., and Amato, G. (2021). Hebbian semi-supervised learning in a sample efficiency setting. Newral Networks, 143:719 731.
Le, Q., Ranzato, M., Monga, R., Devin, M., Chen, K., Corrado, G., Dean, J., and Ng, A. (2012). Building high-level features using large scale unsupervised learning. In Langford, J. and Pincau, J., editors, Proceedings of the 29th International Conference Machine Learning (ICML-12), ICML '12, pages 81 88, New York, NY, USA. Omnipress.
Lee, H., Lee, K., Lee, K., Lee, H., and Shin, J. (2021). Improving transferability of representations via augmentation-an aware self-supervision. Advances in Newral Information Processing Systems, 34.
Lillicrap, P., Cownden, D., Tweed, D. B., and Akerman, C. J. (2016). Random synaptic foedback weights support error backpropagation for deep learning. Nature communications, 7(1):1 10.
Linsker, R. (1992). Local synaptic learning rules suffice to maximize mutual information in linear network. Neural Computation, 4(5):691 702.
Löwe, S., O'Connor, P., and Veeling, B. (2019). Putting an end to end-to-end Gradient- isolated learning of representations. Advances in neural information processing systems, 32.
Madry, A., Makelov, A., Schmidt, I.., Tsipras, D., and Vladu, A. (2017). Towards deep learning models resistant to adversarial attacks. arXiv preprint 1rXiv:1706.00 06083.
Markram, H., Gerstner, W., and Sjöström, P. J. (2011). A history of plasticity. Frontiers in synaptic neuroscience, 3:4.
McInnes, L., Healy, J., and Melville, J. (2018). Umap: Uniform manifold approximation and projection for dimension reduction. ar Xie preprint 1002::180..09.0 03426.
12

---

Published as a conference paper at ICLR 2023
Miconi, T. (2021). Multi-layer hebbian networks with modern deep learning frameworks. arXiv preprint v:Xi::217.00199)
Millidge, B., Tschantz, A., and Buckley, C. L. (2020). Predictive coding approximates backprop along arbitrary computation graphs. arXiv preprint orXiv:2006.04100
Mitrovic, J., McWilliams, B., Walker, J., Buesing, L., and Blundell, C. (2020). Representation learning via invariant causal mechanisms. arXiv preprint arXie:2010. 07922.
Moraitis, T., Sebastian, A., and Eleftheriou, E. (2018). Spiking neural networks enable two-dimensional neurons and unsupervised multi-timescale learning. In 2018 International Joint Conference on Neural Networks (IJCNN), pages 1-8. IEEE.
Moraitis, T., Scbastian, A., and Elefthorion, E. (2020). Short-torm synaptic plasticity optimally models continuous environments.
Moraitis, T., Toichkin, D., Chua, Y., and Guo, Q. (2021). Softhebb: Bayesian inference in unsupervised hebbian soft winner-tako-all networks. arXiv preprint 110::2207005447
Nand, R., Marcille, N., Clopath, C., and Gerstner, W. (2008). Firing patterns in the adaptive exponential integrate-and- model. Biological cybernetics, 99(4):333-34 -47..
Nessler, B., Pfeiffer, M., Buesing, L., and Maass, W. (2013). Bayesian computation emerges in generic cortical microcireuits through spike-timing dependenn plasticity. PLoS computa- lional biology, 9(4)ee1003037
Nessler, B., Pfeiffer, M., and Maass, W. (2009). Stdp enables spiking neurons to detect hidden causes of their inputs. Advances in neural information processing systems, 22:1357-1365.
Nguyen, A., Yosinski, J., and Clune, J. (2015). Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proceedings of the IEEE conference on computer vision and pattern recogrition, pages 427 436.
Nokland, A. (2016). Direct feedback alignment provides learning in deep neural networks. Advances in neural information processing systems, 29.
Nokland, A. and Eidnes, L. H. (2019). Training neural networks with local error signals. In International conference machine learning, pages 4839 4850. PMLR.
Oja, E. (1982). Simplified neuron model as a principal component analyzer. Journal of mathemalical biology, 15(3):267 273.
Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., and Naud, R. (2021). Burst-dependent synaptic plasticity can coordinate learning hierarchical circuits. Nature neuroscience, 24(():1010-I4) 1019..
Pehlevan, C. and Chklovskii, D. (2015). normative theory of adaptive dimensionality reduction in neural networks. In Cortes, C., Lawrence, N., Lee, D., Sugiyama, M., and Garnett, R., editors, Advances in Neural Information Processing Systems, volume 28. Curran Associates, Inc.
Pogodin, R., Mehta, Y., Lillicrap, T. P., and Latham, P. E. (2021). Towards biologically plausible convolutional networks. arXie preprint arXiv:2106. 13031.
Qiao, N., Mostafa, H., Corradi, F., Osswald, M., Stefanini, F., Sumislawska, D., and Indiveri, G. (2015). A reconfigurable on-line learning spiking neuromorphic processor comprising 256 neurons and 128k synapses. Frontiers neuroscience, 9:141.
Rauber, J., Zimmermann, R., Bethge, M. and Brendel, W. (2020). Foolbox native: Fast adversarial attacks to benchmark the robustness of machine learning models in pytorch, tensorflow, and jax. Journal of Open Source Software, 5(53):2607.
13

---

Published as a conference paper at ICLR 2023
Rodriguez, H. G., Guo, Q., and Moraitis, T. (2022). Short-term plasticity neurons learning to learn and forget. In Chaudhuri, K., Jegelka, S., Song, L., Szepesvari, C., Niu, G., and Sabato, S., editors, Proceedings of the 39th International Conference Mochine Learning, volume 162 of Proceedings of Machine Learning Research, pages 18704-18722, PMLR.
Sanger, T. D. (1989). Optimal unsupervised learning in a single-layer linear feedforward neural network. Neural networks, 2(6))459-477 -73..
Sarwat, G., Kersting, B., Moraitis, T., Jonnalagadda, V. P., and Scbastian, A. (2022a). Phase-change mentransistive synapses for mixed-plasticity neural computations. Nature Nanotechnology, pages 1
Sarwat, S. G., Moraitis, T., Wright, C. D., and Bhaskaran, H. (2022b). Chalcogenide optomemristors for multi-factor neuromorphic computation. Nature communicationss 13(1):1 9.
Scellier, B. and Bengio, Y. (2017). Equilibrium propagation: Bridging the gap between energy-basod models and backpropagation. Frontiers in computational neuroscience, 11:24.
Scherr, F., Guo, Q. and Moraitis, T. (2022). Self-supervised learning through efference copies.
Sebastian, A., Le Gallo, M., Khaddam-Aljjaeh, R., and Eleftheriou, E. (2020). Memory devices and applications for in-memory computing. Nature nanotechnologg, 15(7):529 544.
Sejnowski, T. J. (2020). The unreasonable effectiveness of deep learning in artificial intelli- gence. Proceedings of the National Academy of Sciences, 117(48))30033-30) 30038..
Simonyan, K. and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXw preprint arXiv: 1xXxv 10oo::4099...0.0 1556.
Sjöström, P. J., Turrigiano, G. G., and Nelson, S. B. (2001). Rate, timing, and cooperativity jointly determine cortical synaptic plasticity. Neuron, (2()))114--1164.
Stuhr, B. and Brauer, J. (2019). Csnns:
Unsupervised, backpropagation free convolutional neural networks for representation learning. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA), pages 1613-16200 IEEE.
Tavanaei, A. and Maida, A. S. (2016). Bio-inspired spiking convolutional neural network using layer-wise sparse coding and stdp learning- aix preprinl or Xin:1:11 1ax:1611.0000
Tomasev, N., Bica, I., McWilliams, B., Bucsing, L., Pascanu, R., Blundell, C., and Mitrovic, J. (2022). Pushing the limits of self-supervised resnets: Can outperform supervised learning without labels on imagenet? arXiv preprinl arXim:2201.03: 0113001.0111..1 19.
Von der Malsburg, C. (1973). Self-organization of oricntation sensitive cells in the striate cortex. Kybernetik, 14(2):85-100.
14

---

Published as a conference paper at ICLR 2023
A DETAILS TO THE METHODS
A.1 WEIGHT INITIALIZATION
A.1.1 CHOOSING DISTRIBUTION FAMILY
We tested three probability distribution families for initializing the weights randomly; normal, positive, and negative (Equation (3)).
normal (N 0, o²) positive UO, range) negative U(0, -range),
(3)
where U is uniform distribution. We found that they perform similarly to each other in the case of long experiments, where the extensive training examples achieve to align the weights with the imput. However, for shorter experiments, where speed is needed, we found that the positive distribution was better for raw data (input values between 0-1) and normal distribution for data that was normalized into standard scores. The clustering property of SobtHebb learning (Moraitis et al., 2021), where learned features are centroids of clusters in the input space may explain this. In practice, we used normalized data for the reported results, and therefore we used normal distribution for the initial random weights.
A.1.2 CHOOSING DISTRIBUTION PARAMETERS
By varying the distribution parameters, we observed that the norm of the weight vectors i.e. the "radius" R very instructive. We identify three regions of disparate learning (Fig. B.3A); for R smaller than 1, there is no learning with random weight update; for R bigger than 1 and smaller than approximately 2.5, there partial leaning with only percentage of neurons winning and converging while the others stay random; for R bigger than 2.5, all neurons learn. The learning change around comes from the fact that the weight tends to converge to radius of (Moraitis al., 2021).
Determining the initial radius from the weight distribution parameters: We can then derive the distribution parameters from the optimal initial radius using the distribution moment calculation.
R: E(, Σ W.
Σ wa) -B
E(,
E(VN w²) E(VN (u)) VN E(wl)
(4)
Where i is the index of a neuron, is the index of this neuron's 's synapses, N is number of synapses of that neuron, E( is the expected value and so E((ll) is the first absolute moment of distribution. Thus, for the normal distribution E([w() = a v2/T o R V.V. 2N
and
positive distribution E([wo) F(w) range/2 = range R V2. 2
A.2 LEARNING-RATE ADAPTATION
Learning rates that decay linearly with the number of training examples have been extensively used in Hebbian learning (Krotov and Hopfield, 2019; Grinberg et al., 2019; Miconi, 2021; Lagani el al., 2021; Amalo et al., 2019). It is a simple scheduler, reaching convergence with sufficient amount of training example. The linear decay ties the learning rate's value throughout learning to the proportion of the training examples that have been seen, and does so uniformly across neurons. However, the weights may theoretically be able to converge before the full training set's presentation, and they may do so at different stages across neurons. To address this, we tied the learning rate 7i to each neuron's i convergence. Convergence was assessed by the norm of the neuron's weights. Based on previous work on similar learning rules (Oja, 1982; Krotov and Hopficld, 2019) including SoftHebb itself (Moraitis et al., 2021), and weights associated with a convergence to a norm of 1, in case of simple learned features at least. Therefore, used learning rate that stabilizes to as neuron weight. vectors
own observations, the convergence of the neuronal
15

---

Published as a conference paper at ICLR 2023
converge to sphere of radius and is initially big when the weight vectors are large compared to 1:
7 (r:
(5)
where q is a power hyperparameter. Note that this adaptivity has no explicit dimedeppendence but as learning proceeds towards convergence, 1) does decay with time. To account for the case of complex features that do not converge to a norm of 1, a time-dependence can be added top, e-g- usual with linearly decaying factor, multiplying the norm-dependent learning rate, In our experiments we only used the norm-dependencee and simply stopped learning after the first training epoch, i.e. the first presentation of the full training in most cases, or carlier. In practice, our adaptive rate reaches convergence faster (Fig. B.3B) than the lincar time-dependence, by maintaining a scparate learning rate for each neuron and adapting it based on each neuron's 's own convergence status.
A.3 LAYER ARCHITECTURE
Each convolutional layer includes succession of batch normalization, a SoftHebb convolution, pooling operation, and then an activation function. The SoftHebb convolution stride is fixed at 1. Padding is added to the input of the convolutional filters to guarantee that their output has the same size as the unpadded input. The pooling stride was fixed at 2, halving each dimension of the resolution after each layer. In all experiments, the first layer had width of 96 convolutional kernels (Miconi, 2021; Lagani et al., 2021; Amato et al., 2019), except ImageNet experiments where we used 48 kernels. The number of kernels in the subsequent layers was determined according to our width-scaling method, with width factor of 4 (see Appendix A.5).
A.3.1 ACTIVATION FUNCTION
Softmax, i.c. the output of the soft winner-tako-all was used the postsynaptic variable in SoftHebb's plasticity. However, for forward propagation to the subsequent layer, we considered three activation functions; Rectified polynomial unit (RePU), triangle, and softmax. RePU (Equation (6)) was proposed by Krotov and Hopfield (2016) as a generalization of rectified linear units (ReLU) and was also used in follow-up works (Krotov and Hopfield, 2019; Grinberg et al., 2019). Triangle activation was introduced by Coates et al. (2011) for k-means clustering and also appears useful for Hebbian networks in (Miconi, 2021). ). It subtracts the mean activation calculated over all channels each position and then applies a ReLU, Here we generalize Triangle by combining it with RePU (instead of ReLU) through Equation (7):
[u", for
RePU(u) 0. for u &
(6)
Triangle(u,) = RRPU(uy - w
(7)
A.4 HYPERPARAMETER OPTIMIZATION
We systematically investigated the best set of hyperparameters each hidden layer, based on the validation accuracy of a linear classifier trained directly on top of that hidden layer. All grid searches were performed on three different random soeds, varying the batch sampling and the validation set (20% of the training dataset). The classifier is simple linear classifier with a dropout. 0.5 and no other regularisation term. For all searches and final results, we used 96 kernels in the first layer. Subsequent layers scaled with fa. = 4 (see Appendix A.5). However, based on our observations, only the optimal temperature depends on the fw-
For each added layer, grid search was performed in three stages: For the first two stages used square convolutional kernels with a size 5, a max-pooling with a square kernel of size and a Triangle with power of forward activation function.
1. In this stage performed grid search over the remaining hyperparameters (nbeeochss batch size, 77 and of the learning rate scheduler, and temperature 7 of the Hobb softmax). We varied the nb epochs {1, 10, 50, 100, 1000), batch size
16

---

Published as a conference paper at ICLR 2023
layer
operation
hyperparmeters
searched range
found optimum
"
0.00110.12].
0.08
00.25, 0.5, 0.75)
0.5
conv
kernel size
3, 5, 7,
1/T
0.1-100]
pooling
type
AvgPooling, MaxPonning]
MaxPooling
kernelsiize
2, 3, 4]
function
Softmax, RePU, Triangle
Triangle
activation
power
001-10
0.7
"
0.001-0.12].
0.005
[0.25, 0.5, 0.75)
0.5
conv
kernelssize
3, 5, 7, 9]
2
1/T
0.1-100]
0.65
pooling
type
AvgPooling, MaxPooling,,
MaxPooling
kernel_size function
2,3,4]
Softmax, RePU, Triangle
Tringgle
activation
power
0.1-10
1.4
"
0.001-0.12].
0.01
[0225, 0.5, 0.75)
0.5
cony kernelsize
3, 7,
3
1/T
0.1-100)
0.25
pooling
type
AvgPooling, MaxPooling),
AvgPooling
kernel_size
2,3.4]
function power
Softmax, RePU, Triangle
Triangle
activation
00.1-10]
Table Network architecture and hyper-parameters search, and best results on CIFAR-10. More details are provided in section A.4.
{10. 100, 1000). 7 E (0.001,0.004, 0.008, 0.01, 0.04, 0.08, 0.12), 9 {0.25, 0.5, 0.75) and the 1/r E (0.25, 0.5, 1,2,5,
2. A finer grid search over 1/r E (0.15, 0.25, 0.35, 0.5, 0.6, 0.75, 0.85, 1.2, 1.5, 2) and con__kerne_size E (3,5,7,5 using the best result from the previous search.
3. A final grid search over the pooling: pool_type E [AugPooling, MozPooling): pool wewnkernel_aas E (2,3,4), and the activation function: function (RePU, Triangle, Softmar) with for power E 0.31, 0.7, 1, 2,,, 10} (for RePU Triangle) and 7 E {0.1, 0.5, 1, 10, 50, 1003 (for softmax) using the best result from the two previous searches.
A.5 MULTILAYER ARCHITECTURE
The pooling of stride halves each dimension of the resolution after each layer. We stop adding layers when the output resolution becomes at most x 4. The layer where this occurs depends on the original input's resolution. Thus, the multilayer network has three convolutional layers for MNIST or CIFAR-10, four layers for STL-10, and five layers for ImageNet a resolution setting of 160 160 px (Table 4).
A width factor fw characterizes the multilayer network. for links the width of each layer to that of the previous layer, thus determining the depth-dependent architecture. Specifically, the number of filters # Fi in the hidden layer I is far times the number of filters at layer 1: #Fi fo f F1-1. The first hidden layer has 96 filters order to compare with Miconi (2021); Lagani et al. (2021); Amato et al. (2019). We then explored, using CIFAR-10, the impact fo on performance. We tried three different values for fw E 21 4). A value of 1 keeps the same number of filters in all layers, while value of 4 keeps the number of features provided to the classifier head equal to the number of features at the input layer (due to the pooling stride of in each layer). fw bigger than four would substantially increase
17

---

Published as a conference paper at ICLR 2023
# layer
MNIST/CIFAR
STL10
ImageNet
Batchnorm
Batchnorm
Batchnorm
5x5 conv96
5x5 conv96
5x conv48
1
Triangle
Triangle
Triangle
4x4 MaxPool
4x4 MaxPool
4x4 MaxPool
Batchnorm
Batchnorm
Batchnorm
3x conv384
3x3 conv384
3x3 conv192
2
Triangle
Triangle
Triangle
4x MaxPool
4x4 MaxPool
4x MaxPool
Batchnorm
Batchnorm
Batchnorm
3x3 conv1536
3x3 conv1536
3x3 conv768
3
Triangle
Triangle
Triangle
2x2 AvgPool
4x4 MaxPool
4x4 MaxPool
Batchnorm
Batchnorm
3x3 conv6144
3x3 conv3072
4
Triangle
Triangle
2x2 AvgPool
4x4 MaxPool
Batchnorm
5x5 conv12288
5
Triangle
2x2 AvgPool
Table 4: Network architecture (all pooling layers stride of 2). The number of channels also defined, c.g. conv96 means 96 channels. More details can be found Appendix A.5.
the size of the network, and is impractical for deeper networks. We found that to increase performance with depth, the network needs to also grow in width (Fig. 4).
A.6 TRAINING AND EVALUATION PROTOCOLS
Each experiment was performed 4 times, with random initializations, on all datasets except the full ImageNet where only one random seed was tried.
A.6.1 SOFTHEBB TRAINING
The optimal number of Sobb weight update iterations is around 5000 based on CIFAR-10 experiments. Thus, for CIFAR-10 and MNIST (50k training examples), unsupervised training was performed in one epoch with a mini-batch of 10 and 20 for STL-10 unlabelled training set (100k training example). Because of the large number of training examples, randomly select 10% of the ImageNet dataset with a mini-batch of 20. The accuracios we report for SoftHebb are for layers that trained successively, meaning each SoftHebb layer was trained, and then frozen, before the subsequent layer was trained. However, the results are very similar for simultaneous training of all layers, where each training example updates all layers, as it passes forward through the deep network.
A.6.2 SUPERVISED TRAINING
The linear classifier top uses mini-batch of 64 and trains on 50 epochs for MNIST and CIFAR-10, 100 epochs for STL-10, and 200 epochs for ImageNet. For all datasets, the learning-rate has initial value of 0.001 and is halved repeatedly at [20%, 35%, 50%, 60%. 70%, 80%, 90%] of the total number of epochs. Data augmentation (random cropping and flipping) was applied for STL-10 and ImageNet.
18

---

Published as a conference paper at ICLR 2023
A.6.3 BACKPROPAGATTON-TRAAI NETWORKS
Results comparing SoftHebb and Backpropagation are found using the same network archi- tecture. The only difference is the activation function; we found that triangle or softmax activation would be detrimental to BP. The activation function we used for BP-trained networks is ReLU. The learning rate followed the same schedule as described Appendix A.6.2.
A.6.4 FINE-TUNING
In the fine-tuning experiment on STL-10, all Hebbian CNN layers learned using SoftHebb and the large unlabelled training dataset of STL-10; then, output layer was added and the entire network was trained end-ta-end using BP on the full small labelled training subset.
A.7 RECEPTIVE FIELDS
We have visualized the receptive fields (RFs) of hidden layers in the network (Figure 2 and end of Appendix B). The method that we used is activation maximization (Erhan et al., 2009; Le al., 2012: Goodfellow et al., 2014; Nguyen al., 2015). Specifically, we started from a square of random pixels, and we optimized the input through gradient descent (or rather ascent)
maximize the activation of each neuron, under the constraint of an L. norm of 1, i.c. projection to a unit sphere. That is then form of projected gradient descent (PGD), which can also be usod as an adversarial attack, if loss function rather than the activation function is maximized. For this purpose, adversarial allacks, named Foolbox (Rauber et al., 2020). We show RFs that maximize the linear response of the neurons, i.e. the total weighted input. We have tuned the step size of the descent, and we have validated the approach (a) by verifying that the number of iterations suffice for convergence, (b) by confirming that its results at the first layer match the layer's weights, (c) by verifying that the hidden neurons are strongly active if the network is fed with inputs that match the neuron's found RF's, and (d) by seeing that alternative initializations also converge to the same RF. We also tried an alternative method that was used by Miconi (2021). Specifically, we used that paper's available code /oni/Hebbi /eeb/iebiiaae anCNWPyTrrch). We found that the RFs found by PGD activate the neurons more than the RFs found by the alternative method. Moreover, PGD takes into consideration pooling and activation functions, which the other method does not. Therefore we chose present the results from PGD. Example results are presented in Fig. 2 as well as in extended form with more examples at the end of Appendix B.
modified toolbox for
(https:/ thub......... cwm/ conn/ThomaMiicco coni
19

---

Published as a conference paper at ICLR 2023
B ADDITIONAL RESULTS AND ANALYSES
B.1 IMPACT OF TEMPERATURE. SOFTHEBB LEADS TO NEW LEARNING REGIME.
100
accuracy
Accuracy (%) and # of R1 features
R1 features
50
Hard WTA
SoftHebb (without anti-Hlebbaan)
SoftHebb (with anti-Hebbian)
10
10-2
10"
10a
Temperature (i.c. softness)
Figure B.1: Sebb's tomperature-depeneent regimes in single-layer CNN trained on CIFAR-10. With anti-Hebbian plasticity and SoftHebb (but hard WTA), regime cxists where RI and non-R1 features coexist and accuracy is maximal.
100
accuracy
features
Accuracy (%) and R1 features (%)
75
50
25
Hard WTA
SoftHebb (without anti-Hebbian
0
SoftHebb (with anti-Hebbian)
10
10-2
100
102
Temperature (i.e. softness)
Figure B.2: Same as Fig. B.1, but in the 3-hidden-layer nerk,rk, such that all layers use the same temperature. R1 features reported as percentage over all features of all layers.
20

---

Published as a conference paper at ICLR 2023
B.2 IMPACT OF ADAPTIVE LEARNING RATE. IT INCREASES LEARNING ROBUSTNESS AND SPEED.
100
100
No Leuning
Pactial Learning
Numbiee of Ttt fontures
75
Lerming
Aoarayy (56
50
25
25
lincot decay
lineur decay
deperueettttnnt adaptatiion
mert-ddepemeent adaptation
010
0.32
1.00
5.62
31.02
0
25000
50000
Inintal radins
Training excamples
(A) Robustness to weight initialization of our noee-dependent adaptive learning rate scheme, compared to a linear scheduler. The adaptive scheme achieves high accuracies for broad range of weight initializations.
(B) Convergence speed of
dependent adaptive learning rate scheme, compared to a linear scheduler. With the adaptive scheme, neuronal weight vectors converge to sphere of radius 1 (i.e. become R1 features) faster.
Here the initialization is parametrized by the radius of sphere where weight vectors lie initially.
Figure B.3: Effects of adaptive learning rate.
We speculate that low initial radius is problematic (Figure B.3A) because in that regime the balance between excitation (from the imput) and inhibition (from the soft WTA) is overly tilted towards inhibition. Regarding Figure B.3B, see also main text's Section 3, WWeigh--norm-depeendeni adaptive learning rate", and the related paragraph in Section 4.
B.3 IMPACT OF ACTIVATION FUNCTION
The choice of activation function for forward propagation through SoftHebb layers is im- portant. To study this, we compared SoftHebb's performance on CIFAR-10 for various activation functions. In each result, all neurons across all layers used the same activation function hyperparameters, except the case of Triangle. The parameters that were tried were:
Power of RePU: 0.7*, 1.0, 1.4.
Temperature of softmax: 4.0*, 1/0.65, 1.0.
These three values were tried because they were good for individual layers in previous tuning. Asterisk indicates the best parameter value according to validation accuracy. Using the best values produced the test accuracy results reported below, whereas the remaining hyperparameters were not tuned to cach case, but rather were the same, as found for Triangle by the process described in Appendix A.4.
ReLU: (70.68 = 1.07)%
tanh: (56.13 + 0.34)%
RePU: (79.08 = 0.13)%
softmax: (54.05 = 0.55)%
RePU Triangle: (80.31 0.14)%
B.4 IMPACT OF WIDTH (NUMBER OF NEURONS)
The width of the layers is rather impactful, as indicated by varying the width factor of deep layers while keeping the first. layer's width constant (Fig. 4). To further study that impact,
21

---

Published as a conference paper at ICLR 2023
varied the first layer's width and kept the width factor fixed to 4 (which scales all layers). The results are presented in Fig. B.4.
CIFAR10 width comparison
85
Accuracy (1%)
80
75
SoftHebb (ours)
70
48
96
192
Number of neurons 1st layer
Figure B.4: Performance on CIFAR-10 for varying width of the layers. First-layer width is indicated, while subsequent layers scaled by the width factor 4 (see Appendix A.5).
For this control experiment, we did not re-tune the hyperparameters to each width, but rather only to the 96-neuron case. That in contrast Fig. 4, where hyperparameters were tuned to each width factor.
22

---

Published as a conference paper at ICLR 2023
(A) Hard WTA
(B) SoftHebb
(C) Backprop
Figure B.5: Receptive fields of the first convolutional layer's neurons, learned from CIFAR-10
by different algorithms.
B.5 FURTHER STUDY OF HIDDEN REPRESENTATIOOS.
As a control, we perform again the experiment that we presented in Fig. 5, this time in comparison to a network with the initial, untrained, random weights. The results are shown in Fig. B.6 and Fig. B.7.
(A) Trained weights with SofHebb..
(B) Random weights.
Figure B.6: UMAP projection (similar to main text's Fig. 5, top row) of the test set after passing through SoftHebb layers. Here, (A) from trained and (B) from randomly initialized, untrained network.
23

---

Published as a conference paper at ICLR 2023
Layer 1
Layer
Layer
Layer
(A) Trained weights with SofHebb.
(B) Random weights.
Figure B.7: Images and patches that best activate random neurons from SoftHebb Layers 1 and 4 (similar to main text's Fig. 5, bottom row). Here, (A) from trained and (B) from a randomly initialized, untrained network.
B.6 SUPPLEMENTARY RECEPTIVE FIELDS FOUND THROUGH PGD.
For the method, see Appendix A.7. RF's of deeper layers are not all Gabor-like, but rather also include mixtures Gabor filters, and also lake different shapes and textures. In addition, RFs do appear increasingly complex with depth. These results could possibly be expected based on the RFs of the first layer, which are already more complex than the mere Gabor filters that are learned by other Hebbian approaches, such as hard WTA (Figure B.5A). Their mixture in subsequent layers then was unlikely to only produce Gabor filters. It is difficult interpret each RF precisely, but this is common in the hierarchies of deep neural networks.
24

---

Published as a conference paper at ICLR 2023
Figure B.8: All receptive fields of layer 1, learned from STL-10.
25

---

Published as a conference paper at ICLR 2023
Figure B.9: 250 randomly sampled receptive fields of layer 2, learned from STL-10.
26

---

Published as a conference paper at ICLR 2023
Figure B.10: 250 randomly sampled receptive fields of layer 3, learned from 1TL-10.
27

---

Published as a conference paper at ICLR 2023
Figure B.11: 250 randomly sampled receptive fields of layer 4, learned from S0L-10.
28


Reviews:
[{"review_id": "qUWxT591Y3", "reviewer": "Reviewer_39im", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Authors build upon the mitigation of back-propagation and aim at reaching higher accuracies. They first highlight these limits of BP. They then make very strong claims about soft Hebb, an algorithm capable of notably learn in a self-supervised manner or without feedback signals. They evaluate their algorithms or standard benchmarks, which yield satisfactory results, yet with rather shallow networks.\n\n", "summary_of_the_review": "As a summary, the proposed algorithm is well described and analyzed. It however makes many strong claims which are not reached with deep-learning when it is \"very \"deep (see end of page 9). This week point could be well alleviated by comparing to other networks with a similar number of parameters.", "strength_and_weaknesses": "A strength of the proposed paper is that it presents well the limits of BP and the challenges ahead. The proposal to use a softmax normalisation (equation 1) is well-founded and I would have wished to more clearly see the link with Bayesian inference. Results are quite good and it is worth highlighting that this paper is among the few which explore the learned networks to understand why it would behave better (or not).\nA major weakness of the paper is that the claims are too strong with respect to the results obtained. for instance, the studied networks are very shallow, or the kernels shown in figure 2 are not very interpretable. A comparison with stacked sparse auto-encoders or deep predictive coding would have been constructive. Moreover, the model, as described in section 3, seems to involve many different heuristics and parameters. If width scaling is studied (Figure 4), it is not clear how the other parameters are crucial to your results. Finally, the claim \"Soft Hebb learns hierarchical representations\" is perhaps premature, as this would require more layers (rather than distinguishing animals versus non-animals and their sub-categories).\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "jcu6vGHyxQ", "reviewer": "Reviewer_ba7W", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors of the given manuscript extend a bio-plausible backpropagation alternative, “SoftHebb,” to apply to training multiple layers. In biological plausibility, this algorithm bows to the constraints of no weight transport, local plasticity, and no time-locked updates. In addition, the learning is unsupervised. In this restrictive setting, multilayer SoftHebb outperforms competing bio-plausible methods. \n", "summary_of_the_review": "The proposed algorithm operates in a challenging setting and performs well. The paper’s writing and overall presentation are impeccable. The only drawback is that the novelty is somewhat limited. For better reproducibility, the authors should provide code for the reported experiments. Given the code, I would consider increasing my rating.", "strength_and_weaknesses": "Strengths:\n(+) Writing is essentially error-free.\n(+) Clear and informative figures and tables.\n(+) Neuro-inspired mechanism (e.g., lateral inhibition).\n(+) An insightful discussion of limitations, in the end, and also in the main text.\n\nWeaknesses:\n(-) Not accompanied by code for reproducing the experiments.\n(-) Novelty is somewhat limited.\n\nMinor:\n- The motivation for biological learning could have been more robust in the introduction to motivate the reader. Later in the discussion, neuromorphic chips are mentioned as a potential application. Why not in the introduction?\n- Multiple letters for variables such as “lr” and “lrp” may be confusing.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "e6UCRAVUGA", "reviewer": "Reviewer_72et", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes an application of SoftHebb to deep convolutional neural networks. SoftHebb has been proposed before (supposedly by the same authors?), it is a soft winner-take-all version of the Hebbian learning rule implemented using the softmax function. The novelty of the paper lies in making SoftHebb scale to multiple (convolutional) layers. The proposed method compares favorably to other unsupervised counterparts, and exhibits nice properties such as the emergence of hierarchical structures. The method is also posed as biologically plausible, which may be the most important point of the paper.", "summary_of_the_review": "Overall, I think this is a very interesting paper with a lot of potential for impact. Its main limitation is that it has only been tested in the vision domain. However, in the vision domain, deep SoftHebb performed the best on STL-10, which is an unsupervised learning computer vision benchmark. Since convolutions are easily transferable to 1D data, perhaps the method can also be tested on time series data in future?\n\nPlease see below for suggested corrections and questions to the authors.\n\nPage 4: “…model and algorithm that combined achieve good accuracy…” -> …model and algorithm is that combined they achieve good accuracy…\n\nTable 1 is placed in the paper a few pages before its first mention in the text. Figures are also referenced for the first time out of order. I think it does not help the narrative.\n\n“Batch normalization was used, with its standard initial parameters, which we did not train.” - what are the initial parameters, exactly? Such information is crucial for reproducibility.\n\nPage 8: “prior work Table2)” - missing bracket before “Table”.\n\nWhat if SoftHebb was allowed to experience the data for longer than just one epoch? Why was the decision made to stop at one epoch? \n\nPage 9: “it being fully unsupervised, It is also founded…” - second “it” should not be capitalized.", "strength_and_weaknesses": "Strengths:\n* A biologically plausible alternative to back propagation is shown to work on multi-layer network structures.\n* Experiments include complicated tasks such as STL-10 and CIFAR-10, the proposed method compares favorably to existing approaches.\n* The related work section is detailed and beautifully written, it gives an excellent perspective on prior attempts at merging neuroscience insights with deep learning.\n\nWeaknesses:\n* The authors themselves state this as a limitation: SoftHebb is applied to computer vision tasks only, in an exclusively convolutional setting. Moreover, random weights (prior to SoftHebb training) already seem to have expressiveness in the WTA setting. How much of the learning can be attributed to SoftHebb, and how much is due to the convolutional feature extraction?\n* Comparisons lack statistical backing. What are the standard deviations? How consistently does SoftHebb outperform other techniques?\n* All results reported are for classification tasks, i.e. a standard BP is used to fine-tune the weights of the SoftHebb network. Is there any way to evaluate the performance of this unsupervised technique without BP? Perhaps clustering, e.g. unsupervised image segmentation?\n* The authors can be easily guessed based on the prior work cited.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "bvPsYBLFMlD", "reviewer": "Reviewer_ACU4", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a new method for training deep (multi-layered, convolutional) networks with Hebbian-like learning, rather than backpropagation. This is important, because backpropagation is  both computationally demanding and biologically implausible. \n\nAn additional motivation comes from neuroscience. There are many Hebbian-based models that reproduce the type of receptive fields (RFs) observed in primary visual cortex (mostly oriented edge detectors). However, to my knowledge, when applied to multi-layer networks, no Hebbian system has been able to reproduce the emergence  of increasingly complex, selective features in successive layers that is observed in visual cortex - at least not without grossly implausible hacks. This is frustrating, since such an emergence is the intuitively expected result (higher layers composing lower-layer features into increasingly complex detectors). Thus it would be interesting if the authors could demonstrate that their model does learn increasingly complex features in higher layers, using only Hebbian learning with plausible modifications.\n\nThe method produces strong performance on multiple benchmarks, and increasing performance with additional layers (something that is surprisingly difficult to achieve with Hebbian learning).", "summary_of_the_review": "== Updated review == \n\nI thank the authors for their clarifications. I have updated my recommendations accordingly.\n\n== Original review ==\n\n\nThe paper is interesting and potentially important. However one of the claims (increasingly complex and elaborate RFs in higher  layers) needs more substantiation, and the paper requires a few clarifications. I am willing to increase my score if these concerns are addressed.", "strength_and_weaknesses": "- Strengths: The problem is important, the method seems novel and interesting, and the results are strong.\n\n- Weaknesses: The main concern is that one of the paper's strong claims (increasingly complex RFs in higher layers) is not clearly substantiated. \n\nThe supposed better semantic separation in layer 4 than layer 1 in Figure 5a-b is tenuous, at best. Similarly, Figure 5D is highly ambiguous: most of the responses shown in Figure 5D seem compatible with one or two blobs of the right colors at the right locations, i.e. very similar to typical first-layer RFs.\n\nThe claim of increasingly complex RFs in higher areas (which would be very interesting if confirmed) should be better substantiated (see below), or toned down.\n\nThere are more minor problems with the explanations and description, which should be easily fixed (see below).", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%