Hebbian Deep Learning Without Feedback
Anonymous authors
Paper under double-blind review
Abstract
Recent approximations to backpropagation (BP) have mitigated many of
BP’s computational ineﬃciencies and incompatibilities with biology, but important limitations still remain.
Moreover, the approximations signiﬁcantly decrease accuracy in benchmarks, suggesting that an entirely diﬀerent approach may be more fruitful.
Here, grounded on recent theory for Hebbian learning in soft winner-take-all networks, we present multilayer SoftHebb, i.e. an algorithm that trains deep neural networks, without any feedback, target, or error signals.
As a result, it achieves eﬃciency by avoiding weight transport, non-local plasticity, time-locking of layer updates, iterative equilibria, and (self-) supervisory or other feedback signals – which were necessary in other approaches.
Its increased eﬃciency and biological compatibility do not trade oﬀ accuracy compared to state-of-the-art bio-plausible learning, but rather improve it.
With up to ﬁve hidden layers and an added linear classiﬁer, accuracies on MNIST, CIFAR-10, STL-10, and ImageNet, respectively reach 99.4%, 80.3%, 76.2%, and 27.3%.
In conclusion, SoftHebb shows with a radically diﬀerent approach from BP that Deep Learning over few layers may be plausible in the brain and increases the accuracy of bio-plausible machine learning.
Introduction: Backpropagation and its limitations
The core algorithm in deep learning (DL) is backpropagation (BP), which operates by ﬁrst deﬁning an error or loss function between the neural network’s output and the desired output.
Despite its enormous practical utility (Sejnowski, 2020), BP requires operations that make training a highly expensive process computationally, setting limits to its applicability in resource-constrained scenarios.
In addition, the same operations are largely incompatible with biological learning, demanding alternatives.
In the following paragraphs we describe these limitations and their practical and biological signiﬁcance.
Weight transport.
Backpropagating errors involves the transpose matrix of the forward connection weights.
This is not possible in biology, as synapses are unidirectional.
Synaptic conductances cannot be transported to separate backward synapses either.
That is the weight transport problem (Grossberg, 1987; Crick, 1989), and it also prevents learning on energy-eﬃcient hardware substrates (Crafton et al., 2019) such as neuromorphic computing hardware, which is one of the most researched approaches to overcoming the eﬃciency and scaling bottlenecks of the von Neumann architecture that underlies today’s computer chips (Indiveri, 2021).
A key element of the neuromorphic strategy is to place computation within the same devices that also store memories, akin to the biological synapses, which perform computations but also store weights (Sebastian et al., 2020; Sarwat et al., 2022a;b).
However, in BP, the transposition of stored memories is necessary for weight transport, and this implies signiﬁcant circuitry and energy expenses, by preventing the full in-memory implementation of neuromorphic technology (Crafton et al., 2019).
Non-local plasticity.
BP cannot update each weight based only on the immediate activations of the two neurons that the weight connects, i.e. the pre- and post-synaptic neurons, as it requires the error signal, which is computed at a diﬀerent point in time and elsewhere in the network, i.e. at the output.
That makes BP non-local in space and time, which is a critical discrepancy from the locality that is generally believed to govern biological synaptic plasticity (Baldi et al., 2017).
This non-locality implies further computational ineﬃciencies.
Speciﬁcally, forward-passing variables must be memorized, which increases memory
[Table]
a n
[Table]
p o r t - f r ee
L o c a l p l a s t i c i t y
U pd a t eun l o c k e d
U n s up e r v
[Table]
requirements (L¨owe et al., 2019).
Moreover, additional backward signals must be computed and propagated, which increases operations and electrical currents.
It is noteworthy that these aspects are not limiting only future neuromorphic technologies, but even the hardware foundation of today’s DL, i.e. graphical processing units (GPUs), which have their own constraints in memory and FLOPS.
Update locking.
The error credited by BP to a synapse can only be computed after the information has propagated forward and then backward through the entire network.
The weight updates are therefore time-locked to these delays (Czarnecki et al., 2017; Jaderberg et al., 2017; Frenkel et al., 2021).
This slows down learning, so that training examples must be provided at least as slowly as the time to process the propagation through the two directions.
Besides this important practical limitation of BP for DL, it also does not appear plausible that multiple distant neurons in the brain coordinate their processing and learning operations with such precision in time, nor that the brain can only learn from slow successions of training examples.
Global loss function.
BP is commonly applied in the supervised setting, where humans provide descriptive labels of training examples.
This is a costly process, thus supervised
BP cannot exploit most of the available data, which is unlabelled.
In addition, it does not explain how humans or animals can learn without supervisors.
As a result, signiﬁcant research eﬀort has been dedicated to techniques for learning without labels, with increasing success recently, especially from self-supervised learning (SSL) (Chen et al., 2020;
Mitrovic et al., 2020; Lee et al., 2021; Tomasev et al., 2022; Scherr et al., 2022).
In SSL, BP can also use certain supervisory signals generated by the model itself as a global error.
Therefore, while BP does not require labels per se, it does require top-down supervision in the form of a global loss function.
The drawback of this is that learning then becomes specialized to the particular task that is explicitly deﬁned by the minimization of the loss function, as opposed to the learning of generic features.
Practically, this is expressed in DL as overﬁtting, sensitivity to adversarial attacks (Madry et al., 2017; Moraitis et al., 2021), and limited transferability or generalizability of the learned features to other tasks or datasets (Lee et al., 2021).
Moreover, a global optimization scheme such as BP cannot be considered a plausible model of all learning in the brain, because learning in ML benchmark tasks does emerge from highly biological plasticity rules without global (self-)supervision, e.g. through unsupervised
Hebbian-like learning (Diehl and Cook, 2015; Moraitis et al., 2020; 2021; Garcia Rodriguez et al., 2022).
Alternatives to backpropagation and their limitations
Unsupervised Hebbian learning in cortex.
Hebbian-like are those plasticity rules that depend only on the activity of preand post-synaptic neurons.
If such plasticity is combined with between-neuron competition that suppresses weakly activated neurons in a layer, e.g. as an argmax, it can lead to learning of useful features in the absence of any (self-)supervision (Sanger, 1989; Linsker, 1992).
This is a radically diﬀerent approach from BP.
It is not governed by a global optimization process, but rather emerges from local synaptic plasticity as a purely bottom-up self-organization.
Without supervision, feedbacks, or targets, competitive Hebbian learning circumvents all ﬁve limitations of BP, as it does not require any back-propagating signals.
Namely, it is free of weight transport, non-localities, locking problems, and global losses.
These are signiﬁcant practical advantages.
Moreover, these properties make Hebbian learning much more plausible biologically.
Besides, there is abundant evidence for Hebbian-like plasticity in the brain, in forms based on the spiking type of biological neuron activations (Sj¨ostr¨om et al., 2001; Markram et al., 2011; Feldman, 2012).
Even the competitive network connectivity that leads to useful learning through such plasticity is strongly supported by biological observations.
Speciﬁcally, competition between neurons emerges from lateral inhibition and this is found throughout the cortical sheet of the mammalian brain (Douglas et al., 1989; Douglas and
Martin, 2004; Binzegger et al., 2004; 2009), and are commonly described as winner-take-all (WTA).
Furthermore, such competitive learning has been deeply studied computationally and theoretically for a long time under various forms (Von der Malsburg, 1973; Nessler et al., 2013;
Diehl and Cook, 2015; Krotov and Hopﬁeld, 2019; Moraitis et al., 2020).
All these aspects would be important advantages if such learning could underlie DL.
However, unsupervised
Hebbian learning has only been eﬀective in shallow networks.
To be precise, adding layers has failed to show signiﬁcant improvements in standard benchmarks (Amato et al., 2019;
Lagani et al., 2021), except to a certain degree in Miconi (2021) (Fig. 1).
Arguably, this has been the case because emergent bottom-up learning from plasticity is hard to reconcile with top-down learning from a loss function, and the latter approach has been the backbone of DL.
Those competitive Hebbian approaches that were normative, i.e. that derived a plasticity rule from an optimization principle, were based on principles that do not appear compatible with successful DL techniques, either because of dramatically diﬀerent network types (Nessler et al., 2009; 2013) or loss functions (Pehlevan and Chklovskii, 2015).
SoftHebb.
Recent work, however, has advanced the theory behind Hebbian WTA learning in terms that are more compatible with DL.
Speciﬁcally, Moraitis et al.
(2021) used a simple softmax to implement a soft WTA (Equation (1)), which lends a Bayesian interpretation to the network and its learning (see also Nessler et al.
Moraitis et al.
(2021) also derived a Hebbian-like plasticity rule (Equation (2)) that minimizes the Kullback-Leibler divergence of the model’s probability distribution from the input’s, and cross-entropy from the labels under certain assumptions, without having access to those labels.
In one-layer networks, SoftHebb showed increased learning speed and signiﬁcantly higher robustness to noise and adversarial attacks.
Its theoretical results seem important towards deeper networks, but they were not suﬃcient for a practical demonstration.
Here (Section 3), we provide a
SoftHebb-based setup that does achieve a certain depth in Hebbian learning.
Approximations of backpropagation.
Alternatives that are not radically diﬀerent from
BP but rather approximate it, while mitigating some of its issues, have attracted heavy research interest with increasing success.
For example, self-supervised learning enables
BP to learn without labels (Hadsell et al., 2006), with instances such as SimCLR (Chen et al., 2020) and follow-ups (Mitrovic et al., 2020; Grill et al., 2020; He et al., 2020; Caron et al., 2020; Tomasev et al., 2022) reaching rather high performance.
A diﬀerent approach,
Feedback Alignment (FA), (Lillicrap et al., 2016; Frenkel et al., 2021), solves the weighttransport problem.
Direct Feedback Alignment (Nøkland, 2016; Frenkel et al., 2021) as an extension of FA, also avoids non-local computations for weight updates.
The spatial nonlocality of backpropagation has also been addressed by other algorithms such as predictive coding (Hadsell et al., 2006;
Chen et al., 2020), equilibrium propagation (Scellier and Bengio, 2017; Laborieux et al., 2021), burstprop (Payeur et al., 2021).
The work of L¨owe et al.
(2019), and CLAPP by Illing et al.
(2021) are self-supervised algorithms that avoid not only reliance on labels but also spatially non-local learning.
However, they require contrasting of examples known to be of diﬀerent type.
A very recent self-supervised approach avoids this requirement by adding a Hebbian term to learning (Halvagal and Zenke, 2022).
This makes it a rather plausible algorithm for the brain, but it still relies on comparisons between distinctly diﬀerent views of each individual input.
Moreover, its performance is signiﬁcantly worse on CIFARand STL-10 than this paragraph’s previous references.
In fact, all aforementioned approaches cause signiﬁcant drop in accuracy compared to BP, and most cases are not applicable to datasets as complex as ImageNet (Bartunov et al., 2018).
An algorithm that avoids the nonlocality of end-to-end BP and also achieves quite high accuracy in CIFAR-10 was proposed by Nøkland and Eidnes (2019).
However, it relies on an auxiliary trained network added to each layer, which increases complexity, and supervision, which requires labels.
Other works have achieved learning with local plasticity, while avoiding weight transport and the update-locking problem (Crafton et al., 2019; Frenkel et al., 2021).
However, they suﬀer from large performance drops and rely on supervision.
Clearly, signiﬁcant progress compared to standard BP has been made through multiple approaches, but important limitations remain (Table 1).
These limitations would be surmounted if a competitive Hebbian algorithm performed well in multilayer networks and diﬃcult tasks.
Here, we explore the potential of SoftHebb (Moraitis et al., 2021), i.e. the recent algorithm that seems ﬁtting to this goal.
Our results in fact demonstrate a multilayer learning setup, reaching relatively high performance for bio-plausible algorithms on diﬃcult tasks, with high eﬃciency and tight biological constraints.
In the next section we describe this setup.
Overview of multilayer SoftHebb
The key elements of the multilayer SoftHebb model and algorithm that achieve good accuracy without compromising its eﬃciency and biological-plausibility are the following.
SoftHebb (Moraitis et al., 2021) in a layer of K neurons realizes a soft WTA competition through softmax, parametrized by a base b or equivalently a temperature τ :
[Equation]
where u k is the k -th neuron’s total weighted input, and y k is its output after accounting for competition from neurons l .
The second key aspect in the SoftHebb algorithm is the plasticity rule of synaptic weights and neuronal biases.
Biases represent prior probabilities in the probabilistic model realized by the network.
In our experiments here we omitted them, presuming a ﬁxed uniform prior.
The plasticity deﬁned for a synaptic weight w ik from a presynaptic neuron i with activation x i to a neuron k is
[Equation]
Notably, all variables are temporally and spatially local to the synapse.
The rule provably optimizes the model to perform Bayesian inference of the hidden causes of the data (Moraitis et al., 2021)
(see also Section 2).
Soft anti-Hebbian plasticity.
Previous Hebbian-like algorithms have found antiHebbian terms in the plasticity to be helpful in single-layer competitive learning (Krotov and
Hopﬁeld, 2019; Grinberg et al., 2019).
However, such plasticity matched the assumptions of a hard WTA, as opposed to SoftHebb’s distributed activation, and involved additional hyperparameters.
Here we introduce a new, simple form of anti-Hebbian plasticity for soft
WTA networks, that simply negates SoftHebb’s weight update (Equation (2)) in all neurons except the maximally activated one.
Convolutions.
Towards a multilayer architecture, and to represent input information of each layer in a more distributed manner, we used a localized representation through convolutional kernels.
The plasticity rule is readily transferable to such an architecture.
Convolution can be viewed as a data augmentation, where the inputs are no longer the original images but are rather cropped into smaller patches that are presented to a fully connected SoftHebb network.
Convolution with weight sharing between patches is eﬃcient for parallel computing platforms like GPUs, but in its literal sense it is not biologically plausible.
However, this does not fundamentally aﬀect the plausibility of convolutions, because the weights between neurons with diﬀerent localized receptive ﬁelds can become matching through biologically plausible rules (Pogodin et al., 2021).
Alternative activations for forward propagation.
In addition to the softmax involved in the plasticity rule, diﬀerent activation functions can be considered for propagation to each subsequent layer.
In biology, this dual type of activation may be implemented by multiplexing overlapping temporal or rate codes of spiking neurons, which have been studied and modelled extensively (Naud et al., 2008; Kayser et al., 2009; Akam and Kullmann, 2014; Herzfeld et al., 2015; Moraitis et al., 2018; Payeur et al., 2021).
We settled on a combination of rectiﬁed polynomial unit (RePU) (Krotov and Hopﬁeld, 2016; 2019), with Triangle (Appendix A.3.1), which applies lateral inhibition by subtracting the layer’s mean activity.
These perform well (Coates et al., 2011; Miconi, 2021), and oﬀer tunable parametrization.
Weight-norm-dependent adaptive learning rate.
We introduce a per-neuron adaptive learning rate scheme that stabilizes to zero as neuron weight vectors converge to a sphere of radius 1, and is initially big when the weight vectors’ norms are large compared to 1:
[Equation]
, where lrp is a power hyperparameter.
This per-neuron adaptation based on the weights remains a local operation and is reminiscent of another important adaptive learning rate scheme that is individualized per synapse, has biological and theoretical foundations and speeds up learning (Aitchison, 2020; Aitchison et al., 2021).
Ours is arguably simpler, and its relevance is that it increases robustness to hyperparameters and initializations, and, combined with the Bayesian nature of SoftHebb (Section 2), it speeds up learning so that a mere single learning epoch suﬃces (Section 4).
Width scaling.
Each new layer halves the image resolution in each dimension by a pooling operation, while the layer width, i.e. the number of convolutional neurons, is multiplied by a “width factor” hyperparameter.
Our reported benchmarks used a factor of 4.
[Figure]
The most crucial elements for deep representation learning are the soft competition and the corresponding Hebbian plasticity rule that underpin SoftHebb (Figures 3 and B.2), the similarly soft anti-Hebbian plasticity that we introduced (Fig.
B.2), the convolutional neurons, and the width scaling architecture that involves a depth-wise diminishing output resolution (Fig. 4).
The adaptive learning rate signiﬁcantly speeds up the training (Fig. B.3B), such that we only use a single unsupervised learning epoch for the deep network.
The speciﬁc activation function and its tunable parametrization are less crucial but do improve performance (Appendix B).
We arrived at this novel setup grounded on Moraitis et al.
(2021) and through a literature- and intuition-guided search of possible additions.
Results
Summary of experimental protocol.
The ﬁrst layer used 96 convolutional neurons to match related works (Fig. 1), except our ImageNet experiments that used 48 units.
The width of the subsequent layers was determined by the width factor (see previous section).
Unsupervised Hebbian learning received only one presentation of the training set, i.e. epoch.
Each layer was fully trained and frozen before the next one, a common approach known as greedy layer-wise training in such local learning schemes (Bengio et al., 2006;
Tavanaei and
Maida, 2016; L¨owe et al., 2019).
Batch normalization (Ioﬀe and Szegedy, 2015) was used, with its standard initial parameters ( γ = 1, β = 0), which we did not train.
Subsequently, a linear classiﬁer head was trained with cross-entropy loss, using dropout regularization, with mini-batches of 64 examples, and trained for 50, 50, 100, and 200 supervised epochs for MNIST, CIFAR-10, STL-10, and ImageNet accordingly.
We used an NVIDIA Tesla V100 32GB GPU.
All details to the experimental methods are provided in Appendix A, and control experiments, including the hyperparameters’ impact in Appendix B.
Fully connected baselines.
The work of Moraitis et al.
(2021) presented SoftHebb mainly through theoretical analysis.
Experiments showed interesting generative and Bayesian properties of these networks, such as high learning speed and adversarial robustness.
Reported accuracies focused on fully connected single-hidden-layer networks, showing that it was well applicable to MNIST and Fashion-MNIST datasets reaching accuracies of (96.94 ± 0.15)% and (75.14 ± 0.17)%, respectively, using 2000 hidden neurons.
Starting from that work, we found that when moving to more complex datasets such as CIFAR-10 or
STL-10, SoftHebb performance was not competitive.
Speciﬁcally, the same shallow fullyconnected network’s accuracies reached (43.9 ± 0.18)% and (36.9 ± 0.19)% accordingly.
Compared to BP’s (55.7 ± 0.13)% and (50.0 ± 0.16)%, this suggested that single-hiddenlayer networks were insuﬃcient for extraction of meaningful features and separation of input classes.
We then stacked two such fully connected Hebbian WTA layers.
This network actually performed worse, reaching (32.9 ± 0.22)% and (31.5 ± 0.20)% on these tasks.
[Figure]
Convolutional baselines.
Recent research has applied Hebbian plasticity to convolutional hardWTA neural networks (CNN) (Miconi, 2021; Lagani et al., 2021;
Amato et al., 2019).
However, it has not achieved signiﬁcant, if any, improvement through the addition of layers (Fig. 1, green curves).
In our control experiments, we found that these networks with the plasticity rules from the literature do not learn helpful features, as the ﬁxed random weights performed better than the learned ones, also in agreement with results from
Miconi (2021).
Indeed, we ﬁnd that the features learned by such hard-WTA networks are simple
Gabor-like ﬁlters in the ﬁrst layer (Fig.
B.5A) and in deeper ones (see also Miconi (2021)).
A new learning regime.
One way to learn more complex features is by adding an anti-Hebbian term to the plasticity of WTA networks (Krotov and Hopﬁeld, 2019;
Grinberg et al., 2019).
Notably, this method was previously tested only with hard-WTA networks and their associated plasticity rules and not with the recent SoftHebb model and plasticity.
In those cases, anti-Hebbian plasticity was applied to the k-th most active neuron.
Here, we introduced a new, soft type of anti-Hebbian plasticity (see Section 3).
We studied its eﬀect ﬁrst by proxy of the number of “R1” features (Moraitis et al., 2021), i.e. the weight vectors that lie on a unit sphere, according to their norm.
Simple Gabor-like ﬁlters emerge without anti-Hebbian terms in the plasticity (Fig.
B.5A) and are R1.
Even with anti-Hebbian plasticity, in hard WTA networks, the learned features are R1 (Krotov and Hopﬁeld, 2019).
In the case of SoftHebb with our soft type of anti-Hebbian plasticity, we observed that less standard, i.e. non-R1, features emerge (Fig.
B.1 & B.5B).
By measuring the accuracy of the SoftHebb network while varying the temperature, we discovered a regime around τ =(Fig.
B.1) where R1 and non-R1 features co-exist, and accuracy is highest.
This regime only emerges with SoftHebb.
For example, on CIFAR-10 with a single convolutional layer and an added linear classiﬁer, SoftHebb accuracy (71.10 ± 0.06)% signiﬁcantly outperformed hard
WTA (62.69 ± 0.47)% and random weights (63.93 ± 0.22)%, almost reaching the accuracy of BP (72.42 ± 0.24)% on the same two-layer network trained end-to-end.
Convergence in a single epoch.
Adaptive learning rate.
By studying convergence through R1 features and by experimenting with learning rates, we conceived an adaptive learning rate that adapts to the norm of each neuron’s weight vector (Section 3).
We found that it also speeds up learning compared to more conventional learning schedules (Fig.
B.3B)
with respect to the number of training iterations.
The extent of the speed-up is such that we only needed to present the training dataset once, before evaluating the network’s performance.
All results we report on SoftHebb are in fact after just one epoch of unsupervised learning.
The speed-up is in agreement with observations of Moraitis et al.
(2021) that attributed the speed to SoftHebb’s Bayesian nature.
Moreover, we found that with our adaptive learning rate, convergence is robust to the initial conditions, to which such unsupervised learning schemes are usually highly sensitive (Fig. B.3A & B.3B).
(1x, 2x, or 4x)
indi layer increase the number
The architecture’s impact.
The multilayer architecture uses a pooling operation of stride 2, which halves each dimension of the resolution after each layer.
We stop adding layers when the output resolution becomes at most 4 × 4.
The layer where this occurs depends on the original input’s resolution.
Thus, the multilayer network has three hidden convolutional layers for
MNIST or CIFAR-10, four layers for STL-10, and ﬁve layers for ImageNet at a resolution setting of 160 × 160 px.
We used four times more neurons in each layer than in the previous layer.
This architecture on its own, with random initial weights, shows inconsistent increases in classiﬁcation accuracy, up to a variable depth (Fig. 3).
The performance increases caused by the addition of random layers seem surprising but are consistent with the literature where random weights often outperform biologicallyplausible learning rules (Miconi, 2021; Frenkel et al., 2021).
Indeed, by using the more common hard-WTA approach to train such a network, performance not only deteriorated compared to random weights but also failed to increase with added depth (Fig. 3).
Classiﬁcation accuracy (see Tables 1 & 2).
We report that with SoftHebb and the techniques we have described, we achieve
Deep Learning with up to 5 hidden layers.
For example, layer-wise accuracy increases on CIFAR-10 are visible in Fig.
Learning does occur, and the layer-wise accuracy improvement is not merely due to the architecture choice.
That is testiﬁed, ﬁrst, by the fact that the weights do change and the receptive ﬁelds that emerge are meaningful (Fig. 5).
Second, and more concretely, for the end-point task of classiﬁcation, accuracy improves signiﬁcantly compared to the untrained random weights, and this is true in all datasets (Fig. 3).
SoftHebb achieves test accuracies of (99.35 ± 0.03)%, (80.31 ± 0.14)%, (76.23 ± 0.19)%, 27.3% and (80.98 ± 0.43)% on MNIST,
CIFAR-10, STL-10, the full ImageNet, and ImageNette.
We also evaluated the same networks trained in a fully label-supervised manner with end-to-end BP on all datasets.
Due to BP’s resource demands, we could not compare with BP directly on ImageNet but we did apply
BP to the ImageNette subset.
The resulting accuracies are not too distant from SoftHebb’s (Fig. 3).
The BP-trained network reaches (99.45 ± 0.02)%, (83.97 ± 0.07)%, (74.51 ± 0.36)% and (85.30 ± 0.45)% on MNIST, CIFAR-10, STL-10, and ImageNette respectively.
Notably, this performance is very competitive (see Tables 1 & 2).
[Figure]
Layer Layer
[Figure]
[Figure]
[Figure]
patches larger complex features
Evidence for hierarchical representations.
One central question concerning SoftHebb was its capacity to build a hierarchical representation of features, i.e. features that increase in semantic abstraction with increasing depth, and also become more useful for downstream tasks such as object classiﬁcation.
One source of evidence for hierarchy is the layer-wise increase in accuracy that we observe, which shows that the added layers are increasingly useful Fig. 1.
By also using a method from Illing et al.
(2021), we found further evidence (Fig. 5).
In the UMAP projection (McInnes et al., 2018), the classes of inputs appear better separated after four layers than after one (Fig.
5A, B).
Moreover we found the patches in the dataset that maximally activate hidden neurons.
These too appear increasingly complex and abstract (Fig.
5C, D).
Further, we visualized the neurons’ receptive ﬁeld, by numerically optimizing the input to maximize hidden activations, through projected gradient descent, similar to Le et al.
(2012) (see Appendix A for methods).
Again, depth appears to increase the complexity and potential usefulness of the learned features (Fig. 2 and Appendix).
SoftHebb as unsupervised multilayer learning.
In STL-10, SoftHebb (76.23 ± 0.19)% outperforms fully supervised BP (74.51 ± 0.36)% (Fig.
3C).
The very few labelled examples in the training set do not suﬃce to learn good representations through only-supervised
BP.
By ﬁne-tuning SoftHebb with end-to-end BP on the few labelled STL-10 examples,
SoftHebb’s accuracy further improves to (78.5 ± 0.35)%.
This shows that the objective that is implicitly optimized by SoftHebb’s learning is compatible with the explicit objective of cross-entropy, as was more formally theorized and predicted in Moraitis et al.
Comparisons with other important algorithms for learning without labels show that SoftHebb in 4- or 5-hidden-layers-deep networks outperforms the partly biologically-plausible prior work (Table 2).
Of course, BP in more complex models signiﬁcantly outperforms SoftHebb.
[Table]
Discussion
SoftHebb’s accuracy and applicability in diﬃcult tasks challenges several other biologicallyconstrained DL algorithms.
Arguably it is also a highly biologically plausible and computationally eﬃcient method, based on it being free of weight-transport, non-local plasticity, and time-locking of weight updates, it being fully unsupervised, It is also founded on physiological experimental observations in cortical circuits, such as Hebbian plasticity and WTA structure.
Importantly, such Hebbian WTA networks enable non-von Neumann neuromorphic learning chips (Qiao et al., 2015; Kreiser et al., 2017; Sebastian et al., 2020; Indiveri, 2021; Sarwat et al., 2022a).
That is an extremely eﬃcient emerging computing technology, and SoftHebb makes high performance with such hardware more likely.
The algorithm is applicable in tasks such as MNIST, CIFAR-10, STL-10 and even ImageNet where other algorithms with similar goals were either not applicable or have underperformed SoftHebb (Fig. 1, Table 1,
Table 2, & Bartunov et al.
This is despite the fact that most alternatives only address subsets of SoftHebb’s goals of eﬃciency and plausibility (Section 2 & Table 1).
L¨owe et al.
and Burstprop (Payeur et al., 2021) results on STL-10 and ImageNet are not included in Table 2, because the ResNet-50 of L¨owe et al.
(2019) used standard BP through modules of at least 15 layers, and because Payeur et al.
(2021) did not report ImageNet topaccuracy.
SoftHebb did outperform Burstprop and its successor BurstCCN (Greedy et al., 2022) on CIFAR-10 (Table 1).
Beyond neural networks, K-means has also been applied to
CIFAR-10 (Coates et al., 2011), however, without successful stacking of K-means “layers”.
From the perspective of neuroscience, our results suggest that Deep Learning up to a few layers may be plausible in the brain not only with approximations of BP (Payeur et al., 2021;
Illing et al., 2021; Greedy et al., 2022), but also with radically diﬀerent approaches.
Nevertheless, to maximize applicability, biological details such as spiking neurons were avoided in our simulations.
In a ML context, our work has important limitations that should be noted.
For example, we have tested SoftHebb only in computer vision tasks.
In addition, it is unclear how to apply SoftHebb to generic deep network architectures, because thus far we have only used speciﬁcally width-scaled convolutional networks.
Furthermore, our deepest SoftHebb network has only 6 layers in the case of ImageNet, deeper than most bioplausible approaches (see Table 1), but limited.
As a consequence, SoftHebb cannot compete with the true state of the art in ML (see e.g. ResNet-50 SimCLR result in Table 2).
Such networks have been termed “very deep” (Simonyan and Zisserman, 2014) and “extremely deep”
(He et al., 2016).
This distinguishes from the more general term “Deep Learning” that was originally introduced for networks as shallow as ours (Hinton et al., 2006) has continued to be used so (see e.g. Frenkel et al.
(2021) and Table 1), and
its hallmark is the hierarchical representation that appears to emerge in our study.
We propose that SoftHebb is worth practical exploitation of its present advantages, research around its limitations, and search for its possible physiological signatures in the brain.
A Details to the methods A. Weight initialization A.1. Choosing distribution family
We tested three probability distribution families for initializing the weights randomly; normal, positive, and negative (Equation (3)).
normal =
N ( µ = 0 , σ 2 ) positive =
U (0 , range ) negative =
U (0 , − range ) , (3) where U is a uniform distribution.
We found that they perform similarly to each other in the case of long experiments, where the extensive training examples achieve to align the weights with the input.
However, for shorter experiments, where speed is needed, we found that positive distribution was better for raw data (input values between 0-1) and normal distribution for data that was normalized into standard scores.
The clustering property of
SoftHebb learning (Moraitis et al., 2021), where learned features are centroids of clusters in the input space may explain this.
In practice, we used normalized data for the reported results, and therefore we used normal distribution for the initial random weights.
A.1. Choosing distribution parameters
By varying the distribution parameters, we observed that the norm of the weight vector or radius is very instructive.
We identify three regions of disparate learning (Fig. B.3A); for R smaller than 1, there is no learning with random weight update; for R bigger than 1 and smaller than approximately 2.5, there is a partial leaning with only a percentage of neurons winning and converging while the others stay random; for R bigger than 2.5, all neurons learn.
The learning change around 1 comes from the fact that the weight tends to converge to a radius of 1 (Moraitis et al., 2021).
Determining the initial radius from the weight distribution parameters: We can then derive the distribution parameters from the optimal initial radius using the distribution moment calculation.
[Equation]
Where i is the index of a neuron, j is the index of this neuron’s synapses, N is number of synapses of that neuron, E () is the expected value and so E ( | w | ) is the ﬁrst absolute moment of distribution.
Thus, for the normal distribution E ( | w | )
=
σ · (cid:112) 2 /π ⇒ σ
= R · (cid:112) π 2 N and positive distribution E ( | w | )
=
E ( w ) =
range/ 2 ⇒ range =
R · (cid:113) 2 N .
A. Learning-rate adaptation
Learning rates that decay linearly with the number of training examples have been extensively used in Hebbian learning (Krotov and Hopﬁeld, 2019; Grinberg et al., 2019; Miconi, 2021;
Lagani et al., 2021; Amato et al., 2019).
It is a simple scheduler, reaching convergence with a suﬃcient amount of training example.
The linear decay ties the learning rate’s value throughout learning to the proportion of the training examples that have been seen, and it does so uniformly across neurons.
However, the weights may theoretically be able to converge before the full training set’s presentation, and they may do so at diﬀerent stages across neurons.
To address this, we tied the learning rate
lr i to each neuron’s i convergence.
Convergence was assessed by the norm r i of the neuron’s weights.
Based on previous work on similar learning rules (Oja, 1982; Krotov and Hopﬁeld, 2019), on SoftHebb itself Moraitis et al.
(2021), and
on our own observations, the convergence of the neuronal weights is associated with a convergence to a norm of 1, in case of simple learned features at least.
Therefore, we used a learning rate that stabilizes to zero as neuron weight vectors converge to a sphere of radius 1, and is initially big when the weight vectors are large compared to 1:
[Equation]
where lrp is a power hyperparameter.
Note that this adaptivity has no explicit timedependence, but as learning proceeds towards convergence, the lr does decay with time.
To account for the case of complex features that do not converge to a norm of 1, a timedependence can be added on top, e.g. as usual with a linearly decaying factor, multiplying the norm-dependent learning rate.
In our experiments we only used the norm-dependence and simply stopped learning after the ﬁrst training epoch, i.e. the ﬁrst presentation of the full training set in most cases, or earlier.
In practice, our adaptive rate reaches convergence faster (Fig.
B.3B)
than the linear time-dependence, by maintaining a separate learning rate for each neuron and adapting it based on each neuron’s own convergence status.
A. Layer architecture
Each convolutional layer includes a succession of batch normalization, a SoftHebb convolution, a pooling operation, and then an activation function.
The SoftHebb convolution stride is ﬁxed at 1.
Padding is added to the input of the convolutional ﬁlters to guarantee that their output has the same size as the unpadded input.
The pooling stride was ﬁxed at 2, halving each dimension of the resolution after each layer.
In all experiments, the ﬁrst layer had a width of 96 convolutional kernels Miconi (2021); Lagani et al.
(2021); Amato et al.
(2019), except ImageNet experiments where we used 48 kernels.
The number of kernels in the subsequent layers was determined according to our width-scaling method, with a width factor of 4 (see Appendix A.5).
A.3. Activation function
Softmax, i.e. the output of the soft winner-take-all was used as the postsynaptic variable y in SoftHebb’s plasticity.
However, for forward propagation to the subsequent layer, we considered three activation functions; Rectiﬁed polynomial unit (RePU), triangle, and softmax.
RePU (Equation (6)) was proposed by Krotov and Hopﬁeld (2016) as a generalization of rectiﬁed linear units (ReLU) and was also used in follow-up works (Krotov and Hopﬁeld, 2019; Grinberg et al., 2019).
Triangle activation was introduced by Coates et al.
(2011) for k-means clustering and also appears useful for Hebbian networks in (Miconi, 2021).
It subtracts the mean activation calculated over all channels at each position and then applies a ReLU.
Here we generalize Triangle by combining it with RePU (instead of ReLU) through
Equation (7):
[Equation]
A. Hyperparameter optimization
We systematically investigated the best set of hyperparameters at each hidden layer, based on the validation accuracy of a linear classiﬁer trained directly on top of that hidden layer.
All grid searches were performed on three diﬀerent random seeds, varying the batch sampling and the validation set (20% of the training dataset).
The classiﬁer is a simple linear classiﬁer with a dropout of 0.5 and no other regularisation term.
For all searches and ﬁnal results, we used 96 kernels in the ﬁrst layer.
Subsequent layers scaled with a f w = 4 (see Appendix A.5).
However, based on our observations, only the optimal temperature depends on the f w .
For each added layer, grid search was performed in three stages:
For the ﬁrst two stages we used square convolutional kernels with a size of 5, a max-pooling with a square kernel of size 2, and a Triangle with a power of 1 as forward activation function.
In this stage we performed a grid search over the remaining hyperparameters ( nb epochs , batch ize , η and lrp of the learning rate scheduler, and temperature τ of the SoftHebb softmax).
We varied the nb epochs ∈ { 1 , 10 , 50 , 100 , 1000 } ,
[Table]
More details are provided in section A.4.
[Table]
12 } , lrp ∈ { 0 . 25 , 0 . 5 , 0 . 75 } and the 1 /τ ∈ { 0 . 25 , 0 . 5 , 0 .
A ﬁner grid search over 1 /τ ∈ { 0 . 15 , 0 . 25 , 0 . 35 , 0 . 5 , 0 . 6 , 0 . 75 , 0 . 85 , 1 , 1 . 2 , 1 . 5 , 2 } and conv kernel size ∈ { 3 , 5 , 7 , 9 } using the best result from the previous search.
A ﬁnal grid search over the pooling: pool type ∈ { AvgPooling, MaxPooling } ,
pool kernel size
∈ { 2 , 3 , 4 } , and the activation function: function ∈ { RePU, Triangle, Softmax } with for power ∈ { 0 . 1 , 0 . 35 , 0 . 7 , 1 , 1 . 4 , 2 , 5 , 10 } (for
RePU or Triangle) and τ ∈ { 0 . 1 , 0 . 5 , 1 , 2 , 5 , 10 , 50 , 100 } (for softmax) using the best
result from the two previous searches
A. Multilayer architecture
The pooling of stride 2 halves each dimension of the resolution after each layer.
We stop adding layers when the output resolution becomes at most 4 × 4.
The layer where this occurs depends on the original input’s resolution.
Thus, the multilayer network has three convolutional layers for MNIST or CIFAR-10, four layers for STL-10, and ﬁve layers for
ImageNet at a resolution setting of 160 × 160 px ( Table 4).
A width factor f w characterizes the multilayer network.
f w links the width of each layer to that of the previous layer, thus determining the depth-dependent architecture.
Speciﬁcally, the number of ﬁlters # F l in the hidden layer l is f w times the number of ﬁlters at layer l − 1: # F l = f w · # F l − 1 .
The ﬁrst hidden layer has 96 ﬁlters in order to compare with Miconi (2021); Lagani et al.
; Amato et al.
We then explored, using CIFAR-10, the impact of f w on performance.
We tried three diﬀerent values for f w ∈ { 1 , 2 , 4 } .
A value ofkeeps the same number of ﬁlters in all layers, while a value of 4 keeps the number of features provided to the classiﬁer head equal to the number of features at the input layer (due to the pooling stride of 2 in each layer).
A f w bigger than four would substantially increase
[Table]
the size of the network, and is impractical for deeper networks.
We found that to increase performance with depth, the network needs to also grow in width (Fig. 4).
A. Training and evaluation protocols
Each experiment was performed 4 times, with random initializations, on all datasets except the full ImageNet where only one random seed was tried.
A.6. SoftHebb training
The optimal number of SoftHebb weight update iterations is around 5000 based on CIFARexperiments.
Thus, for CIFAR-10 and MNIST (50k training examples), unsupervised training was performed in one epoch with a mini-batch of 10 and 20 for STL-10 unlabelled training set (100k training example).
Because of the large number of training examples, we randomly select 10% of the ImageNet dataset with a mini-batch of 20.
The accuracies we report for SoftHebb are for layers that are trained successively, meaning each SoftHebb layer was trained, and then frozen, before the subsequent layer was trained.
However, the results are very similar for simultaneous training of all layers, where each training example updates all layers, as it passes forward through the deep network.
A.6. Supervised training
The linear classiﬁer on top uses a mini-batch of 64 and trains on 50 epochs for MNIST and CIFAR-10, 100 epochs for STL-10, and 200 epochs for ImageNet.
For all datasets, the learning-rate has an initial value of 0.001 and is halved every [20%, 35%, 50%, 60%, 70%, 80%, 90%] of the total epochs.
Data augmentation (random cropping and ﬂipping) was applied for STL-10 and ImageNet.
A.6. Backpropagation-trained networks
Results comparing SoftHebb and Backpropagation are found using the same network architecture.
The only diﬀerence is the activation function; triangle or softmax activation would be detrimental to BP.
The activation function we used for BP-trained networks is ReLU.
The learning rate followed the same schedule as described Appendix A.6.2.
A.6. Fine-tuning
In the ﬁne-tuning experiment on STL-10, all Hebbian CNN layers learned using SoftHebb and the large unlabelled training dataset of STL-10; then, an output layer was added and the entire network was trained end-to-end using BP on the full small labelled training subset.
A. Receptive fields
We have visualized the receptive ﬁelds (RFs) of hidden layers in the network (Figure 2 and end of Appendix B).
The method that we used is activation maximization (Erhan et al., 2009; Le et al., 2012; Goodfellow et al., 2014; Nguyen et al., 2015).
Speciﬁcally, we started from a square of random pixels, and we optimized the input through gradient descent (or rather ascent) to maximize the activation of each neuron, under the constraint of an L norm of 1, i.e. projection to a unit sphere.
That is then a form of projected gradient descent (PGD), which can also be used as an adversarial attack, if a loss function rather than the activation function is maximized.
For this purpose, we modiﬁed a toolbox for adversarial attacks, named Foolbox (Rauber et al., 2020).
We show RFs that maximize the linear response of the neurons, i.e. the total weighted input.
We have tuned the step size of the descent, and we have validated the approach (a) by verifying that the number of iterations suﬃce for convergence, (b) by conﬁrming that its results at the ﬁrst layer match the layer’s weights, (c) by verifying that the hidden neurons are strongly active if the network is fed with inputs that match the neuron’s found RFs, and (d) by seeing that alternative initializations also converge to the same RF.
We also tried an alternative method that was used by Miconi (2021).
Speciﬁcally, we used that paper’s available code ( https://github.com/ThomasMiconi/HebbianCNNPyTorch ).
We found that the RFs found by PGD activate the neurons more than the RFs found by the alternative method.
Moreover, PGD takes into consideration pooling and activation functions, which the other method does not.
Therefore we chose to present the results from PGD.
Example results are presented in Fig. 2 as well as in extended form with more examples at the end of Appendix B.
B Additional results and analyses B. Impact of temperature. SoftHebb leads to a new learning regime.
B
Impact of adaptive learning rate.
It increases learning robustness and speed.
[Figure]
We speculate that low initial radius is problematic (Figure B.3A) because in that regime the balance between excitation (from the input) and inhibition (from the soft WTA) is overly tilted towards inhibition.
Regarding Figure B.3B, see also main text’s Section 3, “Weight-norm-dependent adaptive learning rate”, and the related paragraph in Section 4.
B. Impact of activation function
The choice of activation function for forward propagation through SoftHebb layers is important.
To study this, we compared SoftHebb’s performance on CIFAR-10 for various activation functions.
In each result, all neurons across all layers used the same activation function hyperparameters, except the case of Triangle.
The parameters that were tried were:
(cid:136) Power of RePU: 0.7*, 1.0, 1.4.
(cid:136) Temperature of softmax: 4.0*, 1/0.65, 1.0.
These three values were tried because they were good for individual layers in previous tuning.
Asterisk indicates the best parameter value according to validation accuracy.
Using the best values produced the test accuracy results reported below, whereas the remaining hyperparameters were not tuned to each case, but rather were the same, as found for Triangle by the process described in Appendix A.4.
(cid:136) ReLU: (70 .
68 ± 1 .
07)% (cid:136) tanh: (56 . 13 ± 0 .
34)% (cid:136)
[Table]
B. Impact of width (number of neurons)
The width of the layers is rather impactful, as indicated by varying the width factor of deep layers while keeping the ﬁrst layer’s width constant (Fig. 4).
To further study that impact, we varied the ﬁrst layer’s width and kept the width factor ﬁxed to 4 (which scales all layers).
The results are presented in Fig.
B.4.
[Figure]
B. Further study of hidden representations.
As a control, we perform again the experiment that we presented in Fig.
5, this time in comparison to a network with the initial, untrained, random weights.
The results are shown in Fig.
B.6 and Fig.
B.7.
[Figure] [Figure]
B. Supplementary receptive fields found through PGD.
For the method, see Appendix A.7.
RFs of deeper layers are not all Gabor-like, but rather also include mixtures of Gabor ﬁlters, and also take diﬀerent shapes and textures.
In addition,
RFs do appear increasingly complex with depth.
These results could possibly be expected based on the RFs of the ﬁrst layer, which are already more complex than the mere Gabor ﬁlters that are learned by other Hebbian approaches, such as hard WTA (Figure B.5A).
Their mixture in subsequent layers then was unlikely to only produce Gabor ﬁlters.
It is diﬃcult to interpret each RF precisely, but this is common in the hierarchies of deep neural networks.