Emergent representations in networks trained with the Forward-Forward algorithm
Authors: Niccolo Tosato
Lorenzo Basile
Emanuele Ballarin
Giuseppe De Alteriis
Alberto Cazzaniga
Alessio ansuini
Venue: Accepted by TMLR ()

Abstract: The Backpropagation algorithm has often been criticised for its lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward-Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal representations obtained by the Forward-Forward algorithm can organise into category-specific ensembles exhibiting high sparsity -- composed of a low number of active units. This situation is reminiscent of what has been observed in cortical sensory areas, where neuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, it can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm.

Published in Transactions on Machine Learning Research (03/2025)
Emergent representations in networks trained with the Forward-Forward algorithm
Niccolò Tosato"
niccolo. tosato Sphd. units.
University of Trieste, Italy
AREA Science Pork, Italy
Lorenzo Basile"
lorenzo. lorenzo.basile basile @phhd. units.
University of Trieste, Iaky AREA Science Pork, Italy
Emanuele Ballarin
smamuudc.bullariinneyyud. unils.
University of Trieste, Italy
Giuseppe de Alteriis
giuseppe.der citespeede_alteriiate allerllleaae www.eriiiikkk..cccoooo
King's College London, UK University College London, UK
Alberto Cazzaniga
AREA Science Park, Raly
Alessio Ansuini
alessio dlessso..nnstirr@@vvvaeeeeeeepe
AREA Science Pork, Italy
Reviewed on OpenReview: https: openreview. net/ forum? id= JdY60ssF3)
Abstract
The Backpropagation algorithu has often been criticised for lack of biological realism. In an attempt to find a more biologically plausible alternative, the recently introduced Forward- Forward-Forwand Forward algorithm replaces the forward and backward passes of Backpropagation with two forward passes. In this work, we show that the internal reprosentations obtained by the Forward-Forward algorithm high sparsity composed of a low number of active units. This situation is reminiscent of what has been obscrved in cortical sensory areas, where ncuronal ensembles are suggested to serve as the functional building blocks for perception and action. Interestingly, while this sparse pattern does not typically arise in models trained with standard Backpropagation, can emerge in networks trained with Backpropagation on the same objective proposed for the Forward-Forward algorithm.
organise into category-specific ensembles exhibiling
1 Introduction
Deep Learning is highly effective approach artificial intelligence, with tremendous implications for science, technology, culture, and society. At its core, there is the Backpropagation (Backprop) algorithm (Rumelhart et al. 1986), which efficiently computes the gradients necessary to optimise the learnable parameters of an artificial neural network. Backprop, however, lacks biological plausibility (Stork 1989) leading to many attempts to address the issue. One of the most recent approaches, the Forward-Forwwe
algorithm (Hinton 2022), eliminates the need to store neural activities and propagate error derivatives along the network. In a standard classification context, the application Forward-Forward requires the designation of positive
'Equal contribution. Code available http:////s..aaoccoocceaeeettttte

---

Published in Transactions on Machine Learning Research (03/2025)
and negative data. For example, to classify images, one could assign positive (or negative) data to those images having their correct (or incorrect, respectively) class label embedded via one-hot encoding at the border (as shown in Figure 1] Panel A). The Forward-Forwary algorithm then learns to discriminate between positive and negative data by optimising a goodness function (e.g., the & norm of the activations), akin contrvstive learning (Chen et al.] 2020 Satisfactory results have been observed [Hinton 2022) for classification tasks on MNIST (Lecun et al. 1998), standard benchmark dataset. This work takes a step beyond performance evaluation, delving into the structure of the hidden representations learned by the Forward-Forwer algorithm, uncovering their spontaneously sparse nature and drawing parallels to neural ensembles observed in the brain (Miller et al. 2014 Yuste et al. 2024).
We organise this paper as follows. In section 2 we set the stage by providing a brief overview of the Forward- Forward algorithm and of neuronal ensembles. Then, |section is dedicated to the description the models and datasets investigated and the methods used analyse representations. Our analysis of the Forward- Forward representations begins in insectioon 4, where we present our key findings. Specifically, in insubsoctio 4.2] we show that. the Forward-Forwwe tificial ensembles, i.e., small sets of highly specialised neurons that consistently co-activate for data in a given class. In subscction 4.3 we demonstrate that these ensembles can overlap, with individual units contributing to multiple ensembles when visual features are shared. Further, subsection 4.4 reveals that ensembles can arise on previously unseen categories, indicating a robust generalization of this representational mechanism. Notably, these ensembles can share units with those associated with seen categories, demonstrating effective integration of new information with concepts learned during training. Finally, in subsection 4.5 we examine the structure of the weights and show that the observed sparsity and ensemble formation arise from sup- pression mochanisms, analogous to the inhibitory processcs mediated by biological neurons (Yuste 2015). These findings are particularly striking because the Berward-Forward algorithm achieves these properties without requiring explicit regularisation to induce sparsity. We observe that, although optimising the cross- entropy loss for the same classification task does not appear to produce the sparse ensembles we observe, the phenomenon may not solely be due to the use the Forward- are obtained by optimising the same goodness function of Forward- Forward, with Backprop instead. This suggests that more focus should be put on the purpose and biological meaning of the loss function rather than the training algorithm (Richards et al. 2019). We discuss our results in section 5
algorithm spontaneously learns sparse representations, organised into ar-
algorithm. In fact, similar results
In summary, our main results are as follows:
The Hewward-Forward algorithm yields sparse representations composed of small groups of highly specific units, which refer to ensembles by analogy with those observed in the cortex.
Ensembles can emerge in a zero-shot manner for classes held out during training and they can share units across visually related categories.
The emergence of sparsity and the formation of ensembles are not unique to Forward-Forward optimisation, as they can also be observed in networks trained with Backpropagation on the same objective function.
2
Related Work
In the section that follows, we summarise key aspects of the Forward-Forward algorithm and the main findings pertaining identification and characterisation of biological neuronal ensembles in the brain.
2.1
Berward-Fowward
The Forward-Porward algorithm (Hinton 2022) is a recently proposed learning algorithm for artificial neural networks, whose main premise the ability to overcome the notorious biological implausibility of Backprop [Rumchhart al., 1986). In fact, while the effectiveness of Backprop makes the standard algorithm for training neural networks, is based on biologically unrealistic assumptions, such as the need propagate information forwards and backwards through the network (Richards et al. 2019
2

---

Published in Transactions on Machine Learning Research (03/2025)
Layer
Layer
Layer
A
3
Class
Class
Class
Class
Class
B
Class
Class
Class
Class
Closs
2.0
Layer 1
C
Classes
Neurons
Figure 1: Activation patterns in a MuyerrLayer Perceptron trained with the Forward-Forwart algorithm, on the MNIST datasct.
Panel A Examples activation patterns in response to a positive input (class label embedded as a one-hot encoding on the top left corner of the image). Images show the activation value for network units, arranged as matrix only for the sake of clarity; darker squares represent more active neurons.
Panel B Activation value of each neuron in the first hidden layer (Layer 1), averaged all images of given class. Neuron index on the x axis; average activation on the y axis. Blue dots indicate units that are considered active according to the leavo-onc-out (LOO) method described in ssussccction 3.5
Panel C Activation map for neurons in Layer for all images, grouped by class. A blue dot in position (z. y) indicates that neuron x is activated by input y; colour scale represents the intensity of such activation. Horizontal bands mark different categories; blue vertical stripes mark active, category-specífic neurons. Each input category activates consistently a specific sets of neurons (ensemble).
Forward-Forward owes its name to the fact that it replaces the backward pass with an additional forward pass. The two forward passes are executed on different data, named positive and negative data. During training, the objective of Forward-Forwand is to maximise so-called goodness function of the neural activations (e.g.,
3

---

Published in Transactions on Machine Learning Research (03/2025)
the Ep norm) positive data and minimise it negative data. In simple image classification setting, such as the one we adopt in this paper, one could encode a class label at the border of images, by one-hot encoding it with a white pixel (as shown in Figure 1 Panel A). Then, following the definition from Hinton (2022), positive data are those for which the encoded label matches the ground truth label, while the opposite holds for negalive data. Layers are trained separately and sequenially, and learn to discriminate bet ween positive and negative data by maximising and minimising their goodness, according to the data presented. Crucially, activations are normalised before being passed the subsequent layer, to prevent layers from relying the goodness computed by their predecessors. At test time, when a new unlabelled sample has to be categorised, many copies of the image are created, each with a different one-hot encoded label. These are then fed into the neural network to obtain a goodness score. Finally, the image gets classified in the category that produced the maximum goodness value.
In the seminal Forward-Forward paper (Hinton 2022,, satisfactory classification results are reported on the standard handwritten digit recognition dataset MNIST, with the definition of positive and negative data described above, and using the f2 or Ei norm of activations as goodness function. In recent. theoretical work, it has been analytically shown that under somewhat mild assumptions sparsity emerges in Forward- Forward layers (Yang) 2023) as a consequence of optimising the Forward-Forward loss. While these formal results are derived for a single layer, they offer a theoretical grounding for our experimental findings. Recent studies inspired by the architectures, notably achieving enhanced performance in Convolutional Neural Networks (CNNs) (Papa- christodoulou et al. 2024 Sun et al., 2025). Additionally, other works have proposed alternative goodness functions and explored the specific contributions of individual neurons to the classification process, shedding light on the interprctability and adaptability of the approach (Terres-Escudert et al. 2024
learning procedure have expanded its applicability across various
We illustrate properties of representations obtained in forward-Forwand networks, that are reminiscent of what is found the neocortex and hippocampus, where ensembles of a few number of units activate consistently in response to similar stimuli. We discuss properties of ncuronal ensembles in the following section.
2.2 Neuronal ensembles
In Neuroscience, neuronal ensembles are defined as sparse groups of neurons that co-activate either spon- taneously or in response to sensory stimuli. These ensembles, rather than individual neurons, have long
been proposed as emergent functional units of cortical activity, playing critical roles in sensory processing,
memory, and behaviour (Miller et al. 2014 Hebb 2005 Harris 2005 György 2010 Hopfield 1982 Carrill-
Reid et al. 2019 Carrillo-Reid & Yustel 2020 Yusle et al. 2024 Recent reviews, such as Yuste el al. (0244)
provide a comprehensive overview of the concept and its implications.
The importance of ensembles has been inereasingly corroborated by experimental studics, enabled by ad- vances in techniques like calcium imaging, which allow for simullaneous recording of large-scale neural activity at single-cell resolution (Carrillo- Reid & Calderon) (2022). For example, Miller et al. (2014) demon- strated that, during visual processing, cortical spiking activity is dominated by ensembles whose properties cannot be explained by the independent activity of individual neurons. These ensembles by sensory stimuli (e.g., visual inputs) and by spontaneous network activity, suggesting that they repres- ent intrinsic functional building blocks of cortical responses. Notably, single neurons often participate in multiple ensembles, thereby enhancing the network's encoding potential (Rigotti et al.] 2013 Fusi et al. 2016). Further evidence from Yoshida & Ohki (2020) showed that sparse ensembles in the primary visual cortex (V1) are clicited by visual stimuli. Images can be decoded reliably from the activity of a small subsct of highly responsive neurons, with additional neurons either failing to improve or even degrading decoding performance. These findings underscore the efficiency of sparse representations, likely facilitated by partially overlapping receptive fields. This arrangement enables robust and efficient encoding of visual information, making sparse ensembles an optimal strategy for downstream processing.
activated both
The presence and functionality of ensembles are not limited to specific species or sensory modalities. Studies in various animal models have revealed their role in diverse neural processes (Dupre & Yusto 2017: Liu & Baraban 2019), and recent findings suggest they may even contribute to conscious experience [ooye et al.; 2023). Moreover, ensembles have demonstrated remarkable stability over time. For instance, Pérez-Ortegal
4

---

Published in Transactions on Machine Learning Research (03/2025)
et al. (021 showed that neuronal ensembles persist for weeks, supporting their potential involvement in long-term representations of perceptual states memories.
Technological advancements have also enabled not just the visualization but the direct stimulation of en- sembles, allowing to "play the piano" with ensembles of neurons Carrillo- Carrillo-Reid Reid & Yuste (2020). All-optical approaches, such as those described by Packer et al. (2015) and Carrillo-Reid al. (2019). have shown that repeatedly stimulating specific groups of neurons in VI can imprint ensembles that remain spontaneously active even after day. These imprinted ensembles exhibit pattern completion, where activating subset of neurons can recall the entire ensemble. Remarkably, this effect persists long after the initial stimulation, and experiments have demonstrated causal links between ensemble activation and behaviour (Carrillo-Rcid & Yuste 2020).
Finally, the concept of neuronal ensembles has inspired computational models. For instance, Doi & Lewicki (2004) demonstrated that sparse and redundant representations are optimal for encoding natural images, particularly when neurons unreliable, & result corroborated by earlier studies (Field 1994 Olshausen & Field 2004). These computational frameworks align with biological observations, suggesting that sparse ensemble representations are both efficient and robust mechanisms for encoding sensory information.
3 Methods
In this work, we investigate and compare the representations produced by three models !
A classifier in the style of that used by Hinton 2022 trained with Forward-Forward (FF);
A classifier identical the above, but trained end-to-end with Backprop optimise the same goodness function (BP/E) /FF);
classifier trained with Backprop on the categorical cross-entropy loss, as customary (BP).
Such different scenarios are described individually in subsection 3.2 respectively.
subsection 3.3 and subsection 3.4
3.1 Data
The datascts we use to train and test the models described so far are MNIST (Lecun al. 1998), FASHIONM-
NIST (Xiao et al. 2017), SVIIN (Netzer et al. 2011 and CIFAR10 AAlex 2009 Details on these datasets are provided in the Appendix B
3.2 Model trained with Forward-Forwart (FF)
Our FF model is inspired by the architecture proposed by Hinton to the Forward-Forwart algorithm. It consists of three fully-connected layers, each composed by 1000 units in the case of MNIST and FASHIONMNIST,, and 3072 units in the case of SVHN and CIFAR10. Each linear layer is followed by elementwise ReLU non-linearities. Both during training and inference, the layer-wise & norm is used as the goodness function of choice; correspondingly, & normalisation is performed between subsequent layers. Additional results obtained using the E norm as goodness function are presented in Appendix K
2022 and likewise trained according
To define positive and negative data, a onc-hot-encoded class vector embedded at the top-left corner of images. Prior to such embedding, these pixels are set to black colour. Then, in the case of positive data, the pixel corresponding to the true class is switched the maximum value elsewhere observed in the image, while in the case of negative examples such value is randomly assigned one of the other pixels of the embedding vector.
During training, the weights are optimised by minimising the loss function L log(1 + Gneeg Gpon) where Gneg and Gos are, respectively, the goodness value for negative and positive data. At inference time, for
From this point on, the model refer the combination network architecture and optimisation algorithm.
5

---

Published in Transactions on Machine Learning Research (03/2025)
each layer, the goodness values corresponding to every possible label are converted into probability using softmax. By performing this step for each layer, they can contribute equally to the prediction.
For comprehensive details on the training procedures of the models discussed in this section and the next two sections, we direct the reader to Appendix C
3.3 Model trained with Backpropagation on the goodness objective (BP/FF)
The architecture of the FF model, while designed to be optimised using the Forward- Forwardd algorithm, be trained scamlessly with Backprop on the same goodness maximisation, minimissatinn objective. Indood, keeping the definition of positive and negative data introduced for FF, one could simply use Backprop to optimise the goodness-based loss from the Forward-Forwall algorithm.
detail, positive and negative data are fed to the network during the forward step, and the overall goodness of the internal representation is evaluated. The backward pass is then executed, and parameters are optimisod to achieve the same goal the FF model. It is worth pointing out that, in this case, the goodness is maximised globally instead of layer-by-layer (i.e., locally).
3.4 Model trained with Backpropagation on the cross-entropy loss (BP)
The FF and BF/FF models are also compared to a standard neural classifier, serving as a baseline. For such purpose, a multi-layer perceptron is employed. The model shares the same number of layers, layerwise neuron count, and non-linear activation function choice with FF and BP/FF: The only architectural difference between the BP model and the other two is the addition of a final softmax layer, to suitably shape and scale the output for the classification task. The model is trained end-to-end with Backprop on the categorical cross-entropy loss.
3.5 Analysis of representations
For each model described, we analyse the internal representation emerging at each layer. We limit our analysis to data belonging the test set (i.e., not seen during training) and correctly classified by the respective model. However, the main results of our analysis, concerning sparse and ensemble-like representations, extend without any modification to training data. Concretely, the representation of single image is a n-dimensional vector composed by the activations (after the ReLU non-lincarity) of all the units in the layer. For each layer, we extract a representation matrix of size (M, n), where M is the total nunber of test images (correctly classified) and n is the number of neurons in the layer considered.
Sparsity
For each representation vector we assign sparsity measure following the notion of sparsity introduced in Hoyer 2004);
S(x) = /n
With this definition, when S(x) = the vector æ contains only one non-zero component representing the case of an extreme sparsity. The other limiting case is the one which all the components of æ are equal in magnitude, in this case S(x) 0. The sparsity function S interpolates smoothly between these two extremes. The sparsity of a layer representation is obtained by averaging the sparsity of its component a vectors S = / ΣM S(z;).
Ensembles
To detect the emergence of -ategory-specific ensembles, within each model and dataset combination, we adopt the following method. The idea that a neuron should be considered active and part of an ensemble if it activates consistently and specifically when the network receives input data that belongs that category. We start by defining a category-specifi representation matrix Xe, of shape (Me, n), where M is the number of

---

Published in Transactions on Machine Learning Research (03/2025)
correctly classified test images of the given category. Then, compute the average activation of each hidden 1 Me (Xe)ey; and the leave-one-out average of the averages I.00j. = n-1 Eny, Fz.c. We then classify a neuron as active (i.e., part of an ensemble) if Zi,c > 2 LO0 We also perform significance test. for these comparisons through permutation test (see Appendix D and Table 3
unit across all samples: Tj. M.
Examples of average activation profiles and of ensembles are reported in Figure
The output of the ensemble computation is a set of active units for each category: E = (ei, .e....e, Ve
{1,2...., C}, where ne is the number of active units for category c. Once the ensembles are defined, it is possible to look at units that are shared across categorics e and by considcring g n The size of the shared units is naturally measured by E n
We can also measure the similarity between two ensembles using the Jaccard similarity index (intersection over union): J(E", = [ ensemble overlap of 30% (15 units), the similarity J is a 0.18. Examples of shared units are reported in Figure 3 (sce Table for typical ensemble sizes in our setting).
|E=uEe'] As an example, for two ensembles composed of 50 units with a substantial
When the sparsity S of representation is low, ensembles are typically ill-defined as too many neurons are significantly active simultaneously and the notion of active unit tends to blur. To set a threshold, we will consider values of S below 0.5 as non-sparse, and in these cases, we do not define ensembles out of the representation.
4 Results
In this section, we describe our findings for the three models introduced, on the MNIST, FASHION SVII and CIFAR10 datasets. In particular, we focus on the properties of representations obtained within the FF model, i.c., a model trained with Forward-Forward on its natural goodness objective. Such properties, such as the emergence of category-specifie ensembles and the presence of shared units across them, establish link between neural networks trained with the Forward-Forward algorithm and biological cortical networks described in Isubeccton 2.2
4.1 Classification accuracy
Before we present the main results this work, we evaluate the performances of our models on the classi- fication tasks at hand. TTable contains results terms of test set classification accuracy for all models we employed FF, BP BP/FF and BP on MNIST, FASHIONNNIST, SVII and CIFAR10. While some of these accuracy values are far from the state-of-the-art. (i.e., respectively, 0.997 (Cireşan et al., 2010), 0.931 (Xiao) al. 2017), 0.860 [Pitsios 2017] and approximately 0.7 (Lin et al. 2015), for fully-connected networks), they are a solid ground on which to build our subsequent investigations. Training details and hyperparameters for all models are reported in Appendix C
Table 1: Test-set classification accuracy for the models considered in our investigation. Results expressed as meon + sld. dev. over 10 runs with independent randomised weight initialisation.
Dataset
FF
BP BP/FF
BP
MNIST
0.94 + 0.008
0.969 + 0.001
0.982 + 0.001
MNIST
0.849 + 0.002
0.877 + 0.002
0.892 + 0.004
SVHN
0.716 + 0.002
0.799
+ 0.004
0.793 + 0.145
CIFARLO
0.484 = 0.004
0.521 + 0.006
0.564 + 0.004
4.2 Forward- Forward elicits sparse neuronal ensembles
The FF and BP/FF models based on the original Forward-Forwand network architccture, and trained
according to the goodness objective dsubsection 3.2 and subsection 3.3) exhibit typically high sparsity
levels in their representations, in clear contrast with BP (see) Figure and Table 2h. While sparsity does not.

---

Published in Transactions on Machine Learning Research (03/2025)
Layer
Layer 2
Layer 3
FF
FF
FF
1.0
1.0
BP/FF BP
BP/FF
BP/FF
BP
0.8
0.8
0.8
Sparsity
0.6
0.6
0.6
0.4
0.4
0.2
0.2
0.0 0123456785 0123456789 0123456789
0.0 0123456789 0123456789 0123456789
0.0
0123456789
0123456789
0123456789
Categories
Categories
Categories
Figure 2: Sparsity of sapegory representations. We report the sparsity of representations computed as described in subsection 3.5 for the three models FF, BP BP/FF and BP on the MNIST dataset. Sparsily values are the average over 10 runs.
spontaneously arise in BP, it can be enforced by means of f regularisation of the activations (Gorriaddis 2019h. An analysis of this setting is presented in Appendix
Table 2: Average sparsity for all combinations of model, dalasel and layer, according to the definition given in ssuseection 3.5) Results are expressed as mean = std. dev. computed over 10 runs with independent random weights initialisation.
Model
Layer
MNIST
MASHI
SVIIN
CIFAR10
1
0.922 + 0.001
0.85 + 0.002
0.83 + 0.001
0.77 + 0.001
FF
2
0.813 + 0.019
0.605 = 0.015
0.706 + 0.001
0.728 + 0.002
3
0.618 + 0.074
0.628 = 0.013
0.489 + 0.004
0.566 + 0.002
1
0.895 + 0.005
0.81 0.007
0.783 0.003
0.753 + 0.004
BP/FF
2
0.747 0.013
0.851 = 0.007
0.95 = 0.003
0.932 = 0.003
0.131 + 0.011 3
0.065 + 0.009
0.133 + 0.011
0.135 + 0.009
1
0.315
0.003
0.352 0.003
0.47 0.02
0.478 + 0.016
BP
2
0.193 + 0.004
0.241 + 0.005
0.524 + 0.212
0.3 + 0.18
3
0.225 + 0.006
0.248 + 0.006
0.232 + 0.106
0.164 + 0.006
When the sparsity level is sufficiently high (S > 0.5) we are able to identify small sets of neurons (ensembles) that consistently co-activate across all the samples of the same class, similar to what has been observed in cortical representations (Yuste 2015: Miller et al. 2014 Harris
2005
Figure (Panels B, C) shows an example of average neuron activations for each class Layer of the FF model trained on MNIST, and showcases the emergence of sparse, category-speeifi, ensembles (sce the Appendix E for similar visualisation for Layers 2 and 3 of the same model and Appendix F for Layer in all the models). These representations typically activate only small fraction of units: ensembles consisting of just few percent of the neurons in layer are commonly observed, whether working with simpler datascts (e.g., MNIST, FASHIONMNIST) or more complex ones (SVII, CIFAR10), with a slight tendency of the FF model to create larger ensembles in the latter case Table 3).
Overall, these findings show that networks trained with the Forward-Forwer Forward objective produce highly sparse representations, characterised by ensembles, f.e., small groups of neurons with category-specins activation patterns.
8

---

Published in Transactions on Machine Learning Research (03/2025)
Table 3: Average fraction of units taking part in ensembles, for all combinations of dataset and layers, in the FF and BP BP/FF models. Ensemble sizes averaged across all categories, divided by the number of neurons in a layer, and then expressed in %. Ensembles are defined according to the LOO method presented in subsection 3.5) Results expressed as mean + std. dev. In the third layer of BP BF/FF, as well as in BP, the representation is non-sparse. With (*) we marked the condition in which ~ 2.6% of the units have a p-value larger than 0.05, see Appendix D for details.
Model
Layer
MNIST
FASHIONMNIST
SVHN
CIFAR10
1
3.69 + 0.09
5.02 + 0.14
10.3 + 0.15
16.08 = 0.09
FF
2
5.31 + 0.35
18.46 + 0.66
21.28 0.23
21.2 + 0.3
3
1.36 + 0.36
20.59 + 0.63
4.48 + 0.52
4.86 + 0.51
1
8.58 + 0.23
13:24 + 0.31
15.07 = 0.16
13.3 0.13
/P/FF
2
13.18 + 0.67
8.45 + 0.47
5.08 + 0.19
5.55 + 0.28(+)
3
4.3 Visually similar classes can elicit ensembles with shared neurons
Drawing parallel with a phenomenon observed in Neuroscience Yoshida & Ohki, 2020 related categorics can be expected to share units of their ensembles. This is indeed what we observe, as shown in Figure Results are reported for FASHIONMNIST, where different classes of clothes or shoes may contain common share of visual features. In this regard, observe clear tendency to share units between similar classes e.g., across representations of pullover, coat and shirt.
In the following section, we provide evidence that a unit can be shared across two ensembles even one of these refers to an unseen category (i.e., excluded from the training set but whose representation, extracted at test time, generates an ensemble), as we show in Figure 4 (Panel C).
These findings indicate that ensembles relative to visually related classes can partially overlap.
4.4 Representations of unseen categories can elicit well-defined ensembles
We investigate the ability of a trained FF model to respond to unseen categories with a coherent activation pattern which is typical of the ensembles we found on the categories seen at training time. To this end, we repeatedly train FF on FASHIONMNIST, removing resentation of the missing category, and verify if ensemble is formed. We find that in all the ten cases, this is indeed the case, and the new ensemble share the same characteristics of the ones emerging for seen categories, apparently with the only exception of lower average activation of their constituent units (see Figure 4 for one example, and the Appendix G for a more detailed account).
category at time. Then,
extract the rep-
In several cases, we also find that the ensembles of unseon categories share units with the ensembles of seen calegories, when endowed with similar visual features (Figure 4 Panel C). A more extensive exploration of these cases is also reported in the Appendix G
These results show that ensemble- like structures can arise in a zero-shot setting on data categories held out of the training set.
4.5 Distribution of excitatory and inhibitory connections
As we observed in subsection 4.2 FF and BP/FF have comparable sparsity levels and, when they are defined, the ensembles have comparable sizes. A more fine-grained inspection of the representations learned by different models al different. layers can be found in Appendix J] The presence of sparse ensembles suggests that a strong inhibition mechanism is at work, leaving only a few neurons active for each data sample. Inhibition in these architectures the result of an interplay between the sign and magnitude of the weights and the those of the biases. Therefore it is natural to wonder whether FF and BP FFFFFF are similar also in this interplay between weights and biases. We find that the answer is no: the FF and BP BPFFF
9

---

Published in Transactions on Machine Learning Research (03/2025)
Layer
A
Coat
B T-shirt
1.0
0.07
Trouser
1.0
Pullover
1.0
0.16
Dress
1.0
Coat
0.16
1.0
0.05
Layer 1
Sandal
Pullover
Shirt
0.07
0.1
0.05
1.0
Sneaker
1.0
0
Bag
Ankle boot
1.0
T-shirt
Trouser
Pullover
Dress
Coat
Sandal
Shirt
Sneaker
Bag
Ankle boot
Figure 3: Visually similar classes in FASHIONMNIST elicit ensembles with shared neurons.
Panel A The ensembles elicited in the first hidden layer of FF by two example inputs. Red circles indicate the active units which are shared between the two categories.
Panel B Element j, of the matrix indicates how many units are shared bet ween the ensembles of category
and calegory j (normalised by the ensemble sizes), by using the Jaccard similarity index: J(E',8) The results are referred a single training run.
(g'ng)
A
B
C
0.20
Active unit
Tolket
Activation value
Trouser
0.15
Pallower
Classes
0.10
Short
0.05
Benokor
Bog
Neuron 1000
Onees
Dan
Sandea
shart
Neurons
Figure 4: The representations of an unseen category form an ensemble in FF trained on FASHION Panel A Activation patterns in response the different categorics the first hidden layer. The unseen calegory (Sandal), surrounded by red lines, produces relatively weaker but well-defined ensemble-like activation pattern.
Panel B Activation value of each neuron, averaged all images of the unseen category. Neuron index the x axis; average activation on the axis. Blue dots indicate units that are considered active according to the method described in subsection 3.5
Panel C Ensembles of unseen categories can share units with the ensembles the other categories. Element j.j of the matrix indicates how many units are shared between the ensembles of category and category j: gi ng The results are referred single training run.
10

---

Published in Transactions on Machine Learning Research (03/2025)
are indeed two profoundly different models that create sparse representations and ensembles with different mechanisms.
To show this, we consider for each neuron i in a layer with width n the fraction of its positive weights w.r.t. the total number of its input connections (e, in the following), and its bias 8,. From these neuron-level quantities we construct. their layer averages: e= = , E at and B = : E, B, The neuron's weights are strongly imbalanced towards inhibition when o = and, viceversa, strongly imbalanced towards excitation when 0. I; when ot 2 0.5 we will say that the neuron's weights are almost perfectly balanced; similar considerations hold for the biases, where a large and negative 3 means strong inhibition for the i-th unit.
Focusing the second hidden layer, observe macroscopic differences in the empirical distribution of o, among the three models (see Figure 5), with 1) dominance of positive weights in the case of FF, 2) bimodal distribution of o, in BP/FF with two populations of imbalanced units in opposite directions, and 3) unimodal and approximately balanced distribution for all the neurons in the BP model.
In FF, we find that the bias distribution is strongly imbalanced towards inhibition with a mean + std. dev, value of 1.334. On the contrary, BP/FF and and BP show a substantial balance (-0.017=00015 and 0.003 + 0.018, respectively). Therefore, although the weights of the FF model appear imbalanced towards excitation, the average bias is very large and negative, and this might explain why, for this model, we observe such high sparsity values. Conversely, the BP/FF model has substantially zero bias (within very small fluctuations), therefore the inhibitory mechanism at work here does not rely upon negative bias, but on the weights' configuration. From these results, we conclude that not only the training objective (i.e., goodness- based vs. categorical cross-entropy minimisation), but the specific training protocols (FF vs FFFFF) are determinant in shaping different interplay between excitation and inhibition, even when similar sparsity levels are observed.
0.45
FF
0.33
BP/FF
0.3
BP
Freguency
0.25
0.15
6.1
0.05
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0"
Figure 5: Distribution of o, in Layer 2 (MNIST dataset). In FF, the distribution is imbalanced, with most of the population of neurons having = 75% of excitatory weights. In BP/FF the distribution bimodal with two populations of neurons: one inmbalanced lowards excitation (right mode) and the other towards inhibition (left mode). The BP model is almost perfectly balanced between excitation and inhibition.
We report the summary statistics et, 3 for all the combinations of models, layers and datasets in Table 4 and Table 5 Tn all settings, we observe a similar picture, with the inhibition mechanisms dominated by negative biases in FF and negative weights in BP/FF.
In summary, these findings illustrate that although FF and BP/FF learn similar representations, they achieve sparsity by relying on fundamentally different neuron inhibition strategies.
11

---

Published in Transactions on Machine Learning Research (03/2025)
Table 4: Average fraction of positive weights (e*) for each combination of models, datasets and layers.
Results are expressed as mean + std. dev, over a single training run.
Dataset
Layer
FF
BP/FF
BP
1
0.661 + 0.031
0.534 + 0.028
0.486 + 0.022
MNIST
2
0.688 = 0.014
0.445 = 0.099
0.523 = 0.019
3
0.535 = 0.071
0.52 = 0.021
1
0.457 0.099
0.509 = 0.018
0.491 + 0.021
MNISIO
2
0.602 + 0.074
0.423 + 0.065
0.52 = 0.02
3
0.416 + 0.191
0.433 + 0.019
0.52 + 0.02
1
0.487 + 0.062
0.499 + 0.009
0.499 = 0.006
SVIN
2
0.521 + 0.033
0.427 0.042
0.504 + 0.011
3
0.583 + 0.096
0.422 + 0.031
0.522 + 0.023
1
0.493 + 0.027
0.502 0.011
0.501 + 0.009
CIFAR10
2
0.489 + 0.044
0.43 + 0.038
0.513 + 0.012
3
0.612 = 0.046
0.408 + 0.034
0.523 = 0.02
Table 5: Average bias (3) for each combination of models, datasets and layers. Results expressed as mean + std. dev. over a single training run.
Dataset
Layer
FF
BP/FF
BP
-1.57 + 2.029
-0.007 + 0.019
0.001 0.021
MNIST
2
-1.66 + 1.534
-0.017
0.003 0.018
3
-0.313 + 0.151
-0.071
0.002 0.018
1
0.694 + 0.706
= 0.007 0.018
0.003 = 0.021
MASHIONMNIST
2
1.858 + 0.489
0.011 0.018
0.003 = 0.019
3
11.37 + 1.213
0.028 + 0.009
0.003 0.019
1
0.662 + 0.219
0.033 + 0.012
-0.004 + 0.013
SVHN
2
0.959 + 0.089
-..05 = 0.011 0.011
0.003 + 0.012
3
0.962 = 0.271
+ 0.016
0.003 = 0.011
1
-0.402 + 0.1
+ 0002200.00
-0.001 + 0.016
CIFAR10
2
0.703 + 0.132
-0.006 0.01
0.003 + 0.016
3
0.873 + 0.076
-0.004 0.013
0.002 = 0.013
5 Discussion and conclusions
In many brain circuits, only small fraction of neurons is active under specific sensory or behavioural conditions. It well established, indeed, that both sensory cortex and the hippocampus exhibit markedly sparse activity. The exact percentages can vary with species, types of stimuli, brain state, measurement technique, and brain area. For example, in rodent primary visual cortex, only about 10 10-97 20% of excitatory neurons respond significantly to visual stimuli of varied complexity, from simple ones, such oriented gratings to complex ones, like movies (Bonin et al. 2011 Miller et al. 2014). In the auditory cortex, the fraction of neurons activated by particular sound can be as low as 5 - 15% (Hromádka et al. 2008 Barth & Poulet 2012). Lastly, hippocampal recordings during animal exploration show that only 20 100-400 40% of neurons become active in different environments (Leutgeb et al.) 2004).. This sparse activity is often organised into ensembles, small groups of neurons that consistently co-activate response to sensory stimuli or during spontaneous activity, and they have boen proposed as functional building blocks of sensory processing,
memory, and behaviour (Miller et al. 2014
Hebb
2005
Harris 2005
György 2010 Hopfield 1982 Carrillo
12

---

Published in Transactions on Machine Learning Research (03/2025)
Reid et al. 2019
Carrillo-Reit & Yuste 2020 Yuste et al. 2024 We discussed the notion of ensemble in
subsection 2.2
The main finding of our work is that artificial neural networks trained with the Forward-Forward algorithm can elicit sparse representations that share intriguing analogies with the neuronal ensembles found in real brains. We started our investigation by collecting and analysing representations from FF networks trained on MNIST, FASHION.MNIST, SVII,, and CIFAR10 and defined, separately for each category, subsets of units (artificial ensembles) that prominently and consistently activated in response to data in such category (sub- section 4.2). These category ensembles turn out to be composed of few active units, which is consistent with the aforementioned experimental findings on the sensory cortex and the hippocampus. Fur- thermore, when image categories are characterised by a certain degree of visual similarity, the corresponding ensembles often share one or more units FFguure 3) Panel B). The fact that single units can appear in multiple ensembles for different categories paralleis the idea of mixed selectivity neurons. Mixed selectivity refers to neurons that respond in complex ways to combinations of charactoristics or stimuli, thus increasing the dimensionality of population activity and allowing for flexible behaviour (Rigotti et al. 2013 Fusi et al., 2016). Neurons that participate in more than one ensemble can be conceptually viewed as exhibiting mixed selectivity, since they contribute to multiple learned representations simultancously.
We then tested the ability of trained FF models to cope with new data, and observed that activations in response to an unseen input category form, in many cases, a new ensemble with sparsity characteristics similar to those formed for other classes during training (Figure 4 Panels A and B). We also noticed that the ensembles of unseen categories often show a high level of similarity and integration with the ensembles of the categories of data encountered during training, realised through the sharing of units (see Figure Panel C and also the results in Appeedix GJ. Beyond this qualitative similarity with the ensembles formed in response to seen categories, we showed, by training linear probes on representations, that the information content in these activation patterns is almost as high as that of seen categories Table 7 While the BP model typically achieves higher decoding performance in this task, it docs so by relying on a dense coding scheme. These findings suggest that the ensembles generated by FF in response to new data can support zero-shot classification tasks, which is particularly relevant in view of the importance of zero/ srro/few-shot learning in human and animal cognitive performance (Lake et al. 2015).
While absent in the BP model, the existence of ensembles composed of a few unils is not unique to FF. It was indeed observed also in BP /PFFF [subsection 4.2), where the ensembles turned out to be of comparable size. This similarity in how representations are organised in FF and BP/FF is also backed by quantitative analysis with established representation similarity metrics Appendix J However, despite their similarity at representation level, the FF and BP/FF models are profoundly diverse, as demonstrated by the different interplay between inhibition and excitation in these models (see subsection 4.5p. We observed in this regard that the excitatory/mnhiiitorr (E/I) balance play key role in the stability of cortical networks and in brain
dynamics Gerssnnr & Kistler 2002 Deco et al. 2014
The sparsity of representations has computational benefits in sensory processing. Olshausen & Field (2004)
emphasised that sparsity may be the optimal encoding strategy for neural networks because it is energy efficient. This is especially important for biological neural networks, which operate under metabolic con- straints. Sparsity also increases the memory-storage capacity and cases readout at subsequent processing layers. [Babadi & Sompolinsky (2014) showed that sparse and expansive coding (i.e., from a lower dimen- sional sensory input space to higher dimensional neural representation) reduced the intra-stimulus vari- ability, maximised the inter-stimulus variability, and allowed optimal and efficient readout of downstream neurons. This is the reason why sparse and expansive transformations are widespread in biology, e.g., in rodents MMmbbeers et al.) 1996) or flies (Turner et al. 2008
The limitations of the present work could be addressed by applying similar analyses to more complex datasets and a variety of tasks. To scale to a more challenging dataset may require the replacement of a fully connected network with more suitable architectures trainable with the forward-Forwal protocol, e.g., the CNNs recently introduced in Paaacchristoooouuuu et al. (2024) and Sun et al. (2025). This approach could provide insights into how data characteristies influence sparsity levels and the resulting ensemble-
Limitations
13

---

Published in Transactions on Machine Learning Research (03/2025)
like structures. Morcover, as highlighted in Yang (0233 the choice of hyperparameters potentially affects representation sparsity, as well as the goodness function of choice.
Future work Many questions are left open and will be addressed in future works. A closer inspection of the activation patterus within each category will be necessary to test for the co-existence of multiple patterns, with one dominant and possibly many subdominant patterns. We have not yet investigated this microstructure (Galán 2008), leaving it to possible extensions of this work. Based on experimental results indicating the presence of small calegory-specific ensembles, a promising avenue for future research in this field encompasses exploring model compression through pruning (Blalock et al. 2020), with the design of new strategics based on the relevance of the ensembles, as well as investigating the dynamic evolution of ensemble size and organisation throughout the training process. An especially intriguing perspective, inspired by recent. work on optimal sparsity in hippocampal memory models (Shah et al., 2025), suggests that sparsity levels are dynamic variables, rather than fixed properties of the network, that can be tuned the compressiiility of sensory imputs to reach optimal performance. Such inquiries hold the potential shed light on the formation, evolution, interactions, and persistence or replacement of ensembles in artificial neural networks. Lastly, from the analysis of representations learned by the Forward-Forwald algorithm, our work suggests novel directions for comparing artificial and biological represenlations (Barrelt et al. 2019 Schrimpf et al.] (018)) particularly for biologically plausible learning algorithms by leveraging a well-sstablised concopt in Neuroscience: that of neuronal ensembles.
Acknowledgements
We thank Alex Rodriguez for the idea of testing the models on categories unseen during training. We also acknowledge the AREA Science Park supercomputing platform ORFEO and thank the staff of the Laboratory of Data Engineering for technical support.
AA, AC and LB were supported by the project "Supporto alla diagnosi di malattie rare tramite l'intelligenza artificiale" CUP: F53C2200177000 and "Valutazione automatica delle immagini diagnostiche tramite l'intellignnza artificiale", CUP: F53C22001180002 AA and AC were supported by the European Union NexteeneraUo within the project PNRR "PRPOCERIC" IR0000028- Mission 4 Component 2 Invest- ment 3.1 Action 3.1.1.
References
Krizhevsky Alex. Learning multiple layers of features from liny images. /ddu//rii//eaarinnngffeettt htta///aaaainn-etuuue---00-----0 pdf, 2009.
htlps://wwer es. loronlo.
Baktash Babadi and Haim Sompolinsky. Sparseness and expansion in sensory representations. Neuron, 83
(5):1213-13) 1226, 2014.
David GT Barrett, Ari S Morcos, and Jakob H Macke. Analyzing biological and artificial neural networks: challenges with opportunities for synergy? Current opinion in neurobiology, 55:55 64, 2019.
Alison L Barth and James FA Poulet. Experimental evidence for sparse firing in the neocortex. Trends in neurosciences, 35:3):345 355, 2012.
Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? Proceedings of machine learning and systems, 2:129 146, 2020.
Vincent Bonin, Mark H Histed, Sergey Yurgenson, and R Clay Reid. Local diversity and fine-scale organiz- alion of receptive fields in mouse visual cortex. Journal of Neuroscience, 31(50):18506 18521, 2011.
Richard Boyee, Robin F Dard, and Rosa Cossart. Cortical neuronal asscmblics coordinate with cog microstate dynamics during resting wakefulness. Cell Reports, 42(2), 2023.
Luis Carrillo-Reid and Vladimir Calderon. Conceptual framework for neuronal ensemble identification and manipulation related to behavior using calcium imaging. Neurophotonics, 9, 2022.
14

---

Published in Transactions on Machine Learning Research (03/2025)
Luis Marrill-RReid and Rafael Yuste. Playing the piano with the cortex: role of neuronal ensembles and pattern completion in perception and behavior. Curvent opinion in neurobiology, 64:89-90 95, 2020.
Luis Carrillo-Reid, Shuting Han, Weijian Yang, Alejandro Akrouh, and Rafacl Yuste. Controlling visually guided behavior by holographic recalling of cortical ensembles. Cell, 178(2):447-457, 2019.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. simple framework for contrastive learning of visual representations. In Proceedings of the International Conference on Machine Learning, 2020.
Dan Claudiu Cireşan, Ucli Meier, Luca Maria Gambardella, and Jürgen Schmidhuber. Deep, big, simple neural nets for handwritten digit recognition. Neural Compulation, 22((2):3207 32012)3307-3220, 12 2010.
Gustavo Deco, Adrián Ponce-Alvarez, Patric Hagmann, Gian Luca Romani, Dante Mantini, and Maurizio inhibitiion ratio impacts the whole brain dynamics. Journal of Newros-
Corbetta. How local cience, 34(23)77886-7895 2014.
Eizaburo Doi and Michael Lewicki. Sparse coding of natural images using an overcomplete set of limited capacity units. Advances in Newral Information Processing Systems, 17, 2004.
Christophe Dupre and Rafael Yuste. Non-overlapping neural networks in hydra vulgaris. Current Biology, 27(8):1085 1097, 2017.
David J Field. What is the goal of sensory coding? Newral Computation, 6(4):559-601. 1994.
Stefano Fusi, Earl K Miller, and Mattia Rigotti. Why neurons mix: high dimensionality for higher cognition. Current opinion in neurobiology, 37:66 74, 2016.
Roberto Galán. On how network architecture determines the dominant patterns of spontancous neural activity. PloS one, 3(5)xe1148, 2008
Georgios Georgiadis. Accelerating convolutional neural networks via activation map compressson. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7085-7095, 2019.
Wulfram Gerstner and Werner M Kistler. Spiking neuron models: Single neurons, populations, plasticity. Cambridge university press, 2002.
Buzsáki György. Neural syntax: cell assemblies, synapsembles, and readers. Newron, 68(3):362 -85,65555,,,. 2010.
Kenneth D Harris. Neural signatures of cell assembly organization. Nature reviews newroscience, 6(5):
399-407, 2005.
Donald Olding Hebb. The organization of behavior: neuropsychological theory. Psychology press, 2005.
Geoffrey Hinton. The forward-forwar algorithm: Some preliminary investigations. arXiv preprint
arXiv:2212. 13345, 2022.
John J Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proceedings of the national academy of sciences, 79(8):2554-255) 25558, 1982.
Patrik O Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of machine learning research, 5(9), 2004.
Tomáš Hromádka, Michael R DeWoese, and Anthony M Zador. Sparse representation of sounds in the unanesthetized auditory cortex. PLoS biology, 6(1):e16, 2008.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Proceedings of the International Conference on Learning Representations, 2017.
Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. Tn International conference machine learning, pp. 3519-3529. PMLR, 2019.
15

---

Published in Transactions on Machine Learning Research (03/2025)
Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332 1338, 2015.
Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-basse learning applied to document recognition. Proceedings of the IEFE, 86(11):2278 22222224,,, 1998. doi: 10.1109/5..0 10.1109/5.726791.
Stefan Leutgeb, Jill K Leutgeb, Alessandro Treves, May-Briut Moser, and Edvard Moser. Distinct ensemble
codes in hippocampal areas ca3 and cal. Science, 305(5688))1295--2288 2004.
Zhouhan Lin, Roland Memisevic, and Kishore Konda. How far can we go without convolution: Improving fully-connected networks. arXiv preprint arXiv:1511.000 1..050000 2015.
Jing Liu and Scott C Baraban. Network properties revealed during multi-scale calcium imaging of seizure activity in zebrafish. Eneuro, 6(1), 2019.
Jac-eun Kang Miller, Inbal Ayzenshtat, Luis Carrillo-Reid, and Rafael Yuste. Visual stimuli recruit intrinsic- ally generated cortical ensembles. Proceedings of the National Academy of Sciences, 111((8):E4053 E4061, 2014.
Peter Mombaerts, Fan Wang, Catherine Dulac, Steve K Chao, Adriana Nemes, Monica Mendelsohn, James Edmondson, and Richard Axel. Visualizing an olfactory sensory map. Cell, 87(4))675 686, 1996.
Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y Ng. Reading digits natural images with unsupervised feature learning. Deep Learning and Unsupervised Feature Learning Workshop, NewrIPS, 2011.
Bruno A Olshausen and David J Field. Sparse coding of sensory inputs. Current Opinion in Neurobiology, 14(4):481 487, 2004.
Adam M Packer, Lloyd E Russell, Henry WP Dalgleish, and Michael Häusser. Simultaneous all-optical manipulation and recording of neural circuit activity with cellular resolution in vivo. Nature methods, 12 (2):140 12):140-166, 2015.
Andreas Papachristodouulo, Christos Kyrkou, Stelios Timotheou, and Theocharis Theocharides. Convo- lutional channel- wise competitive learning for the harward-fowardd algorithm. Proceedings of the AAAI Conference on Artificial Intelligence, 38(13) 13):14536--1544, 2024. doi: 10.1609/ vtt.:/aaai.v.i...222.2 v38.13.2236....
URL
bttts::/ojs... www.:/i..aaaa...... oataai.org//a... wwwgg/indx....e
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library, 2019.
Stamatis Pitsios. SVHN number recognition using deep learning. 2017.
Jesús E. Pérez-Ortega, Tzitzitlini Glarca, and Rafael Yuste. Long-term stability of cortical en- sembles. Elife, 10:c64449, 2021.
Maithra Raghu, Justin Gilmer, Jason Yosinski, and Jascha Sohl-Dickstein. Sveca: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Advances in newral information processing systems, 30, 2017.
Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. A deep learning framework for neuroscience. Nature Nearoscience, 22(11):1661-1770 2019.
Mattia Rigotti, Omri Barak, Mclissa R Warden, Xiao-Jing Wang, Nathaniel D Daw, Earl Miller, and Stefano Fusi. The importance of mixed selectivity in complex cognitive tasks. Nature, 497(7451):5855-90 2013.
16

---

Published in Transactions on Machine Learning Research (03/2025)
David E Rumelhart, Geoffrey E Hinton, and Ronald Williams. Learning representations by back- propagating errors. Nature, 323(688)):333 (33008))333-36, 1986.
Martin Schrimpf, Jonas Kubilius, Ha Hong, Najib J. Majaj, Rishi Rajalingham, Elias B. Issa, Kohitij Kar, Pouya Bashivan, Jonathan Prescott-Roy, Franziska Geiger, Kailyn Schmidt, Daniel L. K. Yamins, and James J. DiCarlo. Brain-score: Which artificial neural network for object rocognition is most brain-like? bioRaiv preprinl, 2018. URL https: http///wwwwbxxxx.ooooooo/ biorxiv www.wwwwwwoooooooiio.oe.oeeeeoee
ortg//onttet//0..011//00000
Abhishek Shah, René Hen, Attila Losonezy, and Stefano Fusi. Optimal sparsity in autoencoder memory models of the hippocampus. bioRriv, pp. 2025-022 01, 2025.
David G. Stork. Is backpropagation biologically plausible? In Proceedings of the International Joinl Con- ference on Neural Networks, 1989.
Liang Sun, Yang Zhang, Jiajun Wen, Linlin Shen, Weicheng Xie, and Weizhao He. Deeperforward: Enhanced forward-forward training for deeper and better performance. In The Thirteenth International Conference on Learning Represenlations, 2025. URL
Gábor J Székely, Maria L Rizzo, and Nail K Bakirov. Mcasuring and testing dependence by correlation of distances. The Annals of Slatistics, pp. 2769 2794, 2007.
Erik B. Terres-Escudero, Javier Del Ser, and Pablo García Bringas. On the improvement generalization and stability of forward-only learning via neural polarization. In Europeon Conference on Artificial Intelligence, 2024. URL stttp:::/api..smmtttccccooo
TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library. https://githut:
com/pytorchhvisiinn 2016.
Glenn C Turner, Maxim Bazhenov, and Gilles Laurent. Olfactory representations by drosophila mushroom body neurons. Journal of neurophysiology, 99(2):734 746, 2008.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mnist: novel image dataset for benchmarking machine learning algorithms, 2017.
Yukun Yang. A theory for the sparsity emerged in the forward forward algorithm. aiv preprint arX1v:2311.05667 2023.
Takashi Yoshida and Kenichi Ohki. Natural images are reliably represented by sparse and variable popula- tions of neurons in visual cortex. Nature communications, 11(1):872, 2020.
Rafael Yuste. From the neuron doctrine to neural networks. Nature Reviews Neuroscience, 16(8):487-497, 2015.
Rafael Yuste, Rosa Cossart, and Emre Yaksi. Neuronal ensembles: Building blocks of neural circuits. Neuron, 2024.
17

---

Published in Transactions on Machine Learning Research (03/2025)
A Computational resources
Training and subsequent experiments were conducted on an NVIDIA DGX A100 system. The system is equipped with 8 NVIDIA A100 GPUs, interconnected by NVLink technology, two AMD EPYC 7742 64-core CPUs, 1TB of RAM, and a 3TB NVME storage configured in RAID-0. Each GPU is equipped with 6912 CUDA corcs, 432 Tensor cores and 40 GB of high-bandwidth memory.
B
Data
The MNIST dataset consists of pictures of handwritten Arabic numerals, from to 9, cach represented grayscale image of size 28 28. FASHIONMNIST has been designed drop-in replacement to MNIST, offering more challenging classification task. It consists of ten classes of clothing items, still represented grayscale images with resolution of 28 x 28. Both datasets provide 60000 training and 10000 test images, balanced in terms of per-class numerosity.
SVHN contains coloured images of digits from house numbers, captured by Google StreetView. The images are composed of 32 32 RGB-encoded pixels. This dataset is slightly larger than the previous two, as it contains 73257 data-points in the training set and 26032 in the test set.
The SVHN images have been cropped in order to center the digit of interest within the frame. However, the presence of adjacent digits and other distracting elements, that have been kept within the images, introduces additional layer of complexity when compared to MNIST and PASHION where the subjects are prominently displayed against uniform black background. The CIFAR10 consists of 60000 coloured natural images categorised in 10 balanced classes. The dataset. is split in 50000 training images and 10000 test images. Each image, like SVIIN has a resolution of 32 x 32 for each channel. Compared to previous datasets, this is the most challenging one for a fully connected network. The dataset split employed is provided by the TORCHVISION framework (TorchVision maintainers and contributors 2016
C Training details
All models (FF, BP/FF and BP), on all datasets (MNIST, FASHIONMNIST, SVHN and CIFAR10), have been optimised using Adam (Kingma & Ba 2017) with 31 0.9 and B2 0.999, implemented in PYTORCH PPaszke et al. 2019). A hyperparameter search has been performed to achieve sufficient accuracy for each model across all datasets. Every model was trained using batches of size 1024.
Table Hyperparameters selected to train our models.
Model
MNIST
FASHIONMNIST
SVHn
CIFAR10
Epochs
1200
100
1000
1000
FF
Learning rate
0.01
0.01
0.0001
0.0001
Epochs
300
300
200
200
BP BP/FF
Learning rate
0.0001
0.0001
0.0001
0.0001
Epochs
80
80
80
BP
Learning rate
0.0001
0.0001
0.0001
0.0001
D Statistical test of ensemble assignment
To assign a unit i to class-ensemble c, we compare its average activation on correct responses Tी, against its leave-one-out average LOO (as described in subsection 3.5). We define
6 = Fi 2 LOO
If d,c 0, the unit is assigned to the ensemble for class To assess the statistical significance of each assignment, compute y-value by building an empirical null distribution. Specifically, shuffle each
18

---

Published in Transactions on Machine Learning Research (03/2025)
relevant representation matrix row-wise 200 times, recalculate ogand for each shuffie, and then define the p-value as the fraction of these shuffled values exceeding the observed oc,-- We applied this test to every unit assigned to an ensemble across all motel/dataset/ (alataste/layerr//un layer/run combinations summarised in Table 3 totaling z 457, 457,000 units. Of these, only 500 (about in 1000) had > 0.05. Most of those higher-p units occurred in Layer BP/FF on CIFAR10, representing Rs 2.6% of the assigned units in that specific configuration.
E Activation patterns in deeper layers
In subsection 4.2 we claimed that in FF and BP/FF the images of a given category activate consistently a small set of units that we named ensembles, that share similaritics to what obscrved sensory cortices. We reported in Figure I the activation map for Layer (the first hidden layer) of FF trained on the MNIST dataset, and observed that very sparse ensembles emerge. In this section show, in similar fashion, the representations for Layers 2 and 3 (Figure 6 and Figure respectively). We find high sparsity also for deeper layers of this specific network; a qualitatively similar conclusion is reached for FF models trained on MNIST,, SVHN and CIFAR10. In BP /FF models similar sparsity levels are observed, with the exception of the last layer that turns out to be non-sparse in all the datasets considered (see Table 30.
Layer 2
Classes
Neurons
Figure 6: Activation patterns in Multi-Layer Perceptron trained with the Forward-Forware algorithm, the MNIST dataset. The image represents the activation map for neurons in Layer 2 for all images, grouped by class. A blue dot in position (x, y) indicates that neuron r is activated by input colour scale represents the intensity of such activation (incorrectly classified samples have been removed). Horizontal bands mark different categories; dark blue vertical lines mark active neurons. Each input category activates consistently spocific sets of neurons (ensemble). The sparsity measured according with the definition provided in subsect ion 3.5 is 0.84.
19

---

Published in Transactions on Machine Learning Research (03/2025)
Layer 3
Classes
Neurons
Figure 7: Activation reported as in Figure 6 for Layer 3. Notice that there are only few units that activates significantly and does not play a role in discriminating categories. The role of this layer, this experiment soems, not related the classification task. Despite the low number of active units, the sparsity level of the representation is lower than that of Layer (S 0.67), due the noise of the inactive units.
F
Activation patterns in different models
In Figure PPanel C) we show the activation patterns Layer 1 of FF trained on MNIST. For the purpose of qualitative comparison, we show here analogous patterns for BP/FF and BP (see Figure
and Figure 9)
Layer 1
Classes
Neurons
Figure 8: Activation pattern in Layer 1 of the BP/FF model trained the MNIST dataset. The sparsity measure is 0.89, comparable with the correspondent first layer of the FF model, reported in Figure Panel
C.
20

---

Published in Transactions on Machine Learning Research (03/2025)
Layer 1
Classes
Neurons
Figure 9: Activation pattern Layer 1 of the BP model trained on the MNIST datasct. The sparsity measure is 0.32 (non-sparse representation), about z of the sparsity level measured in the analogous experiment with FF and BP/FF.
G Further results on representations of unseen categories and their ensembles
We showed in subsection 4.4 that FF model trained on the FASHIONMNIST datasct - deprived of one category can respond at test time to this unseen category with an ensemble (Figure 4).
We report here the results of similar experiments, removing one category at a time. turns out that, in each the ten possible cases (we performed single run for each category), the representations of the unseen category form an ensemble; we show three examples in Figure 10] different from the example shown in Figure 4). It with this situation in mind that we refer to "the ensembles related unseen categories",
Pullover
Coat
Ankle boot
Active unit
Active unit
Active unit
3.20
0.5
Activation value
Activatton value
Activation
0.3
0.3
9.10
2.00
250
420 400
1300
200
120
6.0
1000
4000
600 1002
Nouron
Figure 10: Ensembles elicited by the FF model trained on FASHIONMNIST deprived of one category (we show three examples: Pullover, Coat and Ankle boot). We report for the three categories, the activation value of each neuron in the first hidden layer (Layer 1), averaged on all images of the unseen category. Neuron index on the x axis; average activation on the y axis. Blue dots indicate units that are considered active according to the mothod described in subsccttin 3.5)
When an unseen category forms an ensemble, it generally exhibits a high level of integration with the ensembles associated with the categories encountered during training. This integration implies that it can share common units with ensembles belonging to related categorics. show in infiigre 11 how the ensembles of missing categories (same examples as in Figure 10p integrate by sharing units with the other ensembles.
21

---

Published in Transactions on Machine Learning Research (03/2025)
Tebbir
Teblit
Trouser
16
Trouser
16
Trouser
24
Pollover
Pollover
46
12
Pullower
Sandel
Short
Short
Stot
Sneaker
69
Sneaker
64
Sneaker
127
56
Ackle
boot
bost
Trosser
Patover
Cost
Sanda
Palover
Droos
cast
Sandaa
shirt
Thusse
Da
Figure 11: Shared units between the ensembles of unseen calegories and the ensembles of categories seen during training (stripes delimited by the red lines). The results for Pullover, Coat and Ankle boot are shown.
Overall, these result relates to biological neural networks (Yoshida & Ohki 2020 sembles appear to be the functional building block of brain representations even the absence of known stimuli.
Yuste 2015), where en-
H Performance on unseen categories with linear probes
In this section, we explore the ability of linear probe to discriminate between categories, including those unseen during training, leveraging the models' internal representations. Focusing on the FASHIONMNIST dataset, we trained 10 modols - cach excluding different category for all FF, BP BP/FF, /FF, and BP. A linear classifier was then trained on the training set representations. We note here that for this experiment FF and BP/ BP/FF were provided with the same data as BP, i.e., images without any pixels encoding positive and negative labels. Our findings, reported in Table 7 reveal that the linear probes successfully recover good accuracy levels in almost all cases. This result holds consistently across all categories and models. We also report in Table 8 the performance of linear probes averaged across all categories, seen and unseen, including comparison with baseline models. This shows that the decoding performance of linear probes trained on models in which a category is held out during training is close to the original one (without any held out category).
Table Lincar probe accuracy on the missing category for models trained without that category. Missing category
Model
0
1
2
3
5
6
7
8
FF
0.803
0.901
0.719
0.817
0.714
0.869
0.508
0.876
0.920
0.889
BP BP/FF
0.787
0.939
0.699
0.856
0.742
0.894
0.454
0.836
0.947
0.937
BP
0.863
0.962
0.721
0.933
0.843
0.967
0.632
0.945
0.970
0.957
Table 8: Linear probe accuracy averaged across all the calegories for models trained without one category. The Avg column reports the average across all the ten cases. The Baseline corresponds to the accuracy achieved by the model when all categorics are included during training.
Missing category
Model
0
1
2
3
8
9
Avg
Baseline
FF
0.854
0.857
0.847
0.854
0.849
0.851
0.858
0.862
0.852
BP/FF
0.868
0.869
0.859
0.864
0.863
0.866
0.859
0.856
0.866
0.858
0.864
0.877
BP
0.880
0.890
0.883
0.885
0.883
0.888
0.887
0.887
0.886
0.889
0.886
0.892
22

---

Published in Transactions on Machine Learning Research (03/2025)
Enforcing sparsity in the BP model
In the FF and BP/FF models sparsity emerges without any explicit regularisation or constraint. Instead, if one wanted to promole spanse representations in the BP model, one established way is by means of & regularisation on the activations (Georgiadis) 2019). this section, we present the results obtained by training a BP model using the same lnpperaarameters as those employed the main analysis and & regularisation on the activations, with weight set to 2.5 10-6, We measure the layer-wise sparsity in the MNIST and FASHIONMNIS' datasets, and observe (Table 9) sparsity values comparable and often higher than the ones that spontancously emerge with FF and BP BP/FF FFF, reported in Table 2
Table 9: Sparsity of the BP model with & norm regularisation applied to layer activations.
Model
Layer
MNIST
FASHIONMNIST
0.971
0.955
BP regularised
0.802
0.787
0.813
0.781
However, despite having very high sparsity levels, the reprosentations learned by BP with & regularisation and by unregularised FF display significant differences. Figure 12 reports the Jaccard similarity between MNIST class ensembles (computed using the procedure of subsection 3.5) for the first layer of the regularised BP model (left) and the FF model (right). FF ensembles are highly specific, as they are completely disjoint, while in the case regularised BP there is a non-zero overlap for any pair of classes, regardless of their visual dissimilarity.
1.0
0.02
0.14
0.05
0.04
0.1
0.15
0.08
0.13
0.08
1.0
0.02
1.0
0.12
0.14
0.15
0.12
0.05
0.15
0.14
0.13
1.0
0.14
0.22
1.0
0.21
0.16
0.1
0.16
0.25
0.22
0.32
1.0
9.05
0:14
0.21
1.0
0.07
0.33
0.19
0.24
0.18
1.0
0.15
0.08
0.15
0.13
0.24
0.22
0.08
0.13
0.12
0.18
0.26
0.19
1
2
3
5
9
Figure 12: Jaccard similarity between first-layer ensembles on the MNIST dataset. Left: BP model with & regularisation. Right: FF model.
23

---

Published in Transactions on Machine Learning Research (03/2025)
J Representation similarity
J.1 Model comparison
In this section, we analyze the similarity between representations produced by FF, BP FFFFF and BP. We employ three established representation similarity metrics, namely SVCCA (Raghu et al. 2017). CKA (Korn- blith et al. 2019) and Distance Correlation (dCor) (Székely et al., 2007). We consider training runs with independent weight initialisations each model and compare representations layer-by-layer. The results, reported in Table 10] show that:
CKA and dCor exhibit greater variability than SVCCA across models and layers. We hypothesize that this may be because SVCCA is a linear metric, making it less informative in the presence of nonlinear correlation patterns;
The first layer is almost always the most similar, possibly due to the fact that it is the closest lo the imput, which is shared between models;
Focusing on the second layer, FF and BF/FF are consistently the most similar pair models according to CKA and dCor. The same does not apply for the third layer, which is fact non- sparse BP BP/FF (Table 3).
Table 10:
Representation similarity between models. Results averaged runs with independent
random weight initialisation for each configuration.
FF
BP/FU
FF
BP
BP MP/FFF BP
Dataset
Metric
2
1
2
1
3
MNIST
SVCCA
0.48
0.38
0.45
0.55
0.47
0.52
0.48
0.39
0.45
CKA
0.79
0.53
0.04
0.53
0.49
0.23
0.62
0.43
0.02
dCor
0.85
0.90
0.13
0.61
0.68
0.45
0.70
0.64
0.17
MNISI
SVCCA
0.55
0.35
0.33
0.53
0.36
0.37
0.47
0.37
0.41
CKA
0.60
0.63
0.24
0.51
0.40
0.34
0.60
0.40
0.12
dCor
0.80
0.76
0.10
0.59
0.51
0.50
0.81
0.59
0.11
SVII
SVCCA
0.57
0.55
0.55
0.56
0.53
0.49
0.58
0.50
0.51
CKA
0.47
0.40
0.27
0.32
0.17
0.04
0.72
0.21
0.01
dCor
0.60
0.46
0.09
0.45
0.27
0.13
0.90
0.38
0.03
CIFAR10
SVCCA
0.54
0.53
0.54
0.53
0.50
0.50
0.56
0.46
0.47
CKA
0.46
0.46
0.17
0.37
0.17
0.04
0.65
0.25
0.02
dCor
0.64
0.53
0.06
0.49
0.22
0.07
0.86
0.39
0.06
J.2 Layer comparison
In Figure 13 we report the CKA similarity between different layers of FF models, averaged over 10 inde- pendent training runs. Across all datasets, similarity is noticeably higher for layers and than for layers and 3. The only partial exception FASHIONMNIST, that displays a much narrower gap. These results align with those in Table FASHIONMNIST is the only dataset in which sparsity in the second layer is lower than in the third onc.
K Forward-Forward with L goodness function
We investigated the effect of different choice of goodness function by switching the norm. Con- sequently, adjusted the normalisation for subsequent layers, performed according to the l norm well.
24

---

Published in Transactions on Machine Learning Research (03/2025)
Mnist
FashionMnist
Svhn
Cifar10
1
0.8
1.00
0.33
0.80
0.71
1.00
0.37
0.4
3
0.28
0.33
1.00
0.52
0.71
1.00
0.16
0.32
1.00
0.18
0.37
1.00
2 Layer
2 Layer
2
3
0.0
Layer
Layer
Figure 13: CKA similarity between layers in the FF model, averaged over 10 independent training runs.
We trained 10 instances of FF and of BP /FFFF on MNIST and on FASHION datasets. The hyperpara- meters were set as follows: learning rate: 0,001, epochs: 300, batch size: 1024. With this setup, we train the FF and BP /FFF models on the MNIST and FASHIONMNIST datascts.
Our observations indicate that the accuracy achieved by both the FF and BP BP/FF models is comparable with the & results reported in Table 1. The level of sparsity consistent with the findings presented in Table However, in the case of the BP/FE model, the third layer exhibits siguificantly higher sparsity compared to when using the & norm. Despite this increase, sparsity remains insufficient for recognizing robust and consistent ensembbles. The average fraction of active units per layer is reported in Table 13 Results regarding ensemble overlap between visually similar classes the FF model are reported in Figure 14 of subsectioon 4.3 remain valid when employing the & norm as a goodness function.
The findings
Table 11: Test-set classification accuracy for the models FF and BP/FF, using a goodness function based on the & norm. Results expressed as mean + std. den. over 10 runs with independent randomised weight initialisation.
Datasel
FF
BP BP/FF
MNIST 0.949 + 0.002
0.965 + 0.001
MASHIONMNIST
0.859 + 0.002
0.865 + 0.002
Table 12: Average sparsity for FF and BP/FF with & goodness function, computed according to the definition given in subsection 3.5) Results expressed as mean + std. dev. computed over 10 runs with independent random weights initialisation.
Model
Layer
MNIST
FASHIONMNIST
0.887 + 0.002
0.83 + 0.001
FF
0.61 = 0.005
0.647 + 0.007
0.61 + 0.012
0.532 + 0.012
0.944 0.002
0.921 0.003
BP FFFF
0.915 + 0.005
0.919 + 0.003
0.441 + 0.009
0.408 + 0.011
25

---

Published in Transactions on Machine Learning Research (03/2025)
Table 13: Average fraction of units taking part in ensembles for FF and BP/FF with & goodness function.
Ensemble sizes are averaged across all calegories, divided by the number neurons in a layer, and then expressed in %. Ensembles are defined according to the LOO method presented in subsection 3.5 Results are expressed as mean std. deu. In the third layer of BP BF/FF the reprosentation is non-sparse.
Model
Layer
MNIST
FASHIONMNIST
1
4.22 + 0.09
7.48 + 0.11
FF
2
19.93 + 0.53
19.44 0.53
3
17.24 + 0.85
23.09 = 1.15
1
3.88 + 0.16
5.93 + 0.21
BP/FF
2
3.36 + 0.29
5.26 + 0.32
3
T-shirt
1.0 0.02
0.05
0.04
Trouser
1.0
0.01
Pullover
0.02 1.0
0.01
0.1
0.08
Dress
0.05
0.01
0.01
1.0
0.01
Coat
0.1
0.01
1.0
0.06
Sandal
Shirt
Sneaker
Bag
Ankle
boot
T-shirt
Trouser
Pullover
Dress
Coat
Sandal
Shirt
Sneaker
Bag
Ankle boot
Figure 14: Jaccard similarity index between first-layer ensembles. Results obtained using the & norm as
goodness function in the FF model the FASHIONMNIST dataset.
26


Reviews:
[{"review_id": "NFCfdKfMbX", "reviewer": "Reviewer_3UYx", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1) Timely problem.\n2) Of interest to both ML and Comp Neuro communities\n3) Novel neuroscience-inspired methods that can be used to study internal representations in deep neural networks.\n\nWeaknesses (see requested changes for more detail):\n1) Connection to neuroscience is intriguing but limited (mainly focusing on sparsity and sharing of neurons across stimuli type). \n2) No comparison using several previously established methods for comparing artificial and biological networks\n3) Limited to easier datasets. Intuitions may not hold for larger datasets like ImageNet.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The authors study the emergent internal representations that evolve using the Forward-Forward algorithm using novel techniques inspired by Neuroscience.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Important problem and interesting approach. The paper would be of interest to the ML and Neuroscience Communities but the datasets are too easy for an ML paper and connections to Neuroscience is too limited for a comp neuro paper. I currently recommend a weak accept.\n2. Fig 1: Describe in the fig legend that the class label is embedded on the image itself.\n3. The neuroscience references are rather limited. Please see several recent works by the Stefano Fusi Lab. See “Bio-inspired neural networks implement different recurrent visual processing strategies than task-trained ones do” by Lindsay for insights on metrics to add.\n4. Is ImageNet training possible? if not, please add an estimate of the runtime, or a justification for excluding it. \n5. Fig 3: Is it semantic similarity or visual similarity that causes a higher overlap? Fig 3B: the similarity indices are rather small. There are only 4 off diagonal entries and they are all very small. I am not sure the claim that ‘semantic similarity causes a high overlap’ holds. Sandals and sneakers would be expected to have a higher overlap by this logic. There’s no reason semantic similarity should matter here. The network is not trained on any language input, nor any class co-occurrence data. Visual similarity should be the driver of the high similarity index, not semantic similarity. There are studies that try to tease apart the roles of semantic and visual similarity, e.g. Yeh and Peelen, 2022; Yeh, Thorat, and Peelen, 2024. At least, a simple experiment to quantify visual similarity and semantic similarity should be done. Word embedding distances for the categories could be used for semantic similarity, or something like object2vec from Bonner and Epstein, 2021. For image similarity, the LPIPS score could be used. These may not be the best metric anymore, please choose a reasonable metric.\n6. Use RSA to compare the representations of the three model types and other previously used methods, such as BrainScore.\n7. Carandini and Heeger (C&H) is divisive normalization wrt neighboring neurons. The normalization described in the methods is across all neurons in the layer, which is not the normalization C&H meant (which is division by local neighbors of the neuron). The authors have used C&H to claim biological plausibility, but it is a stretch because what the authors did is not exactly what C&H talk about. \n8. Slight nitpicking but embedding the image class on the image itself is not exactly biologically plausible either.\n9. Table 1: why is the std dev much higher for BP for SVHN, compared to the other two models?\n10. Fig 2: Why does BP/FF layer 3 differ from FF in terms of sparsity, but not the other 2 layers? \n11. Just comparing sparsity index is not very convincing. If citing Yoshida and Ohki, please include some of their analyses, like linearly decodable info.\n12. Fig 5: Why only layer 2?\n13. “The idea is that a neuron should be considered active and part of an ensemble if it activates consistently and selectively when the network receives input data that belongs to that category”. Wouldn’t ‘selectively’ imply that neurons should not be shared across categories, then? Selectivity in other contexts in neuroscience differs than what the authors imply here (e.g. see Rust and DiCarlo, 2010, Madan et al., 2022). \n14. Equation numbering would be good (if allowed by the journal). \n15. The definition of the category-wise means in the section defining ensembles uses a symbol x_j which is defined separately for each category, so the symbol should also specify the category (something like x_j,c)\n16. For defining x_j, why are only the correctly classified images M_c used? One could argue that these neurons are responsible for the network saying that category C is present in an image, but then this computation should include images of categories other than C, where the network falsely detects C.\n17. Classifying a neuron as active in an ensemble compares the mean activations with the means of means, but this comparison could involve significance testing with a p-value threshold\n18. If classifiers are trained based on responses of category ensembles, is the entire ensemble required to get the maximum decoding accuracy for that category?\n19. Fig 4: It makes sense that a network would try to project unseen images to the same latent space that works for most of its seen categories. I would expect backprop-trained models to do so too. Whether Fig 4 tells us something non-trivial can only be assessed by comparing against the other two model types. Additionally, sandal is close enough to categories seen while training, so it makes sense that the network uses a strategy similar to other images. What would the results look like for truly out-of-distribution images, like a spaceship? I am also not sure what the authors mean by a ‘valid’ ensemble.\n20. Across figures the choice of dataset keeps fluctuating between MNIST and FashionMNIST", "broader_impact_concerns": "While no Broader Impact Statement is present, this work doesn't pose any broader ethical concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "Lqw6UMH7Xp", "reviewer": "Reviewer_2pNz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strenghts**\n\n- The paper is well written and simple. \n\n- The paper demonstrates a good empirical analysis of the differences in representation learned between networks trained using FF and BP, showing emergence of sparse ensembles in the former case.\n\n- The emergence of sparsity patterns shows the potential of FF to interpretability and OOD generalization applications.\n\n**Weaknessess** \n\n- Some biological claims are not well enough supported and the connection is loose. E.g. the link with the Neuronal ensembles literature lack deeper discussion nor any theoretical or empirical evidence of brain activations resembling sparse ensembles learned by the FF algorithm. \n\n-  The poor scalability of the FF algorithm to more complex networks and dataset limits the practical impact of future promising direction (e.g. OOD generalization)", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes to analyze the sparsity patterns in the activation space learned by the Forward-Forward (FF) algorithm comparing with the standard back propagation (BP) algorithm. The paper proposes to analyze three algorithms the FF algorithm, the BP algorithm optimizing cross entropy loss and the BP algorithm optimizing the goodness function (sum of squared activations).  The sparsity is mearured on networks trained on MNIST, FashionMNIST, SVHN, and CIFAR10  Its contributions can be summarized as follows: \n\n- The papers demonstrates empirical evidence that optimizing for a goodness loss function both locally (FF algorithm) and globally (BP applied on the goodness loss function) induces sparsity (measured as the l1/l2 ratio of the sample activations), implicitly. \n\n- The sparsity can be measured at the class (category) level,  with individual units contributing to multiple ensembles when visual features are shared. unveiling sparse ensembles w.r.t. to class in activation space.\n\n- The FF-trained networks can form new sparse ensembles for unseen categories, explaining them as a sparse combination of in distribution categories. This supports the potential for zero-shot classification and out of distribution generalization applications.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "**Clarity** \n\nTo improve clarity of the paper, I suggest the following changes: \n\n- List of contributions well specified at the end of the introduction. This helps readers quickly identify the key outcomes of the work and how it contributes to the field. \n\n- A one liner at the end of each experimental section, stating the takeaway of the experiment. This will make the results more accessible.\n\n- A small visualization explaining how an ensemble is computed (while simple) could help in providing a more direct understanding of the section.\n\n**Related work**\n\n- Discuss the relationship between findings and theoretical results in Yang (2023): the paragraph should highlight how your results align with, extend, or contrast the findings in Yang (2023)\n\n**Experiments** \n\nSome of these aspect might be discussed or showed: \n\n- Do the results depend on the goodness function chosen? \n\n- How the sparsity patterns changes across different initializations (seeds)? \n\n- In the ensemble setting, does the sparsity level of each class tell something about the category wise distributions? I.e., e.g. in Figure 2, if one class shows more sparsity than another, what does this tell us? Can it be related with class imbalance? \n\n- broader: \n    - It would be  interesting to see how the performance on unseen categories changes between the FF, BP and BP/FF algorithms, by e.g. training a linear probe on the representations.\n\n    - Can the setting of the FF algorithm and the sparsity result  related with networks trained with weight decay and BP? \n\n   -  How the sparsity pattern is linked to overparametrization condition of the network? I.e. in overparametrized settings is there less of specialized neurons shared across samples (less sparsity) as the network tends to memorize more single examples? One way to see this would be to train the same network on subsamples of the dataset.\n\n\nBy providing answers to some of the points above and incorporating the corresponding analyses, the paper would provide a more comprehensive understanding of the findings and their implications.", "broader_impact_concerns": "There are no ethical implications to be addressed according to my judgement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "sfH6KfuVWq", "reviewer": "Reviewer_DE1Q", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:** \n\n- Forward-forward algorithm is a relatively new and less studied topic. This paper helps improve our understanding of it.\n- The paper is easy to follow and understand\n\n**Weaknesses:** \n\n- The authors only test L2 norm as the goodness function despite mentioning that sparsity seems to be a result of the objective function rather than the FF algorithm itself. There should have been a section in the paper studying the impact of different goodness functions on the three model setups.\n- I would also be interested in seeing how standard regularization methods compare to the implicit regularization in FF models. For example, how sparse are the representations of standard backprop with cross entropy if regularization is used? Are neural ensembles formed if we regularize a standard model? Are the same set of neurons becoming active?\n- The authors only show results for test set, but it would have been interesting to see if the same levels of sparsity and category specific ensembles are being created across the training data too.  There should be a comparative analysis on training vs test data highlighting the differences and similarities.\n- In Fig 3 of the paper, the authors have shown a Jaccard Similarity Matrix highlighting that semantically similar classes have shared neurons. But I feel that the values in the matrix are too low (in the range of 0.07 - 0.16) to have a strong case for being similar. I would have expected a higher similarity index for semantically similar classes like pullover and coat. Is there any explanation to why the values are relatively small?\n- While the authors compare the representations of different model setups using excitatory and inhibitory connections, it would be interesting to see how similar the different model representations are using CCA/CKA similarity measures.\n- As layers are trained separately in case of FF, it would be insightful to have a more in-depth comparison of the differences and similarities between the three layers of the trained FF model", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates the internal representations formed by neural networks when trained using the Forward-Forward algorithm. The authors find out that these internal representations are highly sparse and organized into category specific ensembles mirroring the behavior of neurons in biological brains. They also find that semantically similar classes activate overlapping ensembles with shared neurons. \n\nThey run a set of experiments on three variants of model setups \n\n- Forward forward algorithm (FF)\n- Backprop trained using forward-forward objective (BP/FF)\n- Standard backprop with cross entropy loss (BP)\n\nThey show that FF and BP/FF exhibit high sparsity and form ensembles, while BP does not. \n\nThe main conclusion of this paper is that neural networks trained using Forward-Forward algorithm offers a more biologically plausible learning mechanism than standard backprop algorithm.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "I would recommend the authors to at least address some of the points mentioned in the weaknesses section.", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work investigates the representations that emerge through use of the Forward-Forward training algorithm. This work is of interest to computational neuroscientists and those interested in the science of machine learning. The results are convincing and, while focused on small-scale datasets, will open up new research directions. Given that all 3 reviewers recommend acceptance as is, I am recommending the same.
Decision: Accept (poster)