Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization
Authors: Ravi Francesco Srinivasan
Francesca Mignacco
Martino Sorbaro
Maria Refinetti
Avi Cooper
Gabriel Kreiman
Giorgia Dellaferrera
Venue: ICLR 2024 poster (2024)
Keywords: Forward-only learning
Biologically inspired learning
Artificial neural networks
Analytical characterization
Primary area: applications to neuroscience & cognitive science
TL;DR: We discuss "forward-only" algorithms, provide an analytical characterization and test strategies to improve their performance.

Abstract: "Forward-only" algorithms, which train neural networks while avoiding a backward pass, have recently gained attention as a way of solving the biologically unrealistic aspects of backpropagation. Here, we first address compelling challenges related to the "forward-only" rules, which include reducing the performance gap with backpropagation and providing an analytical understanding of their dynamics. To this end, we show that the forward-only algorithm with top-down feedback is well-approximated by an "adaptive-feedback-alignment" algorithm, and we analytically track its performance during learning in a prototype high-dimensional setting. Then, we compare different versions of forward-only algorithms, focusing on the Forward-Forward and PEPITA frameworks, and we show that they share the same learning principles. Overall, our work unveils the connections between three key neuro-inspired learning rules, providing a link between "forward-only" algorithms, i.e., Forward-Forward and PEPITA, and an approximation of backpropagation, i.e., Feedback Alignment.

Published as a conference paper at ICLR 2024
FORWARD LEARNING WITH TOP-DOWN FEEDBACK: EMPIRICAL AND ANALYTICAL CHARACTERIZATION
Ravi Srinivasan 1.2, Francesca Mignacco Martino Sorbaro 3,6
Maria Refinetti Avi Cooper", Gabriel Kreiman 10 Giorgia D 1,, IBM Research Europe, Zurich, ETH Zurich, Joseph Henry Laboratories of Physics, Princeton University, Initiative for the Theoretical Sciences, Graduate Center, City University of New York, 6 Institute of Neuroinformatics, University of Zurich and ETH Zurich, AI Center, ETH Zurich, ? Laboratoire de Physique de l'Ecole Normale Supéricure, Université PSL, CNRS, Sorbonne Université, Université Paris-Diderol Sorbonne Paris Cité, IdePHICS laboratory, Bcole Fédérale Polytechnique de Lausanne (EPFL.) Massachusetts Institute of Technology. Center for Brains, Minds and Machines, 11 Children's 's Hospital, Harvard Medical School "gabriel. kre sareiman@tcct hww.ard.aaaoooo.ooo "giorgia, aerria.deelln www.leeeeeerreeeeeeeeeeeeeeeeeee com
ABSTRACT
"Forward-only" algorithms, which train neural networks while avoiding a back- ward pass, have recently gained attention as a way of solving the biologically un- realistic aspects of backpropagation. Here, we first address compelling challenges related to the "forward-only" rules, which include reducing the performance gap with backpropagation and providing analytical understanding of their dynam- ics. To this end, we show that the forward-only algorithm with top-down feedback is well-approximated by alytically track its performance during learning in a prototype high-dimensional setting. Then, compare different versions of forward-only algorithms, focusing on the Norward-Forwadd and РЕРITА frameworks, and we show that they share the same learning principles. Overall, work unveils the connections between three key neuro-inspired learning rules, providing a link between "forward-only" algorithms, i.e., Forward-Forward and PEPITA, and approximation of back- propagation, i.e., Feedback Alignment.
algorithm, and we
INTRODUCTION
In machine learning (ML), the credit assignment (CA) problem refers to estimating how much each parameter of a neural network has contributed to the network's output and how the parameter should be adjusted to decrease the network's 's error. The most commonly used solution to CA is the Back- propagation algorithm (BP) (Rumelhart al. 1995). which computes the update of each parameter as a derivative of the loss function. While this strategy is effective in training networks on com- plex tasks, is problematic mechanisms of learning in the brain (Lillicrap et al. 2020). While the lack of biological plausibility does not necessarily represent an issue for ML, may eventually help us understand how address shortcomings of ML, such as the lack of continual learning and robustness, or offer insights into how learning operates biological neural systems (Richards al.) 2019). Second, the backward pass of BP is a challenge for on-hardware implementations with limited resources, due to high memory and power requirements (Khacef et al.) 2022. Kendall et al. 2020).
at least two aspects. First, BP is not compatible with the known
These reasons motivated the development of alternative solutions to CA, relying on learning dy- namics that are more biologically realistic than BP. Among these, several algorithms modified the feedback path carrying the information the error (Lillicrap et al. 201 Nøkland 2016 Akrout al. 2019 Clark et al. 2021) or the target (Lee et al. 2015) Frenkel et al. (021) to each node, while maintaining the alternation between forward and backward pass. More recently, "forward-only" algorithms were developed, which consist of a family of training schemes that replace the backward pass with second forward pass. These approaches include the Forward- Forward algorithm (FF)

---

Published as a conference paper at ICLR 2024
(Hinton 2022) and the Present the Error to Perturb the Input To modulate the Activity learning rule (PEPITA) (Dellaferrera & Kreiman 2022). Both algorithms present a clean input sample in the first forward pass (positive phase for FF, standard pass for PEPITA). In FF, the second forward pass con- sists in presenting the network with a corrupted data sample obtained by merging different samples with masks (negative phase). In PEPITA, instead, in the second forward pass the input is modulated through information about the error of the first forward pass (modulated pass). For simplicity, we will denote the first and second forward pass as clean and modulated pass, respectively, for both FF and PEPITA. These algorithms avoid the issues of weight transport, non-locality, freezing of activ- ity, and, partially, the update locking problem (Lillicrap et al., 2020). Furthermore, as they do not require precise knowledge of the gradients, nor any non-local information, they are well-suited for implementation in neuromorphic hardware. Indeed, the forward path can be treated as a black box during learning and it is sufficient to measure forward activations to compute the weight update.
FA
b
PEPITA
C
PEPITA-WM
d Forward- Forward
y
9
8
9
W5
F
W22
WS
4
A
P=AA
W:
W2
Fe
Different error transportations and WM configurations. Green arrows mark forward paths and orange arrows indicate error paths. (a) Feedback alignment (FA). (b) Present the Error to Perturb the Input To modulate Activity (PEPITA). (c) PEPITA with WM. (d) Forward-Forward (FF).
Figure I:
Despite the promising results, the "forward-only" algorithms are still in their infancy. First, neither Hinton (022) nor Dellaferrera & Kreiman (2022) present an analytical argument explaining why the algorithms work effectively. Second, they present some biologically unrealistic aspects. For instance, the FF requires the generation of corrupted data and their presentation in alternation with clean data, while РЕРITA needs to retain information from the first forward pass until the second forward pass, therefore the algorithm is local in space but not local in time. Third, the algorithms present performance gap when compared with other biologically inspired learning rules, such as feedback alignment (FA). Our work addresses the mentioned limitations of the "forward-only" al- gorithms, focusing on PEPITA, and compares the FF and the PEPITA frameworks. Here, we make the following contributions: (i) We show that PEPITA effectively implements "feedback "feedback-aligmmentt with adaptive feedback (AF) matrix that depends on the upstream weights (Sec. 3.1); (ii) Focus- ing on PEPITA, we use the above result to analytically characterize the online learning dynamics of "forward-only" algorithms and explain the phenomenon of alignment of forward weights and top-down connections observed in original work (Sec. 3.2); (iii) We demonstrate that PEPITA can be applied to networks deeper than those tested by Dellafeerreaa & Kreiman (2022) (Sec. 4 (iv) We propose two strategies to extend the weight mirroring (WM) method to training schemes other than FA and show that PEPITA enhanced with WM achieves better alignment, accuracy, and convergence (Sec. 4 (v) We demonstrate that PEPITA can be approximated to a rule containing sum of Heb- bian and anti-Hebbian terms, allowing space- and time-locality (Sec. [.T and Appendix (B); (vi) We analytically compare the Hebbian approximation of PEPITA with the Berward-Forward algorithm Hinton b022) with top-down feedback (Sec. 5.2)
2 BACKGROUND AND RELEVANT WORK
2.1 BIO-INSPIRED LEARNING ALGORIHTMS
Several training rules for artificial neural networks (ANNs) have been proposed to break the weight
symmetry constraint required by BP (Lillicrap et al. 2016a; Nøkland 2016 Liao al. 2016
Nokland & Eidnes 2019 Frenkel et al. 2021]]azaa Hazan et al. (2088) Kohan et al.
2018 2022 Meule-
mans et al.
2021 Ha alvagal & Zenke 2023 Journ al. 2022 Launay et al.]
2020 Furthermore,
2

---

Published as a conference paper at ICLR 2024
Table 1: Biological properties and test accuracy [%] MNIST achieved by non-exhaustive selec- tion of bio-inspired alternatives to BP. We report the highest accuracy documented in the referenced papers. Legend: x aspect not solved; aspect fully solved: aspect partially solved.
LEARNING RULE
REE
FURWARD ONLY
WEIGHT TRANSPORT
LOCAL
ACTIVITY FREEZING
UPDATE UNLOCKED
MNIST
FA
LILLICRAP ET AL.
2016B
X
X
98.74
DRIP
FRESKEL E1
2
X
98.52
DTP-3
ORORBIA
2019
V
X
97.65 97.62
LRA
ORRRITA
2019
RLRA
ORUKBIA E1 AL.
2020
V
98. 18
EPP
V
PEPITA
DELI N KREIMAN
2022
98.29
FF
HINTON 2022
V
V
98.
OROKBIA MALL 2023n
SP
KOHAN ET 2022
98.62
PEPITA+WM
OURS OURS
98.
PEPIT-HERBIAN
98.05
PEPITA-TL
OURS
92.78
several brain train-insppred algorithms have been shown train ANNs with local information in the frameworks of Predictive Coding (PC) (Rao & Ballard
1999; van den Oord et al.) [219 Millidge
al. 2022 Salvatori et al. 2021 Ororbia Mali 2019 Ororbia et al. 2020) and neural generative
coding (Ororbia & Kifer] 2022, Ororbia & Mali] 2023a Table reports properties of biological
plausibility and accuracy metrics for these algorithms. The FA algorithm (Lillicrap et al., 200 16a) replaces the transposed forward weights W in the feedback path with random, fixed (non-learning) weight matrices to deliver error information to earlier layers, thereby solving the weight trans- port problem (Fig. [T]) (Lillierap al. (2) While FA achieves performance close to BP simple tasks such as MNIST (LeCun & Cortes 2010) and CIFAR- 10 (Krizhevsky et al., a) with rel- atively shallow networks, it fails scale to complex tasks and architectures (Bartunov al. 2018 Xiao et al., 2018: Moskovitz et al., 2018). Moreover, [Hinton & McClelland (1977) proposed the Recirculation algorithm in which the output of an autoencoder fed back into the network as "negative" example. Later, Baldi & Sadowski| (2018) showed that Recirculation is a special case of /aaaancessss/suutttt adaptive FA. To improve FA performances, /Akrout et al. (2019) proposed the Weight-Mirron (WM) algorithm, which is approach to adjust the feedback weights in FA to improve their agreement, al- lowing to train neural networks with complex architectures (ResNet- 18 and ResNet-500 on complex image recognition tasks (ImageNet).
2.2 THEORETICAL ANALYSES
The theoretical study of online learning with 1-hidden-layer neural networks has brought consid- erable insights statistical learning theory (Saad & Solla, 1995a Riegler & Bichl 1995 Goldt ctt al., 2020; Refinetti et al. 2021b). We leverage these works to understand learning with biologi- cally plausible algorithms. Through similar analysis, Refinetti al. (2021a) analytically confirmed the previous results of Lillicrap et al.] (2016b); Nøkland (2016); Frenkel et al. (202T) showing that the key learning with direct feedback alignment (DFA) is the alignment between the network's weights and the feedback matrices, which allows for the DFA gradient to be aligned to the BP gra- dient. The authors further show that the fixed nature of the feedback matrices induces degeneracy- breaking effect where, out of many equally good solutions, a network trained with DFA converges to the one that maximizes the alignment between feedforward and feedback weights. This effect, however, imposes constraints on the structure of the feedback weights for learning, and possibly explains the difficulty of training convolutional neural networks with DFA. Bordelon (2022) derived self-consistent equations for the learning curves of DFA and FA in infinite- width networks via a path integral formulation of the training dynamics and investigated the impact of biologically plausible rules feature learning.
Pehlevan
2.3 THE PEPITA LEARNING RULE
Given a fully connected network with L layers, an input x, and one-hot encoded labels (Fig. [p), the learning rule proposed by Dellaferrera & Kreiman (2022) relies clean and modulated
3

---

Published as a conference paper at ICLR 2024
forward pass through the network. During the clean pass, the hidden unit and output unit activations are computed as:
h ow h (W for 2 < & L,
(1)
where is the non- non-linearity at the output of the hth layer and We is the matrix of weights between layers l - 1 and f. During the modulated pass, the activations are computed as:
hi 00 (W (x Fe)), he oz (W for 2 S SeSL,
(2)
where y is the target output, = hr. denotes the network error, and F is the fixed random matrix used project the error on the input. We denote the output of the network either hg or ÿ. After the two forward passes, the weights are updated according to the PEPITA learning rule:
AW1 = (h1 (x Fe)";
(3)
AW; (he (hgE))
T for 2 < < L;
(4)
AW2 e(hggr,)
(5)
Note that, compared to the original paper, we changed the sign convention to read FFe rather than +e in eqn. 2 and eqn. 3 Because the distribution of entries is symmetric around zero, this has the results. This change will appear useful when describing the WM applied to PEPITA in Section 4 Finally, the updates are applied depending on the chosen optimizer. For example, using stochastic gradient descent with learning rate z/: W + 1) ((L) njW The
no consequence consequent
pseudocode detailing the algorithm is provided in Appendix A
2.4 THE FORWARD-FORWARD ALGORITIIM
Analogously to PEPITA, the FF algorithm removes the need for backward pass and relies on two forward passes (Fig. Dt). The clean pass and the modulated pass operate on real data and on appropriately distorted data, respectively. The latter generated to exhibit very different long-range correlations but very similar short-range correlations. In practice, the modulated samples are hybrid images obtained by adding together one digit image times a mask with large regions of ones and zeros and different digit image times the reverse of the mask. In the clean pass, the weights updated to increase a "goodness" in the hidden layers (Hinton (022) proposes to use the sum of the squared activities, its negation), and in the modulated pass to decrease it. Similarly, the Predictive Forward Forward (PFF) integrates the local synaptic adaptation rule of FF based on goodness measure and contrastive learning, with lateral competition and aspects of PC, such as its local error Hebbian manner of adjusting generative synaptic weights (Ororbia & Mali 2023b).
3 THEORETICAL ANALYSIS OF THE LEARNING DYNAMICS OF PEPITA
In this section, we present a theory for the "forward-only" learning frameworks, focusing specifically on PEPITA in two-layer networks. We propose a useful approximation of the PEPITA update, that we exploit to derive analytic expressions for the learning curves, and investigate its learning mechanisms in a prototype teacher-student setup.
3.1 TAYLOR EXPANSION AND ADAPTIVE FEEDBACK RULE
First, observe that the perturbation applied in the modulation pass is small compared to the input: [ell < ||z||. Indeed, in the experiments, the entries of the feedback matrix F are drawn with standard deviation or = k/ m/vV D. where D is the input dimension - typically large and a constant set by grid search, while the input entries are of order one. Thus, it is reasonable to Taylor-expand the presynaptic term h -ffT, which results in the approximate update rule:
AW Fe) O TT
(6)
where we have used æ instead of (z Fe) since the small perturbation has been found to be negligible for the performance (Dellaferrera & Kreiman 2022). Equ. 6 shows that PEPITA effectively implementing "DFA-like" update (equivalent to FA in lwo- networks), but using an AF matrix where the random term is modulated by the network weights. In light of this observation, it is natural expect the alignment between W, F and Wz. This simple approximation, which we call
4

---

Published as a conference paper at ICLR 2024
Adaptive Feedback Alignment (AFA), is actually very accurate, verify numerically in Fig. 2 displaying experiments on MNIST, and in Fig. [f Appendix(]for the CIFAR- 10 and CIFAR- 100 (Krizhevsky et al. (b) datasets. AFA approximates PEPITA very accurately also in the teacher- student regression task depicted in Fig. D. analyzed in the next section.
3.2 ORDINARY DIFFERENTIAL EQUATIONS FOR ADAPTIVE FEEDBACK ALIGNMENT WITH ONLINE LEARNING FOR TEACHER-STUDERT REGRESSION
To proceed in our theoretical analysis, it is useful to assume a generative model for the data. We fo- cus on the classic Macher-student setup (Gardner & Derrida 1989 /Sung et al. 1992 Watkin al.,
1993 Engel & Van den Broeck 2001: Zdeeorováá & Krzakala 2016). We consider 7> 17-dimensional standard Gaussian input vectors x NO, DD) while the corresponding label y = W2 (Wia) is generated by two-layer teacher network with fixed random weights W1. W2 and activation func- tion (-). The two-layer student network outputs a prediction ş = W2 o (W1rr) and is trained with the AFA rule and online (or one-pass) protocol, i.e., employing a previously unseen example Tu / = I,.... N, at each training step, where N is the number of samples. We characterize the dynamics of the mean-squared generalization error
Ex (
2]
Es [e-] e e y,
in the at finite rate (or sample complexity) N/D (n(1) where the training time is = / D. The hidden-layer size is of order ((1) in both teacher and student. We follow the derivation in the seminal works of Biehl & Schwarze (1995); [aad (olla which has been put on rigorous ground by Goldt et al. (2019), and extend it to include the time-evolution of the AF, As discussed in Appendix L the dynamics of the error eg as a function of training time t fully captured by the evolution of the AF matrix and set of lindenmeniional matrices encoding the teacher-student alignment. We derive a closed set of ordinary differential equations (ODEs) tracking these matrices, and we integrate them to obtain our theoretical predictions. Furthermore, in Appendix F. perform an expansion at early training times t < to elucidate the alignment mechanism and the importance of the "teacher-feedbac" alignment", i.c. W2W, F, as well the norm of (Wl For the sake of the discussion, we consider sigmoidal activations o(-) = (-)) = erf(.), keeping in mind that the symmetry erf(-a) (I) induces degeneracy of solutions.
limit of both input dimension D + oo and number of samples N + oC,
b
d
AdA
APA
1.00
0.75
100
0.50
0.25
0.00
30
75
10
10
10
Figure 2: (a) Test accuracy as a function of epochs for experiments with the MNIST and a 1-hidden- layer network with (1024 hidden units, ReLU). Blue dots mark the "vanilla" PEPITA algorithm
(without momentum) while purple crosses mark the AFA approximation 6 (b) Generalization
error as a function of time for the experiments with PEPITA (blue dots) and AFA (purple crosses),
and the theoretical curves (App. E marked by full lines. Parameters: D = 500, Ir = .05, erf activation, hidden units in both teacher and student. (c) Alignment angle between the teacher and student second-laye weights (dark green), the student and a degenerate solution (light green), and the AF matrix and the student (orange) as a function of time. (d) Direction of the student, the AF and the teacher (and degenerate solutions). Different time shots are marked by vertical dashed lines in panel (c).
Fig. 2b shows our theoretical prediction for the generalization error as a function of time, compared to numerical simulations of the PEPITA and AFA algorithms. We find excellent agreement between our infinite-dimensional theory and experiments, even at moderate system size (D 500).
Fig. P shows the dynamic changes in the alignment of the second-layer student weights W2 with different matrices: the second-layer teacher weights Wq, the second-layer teacher weights of the
5

---

Published as a conference paper at ICLR 2024
closest degenerate solution, and the AF weights W.F. We clearly observe that the error is stuck at plateau until "adaptive-feedback aligument" happens (t ~ 102). Fig. 2 depicts the directions of the wwo-dimensional vectors W2, Wi and W₂ at three different time shots, marked by vertical dashed lines in Fig. - It is crucial to notice that, in this case, the feedhack also evolves in time, giving rise to richer picture, that we further discuss in Appendix [D] In the special case of Fig. 2 while the direction of W F' is almost constant, its norm increases, speeding up the learning process, as shown in Fig. S4 Similar behavior was observed in Dellaferrera & Kreiman (2022). Notice that, in the case of one hidden layer, FA directly implies gradient alignment given that the weight updates become identical if the feedback matches the feed-forward weights.
4 TESTING PEPITA ON DEEPER NETWORKS, WITH WEIGHT MIRRORING, WEIGHT DECAY AND ACTIVATION NORMALIZATION
The theoretical analysis above focuses on two-layer network, as used in the proof-of-princing demonstration of PEPITA's function in the original article. To the best of our knowledge, PEPITA has been tested so far only on two-layer networks. Here, we empirically show that the algorithm can be extended to train deeper networks, Furthermore, we propose three techniques that improve the performance of PEPITA and narrow the gap with BP. First, we enhance PEPITA with standard tech- niques used in ML, namely weight decay (WD) (Krogh & Hertz 1991) and activation normali normalization (as in Hinton| 2022). In addition, we combine PEPITA with the WM algorithm (Akrout et al. 2019)
to train the projection matrix F. Indeed, WM has been shown to greatly improve alignment between the forward and backward connections and the consequent accuracy of FA and, in Section 3. we have shown that PEPITA is closely related to FA. In order to apply WM to PEPITA, a generalization of WM for learning rules where the dimensionalities of the feedback and feedfor- ward weights do not match, such PEPITA, but also including DFA. In our work we focus on fully connected models, and leave the enhancement of convolutional networks for further exploration. Experiments were run on Nvidia V100 GPUs, using custom Python code available here
propose
a
b
POPTA unflre
PER A savol PER Ai
PGPTIA namol PERTTN-O poll PoRIANOAM prill
POPPTA nareel
93.6
52.5
63.0
Actunscy
63.0
W
62.0
62.0
20
81.5
24
50
# layees
# Middin leyore
Thaiiing spoctt
Figure 3: (a), (b) Test accuracy of fully connected networks with increasing depth trained with PEPITA on the CIFAR-10 dataset. (a) PEPITA uniform and PEPITA normal refer to the initialization of the weights and F (Sec. 4. "PEPTTA-Hebbian" refers the learning rule explained in Section
5.1 (b) Effect of weight decay. (c) Alignment angle between F (Eq. (2 and W1 W2 during training
with or without WM. PreM refers to pre-mirroring (Sec. 4). Hyperparameters are reported in Table
S3
The plots indicate mean and standard deviation over 10 independent runs.
4.1 METHODS
We first test two initialization strategies, He uniform (as in Dellaferrera & Kreiman 2022) and He normal. The normal initialization is necessary to factorize the feedback matrix as explained below. The distribution of the entries of F has mean of zero and standard deviation reported in Tab. S3 When not specified, we use the He normal initialization. The hyperparameters are found through grid search and are reported in Table S3
In order to apply WM to PEPITA, we need to modify the original algorithm (Akrout et al. 2019 Indeed, in network trained with FA, the WM algorithm aligns, for each layer Ł, backward matrix F, with the transpose of the corresponding forward matrix W, (Fig. [T.. In order to recover the one-to-one correspondence of WM, we factorize the F matrix into the product of many matrices
6

---

Published as a conference paper at ICLR 2024
the number of layers: F F= -I= (Fig. It). We initialize each of these matrices by sampling the entries from a normal distribution and apply WM layer-wise (see pseudocode in Appendix D. After each standard WM update, we also normalize each F2 to keep the standard deviation of of
the updated F at step & constant at its initialization value of (see Table SS).. Running WM before learning starts (pre-mirroring) brings the random feedback weights into alignment without the of any training data. We run benchmarks both with five epochs of pre-mirroring, and without. We compared PEPITA'S performance to three baselines, BP, FA, and Direct Random Target Projection (DRTP, Frenkel et al., (021)) with and without weight decay. For the baselines, we use the code by Frenkel et al. (2021). Our use of learning rate decay means the grid search returned different hyperparameters, explaining some discrepancies in the accuracies compared to Frenkel et al. 2202
Test accuracy [%] achieved by BP, FA, DRTP, and PEPITA in the experiments, with and without weight decay, normalization of the activations and WM Mean and standard deviation are computed over 10 independent runs. All training schemes are tested with ReLU nonlinearity except DRTP which uses tanh nonlinearity. The hyperparameters obtained through grid search are reported Table S3
Table 2:
Bold fonts refer to the best results exclusively among PEPITA and its improvements.
W. DECAY
NORM.
MIRROR
MNIST
CIFAR-10
CIFAR-100
BP
X
X
98.72-00.06
57.60+00.20
29.36-0.19
X
98.66+0.04
57.92+0.19
29.93+0.27
FA
X
X
98.48=0.05
56.76+0.23
22.75+ 0.28
X
98.35-0.04
57.1940.24
22.62=0.22
DRTP
X
X
95.45-10009
46.92-1.211
16.9910.11
X
95.44=0.09
46.92=0.27
17.59=0.18
PEPITA
X
X
98.02-10.08
52.45110.25
24.69
/
X
X
98.12-10.08
53.05510.23
24.86-10.18
X
J
X
98.41=0.08
53.51+0.23
22.87=0.25
X
X
98.05=0.08
52.63=0.30
27.07=0.11
X
98.10=0.12
53.46100.6
27.04=0.19
X
/
98.42-0005
53.80+0..25
24.20-00.36
4.2 RESULTS
Initialization The He uniform and normal initialization performed equally (Fig. 3h).
Depth Without activation normalization, PEPITA is successful in training networks with up to three hidden layers without drop in accuracy for increasing depth (Fig. Bhp). Furthermore, with activation normalization, PEPITA can train networks with up five hidden layers. However, this comes the cost of a decrease in performance with increasing depth (Fig. (55).
Weight decay and normalization On 1-hidden-layer network trained the CIFAR-10 dataset, weight decay and normalization improved PEPITA's performance by 0.6% and 1.1%, respectively (Fig. 3p and Table 2
Weight mirroring Fig. shows that, when WM is applied, the W and F' matrices achieve a sig- nificantly better alignment than with the standard PEPITA. The improvement of the alignment more significant for 1- and 2-hidden- thermore, the improved alignment obtained with WM is reflected in improved accuracy, especially for 1-hidden-layer networks (Table 2 but also for 2- and 3-hidden-layer networks (Sup. Table ST.. Remarkably, on the CIFAR-I0 dataset, WM combined with activation normalization leads to 53.8%, an improvement of 1.3% compared to the standard PEPITA. On CIFAR-I00 WM improves the accuracy by over 2%, reducing significantly the gap with BP.
than for 3-hidden-layer networks (Sup. Fig. S8). Fur-
Convergence rate Dellaferrera & Kreiman (2022) shows that PEPITA's convergence rate is in be- tween BP (the fastest) and FA (the slowest). By extracting the slowness from the plateau equation
for learning curves (Dellaferrera al. 2022), find that pre-mirroring improves the convergence rate for I-hidden-layer models (Table S2). compatibly with the "align, then memorize" paradigm
proposed for FA (Refinetti al. 2021a
7

---

Published as a conference paper at ICLR 2024
5 ON THE RELATIONSHIP BETWEEN FORWARD-FORWARD AND PEPITA
The goal of this section is to compare PEPITA and a FF framework with top-down connections. To this end, we first propose minor change to the PEPITA update rule that exposes its Hebbian foundation, and which can be applied to train ANNs a time-local fashion. Finally we analytically compare FF and PEPITA in the case where negative samples are based on high-level feedback.
5.1 PEPITA'S FORMULATION AS HEBBIAN AND ANTI- HEBI-HEBBIAN PHASES
In the PEPITA framework, the presynaptic term (hg-i) in the learning rule of eqn. can be replaced interchangeably with the same term, computed during the first forward pass (hę-1) (Dellaferrera & Kreiman 2022). Taking this a step further, we separate the brackets in eqn. 4aand mix the two choices for the presynaptic term obtaining an approximately equivalent rule for hidden layer weights:
(8) The learning rule eqn. S now contains two Hebbian terms, each a product of the activity of the presynaptic and the postsynaptic node, similarly to (Xie & Seung, 2003 Scellier & Bengio 2017; Detorakis et al. 2019). We dub this approximation PEPITA-Hebbian. Fig. Fi shows that the approx- imation in eqn. 8 has negligible impact on the accuracy. We note that these two terms can be used to update the weights online, separating the updates between the first and second pass (Algorithm S2], and allowing alleviate the constraint of storing in memory information of the first pass until the second pass. We name this version PEPITA-time-loce time-local (PEPITA-TL) PEPITA-TE TL exhibits a decrease in accuracy (Sup. Fig. [ST). but it is still the feedback in useful manner (Appendix
AW, = h hg" he-T= ~ h h h3
B We leave the exploration to improve the accuracy of PEPITA-TL. for further work.
5.2
COMPARISON BETWEEN THE WEIGHT UPDATES OF PEPITA AND FORWARD-FORWAND
In FF, the weights are updated increase and decrease the "goodness" in every hidden layer in the clean pass and modulated pass respectively, where the goodness can be the negative sum of the squared neural activities (in our notation, - -llhell? for the clean and || for the modulated pass). Hinton 2022) chooses a loss based on the logistic function a applied to the goodness, mi- nus a threshold, 0: p p= a ([|//]|2 - 0). For simplicity, we work directly on the goodness. This is equivalent to minimizing, each layer l, a loss function defined as:
Je = |hell - 1|
We then compute an update for FF as the derivative of the loss eqn.
9 with respect to the weights:
0J:
1
0
hell
0 1
aw.
aW
aWe
(lco(Wehe-1)) (12h
=
2
OW;
oW.
(Wehee- ( -o (WWa) o (WSM
(d'(Wehe-1) @ hohh - o o'W O ((
= (N, he)h (ha @ @ h
We note that the terms o' and here' = o' (W can be omitted in the common case where is ReLU function because they otherwise. Likewise, they are always equal to if a is linear. In these cases, therefore
if and only if he is already 0, and equal
OJr 2 aw,
heh 1 M
(10)
which is equivalent to eqn. 8 apart from factor 1/2 that can be incorporated in the learning rate. For other choices of a, since Fe & x, the derivative of the activations in the two forward passes are close enough approximate o' o'(Whh,-1
o' oWW,hTT))
By comparing eqn. and eqn. 10, we observe that the FF update rule with loss eqn. approximates the weight update rule of the previously proposed PEPITA. We remark that the comparison between
8

---

Published as a conference paper at ICLR 2024
PEPITA and FF holds for the general FF framework minimizing and maximizing the square of the activities for the clean and modulated passes respectively, which in the experiments by Hinton (2022) provides the best performances. However, the weight update reported by HHnntnn| (2022) presents analytical differences from eqn. [1] due to their use of thresholding and non-linearity applied to the goodness. Another notable difference lies in the way the "modulated"" samples are generated in the two algorithms: as described, in PEPITA's modulated pass, the input is modulated by the error. In the FF algorithm, in contrast, the negative samples are data vectors corrupted by external means, although Hinton (2022) mentions that the negative data may be predicted by the neural net using top-down connections. In contrast, the activations of the two passes are not statistically different in PEPITA, presumably due to the small perturbation of the input in the second pass (Fig. S2)
DISCUSSION
In the quest for biologically inspired learning mechanisms, two of the long-standing challenges include providing theoretical ground to understand the dynamics of the training algorithms and scaling their application to complex networks and datasets. The FF algorithm (Hinton) 2022) and the PEPITA algorithm (Dellaferrera & Kreiman (022) are "forward-only" algorithms that train neural networks with local information, without weight transport, without freezing the network's activity, and without backward locking.
First, we show that PEPITA can be approximated by an FA-like algorithm, where the error is prop- agated to the input layer via the project matrix F' "adapted" through the forward synaptic matrix. By performing theoretical characterization of the generalization dynamics, we provide intuition on the alignment mechanisms, offering novel theoretical insights into the family of forward-only al- gorithms. We then test two standard techniques to improve the performance of PEPITA, i.e. weight decay, and activation normalization. Moreover, in line with our theoretical findings that highlight the importance of the alignment of the feedback and feedforward weights, we test PEPITA in combina- tion with a version of WM tailored to PEPITA. By aligning the feedback weights to the feedforward weights. WM significantly improves the classification accuracy both the 10 and CIFAR- 100 datasets. Finally, we propose PEPITA-hebbian a version of PEPITA in which the update con- tains of Hebbian and anti-Hebbian terms. We compare PEPIIA- the similarities between the two algorithms. While FF uses externally generated negative samples for the second forward pass, Hinton] (2022) himself suggests the possibility that the negative data may be predicted by the neural net using top-down connections. We show that, when choosing the goodness as the negative sum of the squared activities, PEPITA-hebbiar is precisely an example of this possibility. This avoids the biologically unrealistic requirement of corrupting the images en- tailed by FF and is compatible with biological mechanisms of neuromodulation and thalamocortical projections (Mazzoni et al.) 1991 Williams 1992. Clark et al. 2021 Kreiman 202T). Moroever, the separate feedback pathway can be linked to neuromodulatory signals that acts as prusticiitystterriin effects (Roelfsema & Holtmaat 2018), necessary for PEPITA-TL to distinguish between the two forward passes. We envision that further research could build a unified theoretical framework for these and other forward-only algorithms, leading to higher accuracy, network depth, and biological plausibility.
FF, highlighting
7 LIMITATIONS AND FUTURE WORK
While this work provides valuable insights into forward-learinn algorithms from empirical and analytical perspectives, it is important to acknowledge several limitations. First, the algorithms we analyze still lag behind BP terms of performance and scalability. Further work is needed to under- stand the causes of the difference in accuracy between forward-learning algorithms and BP, which may include poor collaboration between layers during training (Lorberbom et al. 2023). Moreover, the factorization of the feedback matrix in the modified version of WM may not correspond to any known biological mechanism. Additional, more bio-plausible alternatives need to be explored. An- other open direction of investigation is the theoretical analysis of the learning dynamics in deep networks. A starting point in this direction could be to consider deeper networks. In conclusion, our work provides starting point for addressing those limitations through further research.
9

---

Published as a conference paper at ICLR 2024
ACKNOWLEDGEMENTS
This work was supported by NIH grant RO1EY026025 and NSF grant CCF-1231216. M.S. was supported by ETH AI Center postdoctoral fellowship.
REFERENCES
Mohamed Akrout, Collin Wilson, Humphreys, T. Lillicrap, and D. Tweed. Deep learning without weight transport. In NeurIPS, 2019.
Pierre Baldi and Peter Sadowski. Learning in the machine: Recirculation is random backpropaga- tion. Neural Networks, 108:479-494, 2018.
Sergey Bartunov, Adam Santoro, Blake A. Richards, Luke Marris, Geoffrey E. Hinton, and Tim- othy P. Lillicrap. Assessing the scalability of biologically-motivtee deep learning algorithms and architectures. In Proceedings the 32nd International Conference Neural Information Processing Systems, NIPS' 18, pp. 9390-9400, 2018.
Michael Biehl and Holm Schwarze. Learning by on-line gradient descent. Journal of Physics A: Mathematical and general, 28(3):643, 1995.
Blake Bordelon and Cengiz Pehlevan. The influence of learning rule on representation dynamics in wide neural networks. arXiv preprint arXiv:2210002157, 2022.
David G. Clark, L. F. Abbott, and Sue Yeon Chung. Credit assignment through broadcasting a global eπror vector, 2021.
Giorgia Dellaferrera and Gabriel Kreiman. Error-driven input modulation: Solving the credit as- signment problem without backward pass. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Con- ference Machine Learning, volume 162 of Proceedings of Machine Learming Research, pp. 4937 4955. PMLR, 17-23 Jul 2022. URL https: httpas////pcceediggs.... www.ssm.rr.ge wtt.sssss.......e.eeemm...l Sellaaerrera22a.11 html
Giorgia Dellaferrera, Stanislaw Wožniak, Giacomo Indiveri, Angeliki Pantazi, and Evangelos Eleft- heriou. Introducing principles of synaptic integration in the optimization of deep neural networks. Nature Communications, 13, 04 2022.
Georgios Detorakis, Travis Bartley, and Emre Neftci. Contrastive hebbian learning with random feedback weights. Neural Networks, 114:1-14, 2019.
Andreas Engel and Christian Van den Broeck. Statistical mechanics of learning. Cambridge Uni- versity Press, 2001.
Charlotte Frenkel, Martin Lefebvre, and David Bol. Learning without feedback: Fixed random learning signals allow for feedforward training of deep neural networks. Frontiers in neuro- science, 15:629892, 2021.
Elizabeth Gardner and Bernard Derrida. Three unfinished works on the optimal storage capacity of networks. Journal Physics A: Mathematical and General, 22(12): 1989.
Sebastian Goldt, Madhu Advani, Andrew Saxe, Florent Krzakala, and Lenka Zdeborová. Dy- namics of stochastic gradient descent for two-layer neural networks in the teacher-student setup. Advances in neural information processing systems, 32, 2019.
Sebastian Goldt, Marc Mézard, Florent Krzakala, and Lenka Zdeborová. Modeling the influence of data structure on learning in neural networks: The hidden manifold model. Physical Review X, 10(4):041044 2020.
Manu Srinath Halvagal and Friedemann Zenke. The combination of hebbian and predictive plasticity learns invariant object representations in deep sensory networks. Nature Neuroscience. 26(11); 1905-1915, 2023.
10

---

Published as a conference paper at ICLR 2024
Hananel Hazan, Daniel J. Saunders, Darpan T. Sanghavi, Hava T. Siegelmann, and Robert Kozma. Unsupervised learning with self-orgamizing spiking neural networks. CoRR, abs/ 1807.0939774 2018.
Geoffrey Hinton. The algoo Some preliminary investigations, 2022. URL https: /tapss///axxivv wttp://pp/p//...ooa///////
Geoffrey E Hinton and James McClelland. Learning representations by recirculation. In Neural information processing systems, 1987.
Bernd Illing, Jean Ventura, Guillaume Bellec, and Wulfram Gerstner. Local plasticity rules can learn deep representations using self-supervised contrastive predictions, 2021.
Adrien Journ,, Hector Garcia Rodriguez, Qinghai Guo, and Timoleon Moraitis. Hebbian deep learning without feedback. arXiv preprint arXiv:2209./1888 1883, 2022.
Jack Kendall, Ross Pantone, Kalpana Manickavasagam, Yoshua Bengio, and Benjamin Scellier. Training end-to-end analog neural networks with equilibrium propagation, 2020. URL https:
hatzxii..rr//bb/220000 01981
Lyes Khacef, Philipp Klein, Matteo Cartiglia, Arianna Rubino, Giacomo Indiveri, and Elisabetta Chicca. Spike-based local synaptic plasticity: A survey of computational models and neuromor- phic circuits, 2022. URL httph:tpp///m /taxxis wwgg/bs/////oooooooee//..oo/ob 2209 15536
Adam Kohan, Edward A. Rietman, and Hava T. Siegelmann. Signal propagation: A framework for learning and inference forward pass, 2022.
Adam A. Kohan, Edward A. Rietman, and Hava T. Siegelmann. Error Nowaardpropagaation Reusing feedforward connections to propagate errors deep learning, 2018. URL https://arxiv.
org abs//bb//it 1808. 03357
G Kreiman. Biological and Computer Vision. Cambridge University Press, 2021.
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-10 (canadian institute for advanced re- search). a. stcttp:/www..r ca. toronto...... wct//kkii// ciirr html
Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. Cifar-100 (canadian institute for advanced research). b. URL
http: //wp:://wwwww.. cs twwonto.
Anders Krogh and John A. Hertz. simple weight decay can improve generalization. In Proceed- ings of the 4th International Conference on Neural Information Processing Systems, NIPS' 91, pp. 950-957, San Francisco, CA, USA, 1991. Morgan Kaufmann Publishers Inc. ISBN 1558602224.
Julien Launay, lacopo Poli, François Boniface, and Florent Krzakala. Direct feedback alignment scales to modern deep learning tasks and architectures. Advances in neural information processing systems, 33:9346-9383 9360, 2020.
Yann LeCun and Corinna Cortes. MNIST handwritten digit database. 2010. BRLhttp:///aan lecun. wwt.c//cceeeeeeo.. http:/exbm/nnisss
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation.
In Joint european conference on machine learning and knowledge discovery in databases, pp- 498-515. Springer, 2015.
Qianli Liao, Joel Z. Leibo, and Tomaso Poggio. How important is weight symmetry in backpropa- gation?, 2016.
Timothy Lillierap, Adam Santoro, Luke Marris, Colin Akerman, and Geoffrey Hinton. Backpropa- gation and the brain. Nature Reviews Neuroscience, 21, 04 2020.
Timothy P. Lillicrap, Daniel Cownden, Douglas Blair Tweed, and Colin J. Akerman. Random synap- tic feedback weights support error backpropagation for deep learning. Nature Communications, 7:13276, 2016a.
11

---

Published as a conference paper at ICLR 2024
T.P. Lillicrap, D. Cownden, D.B. Tweed, and C.J. Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 7:1-10, 2016b.
Guy Lorberbom, Itai Gat, Yossi Adi, Alex Schwing, and Tamir Hazan. Layer collaboration in the forward-forward algorithm. arXiv preprint arXiv:2305. 12393, 2023.
P Mazzoni, R A Andersen, and M Jordan. A more biologically plausible learning rule for neural networks. Proceedings of the National Academy of Sciences, 88(10):443-4447, 1991.
Alexander Meulemans, Matilde Tristany Farinha, Javier Garcia Ordonez, Pau Vilimelis Aceituno, João Sacramento, and Benjamin F. Grewe. Credit assignment in neural networks through deep feedback control. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 4674-4687. Curran Associates, Inc., 2021.
Beren Millidge, Yuhang Song, Tommaso Salvatori, Thomas Lukasiewicz, and Rafal Bogacz. Back- propagation at the infinitesimal inference limit of energy-based models: Unifying predictive cod- ing, equilibrium propagation, and contrastive hebbian learning, 2022.
Theodore H. Moskovitz, Ashok Litwin-Kumar, and L. F. Abbott. Feedback alignment in deep convolutional networks. CoRR, abs/18 hbs/1812.06488, 2018. URL http://arxiv. org/abs/
1812.06483
Suvaditya Mukherjee. Using the Forward-Forwaard Algorithm for Image Classification. https:
7/keras
2023. Online resource.
A. Nøkland. Direct Feedback Alignment Provides Learning in Deep Neural Networks. In Advances in Neural Information Processing Systems 29, 2016.
Arild Nøkland and Lars Hiller Eidnes. Training neural networks with local error signals. In Pro- ceedings of the 36th International Conference on Machine Learning, volume 97, pp. 4839-4850, 09-15 Jun 2019.
Arild Nøkland. Direct feedback alignment provides learning in deep neural networks, 2016.
Alexander Ororbia and Daniel Kifer. The neural coding framework for learning generative models.
Nature Communications, 13:2064, 04 2022. doi: 10. 10.1038//41467--022299333333
Alexander Ororbia and Ankur Mali. Convolutional neural generative coding: Scaling predictive coding to natural images, 2023a.
Alexander Ororbia and Ankur Mali. The predictive forward-forward algorithm, 2023b.
Alexander Ororbia, Ankur Mali, Daniel Kifer, and C. Lee Giles. Large-scale gradient-free deep
learning with recursive local representation alignment, 2020.
Alexander G. Ororbia and Ankur Mali. Biologically motivated algorithms for propagating local target representations. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01): 4651 (658,4658,11 Jul. 2019. doi: 10. 10.1609/aaii 033.0aa....3111...3.3.333333 AAAA/AAAAII /tai
URL wwtphhtp/////// aaai /rg/
Rajesh Rao and Dana Ballard. Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. Nature neuroscience. 2:79-87, 1999. doi: 10.1038/4580.
Maria Refinetti, Stéphane D' Ascoli, Ruben Ohana, and Sebastian Goldt, Align, then memorise: the dynamics of learning with feedback alignment. In Marina Meila and Tong Zhang (eds.), Proceed- ings of the 38th International Conference Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. 9925-8935. PMLR, 18-24 Jul 2021a.
Maria Refinetti, Sebastian Goldt, Florent Krzakala, and Lenka Zdeborová. Classifying high- dimensional gaussian mixtures: Where kemel methods fail and neural networks succeed. In International Conference Machine Learning, pp. 8936-8947, PMLR, 2021b.
12

---

Published as a conference paper at ICLR 2024
Blake A Richards, Timothy P Lillicrap, Philippe Beaudoin, Yoshua Bengio, Rafal Bogacz, Amelia Christensen, Claudia Clopath, Rui Ponte Costa, Archy de Berker, Surya Ganguli, et al. A deep learning framework for neuroscience. Nature neuroscience, 22(11):161-1770, 2019.
Riegler and M. Biehl. On-line backpropagation in two-layered neural networks. Journal of Physics A: Mathematical and General, 28(20), 1995.
Pieter R Roelfsema and Anthony Holtmaat. Control of synaptic plasticity in deep cortical networks.
Nature Reviews Neuroscience, 19(3):166-188) 2018.
David E. Rumelhart, Richard Durbin, Richard Golden, and Yves Chauvin. Backpropagation: The Basic Theory, pp- 1-34. L. Erlbaum Associates Inc., USA, 1995.
D. Saad and S.A. Solla. Exact Solution for On-Line Learning in Multilayer Neural Networks. Phys. Rev. Lett., 74(21):337-4433 1995a.
D. Saad and S.A. Solla. On-line learning in soft committee machines. Phys. Rev. E, 52((4):2255 4243, 1995b.
David Saad and Sara A. Solla. On-line learning in soft committee machines. Phys. Rev. E, 52: 4225 4243, Oct 1995c. doi: 10.100//hy.RevE.5222225 URL httth::t////imm aps org/ 52. 4225
David Saad and Sara A. Solla. Exact solution for on-line learning in multilayer neural networks. Phys. Rev. Lett., 74:4337-4340, May 1995d. doi: 10. Lo. 74.4337. URL https: aps https//rg/.oooo...o..//////eeee. 74t..
Tommaso Salvatori, Yuhang Song. Thomas Lukasiewicz, Rafal Bogacz, and Zhenghua Xu. Predic- tive coding can do exact backpropagation on convolutional and recurrent neural networks, 2021.
Benjamin Scellier and Yoshua Bengio. Equilibrium propagation: Bridging the gap between energy- based models and backpropagation. Frontiers in computational neuroscience, 11:24, 2017.
H. S. Seung, H. Sompolinsky, and N. Tishby. Statistical mechanics of learning from examples. Phys. Rev. A. 45:6056 6091, Apr 1992. doi: 10.1 10.1100//hyseev4445..60 link. aps wwtp//////oo//o///oo//////oo/e// http:///oo....11111//eeeeeee
URL https://
45..555544 6056
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding, 2019.
Rodrigo Veiga, Ludovic STEPHAN, Bruno Loureiro, Florent Krzakala, and Lenka Zdeborova. Phase diagram of stochastic gradient descent in digh-dimensional two-layer neural networks. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neu- ral Information Processing Systems, 2022. URL https://////......... www.rewiew.ww....m net Forum?ic
GL. 3WEdNRM
Timothy L. H. Watkin, Albrecht Rau, and Michael Biehl. The statistical mechanics of learning rule, Rev. Mod. Phys., 65:499-55555 556, Apr 1993, doi: 10.1 wod.//// URL https: .4.999.
7/link. waink.aps.ce www.aap...g.............. ort..oog//oio/oopooppppooo.oo..p /10,
Ronald J. Williams. Simple statistical gralient algorithms for rennectiooisstrinfccoccc learning. Mach. Learn., 8(3-4):229 256, May 1992.
Will Xiao, Honglin Chen, Qianli Liao, and Tomaso Poggio. Biologically-plausible learning algo- rithms can scale to large datasets, 2018.
Xiaohui Xie and H Sebastian Scung. Equivalence of backpropagation and contrastive hebbian learn- ing in layered network. Neural computation, 15(2):441 -44,, 2003.
Lenka Zdeborová and Florent Krzakala. Statistical physics of inference: Thresholds and algorithms. Advances in Physics, 65(5):453-53 2016.
13

---

Published as a conference paper at ICLR 2024
A PSEUDOCODE FOR PEPITA
Algorithm S1 describes the original PEPITA as presented in Dellaferrera & Kreiman (2022).
Algorithm S2 describes our modification of PEPITA, which Hebbian and local both space and in time (Pepita-time-local.
Algorithm $1 Implementation of PEPITA
Algorithm S2 Impl. of PÉPITA-time-loca
Require: Input r and one-hot encoded label y (standard forward pass}
Require: Input x and one-hot encoded label y {standard forward pass}
ho = x for f 1, ..., do h; = ae (Wehee-1)
ho = E for ,. do
h1 = (Welee-1)
end for
AW; = hhh
hy y (modulated forward pass}
WE7 1 We(() - -AW, (apply 1st
update) end for
hor + Fe for L 1....... do
e = h2 y
het or (W
{modulated forward pass}
if then
h = I Fe
AW, (he (KEE))
for 1, ...., do
else
1 = W ME
AW, e(h$=1)
if f 1 then
end if
AW, = -h h
We(t + 1) We(t) - w/W, (apply up-
else
date]
AW = -yh
end for
end if
Wt + 1) = wt (t + 1) (apply
2nd update]
end for
The updates for PEPITA-Hebbian are:
for the hidden layers:
AW. = hoh 112 h
(11)
hehl 1
for the first and last layers
AW = h1x 1 (xr Fe)T; AW, = hihh yh
(12)
These updates are applied at the end of both forward passes for PEPITA-Hcbbian similarly in pseudocode ST
14

---

Published as a conference paper at ICLR 2024
B TRAINING WITH THE TIME-LOCAL RULE
The weight update of the contribute to learning without knowledge of the activity of the other, i.e. learning is time-local. Specifically, the term h can be applied online, immediately as the activations of the first pass are computed. Analogously, the second term - h h can be applied immediately during the second forward pass. To ensure that the hidden-layer updates prescribed by PEPITA are useful, we compared the test curve of PEPITA-TL against a control with F (Fig. [ST). and found that removing the feedback decreases the accuracy from approx. 40% to approx. 18%.
rule in eqn. consists of two separate phases that can
Regarding the localio of FF, the two forward passes can be computed in parallel, as the modu- lated pass does not need to wait for the computation of the error of the first pass. However, according to the available implementations (Mukherjee) 2023) the updates related to both passes are applied together at the end of the second forward pass.
The time-local PEPITA trained on the CIFAR-10 dataset with learning rate 0.01. All the other hyperparameters are the same as the ones reported in Table S3
40
Accuracy (%)
20
PEPMEAAiiee-oo PEP PEPPIAAiiee--ocll. F-0
Training epachs
Figure S1: Test curve for PEРITA in its time-local formulation and time-local PEPITA with F=0 (i.e., only the last layer is trained) on the CIFAR-10 10 dataset. The network has 1 hidden layer with 1024 units. The forward matrices are initialized using the He normal initialization. entries are sampled from a normal distribution with standard deviation 0.5-2 0 66/(22 32.3). We use learning rate 0.0001 and weight decay with X 10-4 The learning is reduced by factor of x0.1 at epoch 50. The plot indicates mean and standard deviation over 10 independent runs. Time-local PEPITA achieves a significantly higher accuracy than the time-local, F=O scheme.
15

---

Published as a conference paper at ICLR 2024
C
DISTRIBUTION OF THE GOODNESS IN PEPITA
b
[1.] 11771
Figure S2: Difference of the norm of the squared activities the first hidden layer between the clean and modulated pass in PEPITA (a) before training, (b) after 50 epochs, and (c) at the end of training. The network is 2-hidden-layer network trained with WD with X = 10 on the CIFAR-10 dataset. The activites are recorded on the test set. We remark that in PEPITA the input of the second forward pass is modulated by the eπror. Since the eπrоr decreases during training, also the difference of the activations in the two passes decreases with training. This explains why the distribution of the difference of the norm of the squared activities has a lower standard deviation in the middle of training (b) and at the end of training (c), than before training
(a). In contrast, the modulation the input in FF is constant during training, and the scope of training is maximising the difference of the goodness two passes.
16

---

Published as a conference paper at ICLR 2024
D ADDITIONAL FIGURES ON THE ADAPTIVE FEEDBACK ALIGNMENT APPROXIMATION
Fig. S3 displays the comparison between the "vanilla" PEPITA algorithm and the AFA approxima- tion introduced in eqn. 6 of the main text. The test accuracy function of training epochs is depicted for the CIFAR-I 1O dataset in the left panel and for the CIFAR- 100 dataset in the right panel.
function of training time. The symbols mark the numerical simulations at dimension D = 500, while the full line represents our theoretical prediction. We observe that, for this run, the of the AF increases over time. We have observed by numerical inspection that this behavior is crucial to speed up the dynamics, as also observed in Dellaferrera & Kreiman 2022
Fig. S4 depicts the norm of the AF matrix f W/F/D
PEPITA
PEPITA
45
AFA
AFA
15
Accuracy
40
35
25
50
75
100
25
75
100
Training epochs
Training spochs.
Figure S3: Comparison between the test accuracy as function of training epochs between the "vanilla" PEPITA algorithm and its AFA approximation (eqn. of the main text) for the CIFAR-10 (left panel) and CIFAR-100 (right panel) datasets.
Smulatioss
Theary
10-1
16
10'
10
10'
16a
16*
10
Figure S4: Norm of the alignment matrix f = F/F as a function of training time, for the same parameters in Fig. of the main text: D 500, ir .05, erf activation, two hidden units in both teacher and student (K M- = 2).
E ORDINARY DIFFERENTIAL EQUATIONS FOR ONLINE LEARNING IN THE TEACHER-STUDENT REGRESSION TASK
In this section, we present the details of the teacher-student model under consideration and we sketch the derivation of the ordinary differential equations tracking the online learning dynamics of the AFA rule. We consider a shallow student network trained with AFA to solve supervised learning task. The input data are random D D-dimensional vectors RD with independent identically distributed (i.i.d)) standard Gaussian entries x, (O, 1), = 1,..., D, and the (scalar) labels are generated as the output of a 1-hidden-layer teacher network with parameters 0 (W1, We, M, 6):
WM
y Σ
WThh (v
(13)
DD
17

---

Published as a conference paper at ICLR 2024
where M is the size of the teacher hidden layer, v denotes the teacher preactivation at unit {1....,M}, and o..) is the activation function. The student is a l-hidden-layer neural network parametrized by 0 = (W1, W2. o) that outputs the prediction
x²= W
ÿ = Σ O2*)
(14)
VD
where K is the size of the student hidden layer, a(-) the student activation function, X the student preactivation at unit & E (1...... K.. For future convenience, we write explicitly the scaling with respect to the input dimension. Therefore, variance with the main text, in this supplemementary section we always rescale the first layer weights as well as the feedback by 1/vD.
We focus on the online (or one-pass) learning protocol, so that at each training time the student network is presented with fresh example Tu. P I, and N/D Op(1). The weights are updated according to the AFA rule defined in eqn. 6
AW = WIF I D
W1 1) (1) 7/ mWWi((A) (A),
ehi
(15)
VD
(2 + W2(u) (2)))())
AW2 - ehi.
(16)
We consider fixed learning rates = 77, = n/D. Different learning rate regimes have been explored in Veiga et al. (2022). is crucial to notice that the mean squared generalization error
1 Fr <g 2
Σ Wh Σ o
(17)
depends on the high-ding digh-dimensiona input expectation only through the low-dimensional expectation over the preactivations { [vm Jm- Notice that, in this online-learning setup, the input is independent of the weights, which are held fixed when taking the expectation. Furthermore, due to
the Gaussianity of the inputs, the preactivations are also jointly Gaussian with zero mean and second moments:
W W D
Q = Ex [>=
(18)
Wh. W
Rkm
E. [>"
(19)
WTn WT
7mn = Ex [UM
(20)
D
The above matrices are named order parameters in the statistical physics literature and play an important role in the interpretation. The matrices Q and capture the self-overlap of the student and teacher networks respectively, while the matrix It encodes the teacher-student overlap. In the infiniteddimensional limit discussed above, the generalization error is only function of the order parameters Q, T, T, R and of the second layer weights W2, W2 of teacher and student respectively. Therefore, by tracking the evolution of these matrices via a set of ODEs their "equations of motion" obtain theoretical predictions for the learning curves. The update equations for Q, R. W2 can be obtained from eqn. [15] according to the following rationale. As an example, we consider the update equation for the matrix Q:
2 + 1) (4)
D (w) -nw [W((M) - (WW((a) 1
1
wFw w(()
(21)
- f
1
where have defined the AF f F/F,D, we have used that ||> 12 D as D do and omitted the / dependence on the right hand side for simplicity. By taking t. = μ/D, as shown in Goldt al. (201)), in the infinitediimensional limit (*) concentrates to the solution of the
following ODE:
dQki
-n [' - [2)() + "
(22)
dt
18

---

Published as a conference paper at ICLR 2024
Similarly, derive ODEs for the evolution of R. W2 and the AF f: df
dR
dw
di - f (o e]"
n (o(t)e)
- (oo (X* (23)
di
dt
where the expectations are taken over the preactivatons and p F x/VD, and we have [x fh, Ey ( p| fr == W2F/ D. qs F F/D. The generalization error be rewritten as
M
K
lim E, c, D-
Σ
w 12 +
Σ
W
12(m, n)
W W (2 m),,
k,1=1
m,R=1
(24)
where 12 generically encodes the averages over the activations
f+ a2 3) = E [c((((")))((())) y = vm if m,n" 0a a if a k,
if k,l
ā a m, (25)
The other averages in eqns. can be expressed in similar way and estimated by Monte Carlo meth- ods. In the case of sigmoidal activation (xx) (xf 222, the function I2 has analytic expres- sion.
2 π V1 Cas CaavI+ C93
C
(2 B) arcsin
Ctkl Q+1, O. Bkmm
C
7 (26)
E. EARLY-TRAINING EXPANSION
As done by Refinetti al. 2021a) for DFA, it instructive consider expansion of the ODEs at early training times. We assume the following initialization: W (0) = 0, Vk E {1...., , K), while the first layer is assumed to be orthogonal to the teacher (+) W = 0 and of fixed norm || (0)]l//D //D qo. Vk {i..... K}. Vm f .M). We also take orthogonal first-layer teacher weights, such that T is the identity matrix. This initialization leads to:
d
d
v2
Re 0,
w dt
Rkm dt
7) 1 (0) W..
(27)
T + 90
t=
1=
We can therefore compute the second-layer update linear order:
2 x2(1+ 1² (²( W t + 0(t9)
dt (t)
(28)
Eqn. 28 shows that the update the second-layer weights at early training times is in the direction of the AF matrix, in agreement with the alignment phase observed in experiments. Crucially, it is necessary that f*(0) # 0 vk in order to have non-zero updates, i.c. the feedback F' must not be orthogonal the first-layer weights at initialization. We now inspect the behavior the FA the beginning of training. We have that the update at time zero is:
d dl
2
(0)
(W2 f),
(29)
t=
V(1
20)
+ 9f) (+())=
and
2
(*(
=² = (0) +
v(1 + go)(1 + 45) ft (0)² (W2 81+
(30)
Eqn. 29 illustrates that the feedback-teachee alignment plays important role in speeding up the
dynamics. Indeed, if [| close to zero, the feedback update slows down inducing long plateaus in the generalization error. similar role is played by the alignment angle between W2 and f.
19

---

Published as a conference paper at ICLR 2024
F PEPITA'S RESULTS COMPARED TO THE BASELINES
Table SI: Test accuracy [%] achieved by BP, FA, DRTP, PEPITA, and PEPITA with WM in the experiments. Mean and standard deviation computed over 10 independent runs. The nonlinearity ReLU for all algorithms except DRTP, for which tanh. WM used in combination with weight decay with 10-1 for the networks trained on the CIFAR-10 dataset and for the 3-hidden-layer networks trained on the CIFAR-100 dataset. The other networks are trained without weight decay. Bold fonts refer to the best results exclusively among PEPITA and its improvements.
HIDDEN LAYERS
HIDDEN LAYERS
MNIST
CIFAR-10
CIFAR-100 100
MNIST
CIFAR-10
CIFAR- 100
BP
98.85+0.06
59.69+0.25
32.28+0.17
98.89++0.04
60.07+00.28
32.80++0.16
FA
98.6410.05 57.7610.39
22.9010.14
97.4810.06
52.99-0.21
22.81+0.21
DRTP
95.36+0.009
47.48+0.19
20.55+0.30
95.74+0.10
47.44+0.19
21.81+0.24
PEPITA
98.1910.07
52.3910.27
24.8811.115
95.0710.11
52.47:0.24
01.00=0.00
PEPITA
98.09=0.07
53.09+0.33
24.6440.24
95.09
52.56+0.31
23.13+0.21
+WD PEPITA
98.13:0.05
53.4440.28
26.95:1.24
96.33++0.12
52.80
23.03::.288
+WM
G SLOWNESS RESULTS
In analysis of convergence rate, we use the plateau equation for learning curves (Dellaferrera
et al. 2022):
max_accuracy epochs
accuracy slowness + epochs
(31)
By fitting the test curve to this equation, we calculate the slowness parameter, which measures how quickly the network reduces error during training. In mathematical terms, the slowness value corresponds to the number of epochs needed to reach half of the maximum accuracy. In practice, lower slowness value indicates faster training.
Table S2 shows that the best convergence rate for PEPITA (i.e., small slowness value) is obtained in general by PEPITA with WM (MNIST, CIFAR-100), Compared to the baselines, PEPITA has better convergence than all the algorithms on MNIST, is the slowest on CIFAR-IO, and the second best after BP on CIFAR-100. However, these results are strongly dependent on the chosen learning rate.
20

---

Published as a conference paper at ICLR 2024
Table S2: Convergence rate in terms of slowness value obtained by BP, FA, DRTP and PEPITA in the experiments for the fully connected models trained on MNIST, CIFAR- 10 and CIFAR-100 (same simulations reported in Table 2 PreM refers pre-mirroring (Sec. D. The smallest the slowness value, the better the convergence rate. The slowness is computed on the first 60 epochs of the test curve (before the learning rate decay), averaged over independent runs. All the networks are trained without weight decay. The slowness result of FA CIFAR-I0 is lower than in Dellaferrera Kreiman (2022) as our grid search returned higher value for the learning rate. Bold fonts refer to the best results exclusively among PEPITA and its improvements.
1x2024 FULLY CONNECTED MODELS
MNIST
CIFAR-10
CIFAR-100
BP FA DRTP
0.061+0.001
0.421
1.406=0.055
0.081+0.002
0.463+0.020
4.946+00.22 123
0.059+0.002
0.362++0.021
12.904+00444
PEPITA
0.05211.000
0.89410.071
2.69510.166
DEPITA+WM
0.047-10.005
0.890-100081
2.333110.102
PEPITA+WM+PERE
0.04010.000
0.85610.005
1.99910.059
H HYPERPARAMETERS
Table S3: 1-hidden-layer network architectures and settings used in the experiments. The nonlinear- ity is ReLU for all algorithms except DRTP, for which is tanh.
HIDDEN LAYER
HIDDEN LAYER NORMALIZATION
MNIST
CIFAR-I0
CIFAR-100
MNIST
CIFAR-10
CIFAR-100
INPUTSIZE
28x
32x32x3 1x1024
32x32x3 Ix1024
28x28x1 1x1024
32x32x3 1x1024
32x32x3
HIDDEN UNITS
1x1024
1x1024
OUTPUT UNITS
10
10
100
10
10
100
BP
0.1
0.01
0.1
7 FA
0.1
0.01
0.01
7) DRTP
0.001
PEPITA
0.1
0.01
0.01
100
10
100
WEIGHT DECAY
10-5
10
10-3
0.0
0.0
X0.1
x0.5
x0.1
DECAY EPOCH
60,90
60,90
60,90
60,90
60,90
60,90
BATCH SIZE
WM
0.1
0.001
0.1
0.001
0.001
0.001
WEIGHT DECAY WM
0.0
0.5
0.1
0.1
OF (UNIFORM)
0.05-2
0.05-2
0.05-2
0.05-2
0.05-2
0.05-2
(0)
(NORMAL.)
0.05
0.05
2
0.05
2
0.05
0.05
0.05
FANK
FANIN
FANTN
PANTH
RANT
FAN
28 28
32 -2
32 32
33
28 28
1
32 32
32 32232
#EPOCHS
100
100
100
100
DROPOUT
10%
10%
10%
10%
10%
10%
21

---

Published as a conference paper at ICLR 2024
Table S4: 2-, 3-hidden-layer network architectures and settings used in the experiments. The nonlin- earity is ReLU for all algorithms except DRTP, for which is tanh. (+) For the 3-hidden-layer network trained with PEPITA on the MNIST dataset, we do not use learning rate decay, as indicated by the grid search.
JIIDDEN LAYERS
JIDDEEN LAYERS
MNIST
CIFAR-10
CIFAR-100
MNIST
CIFAR-10
CIFAR-100
INPUTSIZE
18x28x1
32x32x3
32x32x3
18x28x1
32x32x3
32x32x3
HIDDEN UNITS
2x 1024
2x 1024
1x24
3x1024
3x 1024
3x 1024
OUTPUT UNITS
10
Y BP
0.1
0.01
0.1
0.01
0.1
FA
0.1
0.01
0.01
0.01
0.001
0.01
DRTP
0,001
0..01
0.001
0.001
0.001
0.001
PEPITA
0.1
0.01
0.01
0.001
0.01
0.01
WEIGHT DECAY
10
10
10
(*)
X0.1
x0.1
x0.1
X0.1
x0.1
x0.1
DECAY EPOCH
60,90
60,90
60,90
60,90
60,90
60,90
BATCH SIZE
64
64
64
64
WM
0,00001
1.0
1.0
0.001
0.001
WD WM
0.0
0.1
0.1
0.001
0.1
0.1
(UNIFORM)
0.05-2
0.05-2
0.05 2 PANIS
0.05-2
0.05-2
0.05
PANEN
PAMIN
FANIN
FAMIN
PANKS
OF (NORMAL)
0.05-
0.05
0.05 V
0.05-
0.05
0.05
FAN IN
28 28
32 32 -3
32 32 3
28 25
32 32 -3
32 32 3
#EPOCHS
100
DROPOUT
10%
10%
10%
10%
10%
10%
I TRAINING DEEPER FULLY-CONNECTED MODELS
In the field of biologically inspired learning, reaching convergence networks with more than a couple of hidden layers is a long standing challenge. Two significant reasons for this are difficulty of learning hierarchical representations and the explosion of weight updates and activities in deep layers Ilnin et al.) (22I)) In particular, forward-only learning schemes have been shown to work far maximum of four hidden layers (FF (Hinton 2022)). The original PEPITA paper was only able to train hidden layer models (Dellaferrera & Kreiman, (2022). therefore our strategy reach convergence with up to hidden layers (Fig. (55) represents significant improvement over the previous work.
b
7C
Nourso
02
Noursoo [S]
20
94
82
z Hiddn byyers
# Hiddon layers
#1iddon ayers
Figure S5: Test accuracy obtained with PEPITA and normalization of the activations for 1- to 5- hidden-layer networks on (a) MNIST, (b) CIFAR-10, and (c) CIFAR-100. Note that compared to activation normalization to obtain convergence for
Fig. 3 the trend is decreasing, as here and hidden layers, which we did not use for Fig.
22

---

Published as a conference paper at ICLR 2024
J WEIGHT MIRRORING
Algorithm S3 describes how WM applied to PEPITA. This algorithm is applied at the end of each
training epoch. O' (I+1) refers to the standard deviation of the entries of F(t + 1) = II Fi 1).
Algorithm S3 Implementation of WM
[Mirror weights} for & 1.., do
8, NN,, 0²)
6, = 0r (W 111-1)
Fe(t + Fe(t) 16
end for
(Normalize feedback matrices)
for
= I,.... do
1) oF /0p (o(t1))
(0) 1/L Fr
end for
100
100
nommi
PEPTTK-AM no proM PEPTIA-MM with pesl
PERITA-WM no proMf PEPITA WM with proM
80
F W,W,
60
4
40
40
0
25
50
75
100
25
50
76
100
Training spochs
Training spochs
Figure S6: Alignment angle between F and Wio during training with (pink and purple curves) or without (blue curve) WM for the MNIST (left panel) and CIFAR-100 (right panel) datasets. PreM refers to reermrrroring (Sec. 4. The hyperparameters are reported in Table S3 The plots indicate mean and standard deviation over 10 independent runs.
b
90-
85
HPP Swiit will
75
60
30
36 Thaming
briniing epouls
Figure S7: Alignment angle between F and Wist during training with (pink curve) or without (blue curve) WM for the MNIST dataset for (a) 1-, (b) 2-, (c) 3-hidden-layer networks. The hy- perparameters are reported in Table S3 for the I-hidden-layer network and Table S4 for the 2- and 3-hidden-layer networks. The plots indicate mean and standard deviation over 10 independent runs.
23

---

Published as a conference paper at ICLR 2024
a
b
120
100
100
rETIAANMM zsM
PEPITA paid
60
Figure S8: Alignment angle between F and W... during training with (pink curve) or without (blue curve) WM for the CIFAR-10 10 dataset for (a) 1-, (b) 2-, (c) 3-hidden-layer networks. The hyperparameters are reported in Table for the l-hidden-layer network and Table the and 3 --iddenn-layer networks. The plots indicate mean and standard deviation over 10 independent runs.
24


Reviews:
[{"review_id": "gUrBsPKV1n", "reviewer": "Reviewer_sGVa", "summary": "Forward-only learning algorithms are an alternative to backpropagation (BP), which are potentially more biologically plausible, more memory efficient, and more computationally efficient.\nTwo of these forward-only learning algorithms are Forward-Forward (FF) and PEPITA, the Present the Error to Perturb the Input To modulate the Activity learning rule; both algorithms rely on two forward passes to learn, rather than a forward and a backward pass.\nHowever, the learning dynamics of these algorithms are not well understood, and both still include components that are biologically implausible.\nIn this work, the authors offers an explanation of how PEPITA learns by connecting them to feedback alignment.\nThey then use their insight to improve the performance of PEPITA by combining it with weight mirroring, a technique to improve performance of feedback alignment, as well as weight decay and activity normalization.\nFinally, they propose PEPITA-time-local, an alternative learning rule that improves the biological plausibility of PEPITA by keeping updates local in time (in addition to space), at the cost of some accuracy.", "questions": "- Figure 3 shows the performance of PEPITA-Hebbian, PEPITA + weight decay, and PEPITA + weight mirroring on CIFAR-10. Do the same trends hold for MNIST and CIFAR-100?\n\n- I’m a bit confused on how much the PEPITA-Hebbian approximation impacts the accuracy of the model. Figure 3a shows that the impact is very small on CIFAR-10, maintaining over 50% accuracy on CIFAR-10 for 1, 2, and 3 hidden layers, but in Figure S1 the accuracy of a 1-layer PEPITA-TL model drops to around 40%. Could you explain why this is? If this is due to hyperparameter differences, how well would a non-TL PEPITA model compare in performance under the same hypersparameters/conditions as those used for Figure S1?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "- This paper exposes a novel connection between feedback alignment and forward-only algorithms. This connection is particularly interesting because both type of algorithms have improving biological plausibility as a principal goal; exposing underlying relationships may mutually benefit researchers for both types of algorithms. In fact, the authors have already taken a step in this direction, by empirically showing that weight mirroring, a technique developed for feedback alignment, can also be used to improve the performance of PEPITA.\n- As the authors also point out, while forward-only learning algorithms have gained traction in recent years, and are gradually catching up to the performance of backpropagation on increasingly complex tasks, theoretical understanding of their learning dynamics is mostly lacking; work that shed light on how forward-only learning works is very welcome.\n- The experiments performed are extensive, and the results are convincing.", "weaknesses": "- The theoretical analysis focuses only on shallow 2-layer networks, and I’m uncertain that the Adaptive Feedback rule can be easily extended to analyze deeper networks, as the error signal has to pass through multiple hidden layers either forwards (forward-only) or backwards (feedback alignment). This limits the applicability of the analysis, making it disjoint from the experimental results on multi-layer PEPITA networks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CAFQrSYUkW", "reviewer": "Reviewer_QDBz", "summary": "The paper connects several forward pass-only learning methods as biologically plausible alternatives to backprop, analyses their performance and error dynamics, and also proposes Hebbian and temporally local approximations for one of those methods.", "questions": "If I understand correctly, PEPITA-TL differs from PEPITA-Hebbian by one feature: the anti-Hebbian error term is computed for the updated weights. Based on that, I have two questions:\n1. Would performance of both algorithms match if we reduce the learning rate of PEPITA-TL? Intuitively it should, since the anti-Hebbian error terms would match.\n2. Can we consider PEPITA-Hebbian temporally local since it's a long-term plasticity rule? Since the network is trained with two passes over the same input, the PEPITA-TL approximation adds the first Hebbian term immediately after the first forward pass, so 10s/100s of ms. This would be too short for long-term Hebbian changes (as far as I know), so in your model you can treat both parts of the update rule as if they happen for the same network state -- meaning that PEPITA-Hebbian would be temporally local. (I guess you can account for short-term plasticity as a result of the first forward pass (so non-Hebbian changes proportional to input activity only).)\n\nSmall comments:\n1. Tab1 should have PEPITA-Hebbian performance too.\n2. The appendix can be included in the main pdf in this conference.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The paper is well-written and does a good job articulating its aims and contributions.\n2. The paper provides a theoretical link between several algorithms, and derives error dynamics (for a simple problem) for one of them. \n3. The Hebbian/anti-Hebbian approximation works as well as the PEPITA algorithm, which is a good sign for bio plausibility (of both PEPITA and FF, since they work similarly).", "weaknesses": "1. Performance on CIFAR10/100 of algorithms is lacking compared to backprop (although surprisingly much less so for CIFAR100). This performance gap would likely increase for larger networks.\n2. No experiments with convnets, and I haven't found a justification for that. Since cifar10/100 performance gets much better with introducing convolutions (for backprop), it'd be interesting to look at the performance gap there.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "RzQfnEwCrf", "reviewer": "Reviewer_uKzr", "summary": "This paper analyzes the Present the Error to Perturb the Input To modulate the Activity (PEPITA) learning algorithm, showing its connection to Direct Feedback Alignment (DFA) as well as Forward-Forward (FF) algorithm with negative samples being the “modulated” samples in PEPITA. Some empirical experiments also study PEPITA on deeper networks and with several techniques.", "questions": "1. In Section 3.2, what’s the difference between the analysis and previous analyses for DFA?\n\n2. In Section 5.1, what’s the connection of the derivation with contrastive Hebbian learning [1] or equilibrium propagation [2], which also takes the form of Hebbian and Anti-Hebbian phases?\n\n[1] Equivalence of backpropagation and contrastive Hebbian learning in a layered network. Neural Computation, 2003.\n\n[2] Equilibrium propagation: bridging the gap between energy-based models and backpropagation. Frontiers in Computational Neuroscience, 2017.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "This paper studies the connections between several biologically more plausible algorithms, providing insights from empirical and analytical perspectives.", "weaknesses": "1. I think this paper requires a clearer definition of the considered “forward-only” algorithms. In the abstract, it is defined as “avoiding a backward pass”. However, PEPITA, different from local FF, indeed requires a direct global error feedback to inputs. If PEPITA is considered as “forward-only”, is DFA also “forward-only”, as it shares the similar form of direct global error feedback without layer-by-layer BP? In the sense of “forward-only” algorithms, I don’t think PEPITA is parallel to FF with local learning.\n\n2. In Table 1, why the considered rule has a much worse performance? It is unclear why we should consider it as it does not introduce additional properties over several algorithms while being worse. Additionally, what’s the detailed definition for “local” and “activity freezing”? For PEPITA, it requires global error feedback to inputs, and storage of activations of the first forward propagation until the second forward propagation ends. Why it is considered “local” and solving “activity freezing”? And why PEPITA outperforms DFA in these aspects?\n\n3. In Section 3.1, the similarity to DFA only holds for the first layer. It is unclear for other layers (Eq. (4)).\n\n4. In Section 4, the networks are still quite shallow and experimental results are quite poor. DFA can scale to large datasets and networks [1], and WM also scales to large-scale scenarios [2].\n\n5. Some derivations are not strict and may be contradictory. For example, in Section 3 and Section 5, it is assumed for several times that $Fe$ is much smaller than $x$ so it can be ignored. In Section 5, however, it is also assumed that the “modulated” samples can be used for negative samples in the FF framework. If the modulation is small enough, why can it formulate a valid negative sample? This largely reduces the reliability of the connection between PEPITA and FF.\n\nOverall, I think there is no enough contribution in empirical or theoretical perspectives.\n\n[1] Direct feedback alignment scales to modern deep learning tasks and architectures. NeurIPS, 2020.\n\n[2] Deep learning without weight transport. NeurIPS, 2019.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MQTHAuclRi", "reviewer": "Reviewer_knnT", "summary": "This paper provides further analysis of the PEPITA learning rule, a biologically-inspired \"forward-only\" approach to train neural networks by replacing the backward pass with another forward pass. In connects PEPITA theoretically to two other biologically-inspired approaches, namely an adaptive version of Feedback Alignment and Forward-Forward. Additionally, the paper explores how the initialization of the feedback weights and applying techniques like weight decay, activation normalization, and weight mirroring can improve PEPITA's performance. The experimental findings demonstrate minor improvements through normalization and weight mirroring.", "questions": "- Which approach do the authors considers their primary contribution? Table 1 the authors refer to PEPITA-TL as \"ours\" yet this is only discussed briefly in Section 5.1 and left mostly to future work.\n- Why are the combinations \"weight-decay + normalization + WM\" and \"weight-decay + normalization + no WM\" missing in Table 2? Can weight decay and activation normalization not be combined?\n- In Figure 3c: shouldn't pre-mirroring increase the alignment (thus decreasing the angle) before the first epoch? Yet, in Figure 3c, all three lines seem to start at the same value.", "limitations": "", "rating": 5, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Biologically-inspired and forward-only training approaches are a relevant and timely topic. \n- It would be very useful to unify the multitude of different approaches into a common framework. By connecting three major biologically-inspired approaches, this paper takes an important step towards that goal. \n- Given the still-remaining gap between backpropagation and forward-only approaches, analyzing and improving their performance is essential for their practical relevance.\n- Section 4.2 gives a well-structured summary of the experimental results. It would have been nice, to include a discussion of the impact of weight decay, activation normalization, and weight mirroring", "weaknesses": "- The experimental improvement and evaluation of PEPITA on their own offer only limited novelty. However, together with the analysis connecting PEPITA, FA, and Forward-Forward, this paper amounts to a considerable contribution.\n- experimental evaluation:\n - only small, fully-connected networks, not that suited to more complex vision datasets\n - only image classification (MNIST, CIFAR10, CIFAR100)\n - no experimental comparison to Forward-Forward\n - only minor improvements in accuracy\n- Minor Issues:\n - Figures can be a bit hard to read\n  - in Fig 2 (especially 2b) the overlap of PEPITA and AFA is hard to see without zooming in\n  - Fig 3 would benefit from more contrasting colors\n - the references to Figure 2c in the last paragraph of Section 3.2 seems to be mismatched with the actual order of subfigures in Figure 2\n - Table 1, line 1 seems to have the wrong citation for FA\n - Spelling:\n  - in contribution (vi) \"analitically\"\n  - in the last sentence of Section 1: \"gpus\" should probably be in uppercase, i.e. \"GPUs\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)