Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Authors: Changze Lv
Jingwen Xu
Yiyang Lu
Xiaohua Wang
Zhenghua Wang
Zhibo Xu
Di Yu
Xin Du
Xiaoqing Zheng
Xuanjing Huang
Venue: ICML 2025 poster (2025)
Keywords: Biological Plausibility
Learning Algorithm
Primary area: applications->neuroscience_cognitive_science
TL;DR: We propose Dendritic Localized Learning (DLL), a biologically plausible learning algorithm.

Abstract: Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's success.
However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, as highlighted by the existing literature. 
Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultaneously or yield suboptimal results.
Inspired by the dynamics and plasticity of pyramidal neurons, we propose Dendritic Localized Learning (DLL), a novel learning algorithm designed to overcome these challenges.
Extensive empirical experiments demonstrate that DLL satisfies all three criteria of biological plausibility while achieving state-of-the-art performance among algorithms that meet these requirements.
Furthermore, DLL exhibits strong generalization across a range of architectures, including MLPs, CNNs, and RNNs.
These results, benchmarked against existing biologically plausible learning algorithms, offer valuable empirical insights for future research.
We hope this study can inspire the development of new biologically plausible algorithms for training multilayer networks and advancing progress in both neuroscience and machine learning.
Our code is available at https://github.com/Lvchangze/Dendritic-Localized-Learning.

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Changze Lv*!
Jingwen Xu* Yiyang Lu Xiaohua Wang' Zhenghua Wang' Zhibo Xu Di Yu
Xin Du Xiaoqing Zheng' Xuanjing Huang'
Abstract
et al., 2015), establishing itself as the standard approach for training neural networks. Despite its undeniable success and widespread adoption in various applications ranging from image recognition (He et al., 2016; Dosovitskiy et al., 2020) to natural language processing (Devlin et al., 2019; Brown et al., 2020), the biological plausibility of backpropagation remains a subject of intense debate among researchers in both neuroscience and computational science (Bianchini et al., 1997; Payeur et al., 2021; Zahid et al., 2023).
Backpropagation is the foundational algorithm for training neural networks and a key driver of deep learning's s success. However, its biological plausibility has been challenged due to three primary limitations: weight symmetry, reliance on global error signals, and the dual-phase nature of training, highlighted by the existing literature. Although various alternative learning approaches have been proposed to address these issues, most either fail to satisfy all three criteria simultancously or yield suboptimal results. Inspired by the dynamics and plasticity of pyra- midal neurons, we propose Dendritic Localized Learning (DLL), novel learning algorithm designed to overcome these challenges. Extensive empirical experiments demonstrate that DLL satisfies all three criteria of biological plausibility while achieving state-of-the-art performance among algorithms that meet these requirements. Furthermore, DLL exhibits strong generalization across range of architectures, including MIPss CNNs, and RNNs. These results, benchmarked against existing biologically plausible learning algorithms, offer valuable empirical insights for future research. We hope this study can inspire the development of new biologically plausible algorithms for training multilayer networks and advancing progress in both neuroscience and machine learning. Our code is available at https: / thub. com / Lvchangze/ Loddrittc-Loolizzed Learning.
The primary criticisms of backpropagation's 's biological plau- sibility stem from several unrealistic requirements: the sym- metry of weight updates in the forward and backward passes (Stork, 1989), the computation of global errors that must be propagated backward through all layers (Crick, 1989), and the necessity of dual-phase training process involving dis- tinct forward and backward passes (Guerguiev et al., 2017; Hinton, 2022). These features lack clear analogs in neurobi- ological processes, which operate under the constraints of local information processing. Recognizing these limitations, the research community has made significant strides toward developing alternative training algorithms that could poten- tially align more closely with biological processes. Each of these approaches offers a unique perspective on how synap- tic changes might occur in biologically plausible manner, yet consensus on an effective and biologically plausible training method remains out reach.
In this paper, we first assess existing biologically plausi- ble learning algorithms systematically, including feedback aligmment (Lillicrap et al., 2016), local losses (Marblestone et al., 2016). predictive coding (Rao & Ballard, 1999; Whit- tington & Bogacz, 2017), perturbation learning (Williams, 1992; Werfel et al., 2003), target propaggtion (Bengio, 201 4), Hebbian learning (Hebb, 1949; Munakata & Pfaffly, 2004), STDP (Song et al., 2000), the forward algorithm (Hinton, 2022), and energy-based learning (Hopfield, 1984; Scellier & Bengio, 2017). From this review, we summarize three criteria that any learning algorithm must meet to be considered biologically plausible: C1. Asymmetry of For- ward and Backward Weights - reflecting the inherent lack of symmetry in real synaptic connections; C2. Local Er- ror Representation - ensuring computations are localized, without requiring global error signal: C3. Non-two-stage Training enabling the simultaneous occurrence of infer-
1. Introduction
Backpropagation (Rumelhart et al., 1986) has been instru- mental in the rapid development of deep learning (LeCun
'Equal contribution 'School of Computer Science, Fudan Uni- versity "School of Software Technology, Zhejiang University. Cor- respondence to: Xiaoqing Zheng <zhengxq@ @
Proceedings of the nnnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
:
:
Layer 4
Layer 4
Layer 4
Layer
Output
Target
WA
W
W
Wrand W
We
W
Layer 3
Layer 3
Random
3 3
classifiers
Layer
Iteration
until
Layer L &
Local Error
Gradients
Layer 2
Layer 2
Layer
Layer
local
converge
Layer 1
Layer 1
Layer 1
Layer
Layer 3 &g
(a) Backpropagation
(b) Feedback Alignment
(c) Local Losses
(d) Predictive Coding
:
:
Layer 2 &
Layer
Layer
Classifier
Layer
W
W
W
W
W
Layer 3
Perturb
Layer
Calculate
Last Layer
Layer
W
Update
Layer 1 &
W to W
target
when
Layer 2
Layer 2
from
Layer
Layer
forward
Input
layer
Layer
Layer
Layer
Layer
(i) Dendritic Localized Learning
(e) Perturbation Learning
(f) Target Propagation (g) Hebbian Learning (h) Forward-Forree Forward
Figure/ 1. Illustrations of biologically plausible learning algorithms. (a) Backpropagation; (h) In feedhack alignment, the weight matrix W is replaced with random matrix during backpropagation: (c) In local losses, classic backpropagation is applied layer by layer; (d) In predictive coding, transposed weights W are used iteratively for local convergence: (e) perturbation learning, the weights are randomly perturbed after the forward pass, generating new W for the next iteration; (f) In target propagation, two sets of weights are used: forward weights W and backward weights W, with W used to calculate targets from the last layer; (g) In Hebbian learning, the final classification layer is trained using gradients; (h) The forward-forward forward algorithm updates weights during the forward pass; (i) In DLL, weights W and O are asymmetric and updated simultancously, with local errors being computed within the layer.
ence and training stages. We then conduct experiments to evaluate current algorithms across a variety of network archi- tectures and real-world datasets to assess their performance and biological plausibility. We observe that algorithms that satisfy all three criteria tend demonstrate significantly lower performance compared to backpropagation bench- mark datasets. What's worse, several algorithms may even fail to converge on specific architectures and tasks.
the backward pass, thereby ensuring compliance with the criterion of asymmetry weights. Lastly, in DLL., the infor- mation can be separated in space within a cell, then the two propagation phases, feedforward and feedback, do not require striet temporal segregation and hence could occur simultaneously. Through comprehensive experiments, demonstrate the effectiveness of DLL various bench- marks across diverse model architectures. Furthermore, we implement our DLL algorithm on time-varying recurrent neural networks (RNNs) and give a detailed derivation to support its theoretical rationality. Ultimately, we aim to not only highlight the current strengths and limitations of bio- logically plausible learning algorithms but also to stimulate further research and innovation in this vital area.
These limitations drive our exploration into the development of algorithms that both adhere to biological plausibility and maintain high performance. Inspired by pyramidal neurons (DeFelipe & Fariñas, 1992), which constitute approximately 70-85% of the total population of neurons in the cerebral cortex, we propose Dendritic Localized Learning (DLL), novel learning algorithm that satisfies all three criteria and maintains strong performance. First, we model the pyrami- dal neuron as comprising three distinct compartments: the soma, apical dendrite, and basal dendrite. Evidence (Sprus- ton, 2008) suggests that the apical dendrites of pyramidal neurons receive inputs from other cortical areas and non- specific thalamic sources, while the basal and side branches are primarily driven by inputs from lower-layer cells. Based on this, we propose that sensory input is directed to the basal dendrite, whereas the expected value is routed to the apical dendrite. The local error is then computed within the soma. Second, we propose the use of trainable backward weights to replace the transposed forward weights during
To conclude, our contributions can be summarized as:
We review current biologically plausible learning algo- rithms and summarize 3 criteria for biological plausibil- ity that an ideal learning algorithm should satisfy. We empirically benchmark these algorithms across diverse network architectures and datasets.
We propose Dendritic Localized Learning (DLL), learning algorithm satisfying all criteria of biological plausibility, to train multilayer neural networks.
We conduct extensive experiments on leveraging the DLL algorithm to train MLPs, CNNs, and RNNs across various tasks, including image recognition, text char-
2

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
acter prediction, and time-series forecasting, showing comparable performance to backpropagation.
local loss for layer is L' = ||a' t'1 where at is the activation, and t! is local target. Each layer minimizes its own loss independently: A Aw! = ? aww. This method promotes scalability and modularity by avoiding reliance on global error signals.
2. Criteria for Biological Plausibility
In this section, we first offer three criteria for biological plausibility that an ideal learning algorithm should satisfy, based on the methods reviewed. Secondly, we proceed to assess the current biologically plausible learning algorithms.
Predictive Coding Predictive coding (Rao & Ballard, 1999; W Whittington & Bogacz, 2017; Millidge et al., 2023; Salvatori et al., 2024) posits that the brain minimizes predic- tion error by iteratively updating its internal model to better predict incoming sensory inputs. This translates into hierar- chical networks where each layer predicts the activity of the layer below. The loss function is L -- >, 1 where x' = (uu; (W')7) is the backpropagated activ- ity of the lower layer. Weights are updated by backpropagat- ing the prediction errors: AW' = 7WT. Similar to local losses, although this method utilizes local error signals, it fails to satisfy C1 and C3.
2.1. Criteria
Summarized from existing literature, we propose three cri- teria for evaluating the biological plausibility of learning algorithms:
C1. Asymmetry of Forward and Backward Weights. In conventional neural networks, forward-path neurons transmit their synaptic weights to the feedback path through a process known as weight transpose, which is considered biologically implausible. Real neurons are unlikely to share precise synaptic weights in this manner.
Perturbation Learning Perturbation learning (Williams, 1992) introduces random noise & to the network's weights or inputs and observes the resulting change in output. The gradient is estimated using the finite difference method: A W = L where AC is the change in loss. If W are perturbed by W eG, the loss difference is: AL ~ L(W + <E) - L(W). However, the algorithm's 's reliance on distinct forward and perturbation phases indicates a departure from the simultaneous and seamless integration of inference and learning processes.
C2. Local Error Representation. Biological synapses are thought to adjust their strength based solely on local information, without relying on a global error signal. This is in stark contrast to seain-rule-baser optimization meth- ods, which typically compute error gradients using global information.
C3. too-two-stage Training. Traditional training meth- ods often involve distinct forward (inference) and backward (updating) phases, a feature absent in biological learning. However, the two propagation phases of a biological learn- ing system, i.c., feedforward and backward, do not require strict temporal segregation and can occur simultaneously.
Target Propagation Target propagation (Bengio, 2014) addresses the biological implausibility of backpropaga- tion by replacing error gradients with target activations. Each layer learns to approximate a target output that min- imizes the overall error. The local objective for layer / is L = ||aa - -11 where a' is the current activation, and t' is the computed target. The weight update minimizes this local loss: W + W - WW Although vanilla tar- get propagation satisfies all the criteria, it faces significant challenges in approximating the inverse function, leading to instability in convergence. Difference Target Propagation (Lee et al., 2014) is introduced to address this issue, but it violates C3 because it requires the upper layers to prop- agate two separate values at different times to update the backward and forward weights.
2.2. Assessment on Learning Algorithms
ac'
We show the detailed illustrations of the mentioned existing learning algorithms in Figure 1.
Feedback alignment (Lillicrap et al., 2016; Nøkland, 2016) modifies backpropagation by replacing the weight symmetry requirement with random feedback weights, which the network adapts to align with the true gradients over time. This approach demonstrates that perfect symmetry is not a strict requirement for learn- OL 8- xT. where ac represents the fixed random feedback weights. 5 = B. Ba
Feedback Alignment
Hebbian Learning and STDP Hebbian learning (Hebb, 1949) emphasizes strengthening the connection between neurons that frequently activate together. It forms the basis for synaptic plasticity in the brain. A more refined ver- sion, tee-ttiminnieppeee 2000), adjusts synaptic strengths based on the relative tim- ing of pre and postsynaptic spikes. This mechanism allows networks to capture temporal correlations and has inspired algorithms for unsupervised learning and spiking neural
ing. The gradient update is given by aw
While this algorithm breaks the symmetry requirement of backpropagation, still relies on global error signals and two-stage training process.
plasticity (STDP) (Song et al.,
Local Losses Local losses (Marblestone et al., 2016) method enables decentralized learning by assigning specific objectives to individual layers or regions of a network. The
3

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Pyramidal Neuron
(a) (b)
CI: Asymmetry of Forward and Backward Weights
Layer 1
Layer 2
Layer L
-rriiable W and o
Target: Label
C2: Local Error Representation
O:
Local Error
C3: Non-two-stage Training
No Separate Inference And Training Stages
MM.
-M
-Mi
2
Wi
Loss Curves
Input O
W:
0.05
Dendiiik Localined Leariigg
Beckpoepagatioss
0.07
Prediction
0.70
Input
0.12
5 Local Error
w: Forward Weight
Output
0: Backward Weight
150
340
Fpoch
Figure 2. (a) Overview of Dendritic Localized Learning. (b) Our DLI algorithm satisfied all 3 criteria. (c) Models trained by DLL
successfully converge and achieve comparable performance those trained by backpropagation.
networks. The weight update Hebbian learning is given by Awi n, where x; and Ty are the activations of neurons and j. In STDP, the update depends on the spike timing difference
the network evolves stable states corresponding stored patterns. Equilibrium Propagation (Scellier & Bengio, 2017) trains networks by reaching equilibrium under inputs and then applying small perturbation to the output. The loss minimized in this method may not align with the primary training objective, as reducing energy does not inherently lead to a corresponding decrease in task-specific loss.
Awyy = Ate At//+
At > 0, A-/ At 0,
(1)
where A+, A are scaling factors, and 7+, are time con- stants. These methods lack the ability to utilize supervised learning signals and coordinate weights across layers, with STDP further constrained by its reliance precise spike timing.
3. Dendritic Localized Learning
In this section, will introduce proposed dendritic localized learning (DLL) and its detailed training procedure. Guided by three criteria proposed in Section 2.1, we draw inspiration from the pyramidal neuron (Spruston, 2008) and aim to simulate its calculation mode.
Forward-forward learning (Hinton, 2022) eliminates the need for backward eтrоr propagation, training networks by separately optimizing for positive and negative samples in a forward-only manner. The layer-wise goodness function is defined g ()))), where of is the activation of neuron in layer I. The network maximizes g¹ for positive examples and minimizes it for negative ex- 0g
Forward-Forwand
3.1. Caree-Compartment Neurons
Pyramidal neurons are a type of excitatory neuron com- monly found in the cerebral cortex (DeFelipe & Fariñas, 1992), playing crucial role processes like learning, mem- ory, and higher cognitive functions. We follow Sacramento et al. (2018) to divide pyramidal neuron into three com- partments: soma, apical dendrite, and basal dendrite.
ag' This method imposes OWi
amples: AW' =
OW
significant constraints input and processing, making it challenging extend to other architectures such as CNNs.
To satisfy criterion C2, i.c., local error representation, we define the local error of an individual neuron as & = x u, where u is the sensory input of the neuron and x is the ex- pected value, a.k.a, backpropagated activity, which used to compute the error locally within the neuron. Under our setting, & is calculated in the soma, and and x: stored in the basal and apical dendrite, respectively. This division
based Learning Energy-Bass -Base Learning defines an energy function E(x,y; y; W), where x is the input, y is output, and W are model parameters. The goal is to minimize this energy such that desired outputs correspond low-energy states. The weight updates are computed as ac BE( Hopfield networks (Hopfield, 1984) aw JW
use an energy function to model neural dynamics, where

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
aligns with the structure of pyramidal neurons, where dif- ferent compartments are responsible for processing various types of information, contributing to the local error com- putation that facilitates learning. Therefore, for a neural network consisting of pyramidal neurons, the error of layer i can be computed as:
layer i 1, and We is layer i's weight. For the last layer, the will be valued as the target label.
In the calculation of the backpropagated activity x, we use the differentiation of loss L from x, to obtain Ax,, which is the direction of change for x; x; depends solely on & and Ei+1. as the parameter updates in the DLL algorithm are localized. Consequently, to compute Ax;, it sufficient to use the derivatives of x, with respect to &, and E from C. Ax; is defined as:
& - -
(2)
where x; is the backpropagated activity of layer + 1.
However, due to criterion C1 (asymmetry of forward and backward weights), calculating the backpropagated activity x; from layer + becomes challenging, as the transpose of the forward weights Wi+1 cannot be utilized. To address this, we introduce a special trainable weight θ to replace the traditional forward weight W during updating parameters.
ac
Ax:
dx.
0 E 1
0x;
(5)
@u
- + 8x:
Li+1
Regarding criterion C3 (non-two-stage training), we assume that information can be spatially separated within neuron, allowing the two propagation phases to occur simultane- ously without requiring strict temporal segregation. Addi- tionally, we do not fix the forward weight W, and instead, update both W and θ simultaneously. with supervision from the target label. The detailed training procedure will be discussed in Section 3.2.
- + WT @ (WW...)).
Here, o denotes the Hadamard product. In our DLL algo- rithm, x propagates along the apical of the pyramidal neu- ron, specifically along the path defined by the parameters θ, independent of the parameters W used in the forward pass. Therefore, the calculation formula for Ax; is given by
Ax. = E + 0, 3 f'(,.u)).
(6)
Neurons leverage local information dynamically update synaptic weights between adjacent layers, enabling the net- work to iteratively refine its architecture and functionality. Through this process, the output of each neuron gradually aligns with the intended target, facilitating an efficient redis- tribution of synaptic strength and embodying biologically plausible learning mechanism.
The updated value of x; is calculated using the formula x; + x; + 1/x Ax; where 1/x denotes the learning rate of updating x. Ultimately, when x, approaches stability, Ax; = 0, leading to the expression
E (7 O (WW...)).
(7)
To help readers better understand the mechanism of our DLL algorithm, we provide an illustration in Figure 2, assuming the training of an L-layer multilayer perceptron (MLP).
Finally, we will adjust W. through W. <- W; - 7/ A W,, where yw is the learning rate for updating and the update term A AW; is:
ac
3.2. Training Procedure
AW:
OW.
DLL can be applied to various network architectures, in- cluding MLPs, CNNs, and RNNs We use MLPs with MSE loss as an example to describe our method, then the total loss L of the whole neural network is:
0 -
aW:
(8)
0u
--441(-
aw:
E:41 @ f'(W.u.uu:
L h=
-=
2 1m1
E(x 1m1
(x
)
(3)
For O, we will update it using the following rule: O +
+ Me * AO,, where n/j is the learning rate for updating
where & is the loss of all neurons in layer i, and I. is the number of layers.
+ and AO, is the update term:
1 Oc
At the first epoch, for all layers except the last layer, we will initialize the expected value as the sensory input, written as:
d
ac UE: ag, 80
u f(W.u.) Xi+1 w
(4)
(9)
0 (-))))) 00 [4:1 o (W.))
where f denotes the non-linear linear activation function, u,+1 is sensory input of layer + 1, X,41 is the expected value of
ag,
80?
-6 f f(W...)).
5

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Table 1. We show both the biological plausibility of various algorithms with the proposed criteria and the accuracy of image classification
achieved by various bio plausible learning algorithms. Our proposed DLL achieves the highest performance among the algorithms satisfying all criteria, "CI, C2, C3" stand for three criteria proposed in Section 2.1. All results are averaged across random seeds.
Method
C1
C2
C3
Model
MNIST
MashionMNIST
SVHN
CIFAR-10
Avg.
MLPs 98 60.1
88.54%
40.42%
48.74% 5.56%
74.20%
Backpropagation
CNNs 99. 99.56%==.14%
92.68%+0.428
15.35%
75.10%
90.67%
MLPs
82.16% 14%
54.91%.2o 25%
48. 48.46%.2o 11%
69.35%
Feedback Aligoment
1
x
CNNs 97 000.100 13%
89.74%_..rr 17%%
92. 20.60%..o.o 20%
59 60..00%1....o 40%
84.75%
MLPs
98.56%
88.07%
59. 19.12%o++020 27%
48.58 7 95%
73.58%
Local Losses
x
CNNs
10..0%%00.06%%
91. 11.90%=...om 20%%
95, 08.68 25%
72.18% 10%
89.64%
ML.Ps
98. 08.42%%o..11 19%
88.72%
59.03% 45%
47. 47.34% 24%
73.38%
Predictive Coding
x
CNNS
99. 499.41%60-408
92.03%22..00
94.53% 1-55%
72.94%.2. 5256
89.72%
MLPs
01.44%%.1.400%
68, 68.90% 20.
48.13%
31 17..00%%....... 31%%
59.89%
Perturbation Learning
CNNs
92. 122.61%...440
75. 75.79%
57. 07.69%%2:1..
100.72%6a 38%
66.45%
MLPss
94.0 94.01%
83. =0.0111
54.11%%o..oom
46.10% 10%%
69.38%
Difference Target Propagation
/
/
x
CNNS
96. 96.4%%64..00
90.51 50.51%==.1189 10.18%
69. (9.77%% 32%
50. 80.88% 0.07%
76.88%
ML.Ps
78. 29.02
67. 17.40%%=omm 197800.000
40.80%o22044 44%
19.98%o2o 23%
51.62%
Hebhian Learning
/
CNNS
83.05%.2o. 1256
72.03% 23%
44.77 44.77%220 53%
29. 29.86%22
37.43%
MLPss
77. 77.18%o+o.171
70.03% -0.28%
41.76% 40%
22 68%6
52.91%
R-STDP
/
/
CNNNs 11.67%1000448
74.29%
20.00%
50 00.02%%....o 92%%
33.19%.1 38%
62.29%
Forward Forward
/ v
MLPss
96. 96.99%o4..14 14%
80.51
7..74%
17.52%o=o..o 63%
39.48% 10%
66.12%
CNNs 66.66699900000
10.00%=0.000%
6.70%=o.00m
10.32% 00%
10.67%
MLPS 93. 833.81%.10.188 18%
75.65%%2..a0
22 22.66%61..3000 30%
16.93% 10%
52.25%
Equilihrium Propagation
CNNs
26.73%%t0.223 22%6
30.26% 236%
6. 8..0%%
10.32% 00%
18.50%
MLPs 97. 27.3%% 40%
87.50% 40.4
56. 10.
45 03.88%%=. 10.00%
71.89%
DLL (Ours)
CNNs
98. 87%8%%
90.88%++040%
85 81. +0.1%%
70.89% 10.8%9%%0055.. -0.5%%
86. 86.61%
We present our global algorithm in Appendix A, which outlines the overall training procedure using DLL. Addition- ally, to enhance clarity, we provide detailed derivation for training RNNs with DLL in Appendix B. This derivation highlights its application to sequential tasks and addresses the unique challenges introduced by temporal dependen- cies. The convergence properties and guarantees will be discussed in Appendix C.
Time-Series Forecasting We employ RNNs with various learning algorithms for real-world multivariate time-series forecasting, including Electricity (Lai et al., 2018), Metr-la (Li et al., 2017), and Pems- bay (Li et al., 2017).
To ensure fairness, we use the same model architecture across all learning algorithms for MLPs, CNNs, and RNNs under a certain dataset. Due to the difficulty of achieving convergence with vanilla STDP in this setting, we replace it with an improved version,
STDP (R- STDP) (Mozafari et al., 2018). For detailed dataset statistics, metric explanations, and implementation of models, please refer to Appendix D.
4. Experiments
In this section, we first introduce the experimental settings, including datasets and implementation details. Secondly, we benchmark all biologically plausible learning algorithms, including our proposed DLL, on image classification tasks. Thirdly, we evaluate RNNs trained with DLL on text char- acter prediction and time-series forecasting. Finally, we do an ablation study and analyze its inner properties.
4.2. Image Recognition
We benchmark all previous bio-plausible algorithms and our proposed DLL on image classification tasks in Table 1. As shown in Table 1, we can conclude that:
Current biologically plausible algorithms, while offer- ing valuable insights, often fall short of the high per- formance achieved by traditional backpropagation, par- ticularly complex datasets like SVHN and CIFAR- 10. Algorithms meeting one criterion, including feedback align- ment, local losses, and predictive coding, show competitive performance on simple datasets like MNIST and Fashion- MNIST, indicating their potential viability. However, their reduced effectiveness on more challenging tasks, such as CIFAR-10 10, highlights the need for further advancements in this field. For paradigms that satisfy two criteria, we ob-
4.1. Experimental Settings
Image Classification We utilize several widely used benchmark datasets for image recognition tasks: MNIST, FashionMNIST, SVHN, and CIFAR-10. 10. We take classifica- tion accuracy as the metric.
Text Character Prediction We conduct next-character prediction with RNNs on Harry Potter (Rowling, 2019). We take Prediction Aceuracy (Pred. Acc.) as the metric, which measures the proportion of correctly predicted characters among all predictions.
6

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Table 2.
Performance of RNNs trained with various learning algorithms on text-character prediction and time-series forecasting. The best
results are formatted in bold font format. 1 ()) indicates the higher (lower) the better. All results are averaged across 3 random seeds.
Harry Potter
Electricity
Metr-la
Pems-bay
Method
Pred. Ace. 1
MSE +
MAE
MSE
MAE
MSE 4
MAE 4
Backpropagation
51 11.9%21.0%
0.175400007
0.32440.007
0.131 +0.004
0.21
4400005
0.164=0.001
0.190 1.00000000
Predictive Coding
38.8%
0.162:0.019
0.312
=0040
0.22810.005
0.17810.001
0.002
DLL (Ours)
33.7%220.6%
0.172
0.321
0.15520.005
0.26440.001
0.178:10005
0.224
serve noticeable performance reduction compared those meeting only one criterion. Perturbation learning, which does not require a backward procedure, struggles to achieve adequate convergence.
This task offers insights into DLL's ability to model the internal dynamics of sequential data. Table 2 illustrates that RNNs trained using DLL exhibit competitive performance across a range of tasks, achieving results on par with or surpassing those of backpropagation several metrics, with particularly strong performance on the Electricity dataset. These results underscore DLL's effectiveness in capturing temporal dependencies while maintaining alignment with biologically plausible learning mechanisms.
Our proposed DLL successfully integrates biological plausibility with high performance. Among algorithms that meet all the criteria, MLPs or CNNs trained with DLL converge successfully across all datasets, even achieving performance comparable to backpropagation. This suggests that it is possible to reconcile biological plausibility with high performance. In contrast, methods like Hebbian learn- ing, the forward algorithm, and equilibrium prop- agation struggle to guarantee convergence, especially on more complex architectures (CNNs) or challenging datasets. For instance, MLPs trained with the -orwardd algo- rithm (Hinton, 2022) achieve comparable average accuracy while CNNs trained with that fail to converge across all benchmarks. In comparison, DLL consistently converges quickly and delivers relatively satisfactory results. The results of CIFAR-100 and Ting-ImageNet are shown in Ap- pendix E.
What's ' more, we successfully employ our DLL method to train TexICNNs (Kim, 2014) for text classification tasks. We show the results on text classification in Appendix
4.4. Ablation Study
As mentioned in Section 3, we propose the weights @ and its updating rules based on local errors. Taking the feedback alignment (FA) into consideration, we think is necessary to evaluate how the convergence or performance will be influenced when θ is a random and unchanged matrix. We name this special method as "DIL-FA", i.e., θ will ini- tialize randomly and not participate in the model updating procedure.
4.3. Sequential Tasks
Most biologically plausible learning paradigms mentioned in Section 2.2, such as local losses and target propagation, were primarily designed for discrimination tasks. Therefore, to assess the versatility of our proposed DLL, we follow Mil- lidge et al. (2022) to conduct experiments on training RNNs with our DLL for sequential regression tasks, as shown in Table 2. We compare DLL against several algorithms en- abling model convergence, including backpropagation and predictive coding. Methods that fail converge are not included in the table.
Table 3. Ablation experiments on O "DLL-FA" indicates O ini- tializes randomly and will not be updated, combining our proposed DLL and feedback alignment (FA). Numbers with indicate mod- els fail converge. t (L) indicates the higher (lower) the better.
Metric
DLL
DLL-FA
MLPs on MNIST
97.57%
97.37%
Acc. f
CNNs on CIFAR-10
70.89%
69.85%
RNNs on Harry Potter
Acc. 1
33.70%
0.71%
MSE +
0.172
0.193
RNNs on Electricity
MAE 1
0.321
0.345
MSE 1
0.178
0.198
First, we evaluate RNNs trained with different algorithms on text-character prediction task to investigate their lan- guage processing abilities. Backpropagation outperforms other methods by a significant margin. While backpropa- gation performs best, predictive coding and DLL emerge as promising biologically plausible alternatives. Notably, DLL is the only method that satisfies all the biological plau- sibility criteria while still succeeding in converging during training. Furthermore, we regard real-world multi-variant time-series forecasting as an ideal regression task for eval- uating a model's ability to capture temporal dependencies.
RNNs on Pems-bay
MAE 0.224
0.251
We report the performance comparison between DLL and DLL-FA in Table 3. The results indicate that DLL generally outperforms DLL-FA across a wide range of tasks. On simple datasets like MNIST, DLL achieves marginally higher accuracy (97.57% compared to DLL-FA (97.37%) suggesting that the benefits of DLL are modest in such cases. However, on more complex datasets like CIFAR- 10, DLL consistently demonstrates superior performance, with accuracy improvements of up to one percentage point.
7

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
NLhs with Offeeent Learthg Sate A
2h8ts with Offieett Learring fett
Muet
6020
0.322
23
0200
0.300
24 Eud nia
0:275
24 E1. 1Ml
0.220
DLL H62
mea
30
20
0200
100
64.25
0.275
LeAtiing Sane
LeWn ng Rate
(a)
(b)
(c)
(d)
Figure 3. (a) MLPs trained with DLL by various learning rates. (b) CNNs trained with DIL by various learning rates. (c) Loss curves of RNNs trained with DLL across different sequence lengths, (d) Loss curves of RNNs trained with DLL by different learning rates.
Notably, in sequence modeling tasks, such as RNNs-based predictions on Harry Potter, DLL outperforms DLL-FA by significant margin, with DLL-FA failing to converge in both cases. Furthermore, in time-series forecasting tasks, DLL demonstrates consistent superiority over DLL-FA, achieving lower MSE and MAE across the majority of datasets. These results highlight the critical role of updating O in DLL, which is essential for achieving better performance in both classification and sequential modeling tasks.
metric weight updates across layers, global error signals, and two-stage training, which are not naturally present in bi- ological neural circuits. Currently, many attempts have been made to bridge this gap by exploring biologically plausible learning algorithms that mimic brain mechanisms, as dis- cussed in Section 2.2. While there have been reviews (Weed & Hursting, 1998; Jiao et al., 2022; Millidge et al., 2022; Li et al., 2024; Schmidgall al., 2024) summarizing the strengths, weaknesses, and differences between past learn- ing algorithms, few have empirically benchmarked these algorithms on real-world datasets. In contrast, our study em- pirically evaluates these biologically plausible algorithms across various network architectures and datasets, providing a more comprehensive comparison.
4.5. Analysis
In this section, we perform a convergence and sensitivity analysis for models trained with DLL. Since we update both the weights W and the special parameters θ simultane- ously during training with DLL, the learning rate plays a critical role in maintaining a balance between the trainable parameters during updates. Figures (a) and (b) present the performance of MLPs and CNNs trained with DLL across various learning rates, respectively. We observe that the best performance for most models occurs around 77 1 x 10 indicating an optimal learning rate for stable training. Fig- ures 3 (c) and (d) were evaluated using the Metr-la dataset. In Figure 3 (c), we plot the loss curves of RNNs trained with DLL across different sequence lengths. This suggests that excessively long or short sequence lengths may hinder convergence, making the training process unstable. Finally, Figure (d) demonstrates how RNNs trained with DL.I are affected by the learning rate. We find that a learning rate of 1 x 10-4 is too large, leading to suboptimal performance, likely due to instability in gradient updates. This analysis highlights the importance of carefully tuning the learning rate for stable and effective training across different network architectures and tasks. In Appendix G, we show the time consumption and memory usage of the DLL in image clas- sification tasks. In Appendix H, we evaluate the scalability of our method.
While apical dendrites have been discussed in previous lit- erature for credit assignment and have been utilized for various purposes, our study takes advantage of their prop- erties to design and implement more biologically plausible learning algorithms, which differ significantly from exist- ing approaches. Guerguiev et al. (2017) primarily aimed to explain how deep learning can be achieved using segre- gated dendritic compartments, but they did not propose a specific learning algorithm. As for Bartunov al. (201 8), while their proposed STDP improves upon DTP in perfor- mance and biological plausibility, it fails to resolve the issue of DTP requiring upper layers to propagate two separate values at different times. Payeur et al. (2021) investigated burst-dependent synaptic plasticity. While their work shares conceptual similarities with ours in terms of apical den- dritic processing, the primary objective of their study differs from ours. Our proposed method not only satisfies all three criteria but also achieves higher performance compared to existing biologically plausible learning methods.
In addition, we aim for our proposed DLL to possess general capabilities comparable to those of BP, including the ability to perform both classification and regression tasks, handle diverse modalities such as images and language, and support multi-layer credit assignment. For example, in addition to image recognition tasks, our DLL framework can also be ap- plied to train RNNs for regression tasks (Appendix B), such
5. Related Work
Backpropagation (Rumelhart et al., 1986) has inherent limi- tations in terms of biological plausibility. It requires sym-
8

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
as next-character prediction and time-series forecasting. In contrast, methods like BurstProp (Payeur et al., 2021) and SoftHebb (Journé et al., 2023) may struggle with such tasks, as their designs are not well-suited for recurrent architec- tures. Counter-current Learning (Kao & Hariharan, 2024) violates our third criterion, as it explicitly involves distinct forward and backward phases.
Bengio, Y.
How auto-encoders could provide credit as- signment in deep networks via target propagation. arXiv preprint wrxiiv1407..999 2014.
Bianchini, M., Fanelli, S., Gori, M., and Maggini, M. Ter- minal attractor algorithms: A critical analysis. Neurocom- puting, 15(1):3-13, 1997.
Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances neural information processing systems, 33: 1877-1901, 2020.
6. Conclusion
In this paper, we reviewed the current biologically plausible learning algorithms and summarized three criteria that an ideal learning algorithm should satisfy. Meanwhile, we em- pirically evaluate these algorithms across diverse network architectures and datasets. Secondly, we introduced Den- dritic Localized Learning (DLL), novel learning algorithm designed to meet these criteria while maintaining the effec- tiveness of training MLPs, CNNs, and RNNs. Finally, to validate its performance, we present extensive experimental results across a range of tasks, including image recogni- tion, text character prediction, and time-series forecasting, utilizing MLPss, CNNs, and RNNs. By combining theoret- ical rigor with practical applicability. our work paves the way for future research into biologically plausible learning paradigms, fostering deeper connections between neuro- science and artificial intelligence. The limitations and future directions are discussed in Appendix I.
Crick, F. The recent excitement about neural networks. Nature, 337(62033) (2266)0))199933330 132, 1989.
DeFelipe, and Fariñas, I. The pyramidal neuron of the cerebral cortex: morphological and chemical characteris- tics of the synaptic inputs. Progress in neurobiology, 39 (6):563-607, 1992.
Devlin, J., Chang. M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. In North American Chapter of the Association for Computational Linguistics, 2019.
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. An image is worth 16x 16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2020.
Impact Statement
This work bridges the gap between neuroscience and arti- ficial intelligence by introducing biologically plausible learning algorithm that achieves competitive performance with backpropagation across diverse tasks. By fostering ad- vancements in brain-inspired computing, our research opens new pathways for developing bio-interpretable and scalable machine learning models. We do not think our work will negatively impact ethical aspects or future societal conse- quences.
Guerguiev, J., Lillicrap, T. P., and Richards, B. A. Towards deep learning with segregated dendrites. Elife, 6:e22901, 2017.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 778-778. 2016.
Acknowledgments
Hebb. The organization of behavior. New York, 1949.
The authors would like to thank the anonymous reviewers for their valuable comments. This work was supported by the National Natural Science Foundation of China (No. 62076068).
Hinton, G. The forward-forward algorithm: Some prelim- inary investigations. arXiv preprint arXiv:2212.13344 2022.
Hopfield, J. Neurons with graded response have collective computational properties like those two-state neurons. Proceedings of the national academy of sciences, 81(0)): 3088-3092, 1984.
References
Bartunov, S., Santoro, A., Richards, B. A., Hinton, G. E.. and Lillicrap. T. P. Assessing the scalability of biologically motivateed deep learning algorithms and ar- chitectures. In Neural Information Processing Systems, 2018.
Jiao, L., Yang, Y., Liu, F., Yang, S., and Hou, B. The new generation brain-inspired sparse learning: A comprehen- sive survey. IEEE Transactions on Artificial Intelligence, 3(6):887-907, 2022.

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Journé, A., Rodriguez, H. G., Guo, Q. and Moraitis, T. Hebbian deep learning without feedback. In The Eleventh International Conference Learning Representations, 2023.
Mozafari, M., Kheradpisheh, S. R.,, Masquelier, T., Nowzari- Dalini, A., and Ganjtabesh, M. First-spike-based visual categorization using reward-modulated stdp. IEEE trans- actions on neural networks and learning systems, 29(1)) 6178-6190, 2018
Kao, C. H. and Hariharan, B. Counter-current learning: A biologically plausible dual network approach for deep learning. In The Tiihth-eighth Annual Conference on Neural Information Processing Systems, 2024.
Munakata, Y. and Pfaffly, J. Hebbian learning and develop- ment. Developmental science, 7(2):141-148, 2004.
Nøkland, A. Direct feedback aligmment provides learning in deep neural networks. Advances in neural information processing systems, 29, 2016.
Kim, Y. Convolutional neural networks for sentence classi- fication. In Conference Empirical Methods in Natural Language Processing, 2014.
Pang. B. and Lee, L. Seeing stars: Exploiting class relation- ships for sentiment categorization with respect to rating scales. In ACL, 2005.
Lai, G., Chang, W.-C., Yang, Y., and Liu, H. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR confer- ence on research & development in information retrieval, pp. 95-104, 2018.
Payeur, A., Guerguiev, J., Zenke, F., Richards, B. A., and Naud, R. Burst-dependent synaptic plasticity can coordi- nate learning in hierarchical circuits. Nature neuroscience, 24(7):1010-1019 2021.
LeCun, Y., Bengio, Y., and Hinton, G. Deep learning. nature, 521(7553):436 444, 2015.
Rao, R. P. and Ballard, D. H. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-fied field effects. Nature neuroscience, 2(1):79-87, 1999.
Lee, D.-H., Zhang, S., Fischer, A., and Bengio, Y. Differ- ence target propagation. ECML/PKDD, Dec 2014.
Li, G., Deng, L., Tang, H., Pan, G., Tian, Y., Roy, K., and Maass, W. Brain-inspired computing: systematic survey and future trends. Proceedings of the IEEE, 2024.
Rowling, Harry potter. The 100 Greatest Literary Char- acters, pp. 183, 2019.
Li, Y., Yu, R., Shahabi, C., and Liu, Y. Diffusion con- volutional recurrent neural network: Data-driven traffic forecasting. arXiv preprint arXiv: (70.:/7777.09220 2017.
Rumelhart, D. E., Hinton, G. E., and Williams, R. Learn- ing internal representations by error propagation, parallel distributed processing, explorations in the microstructure of cognition, ed. de rumelhart and j. mcclelland. vol. 1. 1986. Biometrika, 71:599-607, 607, 1986.
Lillicrap, T. P., Cownden, D., Tweed, D. B., and Akerman, C. J. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communica- tions, 7, 2016.
Sacramento, J., Ponte Costa, R., Bengio, Y., and Senn, W. Dendritic cortical microcircuits approximate the back- propagation algorithm. Advances neural information processing systems, 31, 2018
Maass, W. Networks of spiking neurons: the third gener- ation of neural network models. Neural Networks, 14: 1659-1671, 1997.
Salvatori, T., Song, Y., Yordanov, Y., Millidge, B., Sha, L., Emde, C., Xu, Z., Bogacz, R., and Lukasiewiez. T. stable, fast, and fully automatic learning algorithm for predictive coding networks. In The Twelfth International Conference on Learning Representations, 2024.
Marblestone, A. H., Wayne, G., and Körding, K. P. To- ward an integration of deep learning and neuroscience. Frontiers Comput. Neurosci., 10:94, 2016.
Millidge, B., Tschantz, A., and Buckley, C. L. Predictive coding approximates backprop along arbitrary compu- tation graphs. Neural Computation, 34(6):1329-1336) 2022.
Scellier, B. and Bengio, Y. Equilibrium propagation: Bridg- ing the gap between energy-based models and backprop- agation. Frontiers in computational neuroscience, 11:24, 2017.
Millidge, B., Song, Y., Salvatori, T., Lukasiewicz, T., and Bogacz, R. A theoretical framework for inference and learning in predictive coding networks. In The Eleventh International Conference on Learning Representations, 2023.
Schmidgall, S., Ziaci, R., Achterberg, J., Kirsch, L., Ha- jiseyedrazi, S., and Eshraghian, J. Brain-inspired learn- ing in artificial neural networks: a review. APL Machine Learning, 2(2), 2024.
10

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Song, S., Miller, K. D., and Abbott, L. F. Competitive heb- bian learning through spike-timin-deeenddee synaptic plasticity, Nature Neuroscience, 3:919-926, 2000.
Spruston, Pyramidal neurons: dendritic structure and synaptic integration. Nature Reviews Neuroscience, 9(3): 206-221, 2008.
Stork. Is backpropagation biologically plausible? In Inter- national 1989 Joint Conference on Neural Networks, pp. 241-246. IEEE, 1989.
Weed, D. L. and Hursting, S. D. Biologic plausibility in causal inference: current method and practice. American Journal of Epidemiology, 147(5), 1998.
Werfel, J., Xie, X., and Seung, H. S. Learning curves for stochastic gradient descent in linear feedforward net- works. Advances in Neural Information Processing Systems, pp. 1197-1204, 2003.
Whittington, J. C. and Bogacz, R. An approximation of the error backpropagation algorithm in predictive coding network with local hebbian synaptic plasticity. Neural computation, 29(5): 1229- 2017.
Williams, R. J. Simple statistical gradient-following algo- rithms for connectionist reinforcement learning. Machine Learning, 8:229-256, 1992.
Zahid, U., Guo, Q. and Fountas, Z. Predictive coding neuromorphic alternative to backpropagation: A criti- cal evaluation. Neural Computation, 35(12):1881-190) 2023.

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
A. Global Algorithm of Dendritic Localized Learning
In this section, we show the global algorithm dendritic localized learning in pseudo-code style.
Algorithm 1 Algorithm of Dendritic Localized Learning
Input: data D, number of layers L, number neurons per layer N, learning rate 7/w. Tje Initialize W, @ randomly for all layers
for epoch = 0 to max_ cax_ep do for each batch B in D do
Forward Pass (when input stimuli are fed to the basal dendrite):
Assign the input values to the basal dendrites of the input layer neurons: Up = B, and initialize Xo = Up
for = 0 to - 1 do
Compute forward pass u f(( (W),,, where fi is the activation function for the i-th layer Initialize X u,+1
end for
Compute Local Errors (when apical dentrite receives the top-down feedback):
Assign the target values to the apical dendrites of the output layer neurons: X = target
Compute output error {7 = Vu, C(u,,
Compute input error Ço = Xg 11
for down to 1 do
Compute local error &, (T O (W.)). end for
Update Weights and Thetas Simultancously:
for = to L 1 do
E:+1 * then
Update W; + W, + nw (Es+1 ft' f(((,u))u
Update θ; <- θ + ne (-5 @ f ( (W.)))
end if
end for end for
end for
B. Training Recurrent Neural Networks with Dendritic Localized Learning
In the conventional Recurrent Neural Networks (RNNs) framework, the hidden layer consists of single vector, denoted as h. However, our DLL-RNNs model inspired by the structure of pyramidal neurons, incorporating two distinct components within the hidden layer. One component, h", represents sensory input from the basal of the pyramidal neuron, while the other, h", denotes the backpropagated activity from the apical of the pyramidal neuron. Similar to the standard RNNs framework, the DLL-RNNS operates across multiple time steps. At the i-th time step, the hidden state in RNNs is represented as h; whereas in the DLL-RNNS, it is split into two components: hy and h." Specifically, at each time step, we first compute the value of h;, which is then used to initialize h² as:
h = h f(W,h + W.X)).
(10)
Here, x; is the input at time step i for both the traditional RNNs and the DLL-RNNS. The computation involves multiplying the weight matrix Wn by the sensory component the hidden state from the previous time step, h and adding to the product of the weight matrix Wx and the input x; for the current time step our DLL-RNNs. This sum then processed through the activation function f. resulting in the hidden state for the current time step, h We define the local error gh of the hidden layer in the i-th time step as:
h
(11)
We use t; to represent the expected output for the i-th time step, and use y; to denote the actual output for the i-th time step.
12

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
At time step i, the output of the DLL-RNNS is given by ys, which is obtained by multiplying the weight matrix W with the hidden state h,, followed by the application of activation function g (In DIL.-NNN,, the function g is linear function defined g(x) = z),, written as:
y; - o ((Wyb))))) = w
(12)
And We define с as the emror for the output layer in the i-th time step.
t. Y-.
(13)
In our DLL-RNNs, we construct the Mean Squared Error (MSE) loss using the hidden layer and the output layer at each time step i, denoted as e and c.- The MSE loss is expressed as:
C =
1 Σ 2=-2-iehh 2
+ -()))
- (a -h²)² + (t. -y:)
(14)
To compute the backpropagated activity h. differentiate the loss L. with respect to h to derive Ahf, which is the direction of change for h2. We update he through hŞ <- hς + 7h Ah,, where ?h represents the learning rate for updating hf. For the time step i E [1, n - 1], Ahe is defined as:
ac
2h.-
Th
01 3 Di=i (h) h2)²
+ (t, }})
Oh?
0 [ 1 (hg h2)²
1 (t: -.)
a1- 2 -==+1 [hh h2)² + (t, y;,)
0hL41
+
Oh
Ohf+1
Oh!
(15)
OC oy:
7C Oh
-
oy, Th oh oh
0 01 (t, - y:)² Og (Wyh))
oh.
ch
+ Ab 11 ah
oy:
Oh
- W le' og((W, wh [th @ f'W Wxx41)
= ch W W [sh, @ f'W,h + Wxx++1)]
For time step i = n, the formula for Ah is given as:
Ah O
Th
[[-3 1 (hg h; (tn - yn)
Ohh
--hh, h2)) (tn - yn)( [(a) 0yn
(16)
ah
--gh + W ISx C (g((Wh
ch + W
Here, @ denotes the Hadamard product, and since g is linear function defined as g(x) x. its derivative is g'(x) 1. Thus, g'( (W, ) = 1. In our DLL-RNNS, similar to the DLL, we replace Wy with Oy and Wa, with On However, W does not appear in the computation of gh so there is no need for replacement in this context.
In our DLL-RNNs, we updates Ahe as:
ah -ch OT
+ + [e & &(W.h Wxx
(17)
As well, update Ahe, as:
Ah ghh +
(18)
13

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
Similar to DLL-MLPS, we directly assign the value of < as:
c = - Oiih @ f'Wnhf + Wxx+11]l-
(19)
And assign th as:
< = Ox Sm.
(20)
Finally, we will adjust W through W W+** AW. The symbol y) denotes the learning rate for various instances of the weight matrix W. Detailed update rules for the weight matrices A WW are defined as:
Oc
AW,
oW
OL dy:
dy, awy
(21)
Σ (t: (h)))
= Σ E (h²)²² i=1
Then the Wy will be updated as:
Wy Wy t,w AWy
(22)
For A AWx x, we calculate as:
dC
OL dh.
AW
OW
Th OW,
OC
f'(Wnhi + Wxx:)|x;
(23)
The
Σ cch @ f'(Wn, Wxx:)Jx
=1
Then the AW* will be updated as:
Wx - Wx + /w. AW
(24)
And for A W. h. we update it as:
AWn. dc
OWn
" JC dh
Ohe OWn
(25)
> k o f'(Wnh + W
We update Wa as:
Wh - Wn Tw, AW
(26)
A0. Here, n represents the + 1 Similarly, the parameter vector O is updated using the following equation: θ 0 + learning rate, which varies for different instances of the parameter vector O The update term A and the specific update
14

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
formula defined as:
ac T
AO
0C o och 0
(27)
E @ oW(W),)))
Σ (E,)"
Then 6 will be updated as:
oy t 0y + teey AT
(28)
For A0,, we calculate as:
A
ch ch S1+1 2 f'(nn + Wxx++1)
(29)
So the On will be:
On t 0n + 1e,, A(
(30)
C. Discussion on Convergence
The loss function (Equation (3)) designed to minimize the discrepancy between the top-down predictions and bottom-up outputs of each pyramidal neuron in the network. To achieve this, employ local gradient descent-based learning rules and neural plasticity mechanisms update both forward and backward weights. During each iteration, the differences between the network's predictions and the ground truth propagate back through localized errors, effectively coordinating all neurons in an orchestrated manner. As a result, neural responses collectively refine predictions over successive iterations, gradually reducing local errors and driving the network toward convergence. While providing formal convergence proofs remains challenging due to the network's 's nonlinear operations, our empirical results consistently demonstrate a steady decrease loss throughout training, supporting the stability and effectiveness of our approach.
D. Experiment Settings
D.I. Statistics of Datasets
For the image classification task, we utilize several widely used image recognition datasets to evaluate our model's performance across different domains. These datasets include MNIST, FashionMNIST, SVHN, and CIFAR-10.
MNIST. The MNIST dataset consists of 70,000 grayscale images of handwritten digits, each with resolution of 28x28 pixels. It is classic benchmark for evaluating the performance of machine learning models on digit classification tasks, with 60,000 training images and 10,000 test images.
Fashion FashionMNIST is dataset similar to MNIST but consists of images of clothing items, such as shirts, pants, and shoes. It contains 60,000 training images and 10,000 test images, each with resolution of 28x28 pixels. FashionMNIST serves as a more complex alternative to MNIST for testing models on multi-class image classification tasks.
Street View House Numbers (SVHN). SVHN a dataset that consists of over 600,000 labeled digits extracted from strect-level images of house numbers. The dataset includes images of size 32x32 pixels in three color channels (RGB). SVHN designed for digit recognition in real-world, natural scene contexts, making it more challenging than MNIST.
15

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
CIFAR-10. CIFAR-I0 is dataset comprising 60,000 32x32 color images in 10 different categories, with 6,000 images per category. The dataset is split into 50,000 training images and 10,000 test images, representing range of objects including airplanes, automobiles, and animals. CIFAR-I0 is widely used for benchmarking models in object recognition tasks.
For the text character prediction task, we utilize the Harry Potter Series to evaluate our model's performance..
Harry Potter Series. This dataset includes the entire text of the Harry Potter book series, providing unique context for text character prediction tasks. is particularly useful for analyzing themes, character relationships, and genre-specific language, allowing models to be evaluated on their ability to understand narrative structures and stylistic elements.
For the time-series forecasting task, we utilize the Electricity, Metr-la, and Pems-bay.
Electricity. The dataset contains hourly electricity consumption data from 321 clients, covering three years from 2012 to 2014. records 15-minute interval values in kilowatts, with no missing data. Each column represents client, and consumption is set to zero before a client's start date.
Metro Traffic Los Angeles (Metr-la). The Metr-la dataset consists of traffic speed data collected from 207 loop detectors on the highways of Los Angeles. It provides measurements recorded every 5 minutes, capturing temporal patterns in traffic dynamics.
Pems-bay. The Pems-bay dataset is sourced from the Performance Measurement System (PeMS) maintained by California Transportation Agencies (CalTrans). includes traffic data collected from 325 sensors located the Bay Area. The dataset spans six months, from January 1, 2017, to May 31, 2017, providing a detailed temporal view of traffic conditions.
D.2. Implementation Details
1.2.1. DLL-MLPS
Model Architecture Given the distinct input dimensions and varying levels of classification complexity across the four experimental datasets, we designed tailored architectures for each dataset. Specifically, the MNIST and FaShionMNIST datasets consist of single-channel images with size of 28x28 pixels, whereas the SVHN and CIFAR- 10 datasets comprise three-channel color images of 32x32 pixels. To address these differences, we adapted the model architectures to optimize performance. For the MNIST and FashionMNIST tasks, we employed 5-layer DLL-MLPs with layer sizes of 784, 1024, 512, 256, and 10 neurons, respectively. The initial layer size of 784 corresponds to the flattened input vector from the 28x28 single-channel images, while the final layer of 10 neurons represents the output classes. In contrast, for the more complex SVHN and CIFAR-10 datasets, we utilized a deeper 6-layer DI.L--MPPP LPs, with layer sizes of 3072, 4096, 2048, 1024, 256, and 10 neurons. The first layer size of 3072 reflects the flattened input vector from the 32x32x3 three-channel images, and the last layer also comprises 10 neurons for the classification outputs.
Training Hyper-parameters All models were optimized using the Adam optimizer, with a linear learning rate scheduler for weight decay. The hyperbolic tangent activation function was employed in all layers except the output layer. Given the varying sizes and complexities of the datasets, we tailored the hyperparameter configurations to achieve optimal performance for each dataset. For the MNIST dataset, we set the learning rate to 1 10 and used batch size of 128. For FaShionMNIST. considering its increased complexity relative to MNIST, we adjusted the learning rate 5 x 10-4 and used batch size of 64 to better handle the more nuanced classification task. For the SVHN dataset, which presents higher level of complexity, convergence speed. Finally, for CIFAR-10, 10, the most challenging dataset among the four, set the learning rate to 8 x 10-5 and also used batch size of 64 to ensure robust training dynamics.
set the learning rate 5 x 10 and maintained batch size of 64 balance training stability and
D.2.2. DLL-CNNS
Model Architecture Following the same rationale as in the DLL-MLPs model, we employed different architectures for different datasets.
16

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
For the MNIST dataset, the first layer is convolutional layer with 32 filters, a kernel size of 5, and no padding. The second layer is max-pooling layer. The third layer is convolutional layer with 64 filters, kernel size of 3, and no padding. The fourth layer is another max-pooling layer. The fifth layer is convolutional layer with 16 filters, a kernel size of 3, and no padding. The sixth layer is a projection layer with an output size of 200. The seventh and final layer is the output layer, with an output size of 10.
For the MaNHT dataset, the first layer is a convolutional layer with 64 filters, a kernel size of 5, and no padding. The second layer is a max maxxpoooiig layer. The third layer a convolutional layer with 128 filters, a kernel size of 3, and no padding. The fourth layer is an average pooling layer. The fifth layer is a convolutional layer with 64 filters, a kernel size of 3, and no padding. The sixth layer is projection layer with an output size of 128. The seventh and final layer is the output layer, with an output size of 10.
For the SVHN dataset, the first layer is a convolutional layer with 64 filters, kernel size of 3, and padding of 1. The second layer is a max- pax-pooling layer. The third layer is a convolutional layer with 128 filters, a kernel size of 3, and padding of I. The fourth layer is another max-pooling layer. The fifth layer a convolutional layer with 64 filters, a kernel size of 3, and padding of 1. The sixth layer is a projection layer with an output size of 256. The seventh and final layer is the output layer, with an output size of 10.
For the CIFAR-10 10 dataset, the first layer is convolutional layer with 64 filters, kernel size of 3, and padding of 1. The second layer is a max-pooling layer. The third layer is a convolutional layer with 128 filters, kernel size of 3, and padding of 1. The fourth layer is another max max-pooling layer. The fifth layer is a convolutional layer with 64 filters, a kernel size of 3, and padding of 1. The sixth layer is an average pooling layer. The seventh layer is a projection layer with an output size of 256. The eighth and final layer is the output layer, with an output size of 10.
Training
All models were optimized using the Adam optimizer, with a linear learning rate scheduler for weight decay. The hyperbolic tangent activation function was employed in all layers except the output layer. For consistency across all datasets, we standardized the hyperparameters to learning rate of 5 x 10-5 and a batch size of 64.
D.2.3. RLN-
Model Architecture Recurrent Neural Networks (RNNs) are a class of neural networks designed to recognize patterns in sequences of data, such as time series, natural language, or any other sequence data. Unlike traditional feedforward neural networks, RNNs have connections that form directed cycles, allowing them to maintain a hidden state that can capture information about previous time steps. In our DLL-RNNs architecture, the network consists of three layers:
Input Layer:
This layer accepts a one-dimensional tensor. The dimensionality of this tensor is determined by the data format of different datasets.
Hidden Layer: The hidden layer's dimensionality is tunable hyperparameter. We have implemented DLL version of the RNNs, where the hidden layer is divided into two parts: hé. responsible for forward output, and h", responsible for receiving error signals.
Output Layer: This layer outputs a one-dimensional tensor, with its dimensionality also determined by the specific dataset's requirements.
In our DLL-RNNS model, during the output phase at time step i, the h part of the hidden layer receives input x and the input from the previous time step h It then outputs to the current time step y; and the next time step h
Evaluation Metrics We utilize character prediction accuracy to evaluate the performance of the DLL-RNNs model on a text character prediction task. Given character sequence of length n + 1, denoted as [0, n]. the model is designed to operate over time steps. During each time step i, the model receives one-hot encoded tensor corresponding to the i-th character and predicts the one-hot encoded tensor for the (i 1)-th character. This process involves iteratively accepting input and generating predictions. The evaluation metric is based on counting the number of correct predictions made at each step, thereby measuring how well the model learns the sequential patterns and predicts subsequent characters accurately throughout the sequence. We utilize the Mean Squared Error (MSE) loss and the Mean Absolute Error (MAE) loss to evaluate the performance of the DLL--NNs model the context of time-series forecasting tasks. The Mean Squared Error
17

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
(MSE) is defined as:
MSE Σ ==1
U - 8.)2
(31)
In this formula, n represents the total number of observations. The symbol y, denotes the true value of the i-th observation, while y, stands for the backpropagated label of the i-th observation. The term (y: - y:)² is the squared difference between the true and backpropagated labels for the i-th observation. The Mean Absolute Error (MAE) defined as:
n MAE Σ
U Vzl.
(32)
Here, n is the total number of observations. The variable y: indicates the true value of the i-th observation, and ý; represents the predicted value of the i-th observation. The expression [y: y refers to the absolute difference between the true and backpropagated labels for the i-th observation.
Training prrper-prrametrss We conducted a grid search on each dataset to explore the optimal parameter configurations. Specifically, we selected appropriate hidden layer sizes and learning rates tailored to different tasks to achieve the best training outcomes. Additionally, we employed both cosine and linear schedulers to dynamically adjust the learning rate based on the epoch. This approach ensures that the learning process adapts effectively throughout the training phases, optimizing the model's performance across various datasets and tasks. Ultimately, we configured the hidden layer sizes for different datasets as follows: for the Harry Potter Series, the hidden layer size was set to 324; for the Electricity and Metr-la dataset, it was set to 300; and for the Pems-bay dataset, the hidden layer size was set to 384.
E. Experimental Results on CIFAR-100 and Tiny-ImageNet
We trained CNNs with BP and DLL on CIFAR- -10AR-100a0 and TinylmageNet. Consistent with previous work (Bartunov et al., 2018), we report test accuracy for CIFAR-100 and test error rate for Tiny-ImageNet in Table
Table Performance of CNNs trained with BP and DLJ on CIFAR-100 and Tmay-ImageNet. Test accuracy for CIFAR-100 and test error rate for
Method
CIFAR-100
Tiny-ImageNet
BP_ CNN
44.50%
78.601 78.60%
DLL_CNN
38.60%
82.90%
Note that we do not use any additional training techniques such as batch normalization or residual connections. Our CNN architecture is similar to that in Bartunov et al. (2018). For CIFAR-I00, the CNN consists of four convolutional layers with channel configurations of 3-64-64-128 64, followed by two fully connected layers. For TinylmageNet, the CNN consists of five convolutional layers with filter configurations of 3-64-64- 128- 108--1286644,-- followed by two fully connected layers.
F. Training TextCNN with DLL
In this section, we performed experiments on the text classification datasets Subj' and Movie Review (MR) (Pang & Lee, 2005), with results summarized in Table 5.
Table 5. Performance of TexICNN text classification tasks.
Method
Subj
MR
BP TexICNN
88.50%
74. 74.68%
OLL_TeXICNN
84.40%
70.79%
The architecture is identical to the original TexICNN (Kim, 2014). accuracy to those trained with BP on two datasets.
The models trained with DLL. achieve comparable
'sttpp://wwen es cs cornell cadu/poop o htt/paa///oooo revi aw-dat
18

---

Dendritic Localized Learning: Toward Biologically Plausible Algorithm
G. Time Consumption and Memory Usage
We record the training time consumption and GPU memory usage when conducting experiments on image classification. The time consumption and memory usage for these experiments are summarized in Table
Table 6. Training time consumption and GPU memory usage of BP and DLL.
Method
Training Time (s/Epoch)
Memory Usage (MB)
BP_ PP_MLP
31.6
1286.4
BP CNN
99.0
1272.9
DLL MLP
44.7
1595.3
DLL_CNE
169.8
1306.9
To fairly compare time consumption across architectures, we used the CPU instead of the GPU. DLL requires more training time and memory because both the forward weight W and backward weight O are updated simultaneously. Our design not driven by computational or memory efficiency; rather, we prioritize biological plausibility.
H. Scalability of DLL
In this section, we conduct scalability experiments and show the results in Table 7:
Table 7. Scalability of ML.s trained with DLI.
DLL-MLP Architecture
Accuracy on MNIST
784-1024-10
71.15%
784-1024-512-10 784- 1224--1--120
89.61%
97.57%
All MLPs are trained fairly, and the results show the scalability of DLL.
I. Limitations and Future Directions
LII Limitations.
Despite the promising contributions of Dendritic Localized Learning (DLL), several limitations remain. First, the implemen- tation and evaluation of DLL have been restricted to conventional artificial neural networks (ANNs) architectures such as MLPs, CNNs, and RNNs, leaving its applicability to spiking neural networks (SNNs) (Maass, 1997) unexplored. Given the increasing interest in SNNs for energy-efficient and biologically realistic computing, this is a critical area for future work. Second, while DLL achieves biological plausibility by satisfying the proposed criteria, its reliance on unsigned error signals may present challenges in scenarios requiring precise error aligmment or task-specific optimization. Addressing these issues essential for improving both the versatility and robustness of the algorithm.
1.2. Future directions.
Future research will focus on extending the DLL framework to SNNs, leveraging their potential for energy-efficient computation and neuromorphic hardware compatibility. This extension will require adapting the DLL handle the temporal dynamics and discrete spike-based representations inherent SNNs, further aligning the algorithm with biological principles. Additionally, exploring hybrid architectures that integrate DLL with traditional learning mechanisms may enhance their scalability and performance more complex datasets and tasks. Another avenue involves addressing the challenges posed by unsigned error signals, such as developing task-specific adjustments or augmenting the algorithm with mechanisms to improve error precision. Finally, we envision applying DLL to real-world problems in areas like robotics, neuroscience, and autonomous systems, validating its practical utility and impact across diverse domains.
19


Reviews:
[{"review_id": "Z7ubufUAaP", "reviewer": "Reviewer_7p2D", "summary": "The paper focuses on biological plausibility. Looking at prior work, this paper proposes three different metrics for biological plausibility including (i) asymmetry between the forward and the backward weights, (ii) local losses, and (iii) non two-stage training. With that, the paper proposes a new learning system i.e., Dendritic Localized Learning (DDL), which they compare against other local learning approaches as well as end-to-end backpropoagation. The obtained results on computer-vision classification datasets, time-series forecasting datasets, and NLP datasets highlights that potential of DDL in comparison to other local learning approaches.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### 1. Poorly supported claim of 3 criteria without any citation. There are many other criteria explored in prior work.\n\nFirstly, in Section 2.2, we provide a detailed introduction and evaluation of all representative biologically plausible learning algorithms. These algorithms served as the basis for the three criteria, as mentioned in Line 33 (left column). However, we acknowledge that we should have included more explicit citations before discussing these criteria in Section 2.1, and we appreciate the reviewer’s suggestion in this regard.\n\nSecondly, our work specifically focuses on biologically plausible supervised learning algorithms. For instance, when discussing Hebbian Learning and STDP, we highlighted their limitation in leveraging supervised learning signals (Line 188, left column). While we acknowledge that studies such as [4][5] have proposed alternative criteria—such as self-supervised learning and independence between layers—we emphasize that these approaches are often based on unsupervised learning. We fully recognize the value of these criteria in assessing biological plausibility; however, it is challenging to incorporate all possible criteria within a single paper.\n\nTo address this, we will add a discussion and cite relevant studies in our revised manuscript to clarify our motivation. We appreciate the reviewer’s insightful feedback.\n\n### 2. Ignorance of recent related work [3][4] on local learning. Local learning methods satisfy C3 naturally as there is no end-to-end update.\n\nFirstly, as mentioned in the last question, our work focuses on supervised learning algorithms.\n[3] proposed a layer-wise self-supervised pre-training algorithm based on random masking and image recovery, while [4] explored deepening model layers within unsupervised contrastive learning frameworks.\nAlthough these methods help scale local learning approaches to larger settings, their biological plausibility is limited, as they do not fully adhere to Criteria 1 and 3.\nIn this study, our primary goal is to take a step closer to BP while maintaining biological plausibility.\nTo achieve this, we designed the DLL algorithm based on three key criteria derived from existing biologically plausible learning algorithms. \nThat said, we acknowledge the relevance of [3] and [4] and will incorporate a discussion of these works in Section 2.2 and the related work section.\n\nSecondly, we define Criterion 3 as non-two-stage training, meaning there is no strict temporal segregation between forward and backward processes.\nWe believe that not all local learning methods satisfy this criterion. For instance, difference target propagation is a local learning method but does not meet Criterion 3, as we discussed in Line 143 (right column).\nWhile the training methods in [3] and [4] do not involve end-to-end updates, they still exhibit a clear separation between forward and backward processes.\n\n### 3. Writing suggestions: Line 263 and Line 318.\nThanks for pointing them out. We will correct them in our revised manuscript.\n\n### 4. Why is the performance of the selected architectures poor on CIFAR-10?\n\nPlease refer to the first question of reviewer 2B1A.\n\n### 5. How were the architectures selected? How were the hyperparameters tuned? Were they the same for all methods?\n\nYes, as stated in Line 325 (left column) and Appendix C.2, we ensured that all algorithms were evaluated using the same hyperparameters and architectures for a given dataset.\nOur choices for architecture and hyperparameter settings were based on the methodologies outlined in [1][2].\n\n### 6. Can this method be adapted to residual networks?\n\nWe recognize that there is biological evidence supporting inter-regional and inter-layer neuronal connections.\nHowever, whether these connections function equivalently to residual connections in deep learning remains an open question.\nResidual connections were originally introduced to mitigate the gradient vanishing problem in deep learning models, which is less of a concern for local learning-based approaches like ours.\nAs a result, their potential benefits may not be as relevant or impactful in our setting.\n\nWe acknowledge the importance of exploring such architectural enhancements.\nIn future work, we will consider incorporating residual connections to further investigate their potential impact and applicability.\n\n### Reference\n\n[1] Bartunov S, Santoro A, Richards B, et al. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. NeurIPS 2018.\n\n[2] Millidge B, Tschantz A, Buckley C L. Predictive coding approximates backprop along arbitrary computation graphs. Neural Computation, 2022, 34(6): 1329-1368.\n\n[3] Siddiqui S, Krueger D, LeCun Y, et al. Blockwise Self-Supervised Learning at Scale. TMLP.\n\n[4] Xiong Y, Ren M, Urtasun R. Loco: Local contrastive representation learning. NeurIPS, 2020", "claims_and_evidence": "While the general claims are well-supported, the claim regarding the three properties being sufficient is really weak without any citation. I attempted to elaborate on that more in the later sections.", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes. The evaluation and analysis is straightforward as the paper focuses on standard benchmark datasets.", "relation_to_broader_scientific_literature": "The claim of a new contribution based on the analysis of properties of local learning system ignores other work in this space. This is not a unique contribution of the paper as previous papers did the same. Furthermore, the three learning criterions covered in the work can't be considered complete (e.g., see https://arxiv.org/abs/2302.01647 -- I'll cover this more in the weaknesses section). Finally, the paper ignored all of the recent local learning methods, and only focused on old ones. Particularly, the paper ignored all methods that attempt to scale to larger datasets (e.g., see https://arxiv.org/abs/2008.01342 and https://arxiv.org/abs/2302.01647). Local losses are a very active area of research. The covered literature on this is very weak with just one paper.", "essential_references_not_discussed": "Yes, quite a lot of them. The paper didn't cover any of the most recent methods in this space, with almost all comparative analysis limited to papers from the late 2010s. E.g., local losses itself is a very active space, with LoCo being one of the most prominent papers in this space: https://arxiv.org/abs/2008.01342). There are now many extensions of this. I also elaborate on this in my other comments.", "other_strengths_and_weaknesses": "# Strengths\n- Well motivated problem\n- Simple and well-motivated method\n- Well-written paper\n- Comprehensive coverage in terms of evaluation\n# Weakness\n- Poorly supported claim of just three essential properties without any citation from the neuroscience literature that argues about the biological plausibility of these ideas. They seemed to emerge out of the blue. There are many other criterions explored in prior work. E.g., https://arxiv.org/abs/2302.01647 argued that self-supervised learning and no dependence within layers are also essential for biological plausibility. Hence, the contribution of estimating essential properties based on prior work is not the first. Furthermore, it is hard to claim that the list is complete with these three properties.\n- Ignores almost all of the recent related work in this space. Particularly, there has been a lot of focus on recent methods that attempt to make local learning methods scale to large-scale settings (such as ImageNet), while the papers cited in the current work even fail to work on MNIST at times. With correct local learning methods that detach one layer from another, they do satisfy C3 naturally as there is no end-to-end update.\n- Weak architecture selection that already results in poor performance as a start, even with end-to-end BP. E.g., in table 1, the performance of CIFAR-10 with a CNN is just 75%, which is really weak.", "other_comments_or_suggestions": "- Line 263 (left column): mention that 'f' represents the non-linearity (which is clarified in Algorithm 1)\n- Line 318 (left column): \"correctly backpropagated\" -> \"correct predicted\"", "questions_for_authors": "- How were the architectures selected? Why is the performance of the selected architectures poor on CIFAR-10?\n- How were the hyper parameters tuned? Were they the same for all methods?\n- Can this method be adapted to residual networks?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}, {"review_id": "vAr5Xs6YCZ", "reviewer": "Reviewer_2B1A", "summary": "The paper introduces Dendritic Localized Learning (DLL), a biologically plausible learning algorithm inspired by the structure and plasticity of pyramidal neurons. The motivation behind DLL is to address three fundamental biological limitations of backpropagation:\n\n- Weight symmetry – Backprop requires symmetric forward and backward weights, which is not biologically plausible.\n- Global error signals – Biological neurons primarily use local learning rules rather than propagating global error through all the layers.\n- Dual-phase learning – Backprop separates forward and backward passes, whereas biological learning does not have such a strict separation.\n\nThe proposed DLL framework models neurons with three compartments (soma, apical dendrite, basal dendrite) and introduces trainable backward weights  instead of using transposed forward weights in error propagation. The algorithm enables local error computation within neurons and allows for simultaneous weight updates, satisfying biological plausibility criteria.\n\n## update after rebuttal\nI appreciate the authors effort in rebuttal. While I understand the importance of maintaining uniformity across algorithms, I do believe that that for a new learning algorithm, it is also useful to measure or discuss the compatibility with the standard regularization techiniques like weight decay, batch normalization. Also, I don't fully agree that adding dropout for instance makes the approach biologically implausible. Noise is a salient feature of the learning machinery of the brain and dropout, adds a source of noise. \n\nI am not sure if it is a typo, but the authors report DLL to be outperforming on the most challenging dataset (TinyImageNet) which seems highly unlikely and the numbers are quite high compared to Cifar10 and Cifar100. It is also a bit concerning to see model performing better on TinyImageNet compared to Cifar10.\n\nWith these concerns remaining, I will retain my original score and do not feel confident to champion the paper for acceptance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### 1. Performance of BP on CIFAR10 with CNN (75%) is low.\n\nFirstly, as stated in Line 325 (left column), to ensure fair comparisons between different algorithms, we used the same architecture for all methods on a given dataset. \nSpecifically, for CIFAR-10, we employed a CNN with three convolutional layers (see Appendix C.2.2) without incorporating deep learning techniques such as batch normalization, residual connections, or dropout. \nThis design choice was made to maintain full biological plausibility and prevent potential biases that could arise from the selective application of these techniques across different algorithms.\nUnder these conditions, the accuracy achieved by our BP-CNN on CIFAR-10 is reasonable.\n\nSecondly, previous studies like [1][2][3], which also did not incorporate additional training techniques, reported test accuracies of only achieving 60%-70% on CIFAR10 using similar CNN architectures.\nOur primary objective was to establish a fair and unbiased evaluation framework, ensuring that comparisons reflect the intrinsic differences between algorithms rather than the influence of external training enhancements.\n\n### 2. Effect of common regularizations like weight decay or batch normalization. Does DLL work for modern architectures like ResNets?\n\nAs mentioned in the last question, we did not incorporate additional training techniques.\nIf we were to use them, special design considerations would be necessary, as they may violate certain criteria.\nFor example, batch normalization and dropout behave differently during training and testing, and directly applying them to our DLL framework would break Criterion 3 (non-two-stage training).\nWe acknowledge that incorporating such techniques, particularly ResNets, could be beneficial for future optimizations and will consider them in subsequent work.\nHowever, our primary objective in this paper was to explore and evaluate the biological feasibility of the algorithms without relying on external enhancements.\n\n### 3. How does DLL perform on more complex datasets like CIFAR100 or Tiny-ImageNet?\nWe followed your suggestions and trained CNNs with BP and DLL on CIFAR-100 and TinyImageNet.\nConsistent with previous work [1], we report test accuracy for CIFAR-100 and test error rate for TinyImageNet.\n\n|Method|CIFAR100 (test accuracy)|TinyImageNet (test error rate)|\n|-|-|-|\n|BP_CNN|44.5%|78.6%|\n|DLL_CNN|38.6%|82.9%|\n\nNote that we do not use any additional training techniques such as batch normalization or residual connections. Our CNN architecture is similar to that in [1].\nFor CIFAR-100, the CNN consists of four convolutional layers with channel configurations of 3-64-64-128-64, followed by two fully connected layers.\nFor TinyImageNet, the CNN consists of five convolutional layers with filter configurations of 3-64-64-128-128-64, followed by two fully connected layers.\n\n### 4. Discussion on convergence properties and guarantees would make the manuscript much stronger.\n\nThanks for your advice. We will add the following discussion on convergence properties and guarantees:\nThe loss function (Eq. 3) is designed to minimize the discrepancy between the top-down predictions and bottom-up outputs of each pyramidal neuron in the network.\nTo achieve this, we employ local gradient descent–based learning rules and neural plasticity mechanisms to update both forward and backward weights.\nDuring each iteration, the differences between the network’s predictions and the ground truth propagate back through localized errors, effectively coordinating all neurons in an orchestrated manner.\nAs a result, neural responses collectively refine predictions over successive iterations, gradually reducing local errors and driving the network toward convergence.\nWhile providing formal convergence proofs remains challenging due to the network’s nonlinear operations, our empirical results consistently demonstrate a steady decrease in loss throughout training, supporting the stability and effectiveness of our approach.\n\n### 5. How can the performance gap between BP and DLL be bridged?\n\nCurrently, most biologically plausible learning algorithms exhibit a significant performance gap compared to BP. In this study, our primary goal is to move closer to BP while preserving biological plausibility.\nIf the sole objective were performance improvement, specialized training techniques such as normalization and dropout—adapted to our proposed DLL algorithm—could be explored in future work.\n\nReference\n\n[1] Bartunov S, Santoro A, Richards B, et al. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. Advances in neural information processing systems, 2018, 31.\n\n[2] Millidge B, Tschantz A, Buckley C L. Predictive coding approximates backprop along arbitrary computation graphs. Neural Computation, 2022, 34(6): 1329-1368.\n\n[3] Salvatori T, Song Y, Yordanov Y, et al. A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks. ICLR, 2024", "claims_and_evidence": "The authors claim that DLL satisfies all three criteria for biological plausibility while maintaining strong empirical performance compared to prior biologically plausible learning algorithms. They provide:\n- Mathematical derivation of the DLL learning rule and its update equations. \n- Empirical performance on MLPs, CNNs, and RNNs across various datasets.\n- Comparisons with other biologically plausible algorithms, showing that DLL achieves superior accuracy among methods that meet all three criteria.", "methods_and_evaluation_criteria": "The experimental methodology is well-structured and covers different architectures (MLPs, CNN and RNNs) and datasets. This showcases the versatility and applicability of their learning algorithm.", "theoretical_claims": "The paper presents a mathematical derivation of the DLL learning rule and its update equations. Reviewer did not verify the correctness of their derivation.", "experimental_designs_or_analyses": "Overall, the experimental design in sound. I have a few concerns:\n- The performance of backpropagation on cifar10 with CNN (75%) seems quite low. Can the authors provide any justification for this,\n- Authors do not discuss the effect of common regularizations like weight decay or batch normalization. Does it work for modern architectures like ResNets?\n- It would be insightful to see how DLL performs on more complex datasets like Cifar100 or TinyImageNet.", "relation_to_broader_scientific_literature": "The paper clearly situates DLL within the field of biologically plausible learning.", "essential_references_not_discussed": "Not to reviewers' knowledge.", "other_strengths_and_weaknesses": "- DLL provides a promising approach which fulfills biologically plausibility criterion and provides comparable performance. \n- Well written and structured and easy to follow\n\nFor weaknesses, see the concerns mentioned above.", "other_comments_or_suggestions": "In addition to addressing the concerns raised above, manuscript would benefit from\n\n- Discussion on convergence properties and guarantees would make the manuscript much stronger. \n- Discussion on how the authors believe the performance gap between backpropagation and DLL can be bridged and its applicability to modern architectures and complex datasets.", "questions_for_authors": "Q1) Can the author comment on the applicability of DLL on deeper CNNs and modern architectures like ResNets.\n\nQ2) Can the authors explain the low performance with backpropagation on  CIfar10 with CNN. And the effect of weight decay and batch normalization on DLL performance and convergence.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4Qpf62inig", "reviewer": "Reviewer_p1AA", "summary": "The paper introduces Dendritic Localized Learning, as an alternative to backpropagation for training neural networks. The goal is to make learning more biologically realistic by addressing three main issues with backpropagation: the requirement of weight symmetry between the forward and backward passes, the use of global error signals, and the separation of learning into distinct forward and backward phases. The authors propose a model inspired by pyramidal neurons, which has separate compartments for different types of information processing. Instead of using the transposed forward weights for the backward pass, DLL introduces a separate set of trainable backward weights. The authors present experiments that show DLL performing better than other biologically plausible learning algorithms while coming close to backpropagation in accuracy. They apply DLL to multilayer perceptrons, convolutional networks, and recurrent networks and compare it to alternatives like feedback alignment and target propagation.\n\n## update after rebuttal\n\nThanks to the authors for the detailed and thoughtful response. I appreciate the additional experiments on text datasets, the clarification around computational costs, and the effort to demonstrate the stability and scalability of DLL. It’s clear that a lot of work went into addressing the concerns.\n\nThat said, my overall view remains the same. While the new experiments help strengthen the paper, the broader limitations — such as the performance gap on complex tasks, the limited scale of evaluation, and the lack of deeper theoretical guarantees — are still there. I still find DLL to be a promising step toward more biologically plausible learning, and the empirical results are encouraging within the scope tested. So, I’m keeping my original score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. More experimental validation, especially real-time learning, to show simultaneous forward and backward.\n\nOur approach enables real-time learning, as higher-layer neurons propagate signals backward only when a discrepancy between the output and the label is detected. Otherwise, no adjustments are made, and no backpropagation signals are generated.\nWhile this mechanism is theoretically sound, we are uncertain about the best way to empirically validate it.\nWe would greatly appreciate any suggestions on how to design experiments to test and confirm this behavior.\n\n2. Accuracy gap on complex datasets.\n\nWe primarily focus on comparing the biological plausibility of various learning algorithms rather than optimizing for high performance.\nWithin these biological constraints, we propose the DLL algorithm, which further narrows the performance gap with BP.\nAs shown in Table 1, DLL achieves the best performance among algorithms that satisfy all three criteria.\n\n3. Theoretical analysis for convergence.\n\nPlease refer to the 4th question of Reviewer 2B1A.\n\n4. Complex datasets like ImageNet or NLP tasks are ignored.\n\nGiven our current computational resources (four 2080 GPUs with 12GB each), we estimate that conducting experiments on the full ImageNet within 7 days would not be feasible. \nTherefore, we chose to evaluate our methods on Tiny-ImageNet.\nThe results are presented in our response to the third question from reviewer 2B1A.\n\nFor NLP tasks, we have followed [1] to conduct experiments on the \"next-character-prediction\" task in Section 4.3.\nAdditionally, we performed experiments on the text classification datasets Subj and Movie Review (MR), with results summarized below:\n|Method|Subj|MR|\n|-|-|-|\n|BP_TextCNN|88.50%|74.68%|\n|DLL_TextCNN|84.40%|70.79%|\n\nThe architecture is the same as the original TextCNN (Kim, EMNLP2014).\n\n5. Computational cost, memory efficiency, or sensitivity to hyperparameters\n\nThe time consumption and memory usage for these experiments are summarized below:\n|Method|Time Consumption (s/epoch)|Memory Usage (MB)|\n|-|-|-|\n|DLL_MLP|44.7|1595.3|\n|DLL_CNN|169.8|1306.9|\n|BP_MLP|31.6|1286.4|\n|BP_CNN|99.0|1272.9|\n\nTo fairly compare time consumption across architectures, we used the CPU instead of the GPU.\nDLL requires more training time and memory because both the forward weight $\\mathbf{W}$ and backward weight $\\Theta$ are updated simultaneously.\nOur design is not driven by computational or memory efficiency; rather, we prioritize biological plausibility.\n\nAs for sensitivity to hyperparameters, we have included a comparison of various learning rates and sequence lengths in Figure 3.\n\n6. The authors do not establish whether DLL optimizes a well-defined loss function in a way that guarantees stable learning\n\nFigure 2(c) shows that models trained with DLL exhibit a stable decrease in training loss.\nThis figure is based on time-series forecasting experiments using RNNs on the Electricity dataset.\nSimilarly, for MLPs and CNNs, the training losses also show a consistent downward trend.\nWe will release all training logs upon acceptance.\n\n7. No ablation study to determine which aspects of DLL are most responsible for its performance.\n\nOur DLL algorithm is designed based on three criteria that we have summarized from current biologically plausible algorithms.\nIt can be viewed as a faster (without iteration) and more biologically plausible implementation of predictive coding (PC), leveraging the unique properties of pyramidal neurons. Our design removes weight symmetry and separates forward and backward computations.\nAs a result, the performance of DLL is similar to that of PC under certain conditions, and PC has been shown to achieve performance comparable to backpropagation [1][3].\nWe have conducted an ablation study on the backward weight $\\Theta$ in Table 3.\nWe would appreciate any suggestions on additional ablation studies.\n\n8. Code lacks documentation. No details about hyperparameter selection.\n\nUpon acceptance, we will release our code with detailed documentation for reproducibility.\nThe selection of hyperparameters follows the similar choices made in [1][2].\n\n9. Discussion of other learning methods.\n\nWe will add related papers on apical dendrites.\nPlease refer to the 5th question of reviewer Crph.\n\n10. Robustness and scalability?\n\nRobustness is an interesting aspect, and we plan to explore it in future work.\n\nWe conduct scalability experiments:\n\n|DLL-MLP|MNIST|\n|-|-|\n|784-1024-10|71.15%|\n|784-1024-512-10|89.61%|\n|784-1024-512-256-10|97.57%|\n\nAll MLPs are trained fairly, and the results show the scalability of DLL.\n\n[1] Millidge B, et al. Predictive coding approximates backprop along arbitrary computation graphs. Neural Computation, 2022.\n\n[2] Bartunov S, et al. Assessing the scalability of biologically-motivated deep learning algorithms and architectures. NeurIPS, 2018.\n\n[3] Salvatori T, et al. A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks. ICLR, 2024.", "claims_and_evidence": "The authors claim that DLL satisfies all three criteria for biological plausibility and achieves strong performance on benchmark datasets. The claim that DLL removes the requirement of weight symmetry is supported by the introduction of independent backward weights, which is a reasonable approach. The claim that DLL eliminates the need for a global error signal is also valid because local errors are computed at each neuron. The claim that DLL allows simultaneous forward and backward computation is plausible but could use more experimental validation, especially regarding real-time learning. The claim that DLL achieves performance close to backpropagation is somewhat supported by the reported results, but there is still a noticeable accuracy gap, especially on more complex datasets. There is no strong theoretical guarantee provided for convergence or learning efficiency, which weakens the claim that DLL is a robust learning method.", "methods_and_evaluation_criteria": "The proposed method is appropriate for the problem of biologically plausible learning, and the chosen datasets provide a reasonable benchmark. However, the experiments focus mostly on small-scale datasets like MNIST and CIFAR-10. It would be more convincing to see results on more complex datasets like ImageNet or natural language processing tasks. The authors compare DLL against several well-known biologically inspired learning methods, which is a strong aspect of the paper. However, the evaluation mainly considers accuracy, and there is little discussion of computational cost, memory efficiency, or sensitivity to hyperparameters, all of which are important in assessing the practicality of a learning algorithm.", "theoretical_claims": "The paper includes mathematical descriptions of DLL, but it does not provide a formal proof of convergence or an analysis of how DLL behaves in different training conditions. While the authors argue that DLL follows biologically plausible principles, they do not establish whether DLL optimizes a well-defined loss function in a way that guarantees stable learning. The absence of such analysis leaves a gap in understanding whether DLL is a reliable alternative to backpropagation or just an interesting theoretical idea with promising empirical results.", "experimental_designs_or_analyses": "The authors conduct a reasonable set of experiments to compare DLL with other learning algorithms, but there are some issues. The choice of datasets is somewhat limited, as all datasets used are relatively small. There is no evaluation of DLL on tasks requiring deeper networks or larger-scale learning. The comparison to backpropagation is primarily based on accuracy, but efficiency is not analyzed. There is no indication of whether DLL requires significantly more training time or memory compared to standard backpropagation. Additionally, there is no ablation study to determine which aspects of DLL are most responsible for its performance. Without such analysis, it is unclear whether the proposed approach is truly necessary or if simpler modifications to existing biologically plausible learning rules could achieve similar results.", "relation_to_broader_scientific_literature": "The paper positions DLL as an improvement over previous biologically plausible learning algorithms, particularly feedback alignment and predictive coding. The work is connected to existing neuroscience literature on pyramidal neurons and local synaptic plasticity, which supports its biological inspiration. However, there is little discussion of how DLL relates to other learning methods beyond biologically plausible algorithms. It would be useful to compare DLL’s principles with energy-based models or reinforcement learning approaches that also incorporate local learning rules.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "One of the strongest aspects of this paper is its attempt to provide a biologically inspired learning method that is more plausible than backpropagation while maintaining strong performance. The empirical results suggest that DLL is a promising alternative, and the idea of using separate backward weights is an interesting contribution. However, there are some weaknesses that need to be addressed. The paper lacks a theoretical analysis of DLL’s convergence and stability. The experimental evaluation does not include large-scale tasks, so it is unclear how well DLL scales to more complex problems. There is no discussion of the computational efficiency of DLL compared to backpropagation. The code provided is not well-documented, making it difficult to reproduce the results.", "other_comments_or_suggestions": "The authors should include a discussion of DLL’s efficiency and computational cost relative to backpropagation. They should also provide more details on hyperparameter sensitivity and whether DLL requires careful tuning to perform well. The code should be better documented to improve reproducibility. It would also be useful to include a theoretical analysis of DLL’s stability and convergence properties.", "questions_for_authors": "How does DLL compare to backpropagation in terms of training time and memory usage? If DLL is slower, what are the trade-offs in terms of biological plausibility versus efficiency?\nWould DLL generalize well to large-scale datasets like ImageNet or more complex NLP tasks? Have any experiments been conducted to test its scalability?\nDoes DLL have any robustness advantages, such as resistance to adversarial attacks or improved performance on noisy data?\nAre there specific hyperparameters that need to be fine-tuned for DLL to work well, or is it stable across different settings?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "LaDR8uz6HO", "reviewer": "Reviewer_hLAs", "summary": "The article proposes a neural network that is constructed using a local loss and asymmetric weights in the forward and backward passes. The author introduces three criteria for biological plausibility that the neural network should satisfy and demonstrates that the proposed DLL meets these criteria. The author also shows through experiments that DLL outperforms other non-traditional BP networks.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your comments and suggestions, which are valuable for enhancing our paper. We are pleased that our contributions are well recognized.\n\nResponses to your concerns and questions are hereby presented:\n\n### 1. Why are the three criteria for biological plausibility important? More ablation studies are needed to prove that the three criteria contribute to improving the model's performance.\n\nThanks for your suggestion. The three criteria outlined in our paper were summarized from existing biologically plausible learning algorithms, rather than introduced as a means to enhance model performance.\nOur primary objective is to investigate the biological plausibility of learning algorithms. Based on this investigation and the biological limitations of existing methods, we propose a novel approach that satisfies all three criteria—making it more biologically plausible—while achieving performance comparable to backpropagation.\nWe have conducted an ablation study on the backward weight $\\Theta$ in Table 3.\nCould you please provide any suggestions on how to design further ablation experiments?\n\n### 2. Gradient descent is still employed for training. Could this be the main reason why DLL outperforms previous work?\n\n\nThank you for your insightful comment.\nWe agree that STDP is one of the most biologically plausible learning algorithms, as it not only avoids gradient descent but also satisfies the three criteria we summarized.\n\nFrom the perspective of update rules, the high performance of our method stems from the local neuronal plasticity update rules we derived (Equations 8 and 9), rather than relying on a global gradient backpropagation process.\nWe believe that local updates to neuronal plasticity are consistent with the form of computing gradients on local losses, which is a plausible mechanism that could have emerged through long-term evolution.\nSimilar approaches have been employed in other biologically plausible learning methods, such as predictive coding, target propagation, and local losses.\nAlthough gradient descent is not used in STDP, it does not necessarily mean the neuronal plasticity rules can not be derived locally by using gradients.", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "Yes", "theoretical_claims": "Yes, I checked the math and didn't find any obvious issues.", "experimental_designs_or_analyses": "The author demonstrates the advantage of DLL over other non-traditional BP methods through experiments, but I think more ablation studies need to be added to prove that the three design criteria for biological plausibility of DLL contribute to improving the model's performance. This will enhance the soundness of this work.", "relation_to_broader_scientific_literature": "I think the research in this paper is related to brain-inspired computation and computational neuroscience.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths\n    - The article summarizes the characteristics of some non-traditional BP learning methods, extracts three criteria, and designs DLL based on them. Experimental results demonstrate that DLL can outperform previous similar methods.\n    - In today's world where SGD is widely used, exploring new learning methods is refreshing and can increase attention to learning approaches within the field.\nWeaknesses\n    - I think the main issue is that the author should clarify why the three criteria for biological plausibility are important, for example, through ablation studies or theoretical proofs. This would enhance the scientific value of the paper.\n    - Although biological plausible local loss and asymmetric weights are used, gradient descent is still employed for training in this paper. Could this be the main reason why DLL outperforms previous work? An important criterion of brain-inspired learning rules is to abandon GD (such as STDP), because the brain does not directly compute gradients.", "other_comments_or_suggestions": "No.", "questions_for_authors": "Please see weaknesses.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "nI16tuSTUi", "reviewer": "Reviewer_Crph", "summary": "This work proposes a biologically plausible algorithm for training deep neural networks utilizing apical dendrites. The authors apply the proposed algorithms to learning in MLPs, CNNs, and RNNs, and demonstrate that the proposed algorithm outperforms previous biologically plausible algorithms that satisfy all three plausibility criteria set up by the authors.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### 1. Feedback Alignment with MLP on MNIST can achieve 98% test performance.\n\nAs stated in Line 325 (left column), to ensure fairness, we adopted the same architecture (784-1024-512-256-10 FC layers, see Appendix C.2) for all algorithms on MNIST.\nThe architecture used in the paper you mentioned consists of a 256-256-256-256-256-10 FC structure.\nDifferences in network architecture can lead to performance discrepancies, and deeper networks typically achieve higher accuracy.\nFor fair comparison, we avoided additional training technologies (like batch normalization, residual connection) as much as possible because potential biases introduced by these additional techniques may have varying effects across different algorithms.\nTherefore, it is not surprising that our reproduced test performance is slightly lower than theirs.\n\n### 2. The second criterion is problematic considering the neuro-modulatory signals in the brain that deliver global error signals to neurons.\n\nThanks for your suggestion.\nWe are aware of the presence of global signals in the brain, which play an important role in modulating certain mechanisms across neurons.\nHowever, there is currently no clear biological evidence that these signals represent errors corresponding to the difference between expected and actual outputs, nor that they precisely capture the direction and magnitude required for updating each neuron.\nWe agree that further clarification of this criterion is necessary and will add this discussion into Section 2.1.\n\n### 3. In the second line of Eq. 15, the derivatives with respect to i+2, …, n are omitted\n\nThank you for pointing this out. There should indeed be an ellipsis when expanding $\\mathcal{L}$ in the second line of Eq.15. The rest of Eq. 15 remains correct.\n\n### 4. Implementation details of previous algorithms are missing.\n\nAs stated in Line 325 (left column), to ensure fairness, we use the same model architecture across all learning algorithms for MLPs, CNNs, and RNNs under a certain dataset.\nDetailed model specifications and training configurations can be found in Appendix C.2.\n\n### 5. The authors did not discuss works using apical dendrites for credit assignments in the related work section.\n\nThanks for your suggestion.\nWhile apical dendrites have been discussed in previous literature for credit assignment and have been utilized for various purposes, our study takes advantage of their properties to design and implement more biologically plausible learning algorithms, which differ significantly from existing approaches.\nJ Guerguiev et al. (eLife, 2017) primarily aimed to explain how deep learning can be achieved using segregated dendritic compartments, but it did not propose a specific learning algorithm.\nAs for Sacramento et al. (NeurIPS 2018), we have acknowledged in Line 200 (right column) that we followed their division strategy for pyramidal neurons.\nA Payeur et al. (Nat Neurosci 2021) investigated burst-dependent synaptic plasticity.\nWhile their work shares conceptual similarities with ours in terms of apical dendritic processing, the primary objective of their study differs from ours.\nOur proposed method not only satisfies all three criteria but also achieves higher performance compared to existing biologically plausible learning methods.\nThat said, we will add the discussion in the related work section. Thank you!\n\n### 6. If $\\Theta$ and $\\mathbf{W}$ are initialized to be the same and $ξ_i$ in Eq.9 is replaced with $u_i$, the proposed rule becomes equivalent to backprop. I am not sure if this particular approximation was implemented before.\n\nFirst, even if $\\Theta^T$ and $\\mathbf{W}$ are initialized identically, their update directions will differ because $\\xi_i = u_i - x_i$ and $x_i$ from $\\text{layer}_{i+1}$ cannot be exactly $2u_i$ or $0$ since it comes from lable, so $\\xi_i$ and $u_i$ are different in both magnitude and direction. As a result, $\\Theta^T$ and $\\mathbf{W}$ will update in different directions according to Eq.8 and Eq.9. Furthermore, our formula is derived to minimize the loss, leading to the update formula for $\\Theta^T$ in Eq.9. Therefore, replacing $\\xi_i$ in Eq.9 with $u_i$ would result in an incorrect update direction. In conclusion, our method is not equivalent to BP.\n\nSecond, to the best of our knowledge, there are no existing algorithms that simultaneously satisfy all three criteria while also achieving competitive performance with BP.\nIf you could kindly provide the relevant references, we would be pleased to discuss them in our related work section. Thank you!\n\n### 7. L088: 'Pyramidal neurons consist of 70-85% of neurons': This statement is only true for the cortex.\n\nThank you for pointing it out. We will clarify this in our revised manuscript.", "claims_and_evidence": "The manuscript is technically sound except for the issues on evaluation and novelty discussed below.", "methods_and_evaluation_criteria": "The implementation of the previous algorithms appears to be suboptimal. For instance, in table 1, the performance of feedback alignment in MNIST with MLPs is 91.87%. However, previous work has shown that this algorithm can achieve 98% test performance (Bartunov et al., NeurIPS 2018).\n\nAdditionally, I found the second criterion for the biological plausibility to be somewhat problematic considering the ubiquitous presence of neuro-modulatory signals in the brain that deliver global error signals to neurons.", "theoretical_claims": "Yes, I checked. \n\nIn the second line of Eq. 15, the derivatives with respect to i+2, …, n are omitted. Why is that the case?", "experimental_designs_or_analyses": "Implementation details of previous algorithms are missing.", "relation_to_broader_scientific_literature": "I’m not convinced of the novelty of the manuscript. The idea of using apical dendrites for biologically-plausible credit assignments has been discussed previously (J Guerguiev et al., eLife, 2017; J Sacramento et al., NeurIPS 2018; A Payeur et al., Nat Neurosci 2021, …). This work does not appear to offer improvement either in terms of biological plausibility or performance. It is also disappointing that the authors did not discuss any of these works in the related work section. \n\nIf $\\Theta$ and $W$ are initialized to be the same and $\\xi_i$ in Eq. 9 is replaced with $u_i$, the proposed rule becomes equivalent to backprop. Although I am not sure if this particular approximation was implemented before, I’m not surprised by its decent performance, considering previous work explored similar approximations.", "essential_references_not_discussed": "Please see the comment above.", "other_strengths_and_weaknesses": "A thorough comparison with previously proposed algorithms presented in Table 1 is potentially beneficial for the field, though the authors should make sure that all algorithms are evaluated in a fair condition.", "other_comments_or_suggestions": "L088: 'Pyramidal neurons consist of 70-85% of neurons': I believe this statement is only true for the cortex. The granule cell is presumably the most numerous neuron type in the mammalian brain.", "questions_for_authors": "Please see the comments above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: Summarizing all the reviews, my interpretation and summary is the following: 
-- Solid work. No technical flaws. Authors addressed (most of) the questions about benchmarks and other datasets. 
-- Novelty has been questioned. There is a lot of work on dendrites and local learning. Some of this work was not adequately cited in the original submission and the authors acknowledged this and will cite the work. 
-- There was discussion about whether the 3 key criteria outlined by the authors are sufficient and the consensus is that they are important criteria but not sufficient for biological plausibility. 
-- There are questions about computation time and extension to more complex problems. 
There were mixed opinions among reviewers about the overall contribution.
Decision: Accept (poster)