Biologically Plausible Learning via Bidirectional Spike-Based Distillation
Authors: Yifei Wang
Zhangyanxun
Changze Lv
Yiyang Lu
Jingwen Xu
Xiaohua Wang
Di Yu
Xin Du
Xuanjing Huang
Xiaoqing Zheng
Venue: ICLR 2026 Poster (2026)
Keywords: spiking neural networks
learning algorithms
Primary area: applications to neuroscience & cognitive science
TL;DR: We introduce Bidirectional Spike-based Distillation (BSD), a biologically plausible learning method that trains feedforward and backward spiking neural networks.

Abstract: Developing biologically plausible learning algorithms that can achieve performance comparable to error backpropagation remains a longstanding challenge.
Existing approaches often compromise biological plausibility by entirely avoiding the use of spikes for error propagation or relying on both positive and negative learning signals, while the question of how spikes can represent negative values remains unresolved. 
To address these limitations, we introduce Bidirectional Spike-based Distillation (BSD), a novel learning algorithm that jointly trains a feedforward and a backward spiking network.
We formulate learning as a transformation between two spiking representations (i.e., stimulus encoding and concept encoding) so that the feedforward network implements perception and decision-making by mapping stimuli to actions, while the backward network supports memory recall by reconstructing stimuli from concept representations.
Extensive experiments on diverse benchmarks, including image recognition, image generation, and sequential regression, show that BSD achieves performance comparable to networks trained with classical error backpropagation. 
These findings represent a significant step toward biologically grounded, spike-driven learning in neural networks.

Published as a conference paper at ICLR 2026
BIOLOGICALLY PLAUSIBLE LEARNING VIA BIDIRECTIONAL SPIKE-BASED DISTILLATION
Yifei Wang 1.2*
Yanxun Zhang Changze Ly Yiyang Lu Jingwen Xu
Xiaohua Wang1.2 1.2
Di Yu Xin Du Xuanjing Huang Xiaoqing Zheng
College of Computer Science and Artificial Intelligence, Fudan University
2 Shanghai Key Laboratory of Intelligent Information Processing
3 School of Software Technology, Zhejiang University
wangyif22, (atpyyyi22yyaauuaaaaaa yanxunzhang22, czlv24] @m. fudan. edu. 2 thengxq, xihuang e fudan. edu. en
ABSTRACT
Developing biologically plausible learning algorithms that can achieve performance comparable to error backpropagation remains longstanding challenge. Existing approaches often compromise biological plausibility by entirely avoiding the use of spikes for error propagation relying both positive and negative learning signals, while the question of how spikes can represent negative values remains unresolved. To address these limitations, introduce Bidirectional Spike-based Distillation (BSD), a novel learning algorithm that jointly trains a feedforward and backward spiking network. We formulate learning as transformation between two spiking representations (i.e., stimulus encoding and concept encod- ing) that the feedforward network implements perception and decision-making by mapping stimuli to actions, while the backward network supports memory recall by reconstructing stimuli from concept representations. Extensive exper- iments on diverse benchmarks, including image recognition, image generation, and sequential regression, show that BSD achieves performance comparable networks trained with classical error backpropagation. These findings represent a significant step toward biologically grounded, spike-driven learning in neu- ral networks. Our code is available at hhtttp:://githhee com/ alden199/ Bidizectional-Sp.e Soiioonnnl--iieee htt.--ppik--assdddssiiiaaat--
INTRODUCTION
Human learning and cognition cannot be reduced to a simple unidirectional peareeption-to-decisie pipeline. Rather, they emerge from bidirectional processes that integrate bottom-up sensory perception
with top-down memory recall (Caucheteux et al. 2023 Bonetti et al.) 2024).. During visual perception, retinal signals are transformed through hicrarchical neural pathways into high-level conceptual representations stored in memory. Conversely, during recall, the brain can reconstruct partial sensory features from these stored representations.
Kosslyn et al. (1993) showed with positron emission tomography that the same early visual cortical areas (i.e., VI and V2) activated during perception are also engaged when subjects visualize objects with their eyes closed. Later studies demonstrated that stimulus identity can be decoded from early visual cortex activity during both working memory and mental imagery, with patterns resembling those elicited by actual stimulation (Albers et 2013). For instance, consider a learner distinguishing between different bird species. At the outset, early visual areas may encode only basic features such edges color patches, making two similar species appear nearly indistinguishable from each other. As categorical knowledge the species is acquired, higher-level conceptual representations emerge and feed back early visual regions. This feedback sharpens perceptual sensitivity to subtle diagnostic features, such as beak curvature or wing pattern, thereby refining low-level visual representations in accordance with learned category distinctions.
"Equal Contribution. Correspondence Xianqing Zheng (zhengxq@ @ fudan.cd. (udan.edu.cn).

---

Published as a conference paper at ICLR 2026
Inspired by the brain's bidirectional architecture of perception and recall, we introduce bidirectional spike-based distillation (BSD), a novel learning algorithm that frames learning as a transformation between two spiking representations: stimulus encoding and concept encoding. The feedforward pathway performs perception and decision-making by mapping sensory stimuli to conceptual repre- sentations, analogous to the brain's analytical mode, while the feedback pathway facilitates memory recall by reconstructing stimuli from semantic concept encodings, analogous to the brain's 'sii'' imagi- native mode. These feedforward and feedback networks can be trained jointly by distilling feature representations from one another. By integrating perception and recall within unified framework, BSD provides a biologically grounded alternative to conventional unidirectional learning paradigms.
We also show that the proposed bidirectional distillation (implemented via spike trains) yields more biologically plausible learning algorithm. While backpropagation has achieved remarkable success in deep learning (LeCun et al. 2015) Rumelhart et al. 1986), its underlying mechanisms conflict with established neurobiological principles (Crick) 1989 Lillicrap et al., 2020). Key inconsistencies include the requirement for symmetric feedforward and feedback weights, reliance global error signals instead of local synaptic plasticity, a two-stage learning process that clearly separates forward and backward passes, and the use of continuous activations rather than discrete -assed commu- nication. To address these limitations, and building on the three criteria proposed by Ly et al. (2025), we introduce two additional requirements: neurons should communicate using discrete binary spikes for both learning and inference, and learning should rely on unsigned spiking signals only (Hayden) ct al., 2011). The learning algorithm we present demonstrates that all five criteria can be satisfied, whereas existing approaches typically fall short on one or more of these criteria.
Through extensive experiments across range of tasks, including image classification, text character prediction, time-series forecasting, and image generation, and using diverse network architectures such as multi-layer perceptrons, convolutional neural networks, recurrent neural networks, and autoencoders, we demonstrate that BSD achieves performance comparable to backpropagation while satisfying all five criteria for biological plausibility. Our results indicate that more adherence to biological fidelity does not necessarily compromise computational effectiveness.
The contributions of this study can be summarized as follows:
Inspired by the brain's 's bidirectional architecture of perception and recall, we propose novel learn- ing framework in which the feedforward network (stimul-to-decisioon) and the backward network
(concept-to-stimul) are jointly trained by mutually distilling spiking feature representations.
We introduce two additional biological plausibility criteria to complement the three proposed by Lv et al. (2025), resulting in five key principles: asymmetric forward and backward weights, local error representation, non-two-stage learning, spiking neuron models, and unsigned error signals. The BSD algorithm satisfies all five criteria and demonstrates enhanced biological plausibility.
We perform extensive experiments using the BSD algorithm across diverse network architectures and various tasks. The experimental results show that BSD achieves performance comparable to error backpropagation while maintaining greater adherence to biological fidelity.
RELATED WORK
Backpropagation (BP) (Rumelhart et al.) 1986) has long been criticized for its limited biological
plausibility, as it depends on weight symmetry (ttrk 1989). global error signals (Crick) 1989). and
a strictly sequential forward-backward computation process Guerguiev et al. 2017 Hinton 2022)
To address these limitations, numerous alternative approaches have been proposed to improve biologi- cal plausibility (Schmidgall et al. 2024 Jiao et al. [0022] Li et al. (024). Recent advances in spiking
neural networks have introduced attention mechanisms Yao et al. 2023b a) and residual learning capacitt while maintaining spike-based communication. To eliminate the reliance on global error signals, local loss methods (Mostafa et al., 2018) and their variants (Belilovsky et al. 2019; Nokland & Eidnes 2019 Kaiser et al. (020) have been introduced, which typically employ fixed or trainable auxiliary heads to align hidden layers directly with the target. Feedback alignment (Lillicrap et al. 2016) addresses the weight transport problem by replacing the backward weights with fixed, randomly initialized matrices, thereby breaking the symmetry between forward and backward weights. Target propagation (TP) (Bengio) 2014) introduces approximate inverse models to generate layer-wise targets, with weight updates obtained by minimizing the mis-
(Hu et al. 2024) to enhance representational capacity
2

---

Published as a conference paper at ICLR 2026
match between outputs and targets. Although TP employs local losses and avoids weight symmetry, requires each layer to transmit two distinct types of signals at different times. Moreover, TP often suffers from convergence instability. Extensions such Difference Target Propagation (DTP) (Le et al. 2015) and SDTP (Bartunov et al. 2018) improved stability and performance but offered little progress towards greater biological plausibility, and chologicalll-plausiii
Reward Propagation
(BRP) (Zhang et al., 2021), which replaces floating-point interlayer signals in TP with spike- based signals, has not demonstrated reliable generalization across tasks. Predictive coding (Rao & Ballard 1999) offers another influential framework, postulating that the brain continually generates top-down predictions to minimize sensory prediction errors. Motivated by this framework, Error-driven Local Representation Alignment (LRA-E) (Ororbia & Mali, 2019) introduces a mechanism where error signals are projected backward to generate local target representations for hidden layers. The network then learns by minimizing the local discrepancy between actual neuronal activities and these gener- ated targets, thereby cnabling training without global error backpropagation. Other methods, such as Decoupled Neural Interfaces (DNI) (Jaderberg et al. (017). train auxiliary head at each hidden layer to approximate layer-wise gradients yet fail to break weight symmetry. Alternatively, Dendritic Localized Learning (DLL) (Lv al., 2025) employs local error signals for weight updates, yet still relies on transmitting signed floating-point values and generally performs poorly across benchmarks. Counter-Current Learning (CCL) (Kao Hariharan 2024) faces the same limitation of footinn--oiin communication and further lacks demonstrated effectiveness in sequential regression tasks.
Other biologically inspired mechanisms, like Hebbian learning (Donald 1949) and spike-timing: dependent plasticity (STDP) (Song et al. (000). adjust synaptic strength according to correlations in neuronal activity, while burst-dependent synaptic Aasstiitt((ayyeuu al.
202T regulates synaptic plasticity by high-frequee geghffreueecy bursts of spikes. Although fully consistent with biological observations, these rules struggle integrate supervised learning signals, and STDP additionally requires precise temporal resolution. Energy-based learning approaches (LeCun et al. 2006). e.g.. Boltzmann ma- chines (Ackley et al. 1985). Hopfield networks (Hopfield 1984). and contrastive learning frameworks (Hinton (2002), instead optimize energy function. However, minimizing energy does not always correspond to reducing task-specific loss, limiting their utility for general supervised learning. E-prop (Bellec al. 2020) approximates BPTT for SRNNs using eligibility traces; while supporting local, online updates, it remains limited by the use of signed error signals.
In contrast, our approach, which is inspired by the brain's bidirectional interplay between perception and recall, exhibits improved biological plausibility while achieving stable convergence and superior performance on various benchmarks and tasks.
3 PRELIMINARY
3.1 SPIKING NEURONS
We adopt the leaky integrate-and-fire (LIF) neuron model (Maass 1997) in our spiking neural networks (SNNs). The dynamics of the LIF neuron are formulated in discrete time as follows:
Ult H [t] S(t) Ste))
(1)
#[4 = U[L - +1][---]++---] - - - (U] -Ureset)::
(2)
S(t] = O - Unr).
where [[t] represents the input current at time step t. Here, H[] and Utt denote the membrane
potential before and after the trigger of spike Sitt), respectively. The parameter is the membrane time constant, while Unr and Urest specify the firing threshold and reset potential. A spike is generated when the pre-spike potential H [t] exceeds the threshold Unz in which case the neuron fires (S[t) and the membrane potential is reset to Ureset- Additional preliminaries are given in the appendix B
3.2 BIOLOGICAL PLAUSIBILITY CRITERIA
In this work, we mainly build upon three biological plausibility criteria established by Lv et al. (2025): C1. Asymmetric synaptic weights for feedforward and feedback pathways; C2. Local synaptic plasticity based solely locally available information without global error signals; C3. Non-dual-
3

---

Published as a conference paper at ICLR 2026
Layer
Layer
Aplenciiee
Target Spake
3
Ostput Spake
150
ANHP
Inpos
3
Poeslaank poststic wntight.
bettowe sptke
tip daws..pike.
beast
Wey
PO007
30000
Figure 1: (a) Overview of the bidirectional spike-based distillation framework. (b) Illustration of feature alignment using the properties of pyramidal neurons, which receive feedforward and feedback signals through their basal and apical dendrites, respectively. (c) Performance comparison of neural networks trained with BSD, dendritic localized learning (DLL), recently proposed biologically plausible learning algorithm, and backpropagation. The experimental results show that BSD achieves performance comparable to backpropagation.
phase training that eliminates sequential forward-back backward dependencies. We will also introduce two additional criteria in Section further extending the framework of biologically plausible learning.
METHOD
4.1 DESIGN PRINCIPLES
Inspired by the brain's bidirectional processes of perception and recall, we propose Bidirectional Spike-Basee Based Distillation (BSD), which frames learning as a transformation between stimulus encoding and concept encoding through spiking representations. The feedforward pathway maps sensory inputs to concepts for perception and decision-making while the backward pathway reconstructs inputs from concepts, thereby supporting feedforward learning during training.
Before introducing BSD in detail, we extend the three biological plausibility criteria Lv et al. (2025) with two additional principles. Designed to satisfy all five criteria, BSD ensures biologically grounded foundation for learning. The two additional criteria we propose are:
C4. Model of Neurons. Neurons in conventional artificial neural networks produce continuous activations, which are typically interpreted as approximations of average firing rates (Maass 1997). This stands in contrast to the brain's 's actual processing, where biological neurons communicate using discrete action potentials (0-1 spikes) (Gerstner et al. 2014).
C5. Unsigned error signaling. The learning mechanism in most artificial neural networks re- lies signed error signals to propagate directional gradient information for weight adjustments (Rumelhart et al., 1986). This stands contrast with neurophysiological findings, which indicate that neurons encode unsigned reward prediction errors, firing in response surprising outcomes irrespective their positive or negative valence (Hayden et al. (0011)
To facilitate the understanding of the BSD algorithm, we illustrate both the network architecture and the neuronal learning mechanism in Figure [I, using an L-layer MLP configuration. The complete form of the global algorithm is detailed in Algorithm
4.2 MODEL ARCHITECTURE
Neuronal Dynamics. To highlight the biological foundations of BSD, we first describe the neuron model employed in BSD. We follow Sacramento al. 2018) and utilize pyramidal neurons with
4

---

Published as a conference paper at ICLR 2026
citee-compartmentt structure: soma, apical dendrites (carrying backward learning signals), and basal dendrites (receiving feedforward inputs). To satisfy criterion C4 (model of neurons), we use spiking neurons that emit discrete pulses instead of continuous activation values. For each neuron, the membrane potentials arriving at the soma from the basal and apical compartments are denoted e and é, respectively. The apical potential é acts as supervisory signal guiding synaptic plasticity on the basal dendrites, while the basal potential v drives spike generation through = SN(v),, where (N(()) denotes the spiking operation that integrates input voltage and produces spike outputs. In the following sections, we use bold lowercase letters (e.g.. Vi, v,) to denote layer-wise vectors, which are formed by aggregating the single- neuron scalar potentials (u and 0, respectively) from all neurons in layer
Network Architecture. The network comprises L layers of pyramidal neurons, where each layer contains an equal number of two distinct types of neurons. Type 1 neurons receive synaptic inputs from lower-layer neurons and constitute the feedforward pathway, implementing a stimuli-to-decisin process that transforms sensory inputs into conceptual representations. Conversely, Type 2 neurons receive inputs from higher-layer neurons and form the backward pathway, which attempts to recon- struct sensory features from conceptual encodings to assist feedforward learning. During training, the original input x is delivered to bottom-layer Type neurons, while the learning target is first encoded into spike train å and provided to top-layer Type 2 neurons. The two pathways are jointly optimized through mutual distillation of their spiking feature representations, enabling bidirectional information flow that mirrors the brain's perception-recal architecture. To satisfy criterion C1 (asymmetric synaptic weights), we employ independent synaptic weight matrices W and Q for the feedforward and backward pathways, respectively. The feedforward path is described by:
V1 =x;
v, V. W,-18 8, = SV,(v,), i= 2,3, T.,
where v, denotes the somatic membrane potential of Type neurons in layer i from basal dendritic integration, and v denotes the somatic membrane potential of Type 2 neurons in layer arising from apical dendritic integration. W; is the synaptic weight for Type I neurons in layer i, and s; denotes the spike train that Type 1 neurons in layer output.
The backward pathway is described by:
v/2 = $; v/= O s(= i=
(5)
where v denotes the somatic membrane potential of Type 2 neurons in layer arising from basal dendritic integration, v; denotes the somatic membrane potential of Type 1 neurons in layer arising from apical dendritic integration. O the synaptic weight for Type 2 neurons in layer i, and s denotes the spike train that Type 2 neurons in layer output.
4.3 TRAINING PROCEDURE
Motivated by the neuronal least-action principle (Senn et al., 2024), which postulates that pyramidal neurons minimize somato-dendritic mismatch errors through voltage dynamics, and to satisfy criterion C2 (local error computation), our BSD algorithm introduces local loss functions for individual neurons to align basal-received voltage u with apical-received voltage é. Type I and Type 2 neurons respectively receive bottom-up sensory imputs x and top-down target signals 6, which in classification tasks correspond to distinct modalities. Thus, the aligmment between e and è can be viewed as aligning cross-modal embeddings. Inspired by contrastive learning in multimodal representation learning and satisfy C5 (Unsigned Error Signal), we employ the Relaxed Contrastive (ReCo) loss (Lin et al. 2023). which is an unsigned loss function. In our design, eтror computation is localized within each neuron, and no error signals are propagated across the network. Specifically, for Type I neurons in a given layer i (for i = 2,... L..,L- 1), let Vi.k and Vi,, denote the basal and apical membrane voltage for the k-th sample in a batch, respectively. By stacking these vectors across the batch dimension B, we construct matrices V, E BxD, and V, E BRD,, where D. is the number of neurons in layer i. The layer-specific affinity matrix C, E BxB B is then defined by its elements:
Vi Vi.j
[C,]kj
|[ve
The local loss for Type neurons in layer (for I,..., L 1), denoted C,, is defined as:
L. k=1
(1 [Cslax)2
+
Σ (max(0, (C:)e)))²
5

---

Published as a conference paper at ICLR 2026
where À is hyperparameter that controls the penalty strength for suppressing spurious correlations between non-corresponding voltage pairs. The local loss for Type neurons, Li is defined analo- gously, with its formulation provided in Appendix D Compared with InfoNCE loss commonly used in contrastive learning scenarios, ReCo loss avoids penalizing negatively correlated voltage pairs between V, and V,, thereby introducing enhanced flexibility and representational richness to learned embeddings while preserving alignment objectives. We provide more detailed justification for adopting the ReCo loss in Appendix Q Considering criterion C2 (local synaptic plasticity without global error signals), we employ detach () operations truncate the computational graph between layers, ensuring that each neuron's is loss only propagates learning signals to synaptic weights con- nected to its dendrites. As a result, the gradient of the local loss L. with respect to the feedforward is given by:
weights W i-1 (for = 2,
OL:
Σ
2(1 kC,)kt)
1
Vi.
[CilkA
Vi.k |
OW, 1-1
V
[[Ve
(8)
Σ 2\ (aa((0,( jzk
1
Vi.a [|Va
[Ci]ks
|| ) (s 1s)",
Vi,k
where B denotes the batch size, 11-1, represents the spike output vector of Type 1 neurons from layer - 1 for the k-th sample, and L, denotes the local loss for neurons in layer i. This locally computed gradient is then used to update the feedforward synaptic weights via a standard gradient descent step for i 2, 2....................... L 1:
Oc,
NEW
Wold 1-1
TW OW,-1
(9)
where the superscripts 'new' and 'old' denote the weights after and before the update, and 7/w the learning rate for the feedforward weights. A symmetric update rule, also based on its corresponding local gradient, is applied to the backward weights @ The detailed gradient derivations for both W and ( are presented in Appendix D The total loss for Type I neurons is defined as:
Listal Σ Li + Lup. F=1
(10)
where the top-layer loss is defined as Luop = ER= entropy loss. The total loss for Type 2 neurons, L' totals is defined symmetrically. As all information processing can be separated within single neuronal compartment, the feedforward and backward distillation processes do not require strict temporal separation and can learn simultaneously, thereby satisfying C3 (non-two-stage training). For classification tasks, during inference, the predicted label for input x is determined by computing cosine similarity between the ensemble output spikes sy and the spike trains corresponding all candidate labels. The detailed training and inference procedures for RNN architectures are presented in Appendix C
E==g CCE(VV.L VL,K), where Lce represents the cross-
4.4 LEARNING FOR GENERATION TASKS
We employ autoencoder architecture for generation tasks, where both bottom-up and top-down inputs are images x. To capture fine-grained edge details while reducing noise artifacts, we apply Fast Fourier Transform (FFT) decomposition to the input images as well as to all basal and apical membrane voltages. This rrequency-doman representation separates low- and high-frequency components, enabling adaptive loss computation. Within each layer, the regularization parameter is adjusted according to frequency content: larger values are assigned to sigh-freqeency components to suppress spurious correlations and preserve edge fidelity, whereas smaller values are used for low-frequency components to avoid noise amplification and maintain structural coherence. At the top layer, mean squared error employed to compute the loss berween basal and apical voltages. The detailed mechanism of the FFT decomposition is explained in Appendix M
5 EXPERIMENTS
In this section, we first present the experimental settings and implementation details. We then evaluate our proposed BSD algorithm image classification tasks, comparing it against other biologically

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of various learning algorithms in terms of biological plausibility criteria and image classification performance. Our proposed BSD algorithm satisfies all five criteria of biological plausibility (C1- C5) while achieving performance comparable to backpropagation, "CI, C2, C3, C4, C5 refer to the criteria defined in Section 4.1
Results are averaged over four random seeds.
Method
C2
CS
Model
MINST
IIshiomMNIST
SVHN
CIFAR-10
CIFARIOO
Backgropupation
MLPs
16..77%
89. 20..59%
61. a4.44
57. 67.65%
27 97.92%% 20.7%%
67. 67.125
on ANNI
CNNS
90.500 50.566 158
912 48% an13
944
17.12% 2008
67. 67.76% avR
88. 2915
Backgrogugation
MLPS
08.57% 90%
58 577
11.25%% 80%
00.0375 s0r6
25.52% 47%
64.40%
K
x
SNNs
CNNs
57.02%
67.21%%
MLPS
98.42%
88. 72%
59.05%
47.34 17.3475 a.
19.72%
62.655
Prodictive Coding
CNNS
19.41%% ors
32. 005%
94.500% as
72.944%
58.08%% uo..0.%.ooooo axs
82, 411%
CCL
1
MLPS
M8.155% 105
88. 5855 carg
600 60.955 en cars
02.7
21. 71.76%% c5 2ors
64.44%
CNNS
96,37%% nos
55. 7V%
88.
52,94%
u6.2%%%
MLPS
97.57%
N7 54/5
56.80%
46,87%
18.24%
61.16%
DLL
CNNS
98.87% are
900 55.5855
85.81
70.8PP en GArA
38.60% avs
77.01%
R-STDP
v
MLPSs
77. 77.188 1e%
70 (055 nert
41. 11.765% oor
22, 62.669% anrk
1.35% 1936
42.68%
CNNs
01.67%aa udds
74 14.295
20.0299 sorls
10% 28r4
1.40%
50.10%
BSD (Ours)
1
MLPS
05 62%
86 39.5%% es 10%
60 40% an 1o%k
48 90% wit
22 10 2014
62. 68%
CNNS
nors
13%
7856
plausible learning algorithms. We assess BSD-trained RNNs sequential regression tasks and evaluate BSD-trained autoencoders on image generation tasks. Finally, we conduct ablation studies and analyze the network's convergence properties and performance.
5.1 EXPERIMENTAL SETTINGS
Image Classification. We evaluate BSD algorithm widely used benchmarks including MNIST, FashionMNISTT, SVHN, CIFAR-10, 10, and CIFAR-100, using classification accuracy as the metric.
Text Character Prediction. We conduct next-character prediction experiments with BSD-trained RNNs on the Harry Potter text corpus (Plath et al. 2019). reporting prediction accuracy.
Time-Series Forecasting. BSD-trained RNNs evaluated on three widely used world multi- variate time-series forecasting datasets: Electricity (Lai et al. 2018), Metr-la (Li et al. 2018) 8). and Pems-bay (L et al. 2018), with performance primarily measured by mean squared error (MSE).
Image Generation. BSD-trained autoencoders are applied to generation tasks on MNIST, Fash- ionMNIST, and CIFAR-10, with generation quality assessed using Fréchet Inception Distance (FID) (Heusel et al. 2017). Additional results and visualizations are provided in Appendix
5.2 IMPLEMENTATION DETAILS
To ensure fair comparison, identical network architectures are used across all learning algorithms for each task. For MLPs, CNNs, RNNs, and autoencoders, the same configurations are applied when comparing methods on a given dataset. Detailed dataset descriptions, architectural specifications, hyperparameter settings, and evaluation metrics are provided in Appendix E A comprehensive analysis of the computational costs, including training and inference memory consumption, is provided in Appendix
5.3 IMAGE CLASSIFICATION
We benchmark the performance of backpropagation on Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs), Predictive Coding, Reward-modulated Biikke-Tiiing-Deeedeen Plasticity (R-STDP) (Izhikevich) 2007), Counter-Current Learning (CCL., Dendritic Localized Learning (DLL), and our BSD algorithm image classification tasks. Comprchensive results are presented in Table I with additional implementation details provided in Appendix E.2
Spiking neuron outputs degrade performance compared to continuous activations. When trained with backpropagation, Spiking Neural Networks (SNNs) consistently exhibit inferior performance compared to Artificial Neural Networks (ANNs) across all evaluated datasets and network architec- tures. This performance degradation highlights the disadvantage of using spiking neurons, which emit binary 0-1 spikes, opposed neurons that produce continuous-vall valued activations.
7

---

Published as a conference paper at ICLR 2026
Table
Comparison of different learning algorithms for RNN training on text character prediction and time-series forecasting tasks. Our proposed BSD algorithm achieves performance comparable backpropagation. t (4) denotes higher (lower) values indicate better performance. All results are averaged across 4 random seeds. The best results and the results of BSD are presented in bold.
Harry Potter
Electricity
Metr-la
Pems-bay
Method
Pred. Ace.
MSE 4
MAE
MSE 4
MAB
MSE 4
MAE
Backpropagation ANNs
51 9%
0.175a0.00r
0.324a0.07
0.131
0.21440.006
0.164=0.001
=.190=n.002
Backpropagation SNNs
27. 37.8%
0.16920.018
0.316s0.021
0.15420.014
0.243ao.02
0.166120.00
Predictive Coding
38. 8..81.5
0.162
0 0.312
0.141
2.228
0.178aa.0M
0.2022c.0m
DLL
33. 1% ens
0. 1.22200.01
0.32 20.01a
0. 15.165 oo0
0. 264 oo1
0.
17.17.00000 000
0.22440 001
BSD (Ours)
88 196
0. 1.1651
0.31410oss
0. c0
0.197 10
0.174 0o7
0.206.0.013
Our proposed BSD algorithm reconciles biological plausibility with competitive performance. BSD satisfies all five criteria of biological plausibility (C1-C5) while achieving performance compara- ble to backpropagation, with stable convergence across diverse datasets. In particular, on challenging datasets such as SVHN, CIFAR-10, and CIFAR-100, and on more complex architectures including CNNs, BSD consistently delivers robust performance. Taken together, these findings underscore the feasibility of attaining biological plausibility without sacrificing task performance. To further assess the scalability of our approach, we conducted experiments the more complex Tiny- Tiny--mageNet dataset, with the results presented in Appendix We also evaluate the robustness of our models against imput noise Appendix L
5.4 SEQUENTIAL REGRESSION TASKS
We evaluate BSD four representative sequential regression tasks, comparing its performance against backpropagation on ANNs and SNNs, Predictive Coding, and Dendritic Localized Learning (DLL). Comprehensive results reported in Table
For time-series forecasting, we utilize the Electricity, Metr-la, and Pems-bay datasets. Predictive Coding satisfies only the second biological plausibility criterion, whereas DLL meets the first three. In contrast, BSD satisfies all five criteria while achieving performance comparable to backpropagation, with RNNs trained under BSD converging reliably across all forecasting tasks.
For text character prediction, experiments are conducted on the Harry Potter corpus. As shown in Table 2 BSD again converges successfully, outperforming both Predictive Coding and DLL.
Together, these results demonstrate that BSD effectively captures temporal dependencies sequential regression tasks while remaining consistent with biologically plausible learning principles.
Figure 2: Results on image generation tasks using autoencoders trained with backpropagation and BSD. GT denotes the ground truth, BP indicates backpropagation, and BSD refers to our proposed method. The reconstruction quality demonstrates that BSD attains performance comparable to backpropagation on generation tasks, while preserving biological plausibility
5.5 IMAGE GENERATION
Figure 2 presents comparison of image reconstruction on CIFAR-10 10 across three approaches: BP-trained autoencoders, the Fully Spiking Variational Autoencoder (FSVAE) (Kamata et al. 2022). and BSD-trained autoencoders. Generation quality is evaluated using the Fréchet Inception Distance (FID) (Heusel al. 2017), a widely adopted metric for assessing the quality of generative models.
8

---

Published as a conference paper at ICLR 2026
Leamning Curves for Different A Values
effect of Firing Threshold
Vidirecional spike Aligmment
0.5
2.50
0.41
3.0
0.7
0.8%
60
0.6
3.00
Layer
0,888
0.811
0.888
8.77
0.
0.4
0.809
0.313
8
3.70
70
570 3.5
030 2.25 EM0
6.95
(a)
(b)
Figure 3: Analysis of BSD algorithm parameters and bidirectional spike dynamics. (a) Test accuracy curves of CNNs trained with BSD using different À values. (b) Impact of firing threshold on network performance. (c) Alignment of spikes emitted by Type I and Type 2 neurons within the same layer.
The visual results suggest that BSD adapts ro- bustly to generative tasks and produces reconstruc- tion quality closely comparable to that achieved with backpropagation. Table reports FID scores on CIFAR-10 and MNIST, indicating that BSD at- tains competitive performance across both datasets and thereby further demonstrating its suitability for generative modeling while maintaining bio- logical plausibility. Additional visualizations and results, including experiments on FNshionMNIST, are presented in Appendix F.1
Table 3: Performance comparison of different algorithms on image generation tasks, & denotes lower values indicate better performance.
Dataset
Model
ANN-BP
127 34
CIFAR-10
ESVAE
175.5
BSD (Ours)
168.12
ANN BP
49.56
MNIST
ESVAE
97.06
BSD (Ours)
72.39
5.6 ABLATION STUDY
inspired by contrastive learning, we adopt ReCo loss our layer-wise loss function. Here, we delve deeper into the impact of layer-wise loss function selection, analyzing how BSD performs when using MSE or InfoNCE (van den Oord al. 2018), a loss commonly used in contrastive learning scenarios, for aligning intraneuronal voltages. We term the method using MSE as layer-wise loss "BSD-MSE" and the method using InfoNCE, as "BSD-InfoNCE," and conduct. ablation experiments on both image classification and sequential regression tasks.
As mentioned in Section 4
Table compares the performance of BSD, BSD- MSE, and BSD-InfoNCE on image classification tasks. BSD-MSE yields markedly inferior results: MLPs attain only 19.11% accuracy on SVHN and fail to converge on other datasets, while CNNs exhibit substantial performance gaps across all tasks and do not converge on CIFAR- 100. BSD- InfoNCE converges reliably on all datasets, but BSD with ReCo loss consistently achieves higher accuracy. On complex benchmarks such CIFAR- 100, BSD surpasses BSD-IIfoNCE, by more than 15 accuracy points. Additional ablation experiments for text character prediction, time-series forecasting, and image generation tasks are provided in Appendix
Table 4: Ablation study of loss functions on image classification tasks.
Model
MNIST
MNsH
SVHN
CIFAR-10
CIFAR-100
ML.S
12.51%
13.72%
19.11%
11.49%
1.39%
BSD-MSE
CNNs
21.10%
29.31%
19.46%
16.00%
1.58%
ML.Ps
94.56%
85.71%
57.33%
43.77%
19.25%
BSD-IIGNNCE
98.77%
88.97%
83.27%
72.38%
38.06%
MLPs
95 62%
86.
86.39%
60.40%
18. 90.990%
22.10%
BSD
CNNs
41%
91.05%
90.81%
81.13%
53.48%
To conclude, employing MSE loss for intraneuronal voltage alignment leads to convergence dif- ficulties, indicating that it is ill-suited for BSD training. Moreover, compared to InfoNCE, ReCo loss offers a distinct advantage by not penalizing unpaired samples that are already orthogonal or negatively correlated, which allows for more flexible alignment and richer feature representations. We also investigate the impact of the number of timesteps on model performance in Appendix O An ablation study demonstrating the importance of Batch Normalization is presented in Appendix P

---

Published as a conference paper at ICLR 2026
5.7 TRAINING ANALYSIS
In this section, we analyze the convergence behavior of BSD-trained models and the properties of Type and Type 2 neurons during training. Since BSD applies ReCo loss to all layers except the top one and relies on spikes for inter-neuronal communication, both the penalty weight À in ReCo loss and the neuronal firing threshold are critical factors influencing its convergence. Additionally, we investigate the sensitivity of our framework to batch size in Appendix as contrastive learning methods often depend on sufficiently large batches. detailed analysis of the energy efficiency of our BSD-trained models during inference is provided in Appendix K highlighting the benefits of spike-based computation.
Figure 3 a) shows the effect of the penalty weight À on BSD-trained CNNs evaluated the SVHN dataset. Performance drops markedly when À == O, underscoring the role of À promoting separation among sample representations and thereby enlarging the representational space. We further observe that smaller values of accelerate convergence, while the best final performance is achieved at À 0.6. Figure bb) illustrates the relationship among neuronal firing threshold, spike sparsity, and network performance on SVHN. The network attains its best performance at a firing threshold of 0.6, suggesting that thresholds that are either too low or too high impair effective learning. Figure ( shows the Hamming similarity between the spike trains of Type and Type 2 neurons within the same layer. The similarity rises quickly during training and exceeds 0.85 by epoch 20, demonstrating that the feedforward achieve mutual alignment of their spiking feature representations through bidirectional distillation. therehy validating the effectiveness of our joint training framework.
and backward (concept-ter to-pt--o-stiie stimmilli pathways successfully
For completeness, we provide t-SNE visualizations of representations on BSD-trained CNNs in Appendix H and examine the degree of alignment between weights W and O in Appendix G
6 CONCLUSION
Human learning and cognition emerge from bidirectional processes that integrate bottom-up sensory perception with top-down memory recall. In this framework, the feedforward network supports per- ception and decision- pociiion-makiigge by transforming sensory stimuli into conceptual representations, while the feedback network facilitates memory recall by reconstructing stimuli from semantic concepts. Inspired by this principle, we propose a novel bidirectional learning framework in which the feedfor- ward and feedback networks are jointly trained by distilling their hidden feature representations from one another. This approach leverages the properties of pyramidal neurons, which receive feedforward (perception) and feedback (learning) signals through their basal and apical dendrites, respectively. Extensive experiments across diverse network architectures and tasks show that the resulting learning algorithm achieves performance comparable to error backpropagation while yielding stronger adher- ence to biological fidelity. Specifically, BSD achieves competitive performance across tasks: within 3% of backpropagation on MNIST (99.44% vs 99.56%) and CIFAR-10 10 (84.13% vs 87.12%% and superior MSE on time-series forecasting tasks like Electricity 165 0. 175), demonstrating the effectiveness of our approach. The limitations and future directions are discussed in Appendix R
ETHICS STATEMENT
This research presents a biologically plausible learning algorithm and docs not involve human subjects sensitive data. All experiments were conducted publicly available datasets that widely used in machine learning research. The proposed BSD algorithm is general-purpose learning method without inherent bias, unethical practices, discriminatory applications. The authors declare conflicts of interest or competing financial interests related to this work. The research methodology adheres well-established standard practices in machine learning and computational science, with no foresecable harmful implications or misuse potential.
REPRODUCIBILITY STATEMENT
The authors have made extensive efforts to ensure the reproducibility the empirical results reported in this paper. First, detailed dataset characteristics documented in Appendix E.T Second,
10

---

Published as a conference paper at ICLR 2026
comprehensive implementation details, covering network architectures, hyperparameter settings, training procedures, and evaluation metric explanations, are provided in Appendix E.2
ACKNOWLEDGMENTS
The authors would like to thank the anonymous reviewers for their valuable comments. This work was supported by the National Natural Science Foundation of China (No. 62076068).
REFERENCES
David H. Ackley, Geoffrey E. Hinton, and Terrence Sejnowski. A Learning Algorithm for Boltzmann Machines. Cognitive Science, 9:147-169, 1985. doi: 10.1207615116700eo0999....
Alexandra Albers, Peter Kok, Ivan Toni, H Chris Dijkerman, and Floris de Lange. Shared representations for working memory and mental imagery in early visual cortex. Current Biology, 23(15):1427-1431, 2013.
Sergey Bartunov, Adam Santoro, Blake A Richards, Luke Marris, Geoffrey E Hinton, and Timo- thy Lillicrap. Assessing the scalability of hologically-motivvee deep learning algorithms and architectures. Advances in Neural Information Processing Systems, 31, 2018.
Eugene Belilovsky, Michael Eickenberg, and Edouard Oyallon. Greedy layerwise learning can scale to imagenet. In International conference on machine learning, pp. 583-593. PMLR, 2019.
Guillaume Bellec, Franz Scherr, Anand Subramoney, Elias Hajek, Darjan Salaj, Robert Legenstein, and Wolfgang Maass. A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 1(1):3625, 2020.
Yoshua Bengio. How auto-encoders could provide credit assignment in deep networks via target 2014.
propagation. arXiv preprint
Leonardo Bonetti, Gemma Fernández-Ruhie Francesca Carlomagno, Mathias Dietz, Dimitrios Pantazis, Peter Vuust, and Morten L Kringelbach. Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding. Nature Communications, 15(1):4313, 2024.
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King. Evidence of predictive coding hierarchy in the human brain listening to speech. Nature Human Behaviour, 7(3):430-44) 2023.
Francis Crick. The recent excitement about neural networks. Nature, 337(6203) 1989.
Ekin Dogus Cubuk, Barret Zoph, Jonathon Shlens, and Quoe V. Le. Randaugment: Practical automated data augmentation with a reduced search space. In Advances in Neural Information Processing Systems, volume 33, 2020.
Hebb Donald. The organization of behavior. New York 1952 Donald The Organization of Behuviour 1952, 1949.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier. Tiejun Huang, and Yonghong Tian. Incorporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEFF/CVF international conference on computer vision, pp. 2661-2671, 2021.
Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski. Neuronal dynamics: from single neurons to networks and models of cognition. Cambridge University Press, 2014.
Jordan Guerguiev, Timothy Lillicrap. and Blake A Richards. Towards deep learning with segregated dendrites. eLife, 6:e22566, 2017.
Benjamin Y Hayden, Sarah R Heilbronner, John M Pearson, and Michael L Platt. Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction emrors driving adjustment in behavior. Journal of Neuroscience, (1(1)):478-4118 2011.

---

Published as a conference paper at ICLR 2026
Avi Hazan and Elishai Ezra Tsur. Neuromorphic analog implementation of neural engineering -nnpewor--inspired spiking neuron for digennimensional representation. Frontiers in Neuroscience, 15:627221, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. In Advances in neural information processing systems, pp. 6626-6637, 2017.
Geoffrey Hinton. The forward-fowwart algorithm: Some preliminary investigations. arXiv preprint vrXi::2212.13335, 2(3):5, 2022.
Geoffrey E. Hinton. Training Products of Experts by Minimizing Contrastive Divergence. Neural Computation, 14(8):1771-1800, 2002. doi: 10. 00011620089766022611330001
John J Hopfield. Neurons with graded response have collective computational properties like those of two-state neurons. Proceedings of the national academy of sciences, 81 (1100:308883302) 1984.
Yifan Hu, Lei Deng, Yujie Wu, Man Yao, and Guoqi Li. Advancing spiking neural networks toward deep residual learning. IEEE Transactions on Neural Networks and Learning Systems, 36: 2353-2367, 2024. doi: (TW.0//TNNNLLS22233353555
Eugene M Izhikevich. Solving the distal reward problem through linkage of stdp and dopamine signaling. Cerebral Cortex, 17(10):2443-2455 2007.
Max Jaderberg, Wojeiech M Czarnecki, Simon Osindero, Oriol Vinyals, Alex Graves, David Silver, and Koray Kavukcuoglu. Decoupled neural interfaces using synthetic gradients. Proceedings of the 34th International Conference Machine Learning, 70:1627-1635, 2017.
Hou Jiao al. The new generation brain-inspired sparse learning: A comprehensive survey. Informa- tion Fusion, 89:35-57, 2022.
Jacques Kaiser, Hesham Mostafa, and Emre Neftei. Synaptic plasticity dynamics for deep continuous local learning (decolle). Frontiers Neuroxcience, 14:424, 2020.
Hiromichi Kamata, Yusuke Mukuta, and Tatsuya Harada. Fully spiking variational autoencoder. In Proceedings of the AAAI conference artificial intelligence, volume 36, pp. 7059-7067, 2022.
Chia-Hse thia-Hsiang Kao and Bharath Hariharan. Counter-current learning: A biologically plausible dual network approach for deep learning. Advances in Neural Information Processing Systems, 37: 70905-70925, 2024.
Stephen M Kosslyn, Nathaniel M Alpert, William L Thompson, Vera Maljkovic, Steven B Weise, Christopher F Chabris, Susan E Hamilton, Scott L Rauch, and Fabio S Buonanno. Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study, Proceedings of the National Academy of Sciences, 90(24): 11112112111111 1805, 1993.
Guokun Lai, Wei-Cheng Chang. Yiming Yang, and Hanxiao Liu. Modeling long-and short-term temporal patterns with deep neural networks. In The 41st international ACM SIGIR conference on research & development in information retrieval, pp. 104, 2018.
Yann LeCun, Sumit Chopra, Raia Hadsell, Mare' Aurelio Ranzato, and Fu Jie Huang. A Tutorial on Energy-Basse -Baed Learning. In Gökhan Bakir, Thomas Hofmann, Bernhard Schölkopf, Alexander Smola, and Ben Taskar (eds.), Predicting Structured Data. MIT Press, 2006. An extended tutorial introducing energy-based models and their relation to Hopfield networks and Boltzmann machines.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 52167553) 2015.
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio. Difference target propagation. In Joint european conference on machine learning and knowledge discovery in databases, pp. 498-515. Springer, 2015.
X Li et al. A review of learning in biologically plausible spiking neural networks. Frontiers in Neuroscience, 2024.
12

---

Published as a conference paper at ICLR 2026
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu. Diffusion convolutional recurrent neural network:
Data-driven traffic forecasting. In International conference on learning representations. 2018.
Timothy P Lillicrap, Daniel Cownden, Douglas B Tweed, and Colin J Akerman. Random synaptic feedback weights support error backpropagation for deep learning. Nature Communications, 7(1): 13276, 2016.
Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J AAermaa, and Geoffrey Hinton. Backprop- agation and the brain. Nature Reviews Neuroscience, 21(6):33--34, 2020.
Zudi Lin, Erhan Bas, Kunwar Yashraj Singh, Gurumurthy Swaminathan, and Rahul Bhotika. Relaxing contrastiveness in multimodal representation learning. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 2227-2236, 2023.
Changze Lv, Jingwen Xu, Yiyang Lu, Xiaohua Wang. Zhenghua Wang. Zhibo Xu, Di Yu, Xin Du, Xiaoqing Zheng, and Xuanjing Huang. Dendritic localized learning: Toward biologically plausible algorithm. In Forty-second International Conference on Machine Learning. 2025.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural Networks, 1099):1659-1671, 1997.
Hesham Mostafa, Vishwajith Ramesh, and Gert Cauwenberghs. Deep supervised learning using local errors. Frontiers in neuroscience, 12:608, 2018.
Arild Nøkland and Lars Hiller Eidnes. Training neural networks with local error signals. In International conference muchine learning, pp. 4839-4850. PMLR, 2019.
Alexander G Ororbia and Ankur Mali. Biologically motivated algorithms for propagating local target representations. In Proceedings of the aaai conference on artificial intelligence, volume 33, pp. 4651-4658, 2019.
Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud. Burst- dependent synaptic plasticity can coordinate learning in hierarchical circuits. Nature Neuroscience, 24:1010-1019, 2021.
Michael Pfeiffer and Thomas Pfcil. Deep learning with spiking neurons: Opportunities and challenges. Frontiers in neuroscience, 12:409662, 2018
James Plath, Gail Sinclair, and Kirk Curnutt. The 100 Greatest Literary Characters. Rowman & Littlefield, 2019.
Rajesh PN Rao and Dana H Ballard. Predictive coding in the visual cortex: functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79-87, 1999.
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by
back- bockkpropagating errors. Nature, 323(6088):533 33633333336111 1986.
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn. Dendritic cortical microcircuits approximate the backpropagation algorithm. Advances in Neural Information Processing Systems, 31, 2018.
Samuel Schmidgall, Rojin Ziaci, Jascha Achterberg, Louis Kirsch, S Hajiseyedrazi, and Jason Eshraghian. Brain-inspired learning in artificial neural networks: review. APL Machine Learning, 2(2), 2024.
Walter Senn. Dominik Dold, Akos F Kungl, Benjamin Ellenberger, Jakob Jordan, Yoshua Bengio, João Sacramento, and Mihai A Petrovici. A neuronal least-action principle for real-time learning in cortical circuits. eLife, 12:RP89674, 2024.
Sen Song, Kenneth D Miller, and Larry F Abbott. Competitive hebbian learning through spike-timing- dependent synaptic plasticity. Nature neuroscience, 3(9):919-926, 2000.
Stork. backpropagation biologically plausible. In International 1989 Joint Conference on Neural Networks, pp. 241-246. IEEE, 1989.
13

---

Published as a conference paper at ICLR 2026
Aaron den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXi::1807.03745 2018.
Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang. Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing. IEEE Transactions on Neural Networks and Learning Systems, 33( 1):6249-6262, 2021.
Man Yao, JiaKui Hu, Zhaokun Zhou, Li Yuan, Yonghong Tian, Bo XU, and Guoqi Li. Spike-driven transformer. In Thirty-seventh Conference on Neural Information Processing Systems, 2023a.
Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE Transactions Pattern Analysis and Machine Intelligence, 45(8):9333-944 2023b. doi: 10.1 109/TPAML.2023.224120000
Tielin Zhang, Shuncheng Jia, Xiang Cheng, and Bo Xu. Tuming convolutional spiking neural network with biologically plausible reward propagation. IEEE Transactions on Neural Networks and Learning Systems, 33(12)77621-761, 2021.
14

---

Published as a conference paper at ICLR 2026
A GLOBAL ALGORITHM OF BSD
Algorithm Spike Bidirectional Distillation Algorithm
1: Input: sequence of data D, layers L, learning rates 1/w. 7|e
2: Initialize i....., W L+1 and 1I..... L+1 randomly
3: for epoch = ..... max epochs do
4: for each batch x, D do
5:
Forward (rth(tranfforms the low-level signals into high-level representations): Assign the input values the voltage V1 of the input layer neurons. Inputs can be viewed as analog signals arising from low-level perceptual neurons.
6:
7:
8:
Vi X S1 < (V1(v)).
9:
where (N( denotes the leaky integrate-anc and-fiee (LIF) spike generator, taking v; as input and yielding the spike train s; for layer
10:
for i 2..... L do
When spikes pass through the synaptic cleft, they are multiplied by W. which determined by the strength of the connection:
V V, < W;-18i-1, S; SN((s:
13:
end for
14:
15:
Backward (ransfooos the high-level encoding to low-level stimuli): ví. < where š the encoding of target y St SN(v',,
17:
for i L I ,,..., 1..,1d do
19:
V; +- v t O +
20:
end for
21:
22:
Loss (omputation:: local loss)
23:
We present the loss defined for Type I neurons, with the loss for Type 2 neurons following symmetrically. do L; + LReco vz)
24:
for i = l.....
26:
where LReco denotes the Relaxed Contrastive (ReCo) loss described in Section 4.3 end for
27:
28:
We calculate the loss of the last layer using cross-entropy: Lup +- LCu(VE.
29:
30:
Local Gradients:
31:
for i = 2,..., L - do
32:
The calculation of the weight gradients only involves the local loss:
33:
V aw. Ve, dc, end for
8c.
80,
35:
Ve,
0C
80
36:
V W JWL-1
37:
Parameter Updates:
39: for i 1. ...., L do
40:
W; f- W, 1/W Vw, O (-60) (0,;
-/e Vo,
41:
end for
42:
end for
43:
end for
B ADDITIONAL PRELIMINARIES
Spiking neurons can be categorized into two types based on their signal propagation mechanisms. One category, referred to based neurons, conveys information through spike trains. Discrete Spiking Neural Networks (SNNs) integrate spike inputs at each time step and generate a binary spike output when the integrated value exceeds threshold. Continuous analog input signals typically
15

---

Published as a conference paper at ICLR 2026
encoded (e.g., rate encoding, time encoding, A-encoding) into spike sequences. Representative models in this category include IF, PLIF(Fang et al. (02T). and LIF(Maass 1997). Discrete SNNs compatible with existing digital design processes, allowing the grouping and virtualization of large numbers of neurons on computational cores to achieve parallel architectures. These systems exhibit high scalability and efficient trainability, benefiting from sparse, event-driven computation with low power consumption during the inference phase, However, the discrete nature of spike signals limits their information capacity relative to traditional neuron models, which transmit continuous matiigg-poin values directly (Peiffeer & Pfeil 2018).
The second type of spiking neuron architecture uses similar temporal equations as spiking neurons but propagates analog signals, referred to as mnloog-bssed neurons, such as LIAF(Wu et al., (021). These signals can encode more information and exhibit lower response latency in certain tasks. However, hardware implementations of such neurons require analog circuits and because computations cannot be skipped during inference, the system exhibits higher power consumption (Hazan & Ezra Tsur 1021
In Section 3.1 we provide the time dynamics equation for LIF. The equation D and equation 3 for different types of spiking neurons are similar, while equation 2 exhibits multiple variations. For example, the equation for is defined as follows:
H = U[t 1] + [[t]
(11)
The primary distinction between IF and LIF lies in the inclusion of a voltage leakage mechanism in LIF, which more closely resembles the behavior of biological neurons. Other variants, such as PLIF, consider the parameter in equation 2 to be learnable, while LIAF replaces θ in equation with ReLU, thereby enabling the transmission of continuous floating point values.
C THE MODEL ARCHITECTURE OF RNNS
The network comprises one layer of auto-regressive pyramidal neurons, which contains a pair of two distinct types of neurons. Type I neurons receive synaptic inputs on their basal dendrites from axons of lower-layer neurons, while Type 2 neurons receive basal dendritic inputs from axons of higher-layer neurons. X1, X2,..., XN represent the input sequence where is the sequence length. The training objective is to align the output o, of each timestep Type 1 neurons with ý, at each timestep i. During learning, the original input x; is delivered to the basal dendrites of Type neurons at timestep i, while the target Y; is provided to the basal dendrites of Type 2 neurons.
We employ synaptic weight matrices Wh.. Whhh and Who ho in the feedforward pathway corresponding to the input to the hidden layer weight, the spike of the previous time to the hidden layer weight and the hidden layer to the output weight respectively. Similar to the feedforward pathway, O e and Oni in the backward pathways corresponding the output target to the hidden layer weight, the spike of the previous time to the hidden layer weight and the hidden layer to the reconstructed input weight respectively.
The feedforward process is described by:
(12) where h, denotes the somatic membrane potential of Type neurons at timestep i arising from basal dendritic integration. h denotes the somatic membrane potential of Type 2 neurons at timestep i arising from apical dendritic integration. s; denotes the spike train that Type neurons at timestep i output and s = 0.
h; h = X Wm1 O, W Wio,; S; = SN 2,
The backward process is governed by:
h = h, in + Oms//41 = O $s = SN(hh, i= 1,2, - (13)
where h denotes the somatic membrane potential of Type 2 neurons at timestep i arising from basal dendritic integration. h, denotes the somatic membrane potential of Type 1 neurons at timestep i arising from apical dendritic integration. θ is the synaptic weight for Type 2 neurons in layer i, and si denotes the spike train that Type 2 neurons at timestep i output and s'y 0.
16

---

Published as a conference paper at ICLR 2026
D GRADIENTS DERIVATION FOR BSD
This section details the derivation of the gradients for the local loss functions with respect to the feedforward weights W and backward weights 6.
D.1 GRADIENT WITH RESPECT FEEDFORWARD WEIGHTS W.-1 (TYPE NEURONS)
For Type neurons in layer (for i = 2..... 1), the local loss L. is defined as:
4 (1-Cml) +
Σ Σ (max(0, (C,ls)))²
(14)
k=1
where [Cilky is the cosine similarity between Vi, ii and V.,, Vag
The gradient with respect to W is found via the chain rule. The resulting gradient computed as sum of outer products over the batch:
OC.
B
0Li
(s (s;-
(15)
OW
0van
k=1
The gradient with respect the basal voltage Vi,k is:
Oc, 2(1 - (C:Jus) Ovin
O.C.lu Σ
OCCcks
2A max(0, max(0,|(Cll))) [Cilas)
(16)
0vi,
iv.
The derivative of the cosine similarity term is:
1
Vi.j
Vi.k [Cilks
(17)
Ov.
|ivalll
[
Combining these terms yields the final gradient for W,-1
dC,
B
- 2(1
(Cilax)
Vi.k Viall
[Ci|en
Vi.k
OW,-1
Villel
[V
km
(18)
+ 2A m0 (Ciles)
Vi.5
- [Cilks ||va,ell) (8)
Vi
)
|/s,all
j
D.2 GRADIENT WITH RESPECT BACKWARD WEIGHTS O (TYPE NEURONS)
For Type neurons in layer (for i =,, .,.... L
1), the local loss C is defined symmetrically:
B C
B
Σ (1- (C) +
ΣΣ
(max(0,
[C))))))
(19)
k=1
k= jo
where [Cj]ky is the cosine similarity between V = Oss' The derivation for O follows the same structure, yielding a sum of outer products over the batch:
Vigi and
V
+1.
Oc
Σ
(s(41,))²
(20)
00
The gradient with respect to vL is:
0C
2 CC
&C.kks
2(1 - [C]]
Σ 22 max(0, (C]]ks)
(21)
OV.K
Ov
where the derivative the cosine similarity term is:
a|Ckks
1
V
V [C
(22)
0v
|
[
17

---

Published as a conference paper at ICLR 2026
This gives the final gradient for D,
ac
B
V
Ve.e
2(1
[C]]
Vi
|
V
(23)
Vis
[Cilas ViLk
) (5(+1*)*
+ Σ 2A max(O, (Cilks)
|v |l
||
1fth
E EXPERIMENTAL SETTINGS
E.1 STATISTICS OF DATASETS
We evaluate our proposed Bidirectional Spike-based Distillation (BSD) algorithm across diverse range of tasks. The datasets employed in our experiments detailed below, categorized by task.
Image Classification. For the image classification tasks, conducted experiments five widely- used benchmarks to assess the model's performance visual recognition.
MNIST.
The Modified National Institute of Standards and Technology (MNIST) dataset is cornerstone benchmark comprising 60,000 training and 10,000 testing images of handwritten digits (0-9). Each image is a 28 28 pixel grayscale representation.
FashionMNIST. As more challenging drop-in replacement for MNIST, the FashionM- NIST dataset contains 60,000 training and 10,000 testing examples of clothing items and accessories across 10 classes, each being a 28 x 28 grayscale image.
Street View House Numbers (SVHN). The SVHN dataset consists 32 x 32 color images of house numbers from a real-world setting. We use the standard cropped version, which includes 73,257 digits for training and 26,032 for testing, categorized into 10 classes.
CIFAR-10. The Canadian Institute for Advanced Research (CIFAR-10) dataset contains 50,000 training and 10.000 testing 32 x 32 color images across 10 mutually exclusive object classes (e.g., airplane, automobile, bird).
CIFAR-100. The CIFAR-100 dataset is collection of 60,000 32 x 32 color images, designed for fine- ame-ggained object recognition tasks. contains 100 distinct classes that are hierarchically grouped into 20 superclasses. For each class, there are 500 training images and 100 testing images.
Image Generation. To evaluate the generative performance of BSD, we employed an autoencoder architecture for image reconstruction on the MNIST, FashionMNIST, and CIFAR- 10 datasets. The quality of the generated images was quantitatively assessed using the Fréchet Inception Distance (FID) score, which measures the similarity between the distributions of the reconstructed and original images.
Text Character Prediction. For the sequential prediction task, we utilized natural language corpus to evaluate the model's ability to capture temporal dependencies at the character level.
Harry Potter Corpus. This dataset is text corpus derived from the Harry Potter book series (Plath et al. 2019). used for next-character prediction experiments.
Time-Series Forecasting. For evaluating performance multivariate time-series forecasting, employed three standard real-world datasets from different domains.
Electricity. The Electricity dataset (Lai et al. 2018) contains hourly electricity consumption data for 321 clients from 2012 to 2014, serving benchmark for long-term forecasting.
Metr-la. This traffic forecasting dataset (Li et al. 2018) contains traffic speed data from 207 sensors on highways in Los Angeles County, aggregated every 5 minutes over four-month period.
18

---

Published as a conference paper at ICLR 2026
Pems-bay. Sourced from the Caltrans Performance Measurement System (PeMS), this dataset (Li et al. 2018) comprises traffic speed information from 325 sensors in the San Francisco Bay Area over six months, providing another challenging forecasting benchmark.
E.2 IMPLEMENTATION DETAILS
E.2.1 BSD-MLPS
Model Architecture. For the feedforward path composed of Type neurons, we design distinct Multi-Layer Perceptron (MIP) architectures tailored to each task, given the heterogeneity of input dimensions and class cardinalities across the five datasets. For each layer, the synaptic weight configurations of the backward path are designed to ensure that the number of neurons in both the Type feedforward and Type 2 feedback paths remain consistent across all layers MNIST and FashionMNIST consist of single-channel 28 x 28 grayscale images, whereas SVHN, CIFAR- 10, and CIFAR-100 comprise MNIST, FaSS SVHN, and CIFAR- 10 are 10-class tasks, while CIFAR- 100 involves 100 classes. To accommodate these differences and ensure optimal performance, we adopt dataset-specific MI.P configurations. For MNIST and MNSST we employ a six six-layer MLP with layer sizes 1784, 1024, 1024, 512, 256, 10]. where the input dimension of 784 corresponds to the flattened 28 x 28 x images and the output dimension of 10 reflects the number of classes. For SVHN and CIFAR-10, we adopt wider six-layer MLP with [3072, 4096, 2048, 1024, 512, 10] neurons per layer, where the imput dimension of 3072 is derived from flattening the 32 x 32 x 3 images. For CIFAR- 100, maintain the same six-layer depth but adapt the final output 100 classes, yielding layer sizes of [3072, 4096, 2048, 1024, 512, 100]. For the backward pathway, class labels are converted into spike-encoded targets: given a classification task with C categories, the target for sample of class j is represented by categorical code inspired by one-hot encoding, consisting of binary vector of length C' with single active entry the j-th position and all others inactive. This vector is then repeated along the temporal dimension according to the number of time steps and delivered to the basal dendrites of top-layer Type 2 neurons. In all architectures, neuronal dynamics for membrane potential integration and spike generation are modeled using the Leaky Integrate- and-FFre (LIF) framework. The feedback path, composed of Type 2 neurons, follows a symmetric architecture to the feedforward path composed of Type neurons.
32 x 32 color images. In terms of classification objectives,
Training Hyperparameters. All MIPs are optimized using AdamW with cosine annealing learning-rate schedule. The simulation length is fixed at 7-4 time steps. For LIF neurons, we employ the ATan surrogate function provided in the SpikingJelly framework. To ensure that spike sparsity remains within a regime conducive to effective learning, the firing thresholds are set to 0.2 for Type I neurons and 0.1 for Type 2 neurons. Across all datasets, we adopt a learning rate of 1x10-4 batch size of 128, and apply RandAugment for data augmentation (Cubuk et al. 2020). To stabilize optimization, gradient clipping is applied with threshold of 0.3. Finally, for all non-top layers, the penalty weight in the ReCo loss is fixed at A
E.2.2 BSD-CNNS
Model Architecture. For the feedforward path composed of Type I neurons, we adopt a unified CNN architecture for all tasks. The architecture consists of five convolutional layers followed by a fully connected output layer. Specifically, the first convolutional layer maps the input channels to 128 channels using x 3 kernels, followed by 2 max moo-lno.ing. The second convolutional layer maintains 128 channels with the same 3 x 3 kernel and 2 2 max-pooling. The third and fourth convolutional layers expand the representation to 256 channels each, again using 3 x kernels with 2 x 2 max-pos max-pooling. The fifth convolutional layer further increases the dimensionality to 512 channels, also with x 3 kernels and 2 x max-pon maxxpoooling The feature maps are then flattened into a 512-dimensional vector, which is connected to the final output layer whose dimension is aligned with the spike-encoded target S.
As in the MLP case, class labels are converted into spike-coded targets and delivered to the basal dendrites of top-layer Type 2 neurons. For a classification task with C' categories, the target of a sample from class j is represented by categorical spike code inspired by one-hot encoding: a binary
19

---

Published as a conference paper at ICLR 2026
vector of length C with single active entry corresponding class j. repeated along the temporal dimension to match the simulation time steps.
To stabilize neuronal spiking activity across layers, we insert batch- normalizataninnnn layer after each convolutional operation. This normalization constrains the membrane potentials to a comparable range across layers, thereby ensuring that spike sparsity remains consistent and that neuronal firing thresholds are properly regulated throughout the network.
Similarly to the MLP case, the feedback path, composed of Type 2 neurons, mirrors the structure of the feedforward path composed of Type neurons. In the feedback path, we use upsampling as the reverse operation of the pooling used in the feedforward path, ensuring that the number of Type and Type 2 neurons in feedforward and feedback paths remains equal across all layers.
Training Hy Hyperparameters. For all tasks, we optimize our models using the AdamW optimizer and employ a cosine learning rate scheduler. To ensure that the neuronal firing sparsity remains within an ideal range, we set the firing thresholds of both Type I and Type neurons to 1.0. The number of time steps is set to 4, with warm-up period of 100 steps. For data augmentation, we use Rannent with a magnitude of for MNIST, FashionMNIST. CIFAR-10 10, and CIFAR-100, while on SVHN, we apply only RandomCrop. All images are normalized prior processing. The batch size is set to 128 across all datasets. For MNIST, FaSHiOnMNIST, CIFAR-10, and CIFAR-100 100, the learning rates for both Type I and Type 2 neurons are set to 1 x 10-3, whereas for SVHN we reduce the learning rate to 1 x 10-4. For the layer-wise ReCo loss applied to all layers except the top layer, we set the penalty weight À to 0.6, consistent with the configuration used in MLPs.
E.2.3 BSD-RNNS
Model Architecture. For all sequential regression datasets, we employed the same RNN model architecture which is a single-layer RNN with a hidden size of 300, where the input and output di- mensions corresponded to those of the respective datasets: Harry Notter(103,1053 Electricity Metr-la(206,1)) 1), tems-bay(324, 1), where the dimensions are presented in the form of (input dimen- sions, output dimensions).
Training Hyperparameters. Similarly to the classification tasks, we utilized the AdamW W optimizer along with cosine learning rate scheduler across all sequential regression datasets. To identify optimal performance in different datasets, we maintained identical activation thresholds for both Type and Type 2 neurons and fixed the number of timesteps at 4 for all tasks. However, we varied the activation threshold according to each dataset: 1.0 for the Harry Potter and Metr-la datasets, 0.8 for Electricity, and 0.5 for Pems-bay. All weights were initialized with PyTorrhh's 's default weight initialization. The batch size was set to 128 for the Electricity dataset and 64 for the remaining sequential regression datasets, with the sequence length fixed to 32 across all datasets. The learning rate set to 0.001 for the Harry Potter, Electricity and Metr-la datasets, and 0.00015 for Pems-bay Due to the larger dataset size of Harry Potter, training was limited to 10 epochs, whereas the other datasets were trained for 100 epochs. To stabilize training, gradient clipping with upper bound of 1 was employed for all sequential regression datasets, and batch normalization was applied to the hidden layer in Pems-bay to normalize each batch to zero mean and unit variance.
Metric. For time-series forecasting datasets (Electricity, Metr-la, and Pems-bay), the mean squared error (MSE) and the mean absolute error (MAE) were used as performance metrics. The formulas are as follows:
D
MSE
(~v.1.d Vo,e,a)
(24)
BxTx
D
B
T
D
MAE- = BXTXD
10 Vb.e.ll.
(25)
where B,T, T, D represents the batch size, sequence length, and output dimensions, respectively. While yb, is the value of the d-th feature of the output at the f-th time step for the b-th sample in the batch and aa is its target value. We normalized each column of the raw data to zero mean and unit variance before calculating MSE and MAE, consistent with the preprocessing described in (Lv et al.
20

---

Published as a conference paper at ICLR 2026
2025). We selected the model with the lowest MSE as the final recorded performance metric. For the text character prediction dataset (Harry Potter), element-wise accuracy was used as the performance metric. In preprocessing, all whitespace characters in the text were replaced with single space, and performance was evaluated on the processed dataset.
E.2.4 BSD-AUTOONCODEES
Model Architecture. We address the task of image reconstruction on the MNIST, FashionMNIST, and CIFAR- 10 datasets using a convolutional autoencoder. The architecture, which remains consistent across all datasets, is composed of feedforward path (Type neurons) and symmetric feedback path (Type 2 neurons). To maintain an identical neuron count between the Type and Type populations each corresponding layer, the architecture employs inverse spatial operations: where one path uses a strided convolution, the other utilizes an upsampling layer, and vice versa. To ensure stable and sparse spiking activity, each layer in both paths incorporates batch normalization step after convolution and upsampling.
The encoder section systematically compresses the input into latent representation through a cascade of three convolutional layers. The first layer applies 3 x 3 convolution with a stride of 2. halving the input's spatial dimensions while expanding the feature maps 128 channels. The second layer repeats this operation, again using stride of 2 to halve the spatial resolution and increasing the channel depth from 128 to 256. The final encoding layer performs third stride-2 convolution, mapping the features 512-channel bottleneck representation.
Symmetrically, the decoder reconstructs the image from this bottleneck representation. Each decoding layer first doubles the spatial dimensions of its input via nearest-neighhor upsampling operation. Following this, a 3 x 3 convolution with a stride of 1 is applied to refine the features. This two-step process is repeated three times: the first block reduces the channel count from 512 to 256, the second from 256 to 128, and the final block restores the feature map to the original input resolution while matching the channel count of the source image.
The source image is provided as the basal input to both the bottom-layer Type neurons and the top-layer Type 2 neurons. Thus, the top-layer Type 2 neurons function as dedicated transducer for the backward path, converting the source image into a spike-based representation.
Training Hyperparameters. The model is optimized across all datasets using the Adam W opti- mizer, with learning rate of 0.001 managed by cosine annealing schedule. The spiking threshold for both Type 1 and Type 2 neurons is uniformly set to 0.4, and the network is simulated for total of timesteps. To apply differential penalties to the reconstruction error, we decompose the error in the frequency domain. This is achieved by creating binary mask based on the image's Fourier transform. A circular region in the frequency spectrum, centered at the zero frequency, is defined as the low-frequency domain, while the area outside this region constitutes the high- frequency domain. The boundary is controlled by a freq_cutoff.ree io typerparameter, representing the normalized radius of this low-frequency circle. This allows us to assign distinct À weights for the components.
Reconstruction-on. caanssrructioooon-Conttutittiiit Construction (ReCo) loss to the low-frequecc and For the MNIST and FashionMNIST datasets, the model is trained with batch size of 32. The freq. cut.cctoooooo ff rat io o is set to 0.6, with the ReCo loss À configured to 0.005 for low-frequency components and 0.01 for high-frequency components. For the more complex CIFAR- 10 dataset, the batch size is increased to 64. The Freq_cutoff.rat ReCo loss weights are set 0.005 for the low-frequency band and increased to 0.05 for the high- frequency band, placing a greater penalty on the reconstruction of fine-grained details. To improve model generalization on this dataset, we also apply RandAugment for data augmentation during training.
is adjusted to 0.7, and the corresponding
Evaluation Metrics For autoencoder-based image generation tasks, we employ Fréchet Inception Distance (FID) (Heusel et al.) 2017) as our evaluation metric. FID measures the distributional similarity between real and generated images by computing the distance between their feature representations the Inception-v3 network's 's activation space. Specifically, the metric calculates the Fréchet distance between two multivariate Gaussian distributions fitted to the feature vectors of real and synthetic images extracted from the final pooling layer of Inception-v3. Lower FID scores indicate higher similarity between generated and real image distributions, with perfect score
21

---

Published as a conference paper at ICLR 2026
of O representing identical distributions. This metric provides reliable quantitative assessment of generation quality that correlates well with human perceptual judgment, making particularly suitable for evaluating the generative capabilities of our BSD-trained autoencoders.
F
ADDITIONAL EXPERIMENTS
F.1
SUPPLEMENTARY RESULTS FOR IMAGE GENERATION
7
2
1
7
9
7
2
/
4
/
4
9
000
7
2
/
4
/
4
9
Figure 4: Comparison of image reconstruction on the MNIST dataset by autoencoders trained with Backpropagation (BP) and our proposed Bidirectional Spike-based Distillation (BSD).
Figure 5: Visual comparison of FashionMNIST reconstructions from autoencoders trained with Backpropagation (BP) and our Bidirectional Spike-based Distillation (BSD) method..
Figures and 5 present a qualitative comparison of image reconstructions on the MNIST and Fashion- MNIST datasets, showcasing outputs from autoencoders trained with conventional Backpropagation (BP) alongside those trained with proposed Bidirectional Spike-based Distillation (BSD). Table 5 provides a complementary quantitative analysis on the FashionMNIST dataset, benchmarking the performance of BSD against BP and the Fully Spiking Variational Autoencoder (FSVAE) using the Fréchet Inception Distance (FID) metric. While BSD-trained autoencoders demonstrate proficient generative capabilities by successfully capturing the salient global structure of the input images, compared to BP-trained ANNs, BSD BSD-ggneraaed images tend to exhibit loss of high-frequency detail. This suggests that while BSD establishes robust framework for generative modeling, exploring further refinements or architectural adaptations could be beneficial for enhancing the preservation of fine-grained textures and improving the overall sharpness the generated images.
Table 5: Quantitative comparison of image generation performance the FashionMNIST dataset, evaluated using Fréchet Inception Distance (HID). + denotes that lower scores indicate better perfor- mance.
Dataset
Model
FID
ANN-BP
29.07
FashionMNIST
ESVAE
90.12
BSD (Ours)
112.97
22

---

Published as a conference paper at ICLR 2026
Table 6: Ablation study on character prediction and time-series forecasting tasks.
Harry Potter
Electricity
Metr-la
Pems-bay
Method
Pred. Acc.
MSE
MAE
MSE
MAE J
MSE +
MAE 4
BSD-MSE
29.6%
0.179
0.328
0.160
0.247
0.180
0.221
BSD-
29.9%
0.172
0.323
0.159
0.257
0.190
0.230
BSD
41.8%
0.165
0.314
0.125
0.197
0.174
0.206
F.2
ABLATION STUDY ON RNNS
Table 6 compares the performance of BSD-ReCo, BSD-MSE, and BSD-infoNCE on sequential regression tasks. In all tasks, the ReCo loss consistently demonstrated the highest performance. In contrast to the outcomes from the ablation studies on image classification tasks, all inter-layer loss functions achieved stable convergence on the sequential regression datasets, which may be attributed to the relatively shallow depth of the RNNs.
In conclusion, for sequential regression tasks, MSE, infoNCE, and ReCo demonstrated stable convergence, but ReCo remains the most optimal inter-layer loss, with its performance advantage being particularly evident on text character prediction datasets.
F.3 ABLATION STUDY ON AUTOENCODERS
E5D. pamido
BSD
SED
Figure 6: Visual examples of image reconstructions on the CIFAR-10 dataset, showcasing the performance of autoencoders trained with BSD using various layer-wise loss function configurations.
Dataset
Model
FID J
BSD
168.12
BSD-primitive
185.33
CIFAR-10
BSD-InfONCE
190.95
BSD-MSE
177.66
Table Ablation study on CIFAR-10 image generation, evaluating the impact of different layer- wise loss functions and frequency decomposition strategies, with performance measured by Fréchet Inception Distance (FID). + denotes that lower FID values indicate better performance. BSD- primitive: BSD without adaptive frequency domain decomposition. BSD-InfONCE. BSD employing InfoNCE loss for intra neuronal voltage alignment. BSD-MSE: BSD utilizing Mcan Squared Error (MSE) loss for
voltage alignment.
23

---

Published as a conference paper at ICLR 2026
To investigate the impact of different layer-wise loss functions and frequency decomposition strategies on image generation, we conduct an ablation study using BSD-trained autoencoders on the CIFAR-10 dataset. Figure 3 presents visual comparisons of image reconstructions, and Table 7 details the corresponding FID scores. Our ablation study includes the following configurations: BSD: our proposed method incorporating ReCo loss and adaptive frequency decomposition; BSD-primitive: BSD using ReCo loss but without adaptive frequency domain decomposition, applying a uniform across all frequency components: BSD-InfoNCE: BSD that substitutes ReCo loss with InfoNCE loss for intra-neuronal voltage alignment; and BSD-MSE: BSD that replaces ReCo loss with Mean Squared Error (MSE) loss for intra-neuronal voltage aligmment. From Table 7 it is evident that performing frequency domain segmentation using FFT and applying distinct À penalty weights for low and high-frequency regions significantly enhances the performance of BSD-trained autoencoders. As result, BSD achieves the best FID score of 168.12 notably outperforming BSD BSD-primitive (FII 185.33). Furthermore, for image generation tasks, BSD-InfONCE (FID 190.95), which employs InfoNCE for intra-ncuronal voltage alignment, yields inferior results compared to BSD (using ReCo Loss for intra-neuronal voltage alignment). ability capture salient structural features and approximate pixel locations of objects, it nonetheless exhibits obvious deficiencies in both color fidelity and detail retention. Compared to the recon- structions from BSD (which utilizes ReCo Loss and adaptive frequency decomposition), the images produced by BSD-MSE appear significantly blurred and desaturated, losing fine textures and distinct chromatic attributes present in the ground truth (GT) images. These results collectively affirm the effectiveness of our proposed design, which integrates ReCo Loss as the layer-wise loss function, performs frequency domain segmentation via FFT, and applies adaptive penalty weights for distinct low and high-frequency regions.
shows that while BSD-MSE demonstrates an
G DYNAMICS OF SYNAPTIC WEIGHT ALIGNMENT
Weight Alignment Between Forward and Backward Networks
enc1
0.6
Cosine Similanty
enc2
0.5
enc3
enc4
0.3
enc5
0.2
output
0.0
Epoch
Figure 7: Alignment dynamics of synaptic weights (W and Q) between the feedforward and backward pathways in a BSD-trained convolutional neural network for image classification on the CIFAR-10 10 dataset. The figure presents the cosine similarity of weights across network layers as function of training epochs.
In this section, we examine the alignment dynamics between the feedforward synaptic weights (W) and the backward synaptic weights (θ) within a convolutional neural network trained using the BSD algorithm. The network was tasked with image classification on the CIFAR-10 dataset. We measured the cosine similarity between the weights connected to the basal dendrites of Type I and Type 2 neurons to quantify their alignment in each layer. In Figure 7 "enel" through "ene5" denote the five feedforward convolutional layers and their corresponding backward upsampling layers, progressing from the shallowest to the deepest. The "outpuu" label refers to the final fully connected layer. Figure 7 reveals that the weight alignment for the final fully connected layer undergoes rapid initial increase, peaking within the first few epochs before stabilizing at a high cosine similarity value of approximately 0.7. In contrast, the weights of the convolutional and upsampling layers ("enc1" to "enc5") consistently exhibit negligible alignment, with their cosine similarity values remaining close throughout the entire training process. Such alignment behavior demonstrates that
24

---

Published as a conference paper at ICLR 2026
the feedforward and backward pathways do not converge toward symmetric weights under BSD. Thereby we confirm that the BSD algorithm adheres to criterion C1: Asymmetric synaptic weights for feedforward and feedback pathways.
H T-SNE VISUALIZATION OF LEARNED FEATURE REPRESENTATIONS IN BSD-TRAINED CNNS
After 5 apochs
After 10 apachs
After 15 opochs
After 20 spocks
Lawer
CIFAR- 10 Classes
airpiane
automebile bird
cal
deer
frog
ship
Layer
Figure 8: t-SNE visualizations illustrating the evolution of learned feature representations from the final two convolutional layers of the feedforward path in BSD-trained CNNs, for an image classification task on the CIFAR-10 10 dataset across training epochs.
Figure shows t-distributed stochastic neighbor embedding (t-SNE) projections of feature represen- tations from the final two convolutional layers (Layers and 5) of the feedforward path at epochs 10, 15, and 20 during BSD training on CIFAR-10. As training proceeds, class-specific structure becomes increasingly separated; by epoch 20, Layer exhibits distinct, compact clusters for most classes. This progression indicates that BSD progressively shapes intermediate features into more olass-discimminatie representations.
I EVALUATION ON TINY-IMAGENET
To evaluate the scalability our proposed algorithm, we conducted experiments on the Tiny- ImageNet dataset. We compared our proposed BSD algorithm against both a standard backpropaga- tion (BP) baseline and Dendritic Localized Learning (DILL). For fair comparison, we used the same 5-layer CNN architecture across all three methods.
Table 8: Performance comparison on the Tiny-ImageNet dataset.
Method
Accuracy (%)
BP
41.07
Dendritic Localized Learning (DLL)
17.10
BSD (Ours)
35.34
The results are summarized Table g
As shown, our BSD algorithm achieves an accuracy of 35.34%, substantially outperforming the other biologically plausible method, DLI..
J ABLATION STUDY ON BATCH SIZE
To investigate the sensitivity our BSD framework to the batch size, we conducted an ablation study using the BSD-trained CNN across the SVHN, CIFAR-10, and CIFAR-100 datasets. All other
25

---

Published as a conference paper at ICLR 2026
hyperparameters were kept consistent with the main experiments detailed in Appendix E.2 The results are summarized in Table
Table 9: Impact of varying batch size on the performance of the BSD-trained CNN across multiple datasets.
Batch Size
SVHN (%)
CIFAR-10 (%)
CIFAR-100 (%)
16
71.55
68.96
34.99
32
78.18
78.07
42.53
64
90.97
81.54
49.15
128
90.81
84.53
53.11
256
88.97
83.85
53.49
The results demonstrate clear trend: the performance of our BSD framework generally improves as the batch size increases. At smaller batch sizes, the model's performance is limited. This is an expected behavior for contrastive learning methods like ReCo, which require sufficiently large and diverse set of negative samples within each batch to effectively shape the representation space. As the batch size increases, provide a more robust estimate of the data distribution and richer set of negative examples for the contrastive objective. We also observe that the batch size reaches 128, the performance begins to stabilize. Further increasing the batch size to 256 yields minor fluctuations in accuracy. This suggests that batch size of approximately 128 provides sufficient number of negative samples for the model to achieve robust performance on these datasets. These findings confirm that our BSD framework scales favorably with batch size, which consistent with the principles of contrastive learning.
observe substantial performance gains across all datasets, larger batches
K ENERGY CONSUMPTION ANALYSIS
To quantify the energy efficiency of our trained models, conducted theoretical analysis of energy consumption during inference. The primary energy advantage of SNNs is realized in this phase, particularly on specialized neuromorphic hardware. Our BSD algorithm training methodology: the resulting model used for inference is standard SNN. Therefore, a model trained with BSD fully retains the inherent energy-saving characteristics of SNNs.
We estimate the theoretical energy consumption per sample by following the methodology proposed by Yao al. (023hb) For given SNN layer 1, the energy consumption estimated as:
Ene = EAC x (T x 7 X FLOPs((),
(26)
where 7 is the number of timesteps, " is the layer's 's average firing rate, and EAC is the energy per accumulate operation. For equivalent ANN layer 1, the formula is:
EnergyANNi (1) EMAC FLOPs(!),
(27)
where EMAC is the energy per multiply-accumulate operation. The total energy for each model the sum over all layers. We use the energy constants for 45mm process from the literature (Eac 0.9 pJ. EMAC = 4.6 pJ) for our analysis Yao et al., (2023)). The analysis was performed on our BSD-trained 5-layer CNN and its equivalent ANN counterpart the CIFAR-10 task. A laser breakdown of the energy consumption for both models is provided in Table 10 and Table II with final summary in Table
26

---

Published as a conference paper at ICLR 2026
Table 10: Layer-wise energy consumption analysis for the BSD-trained SNN during inference
single sample from CIFAR-10.
Layer Name
FLOPs
Avg Firing Rate (n)
SOPs
Energy (J)
Layer (Conv2d)
3.54e+06
0.5006
7.09e+06
6.38e-06
Layer (Conv2d)
3.77e+07
0.1519
2.29e+07
2.06e-05
Layer (Conv2d)
1.89c+07
0.1815
1.37c+07
1.23e-05
Layer (Conv2d)
9.44e+06
0.1943
7.34e+06
6.60e-06
Layer (Conv2d)
4.72e+066
0.2037
3.85e+06
3.46e-06
Layer (Linear) 5.12e+03
0.0803
1.65e+03
1.48e-09
Table 11: wase energy consumption analysis for the equivalent ANN-BP model during inference
on single sample from CIFAR-10.
Layer Name
FLOPs
Energy (J)
Layer (Conv2d)
3.54c+06
1.63e-05
Layer 2 (Conv2d)
3.77e+07
1.74e-04
Layer 3 (Conv2d)
1.89e+07
8.68e-05
Layer 4 (Conv2d)
9,44e+06
4.34e-05
Layer 5 (Conv2d)
4.72e+06
2.17e-05
Layer 6 (Linear)
5.12c+03
2.36e-08
Table 12: Total energy consumption comparison per inference sample.
Model
Total Operations per Sample
Energy per Sample (J)
Energy Reduction
ANN (BP)
7.43e+07 (FLOPs)
3.42e-04
SNN (BSD)
5.49e+07 (SOPs)
4.94e-05
85.5% +
This quantitative analysis confirms the substantial energy advantage of the SNN model. By leveraging sparse, event-driven computation, the SNN trained with our BSD method approximately 85.5% more energy-efficient during inference than its architecturally identical ANN counterpart. This highlights that our biologically plausible training method yields models that are not only performant but also highly efficient for deployment.
L ROBUSTNESS TO INPUT NOISE
To evaluate the robustness of our BSD framework against corrupted data, we conducted an experiment assess the resilience of our algorithm input noise. We used the model checkpoints pre-trained on the clean training sets of CIFAR- 10 and SVHN. During the inference phase, we introduced corruption by adding random Gaussian noise (mean=0, std=0.05) to the input images. We then evaluated the performance of these clean-trained models on the noisy test data and compared it to standard backpropagation (BP) baseline. The same 5-layer CNN architecture was used for both methods to ensure fair comparison.
The results, summarized Table corrupted test set.
show the accuracy on both the original clean test set and the
As expected, the performance both methods degrades under noisy conditions. On both CIFAR- 10 and SVHN, the drop in accuracy for BSD is comparable that of BP. The results indicate that our approach learns robust and effective features.
We attribute this competitive robustness the dual-objective nature of the BSD training process. The final task loss at the output layer pushes the network to extract discriminative information, while the layer-wise alignment objective simultaneously encourages the preservation of rich generative
27

---

Published as a conference paper at ICLR 2026
Table 13: Comparison of model accuracy on clean and noisy data for CIFAR-IO and SVHN. Models were trained only on clean data. "Noisy Accuracy" corresponds to Accuracy Under Attack (AUA).
Dataset
Method
Clean Accuracy (%)
Noisy Accuracy (%)
BP
87.18
70.89
CIFAR-10
BSD
83.67
67.06
BP
94.31
89.24
SVHN
BSD
90.81
85,40
information from the input. This balance between two complementary goals acts as form of regularization, guiding the model to learn more generalizable representations that are inherently more resilient input perturbations, rather than learning brittle features that are only optimal for the clean data distribution.
M FFT DECOMPOSITION FOR ADAPTIVE LOSS
The FFT decomposition. used in our generation tasks, is a method to separate the low-frequency and high-frequency components of image membrane voltage map, allowing to apply adaptive loss function. The process is as follows:
First, apply 2D Fast Fourier Transform the spatial tensor (e.g., batch of membrane voltage maps of shape [B, C, H, WD. converting it into its rreuennc-ddmmii representation.
Second, isolate these components, we construct frequency-domain masks. A low-pass filter is cre- ated by defining a circular region centered at the zero-frequency origin of the spectrum. Frequencies falling inside this radius are designated as low-frequency components, which represent the global structure and smooth areas of the image. Conversely, all frequencies outside this radius are designated as high-frequency components, which correspond edges and fine-grained textures.
Finally, by element-wise multiplying the dremanncy-domain representation with these two masks, we separate it into two distinct tensors: one containing only low frequencies and another containing only high frequencies. This separation enables the adaptive loss computation mentioned in the main text. We calculate the ReCo loss independently on these two components but use a different penalty weight À for each. A smaller À is applied to the coherence, while a larger À is applied to the low-freguee www-freuueccy components maintain structural components to more strongly enforce the preservation of sharp details and edge fidelity. This approach allows the model to better preserve fine details without sacrificing the image's structural coherence.
N COMPUTATIONAL COST ANALYSIS
To provide a comprehensive assessment the computational costs associated with our proposed BSD algorithm, we conducted a comparative analysis against a standard Backpropagation (BP) baseline for SNNs. All experiments were performed on the CIFAR-10 dataset using single NVIDIA GeForce RTX 2080 Ti GPU. We profiled memory consumption across different batch sizes to evaluate scalability, For both methods, we employed the 5-layer convolutional network architecture detailed in Appendix E.2 ensure fair comparison. The results for inference and training are presented separately below.
Inference Performance. During inference for classification tasks, the BSD framework utilizes only its feedforward pathway (weights W). Since this pathway is architecturally identical to BP-trained SNN of the same configuration, the computational graph and parameters used during a forward pass the same. As empirically confirmed in Table 14] this results in identical memory consumption across various batch sizes. This demonstrates that our biologically plausible training method introduces computational overhead during the inference phase.
28

---

Published as a conference paper at ICLR 2026
Table 14: Comparison of memory consumption during inference on CIFAR-I0 across varying batch sizes.
Batch Size
Method
Inference Memory Consumption (MB)
BP
344
32
BSD
344
BP
944
64
BSD
944
BP
1826
128
BSD
1826
Training Performance. During training, BSD must maintain two sets of weights (W and )) and store the activations for both the feedforward and backward pathways. Furthermore, the ReCo loss constructs B B affinity matrix, which introduces an additional memory usage component that scales quadratically with the batch size (O(B²)) However, the primary driver of BSD's increased memory consumption the necessity of storing the parameters and full activation maps for the second (backward) pathway, characteristic inherent to dual-network design. As detailed in Table 15] this architectural requirement leads to higher memory consumption compared to BP, which only stores one of weights and the activations required for its backward pass.
Table 15: Comparison of memory consumption during training on CIFAR-10 across varying batch sizes.
Batch Size
Method
Training Memory Consumption (MB)
BP
950
32
BSD
1154
BP
1844
64
BSD
2852
BP
3324
128
BSD
5034
Time Complexity Analysis. The empirical results are consistent with the theoretical time complex- ity of the algorithms. For network with L layers, T timesteps, batch size of B, and an average layer width of D, the complexities are as follows:
BPTT: The complexity is O(T T B D2), derived from the forward pass and symmetric
backward pass through the unrolled computation graph.
1322. D). The first term, OLL B- D²), accounts for propagation through the two pathways and comparable to BPTT. The second term, O(L- -2 arises from computing the B affinity matrix for the boser-wise ReCo loss, making our training complexity more sensitive to batch size.
BSD: The complexity (L
B-2 --
O ABLATION STUDY ON THE NUMBER OF TIMESTEPS
To more comprehensively investigate the influence of the number of timesteps (T) our model's performance, we expanded our ablation study across multiple datasets for both our CNN and RNN architectures. All other hyperparameters were kept consistent with those described in main experiments. The results are summarized Table 16 for image classification tasks and Table for sequential regression tasks.
For both CNN and RNN models, performance generally improves when increasing the number of timesteps from T 2 to T 4. For timesteps greater than four (T 4), performance tends to stabilize, exhibiting only minor fluctuations across most datasets and metrics. This suggests that four timesteps provide robust and effective balance, allowing neurons to integrate sufficient information
29

---

Published as a conference paper at ICLR 2026
Table 16: Impact of varying timesteps (1) the performance of BSD-trained CNNs across image classification datasets.
Timesteps (2)
CIFAR-100 (%)
SVHN (%)
CIFAR-10 (%)
2
47.12
82.54
78.09
4
53.48
90.81
84.53
6
53.11
90.35
82.28
8
53.24
90.94
81.51
Table 17: Impact of varying timesteps (T) on the performance of BSD-trained RNNs across sequential regression tasks.
Timesteps (7)
Harry Potter (ace f)
Metr-la (mse І)
2
0.4185
0.
4
0.4169
0,1245
0.4168
0.1248
8
0.4212
0.1238
to form rich representations without introducing excessive temporal complexity. These expanded findings empirically justify our choice 4 as a robust and efficient setting across a variety of tasks and architectures.
P ABLATION STUDY ON BATCH NORMALIZATION
To more broadly investigate the specific impact of batch normalization (BN), expanded our ablation study to include the SVHN, CIFAR-10, and CIFAR- datasets. The experiments were conducted on -sD-nrained CNN, and the results presented in Table 18
Table 18:
Impact of Batch Normalization (BN) on the performance of the BSD-trained CNN across multiple datasets.
Method
SVHN (%)
CIFAR-10 (%)
CIFAR-100 (%)
BSD without BN
80.68
80.75
53.31
BSD with BN
90.81
84.53
53.48
The results show that incorporating BN improves performance across all tested datasets. This benefit is attributed to the ability of BN to stabilize the distribution of membrane potentials across layers. The normalization ensures that spike sparsity remains consistent, leading to more stable and effective learning.
Q JUSTIFICATION FOR THE CHOICE OF RECO LOSS
Here, we provide a more detailed justification for our choice of the Relaxed Contrastive (ReCo) loss over other contrastive alternatives like InfoNCE.
Our ablation study in Section 5.6 provides the empirical evidence, showing that BSD with ReCo loss consistently and significantly outperforms BSD-InfONCE, This is particularly evident on complex datasets like CIFAR- 100, where the performance margin over 15 percentage points. The theoretical rationale for this superiority, especially for aligning the voltage signals in our BSD algorithm, is as follows.
The core objective at each hidden layer to align the pre-spike membrane potential from the feedforward path (Vr,)) with its corresponding supervisory voltage from the backward path (Vr,)). Simultancously, the network must distinguish this pair from all negative pairs, where the voltage Vi.k paired with supervisory signals from other samples in the batch (V,,) for j + k).
30

---

Published as a conference paper at ICLR 2026
A standard InfoNCE loss enforces this distinction by applying uniform repulsive force all negative pairs. This compels the membrane voltage pattern generated for one sample to be strictly anti-correlated with the supervisory voltage patterns of all other samples. This is overly restrictive constraint for high-dimensional membrane potentials, which can make the learning task unnecessarily difficult and potentially warp the feature space.
In contrast, the ReCo loss adopts a more targeted and flexible objective. It applies penalty only when the voltage for one sample is confusingly similar to the supervisory signal of another (i.e., has a positive cosine similarity). If a negative pair's voltage representations are already orthogonal or dissimilar, they incur no loss penalty. This allows the network focus its learning capacity on separating the most confusable voltage signals, rather than expending effort pushing already distinct representations further apart. This fosters a richer and more flexible representational space for the membrane potentials, leading to the improved final performance that is empirically validated by our results.
R LIMITATIONS AND FUTURE DIRECTIONS
R.1 LIMITATIONS
Despite its promising performance and significant contributions, the Bidirectional Spike-Based Distillation (BSD) framework also presents opportunities for further research. The current BSD framework has been evaluated on conventional neural network architectures such as Multi- Perceptrons (MLPs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and autoencoders. Expanding the applicability of BSD broader range of network architectures, such as those that incorporate residual connections or attention mechanisms, can further enhance its capacity address complicated tasks.
R.2 FUTURE DIRECTIONS
Future research can explore extending the BSD framework to support wider variety of network architectures, including those that employ residual connections and attention mechanisms, to handle increasingly complex datasets and challenging tasks.
THE USE OF LARGE LANGUAGE MODELS
The authors acknowledge the use of Large Language Model (LLM) in this work. Its role strictly confined to polish the writing and correcting grammatical errors to enhance the overall clarity and readability of the paper.
31


Reviews:
[{"review_id": "TkMRb4cc1E", "reviewer": "Reviewer_5Dww", "summary": "This paper introduces Bidirectional Spike-based Distillation (BSD), a novel biologically plausible learning algorithm for neural networks. The work addresses a fundamental challenge in neural network research: developing learning methods that satisfy biological plausibility criteria while achieving competitive performance with backpropagation. BSD is inspired by the brain's bidirectional architecture integrating bottom-up sensory perception with top-down memory recall. The algorithm jointly trains feedforward and backward spiking networks, where the feedforward pathway maps stimuli to conceptual representations (analogous to perception and decision-making), while the backward pathway reconstructs stimuli from concepts (analogous to memory recall). The authors demonstrate that BSD satisfies five biological plausibility criteria: asymmetric forward and backward weights, local synaptic plasticity, single phase learning (eliminating sequential forward-backward passes), spiking neurons, and unsigned error signals. They validate the algorithm across diverse architectures and tasks.", "questions": "See weaknesses. Moreover, could you expand the observation that \"The network attains its best performance at a firing threshold of 0.6, suggesting that thresholds that are either too low or too high impair effective learning\" (line 461) with a possible explanation for this phenomenon?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Biological motivation and plausibility:** The paper draws inspiration from the neuroscience principle of bidirectional processes that integrate bottom-up sensory perception with top-down memory recall. BSD implements this concept with high fidelity and satisfies five relevant criteria of biological plausibility, extending beyond the three criteria proposed by prior work.\n\n**Comprehensive experimental validation:** The authors test the algorithm on multiple architectures (MLPs, CNNs, RNNs, autoencoders) and diverse datasets spanning image classification (MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100), text prediction and time-series forecasting (Harry Potter, Electricity, Metr-la, Pems-bay), and image generation tasks, demonstrating the generalizability of the approach.\n\n**Novel algorithmic contributions:** The integration of bidirectional spiking networks trained via mutual distillation represents a unique approach to biologically plausible learning, distinguishing it from existing methods that typically compromise on some biological plausibility criteria. Moreover, BSD achieves performance comparable to backpropagation on all introduced tasks.\n\n**Ablation study:** The paper includes an ablation experiment examining the impact of different loss functions (MSE, InfoNCE) and demonstrates that ReCo loss consistently outperforms alternatives.", "weaknesses": "**Limited discussion of practical implications and broader impact:** The authors do not adequately address the practical implications of their work. Key questions remain unanswered:\n- Will BSD be adopted by the machine learning community, and if so, for what primary applications?\n- Will the code be released as an integrated Python library compatible with current frameworks (PyTorch, JAX)?\n- Alternatively, will BSD primarily serve the neuroscience community for computational experiments addressing neuroscience-related problems?\n\n**Insufficient computational performance analysis:** Performance metrics including time complexity, memory consumption, and GPU usage should be clearly stated and compared to backpropagation in the main text.\n\n**Minor:**\n- In the code, clear instructions for reproducing the experimental results are not given.\n- In Section 4.2, the notation is difficult to follow and contains apparent errors. Specifically, $\\hat{v}'$ is never defined, while $v'$ and $\\hat{v}$ are defined but never used in equations. The same symbol $v'$ is used to define Type 1 neurons in Equation 4 and Type 2 neurons in Equation 5, and vice versa for symbol $\\hat{v}$. Additionally, in Section 4.3 (line 239), matrices of membrane voltages are defined but never used in subsequent equations.\n- The application of FFT decomposition to input images and membrane voltages is mentioned but not adequately explained. This technique appears important for the method's performance on generation tasks, yet it receives minimal emphasis and should be expanded at least in the appendix.\n- Font sizes in figures (especially Figure 1c and Figure 3) are excessively small and difficult to read while going through the main text.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QE9x7PGYWM", "reviewer": "Reviewer_xq86", "summary": "This paper introduces Bidirectional Spike-Based Distillation (BSD), a biologically plausible learning algorithm inspired by the brain's bidirectional perception-recall architecture. BSD transforms spiking representations between stimulus encoding (feedforward path for perception) and concept encoding (feedback path for recall), satisfying five biological plausibility criteria (e.g., asymmetric weights, local plasticity). Experiments on image classification, sequential regression, and generation tasks demonstrate that BSD achieves performance comparable to error backpropagation while maintaining stronger neurobiological fidelity.", "questions": "See Weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. BSD fulfills all five proposed biological criteria, i.e., asymmetric weights, local error signals, non-two-stage learning, spiking neuron models, and unsigned error encoding, making it more aligned with neurobiological principles than backpropagation-based methods.   \n\n2. BSD achieves accuracy close to backpropagation on diverse benchmarks and robustly handles sequential regression and image generation, proving practicality without sacrificing biological grounding.", "weaknesses": "1. Discrete spike-based communication reduces information capacity compared to continuous activations, leading to performance degradation in SNNs across all tasks.   \n\n2. The description of the spike generation and information encoding lacks depth. While the LIF neuron dynamics are provided, the paper does not specify the exact strategy for converting inputs (like images) and targets into the initial spike trains (s1 and $\\hat{s}$) for the feedforward and backward paths. The choice of encoding scheme (e.g., rate coding, temporal coding) can significantly impact performance, and its omission is a notable gap. Furthermore, the distillation process between the two pathways is described at a high level as feature alignment using the ReCo loss, but the precise mechanisms of how the error signals from this loss guide the updates of the distinct weight matrices (W and $\\theta$) need a more detailed explanation, potentially including a mathematical derivation.\n\n3. The selection of the Relaxed Contrastive (ReCo) loss over other alternatives is justified by its unsigned nature and performance in ablations. However, a more rigorous theoretical or empirical justification for why it is particularly suited for aligning voltage signals in spiking neurons, as opposed to other contrastive losses, would strengthen the argument. Similarly, the choice of specific hyperparameters, such as the penalty weight $\\lambda$ = 0.6 and the firing thresholds for different neuron types and architectures, appears to be determined empirically. The paper would benefit from discussing the sensitivity of the results to these choices or providing a principled rationale for them.\n\n4. The paper introduces two new biological plausibility criteria (C4 and C5) and claims BSD satisfies all five. However, the argument for satisfying C2 (local synaptic plasticity) is somewhat superficial. It states that error computation is localized using the ReCo loss, but a more detailed analysis is needed to explain how the weight updates for W and $\\theta$ rely only on locally available information (e.g., pre- and post-synaptic spikes and local voltages) without any implicit non-local dependencies. A detailed diagram or step-by-step explanation of the local update rule for a single synapse would make this claim more credible.\n\n5.  The experiments demonstrate performance on standard benchmarks but do not thoroughly test the robustness of the BSD-trained models. For instance, how do these models perform on corrupted data, out-of-distribution examples, or under varying noise conditions? Including such robustness evaluations would be crucial for claiming the practical viability of the algorithm. Furthermore, the scalability of BSD is asserted but not conclusively proven. Testing on a larger-scale, more complex dataset (e.g., ImageNet) would provide much stronger evidence for its scalability beyond relatively small datasets like CIFAR-100.\n\n6. A key claim of the paper is that BSD is more biologically plausible. However, the experiments primarily focus on task performance metrics (accuracy, FID, MSE). There is a lack of direct analysis and quantification of the purported biological properties themselves. For example, the paper could analyze the degree of weight symmetry/alignment between the forward (W) and backward ($\\theta$) pathways during training. Figure 7 shows this alignment is low for most layers, which supports adherence to C1, but a more quantitative discussion of this property across different tasks and architectures would be valuable. Similarly, the energy efficiency of the spiking models, a major claimed advantage of SNNs, is not measured or compared to analog networks.\n\n7. In the image generation task, the qualitative results (Figure 2) and FID scores are presented. However, the FID scores for BSD are generally worse than those for BP-trained models. The paper states the quality is comparable, but a more critical discussion of the performance gap and the specific challenges BSD faces in generative tasks (e.g., possibly due to the discrete nature of spikes) would be more balanced. The visual samples in Figure 2 also lack a detailed description of what specific shortcomings (e.g., blurriness, lack of detail) are observed in the BSD-generated images compared to the BP baseline.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7uanxca93C", "reviewer": "Reviewer_SDT5", "summary": "This paper proposes the Bidirectional Spike-based Distillation (BSD) algorithm. Inspired by the brain’s bidirectional perception-recall mechanism, BSD jointly trains a feedforward spiking network (for stimulus-to-concept perception and decision-making) and a backward spiking network (for concept-to-stimulus memory reconstruction), framing learning as a transformation between stimulus encoding and concept encoding. Through experiments across multiple tasks (image classification, image generation, sequential regression) and network architectures (MLPs, CNNs, RNNs, autoencoders), BSD is verified to achieve performance comparable to classical error backpropagation while satisfying all five biological plausibility criteria.", "questions": "1. The paper mentions that the number of time steps is set to 4 for MLP, CNN, and RNN tasks (e.g., Section E.2.2, E.2.3) and 8 for autoencoder-based image generation tasks (Section E.2.4), but no analysis of the impact of different time step values on performance is provided. Do you plan to supplement experiments with varying time steps (e.g., 2, 6, 10) to explore how time step adjustments affect BSD’s performance across different tasks (image classification, image generation, sequential regression)?\n\n2. Batch normalization is applied in BSD-CNNs (after each convolutional layer, Section E.2.2) and BSD-RNNs (for the hidden layer in Pems-bay, Section E.2.3) to stabilize spiking activity, but the paper lacks quantitative and qualitative discussions on its specific impact. Could you provide experimental results comparing the convergence speed (e.g., number of epochs to reach stable performance) and final task performance (e.g., classification accuracy, FID, MSE) of BSD with and without batch normalization?\n\n3. The BSD algorithm adopts dual forward (Type 1 neurons) and backward (Type 2 neurons) branches with independent weight matrices (W for forward, Θ for backward, Section 4.2), which may increase GPU memory consumption. Have you quantified the GPU memory usage of BSD compared to classical error backpropagation (on ANNs/SNNs) and other biologically plausible algorithms (e.g., DLL, CCL) under the same network architecture and batch size?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. BSD is the an algorithm to simultaneously meet five core biological plausibility criteria, resolving conflicts between existing methods and neurobiological principles in aspects such as spike signal utilization and error signal type, thus providing an effective framework for biologically grounded spike-driven learning.\n\n2. Experiments cover multiple tasks (e.g., image classification on MNIST/CIFAR-10, image generation evaluated via FID, sequential regression including text prediction and traffic forecasting) and various network architectures, verifying BSD’s strong generalization ability and task adaptability.\n\n3. The algorithm design is closely aligned with the brain’s bidirectional mechanism of bottom-up perception and top-down recall, utilizing the basal and apical dendrites of pyramidal neurons to process feedforward and feedback signals respectively, resulting in a theoretical foundation highly consistent with neuroscientific observations.", "weaknesses": "1. It would be beneficial to investigate the influence of different timesteps on the BSD network's performance through additional experiments.\n\n2. Further discussions on the impact of batch normalization on the convergence and overall performance of the BSD algorithm could be included.\n\n3. Considering the BSD algorithm's dual forward and backward branches, it presumably requires a larger amount of GPU memory, which could be a relevant consideration for practical applications.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yh0YGLshCZ", "reviewer": "Reviewer_MvQY", "summary": "The authors present a layer-wise local learning rule for two-compartment spiking neurons where each layer tries to minimize a Relaxed Contrastive (ReCo) loss between the apical and basal activation. The network is constructed with a bottom-up discriminative and top-down generative stream, where each stream has its own classification/regression task. The authors argue that in this network, just optimizing these within-layer errors inside the spiking neurons results in competitive performance on various tasks.", "questions": "What is the effect of having spikes? In 5.3 it is noted that spikes degrade accuracy compared to ANNs. For the proposed architecture, it would seem the proposed approach would also work for identically structured non-spiking RNNs. Does this improve performance? \n\nHow does the approach scale? All examples are relatively small/shallow  networks, whereas it is known that similar ELBO-like losses tend to not scale well. Can you comment Deeper networks?\n\nHow does ReCo scale with batch size?\n\nCan you demonstrate why layer-wise ReCo works as well as it does, compared to e.g. non detached ReCo?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 4, "strengths": "The experiments are extensive and well done. The loss as described seems to be a novel application for spiking neural networks. If correct (see below), the stability of the local learning approach is amazing and would be a big step in general towards local learning.", "weaknesses": "I have a number of smaller issues, and one main one:\n\nwhile the ReCo loss has been presented before, I am not aware of its use in a setting where there is only layer-wise learning. Ie in Lin et al 2023, this loss seems to be backpropagated through a deep network. It is not obvious why this loss still works when indeed the errors are not backpropagated from the output layer. As it is now, the approach seems to mimic a layer-wise ELBO loss, with some nudging of only the final layer from errors calculated from outputs. The issue is that such layer-wise ELBO losses typically collapse and need measure to learn properly. I do not see either the mathematical argument why the proposed approach should work or the ablation study that demonstrates how this works compared to not detaching. Checking the code, I am also not convinced that necessarily the detach() operation is applied correctly.  I would very much like to see convincing arguments why the described approach works, and that it works correctly. \n\nSmaller issues:\n\nThe text is many places is not particularly clear. Eg. where the loss is described in 4.3 (l234), it is not immediately clear that the ReCo loss is being explained. \n\nFigures 1 and 3 dont print well.\n\nThe term \"distillation\" is prominently mentioned but it is not explained what is meant by this term.  Similarly for \"embeddings\" \n\nLayer-wise learning is not necessarily local, it should be explained in detail why the learning rule is neuron-local or instead layer-local.\n\n\"predictive coding\" is a closely related approach and should be discussed. It is also shown in Table 1 but not cited, so the particular predictive coding approach is unclear. \n\nthe approach claims biological plausibility, but the ReCo contrastive loss relies on the batch to provide negative samples, which is not a quantity locally known by the neuron. \n\nI do not agree with all stated biological constraints: as is well noted in the literature, symmetric forward and backward weights can arise naturally provided that the same local learning is present at forward and backward site, and weight-decay is applied (easy to see). \n\nThere is a misunderstanding of how the brain seems to employ global error signals: first, multiple neuromodulatory signals are thought to be quite a-specific, like dopamine being broadcasted widely and non-targeted. Moreover, to the degree that dopamine signals a global reward prediction error, negative errors are believed to be encoded as a decrease from a steady baseline, thus resolving the issue with error-signs.\n\ntypo l054: \"spiking trains\" ??", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hlOQDTgzke", "reviewer": "Reviewer_E6E3", "summary": "This paper presents Bidirectional Spike-based Distillation (BSD), a biologically motivated training recipe for spiking neural networks. BSD jointly trains a bottom-up “perception” SNN and a top-down “recall” SNN, framing learning as the alignment of basal (forward) and apical (feedback) membrane potentials. Hidden layers are optimized with a layer-local, unsigned Relaxed Contrastive (ReCo) loss that increases same-sample similarity and suppresses positive cross-sample correlations, while the top layer uses a standard supervised objective. The method is claimed to satisfy five criteria for biological plausibility (asymmetric weights, layer-local plasticity, non-dual-phase training, spiking neurons, and unsigned signals at hidden layers). Experiments on image classification, sequence regression, and simple autoencoding show performance close to BP on modest benchmarks with MLP/CNN/RNN backbones. Conceptually, the novelty appears moderate: BSD primarily assembles known ideas from local/decoupled learning and feedback/target-prop-style alignment into an all-spiking, two-pathway realization. Moreover, the advertised “top-down memory recall” is instantiated as label-driven feedback reconstruction rather than an addressable memory or retrieval mechanism. Finally, the work lacks validation on modern architectures and large-scale datasets and does not compare against strong local-learning baselines for SNNs, so the contribution is best viewed as an effective integrated recipe rather than a fundamentally new paradigm.", "questions": "Q1. Can you report a ImageNet-1k experiment (e.g., ResNet-18/50) with accuracy and training cost versus BP and the above baselines?\n\nQ2. How sensitive is BSD to batch size, especially small-B/micro-batch or streaming regimes where the B×B affinity becomes ill-conditioned?\n\nQ3. How do you reconcile C5 with a signed top-layer cross-entropy; is C5 intended only for hidden layers, or can you replace CE with an unsigned alternative and report the impact?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "This work presents a well-motivated learning algorithm inspired by the brain's perception-recall mechanisms. The paper clearly defines its contribution by introducing two new criteria for biological plausibility and demonstrating that the proposed BSD algorithm satisfies all five. The joint training of feedforward and backward networks via mutual distillation is an elegant approach to implement local learning. The experimental evaluation covers a diverse set of tasks and network architectures, which supports the algorithm's versatility. The results show that BSD can achieve performance competitive with backpropagation while adhering to stricter biological constraints.", "weaknesses": "The paper positions layer-local losses and decoupled feedback as distinctive, yet these ideas substantially overlap with established families (local errors/greedy layerwise, synthetic gradients, feedback alignment, target/representation propagation; in SNNs, DECOLLE and e-prop), and the batch-correlation alignment mirrors redundancy-reduction/self-correlation objectives (Barlow Twins/VICReg), so the conceptual novelty appears moderate without head-to-head baselines. \n\nClaim–implementation gaps further temper the contribution: the “top-down memory recall” component is effectively label-driven feedback reconstruction (no addressable memory, retrieval/gating, replay, or partial-cue/label-free recall). \n\nScalability is unclear because ReCo forms a B×B cosine matrix per layer (O(B²) compute/memory) with no memory profiling or scalable approximations, leaving open questions about large-batch, streaming/micro-batch, and deeper/wider models. \n\nExternal validity is limited to small datasets and shallow backbones (MNIST/SVHN/CIFAR; 6-layer MLP/5-layer CNN), with no results on modern architectures (ResNet) or large-scale data (ImageNet), so it is difficult to judge whether the gains persist at realistic scale or against strong local-learning baselines for SNNs.\n\n---\n\nReferences \n\nDeep Supervised Learning Using Local Errors, https://pmc.ncbi.nlm.nih.gov/articles/PMC6127296/\n\nGreedy Layerwise Learning Can Scale to ImageNet, https://proceedings.mlr.press/v97/belilovsky19a/belilovsky19a.pdf\n\nDecoupled Neural Interfaces using Synthetic Gradients, https://arxiv.org/abs/1608.05343\n\nRandom synaptic feedback weights support error backpropagation for deep learning, https://www.nature.com/articles/ncomms13276\n\nTraining Neural Networks with Local Error Signals, https://www.gatsby.ucl.ac.uk/~pel/tnlectures/papers/local2019.pdf\n\nDifference Target Propagation, https://arxiv.org/abs/1412.7525\n\nBiologically Motivated Algorithms for Propagating Local Target Representations (LRA-E), https://ojs.aaai.org/index.php/AAAI/article/view/4389/4267\n\nSynaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE), https://www.frontiersin.org/articles/10.3389/fnins.2020.00424/full\n\nA solution to the learning dilemma for recurrent networks of spiking neurons (e-prop), https://www.nature.com/articles/s41467-020-17236-y\n\nBarlow Twins: Self-Supervised Learning via Redundancy Reduction, https://proceedings.mlr.press/v139/zbontar21a.html\n\nVICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, https://arxiv.org/abs/2105.04906", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)