Biologically Plausible Learning via Bidirectional Spike-Based Distillation

## Abstract

Abstract Developing biologically plausible learning algorithms that can achieve performance comparable to error backpropagation remains a longstanding challenge.
Existing approaches often compromise biological plausibility by entirely avoiding the use of spikes for error propagation or relying on both positive and negative learning signals, while the question of how spikes can represent negative values remains unresolved.
To address these limitations, we introduce Bidirectional Spike-based Distillation (BSD), a novel learning algorithm that jointly trains a feedforward and a backward spiking network.
We formulate learning as a transformation between two spiking representations (i.e., stimulus encoding and concept encoding) so that the feedforward network implements perception and decision-making by mapping stimuli to actions, while the backward network supports memory recall by reconstructing stimuli from concept representations.
Extensive experiments on diverse benchmarks, including image recognition, image generation, and sequential regression, show that BSD achieves performance comparable to networks trained with classical error backpropagation.
These findings represent a significant step toward biologically grounded, spike-driven learning in neural networks.

## 1 Introduction

Human learning and cognition cannot be reduced to a simple unidirectional “perception-to-decision” pipeline.
Rather, they emerge from bidirectional processes that integrate bottom-up sensory perception with top-down memory recall
(Caucheteux et al., 2023; Bonetti et al., 2024).
During visual perception, retinal signals are transformed through hierarchical neural pathways into high-level conceptual representations stored in memory.
Conversely, during recall, the brain can reconstruct partial sensory features from these stored representations.

Kosslyn et al. (1993) showed with positron emission tomography that the same early visual cortical areas (i.e., V1 and V2) activated during perception are also engaged when subjects visualize objects with their eyes closed.
Later studies demonstrated that stimulus identity can be decoded from early visual cortex activity during both working memory and mental imagery, with patterns resembling those elicited by actual stimulation (Albers et al., 2013).
For instance, consider a learner distinguishing between different bird species.
At the outset, early visual areas may encode only basic features such as edges or color patches, making two similar species appear nearly indistinguishable from each other.
As categorical knowledge of the species is acquired, higher-level conceptual representations emerge and feed back to early visual regions.
This feedback sharpens perceptual sensitivity to subtle diagnostic features, such as beak curvature or wing pattern, thereby refining low-level visual representations in accordance with learned category distinctions.

Inspired by the brain’s bidirectional architecture of perception and recall, we introduce bidirectional spike-based distillation (BSD), a novel learning algorithm that frames learning as a transformation between two spiking representations: stimulus encoding and concept encoding. The feedforward pathway performs perception and decision-making by mapping sensory stimuli to conceptual representations, analogous to the brain’s analytical mode, while the feedback pathway facilitates memory recall by reconstructing stimuli from semantic concept encodings, analogous to the brain’s imaginative mode.
These feedforward and feedback networks can be trained jointly by distilling feature representations from one another.
By integrating perception and recall within a unified framework, BSD provides a biologically grounded alternative to conventional unidirectional learning paradigms.

We also show that the proposed bidirectional distillation (implemented via spiking trains) yields a more biologically plausible learning algorithm.
While backpropagation has achieved remarkable success in deep learning (LeCun et al., 2015; Rumelhart et al., 1986), its underlying mechanisms conflict with established neurobiological principles (Crick, 1989; Lillicrap et al., 2020).
Key inconsistencies include the requirement for symmetric feedforward and feedback weights, reliance on global error signals instead of local synaptic plasticity, a two-stage learning process that clearly separates forward and backward passes, and the use of continuous activations rather than discrete spike-based communication.
To address these limitations, and building on the three criteria proposed by Lv et al. (2025), we introduce two additional requirements: neurons should communicate using discrete binary spikes for both learning and inference, and learning should rely on unsigned spiking signals only (Hayden et al., 2011).
The learning algorithm we present demonstrates that all five criteria can be satisfied, whereas existing approaches typically fall short on one or more of these criteria.

Through extensive experiments across a range of tasks, including image classification, text character prediction, time-series forecasting, and image generation, and using diverse network architectures such as multi-layer perceptrons, convolutional neural networks, recurrent neural networks, and autoencoders, we demonstrate that BSD achieves performance comparable to backpropagation while satisfying all five criteria for biological plausibility.
Our results indicate that more adherence to biological fidelity does not necessarily compromise computational effectiveness.

The contributions of this study can be summarized as follows:

- •
Inspired by the brain’s bidirectional architecture of perception and recall, we propose a novel learning framework in which the feedforward network (stimuli-to-decision) and the backward network (concept-to-stimuli) are jointly trained by mutually distilling spiking feature representations.
- •
We introduce two additional biological plausibility criteria to complement the three proposed by Lv et al. (2025), resulting in five key principles: asymmetric forward and backward weights, local error representation, non-two-stage learning, spiking neuron models, and unsigned error signals. The BSD algorithm satisfies all five criteria and demonstrates enhanced biological plausibility.
- •
We perform extensive experiments using the BSD algorithm across diverse network architectures and various tasks.
The experimental results show that BSD achieves performance comparable to error backpropagation while maintaining greater adherence to biological fidelity.

## 2 Related Work

Backpropagation (BP) (Rumelhart et al., 1986) has long been criticized for its limited biological plausibility, as it depends on weight symmetry (Stork, 1989), global error signals (Crick, 1989), and a strictly sequential forward–backward computation process (Guerguiev et al., 2017; Hinton, 2022).

To address these limitations, numerous alternative approaches have been proposed to improve biological plausibility (Schmidgall et al., 2024; Jiao et al., 2022; Li et al., 2024). However, our method uniquely satisfies established biological constraints while maintaining strong performance and broad applicability across model architectures and tasks. For example, target propagation (TP) (Bengio, 2014) introduces approximate inverse models to generate layer-wise targets, with weight updates obtained by minimizing the mismatch between outputs and targets. Although TP employs local losses and avoids weight symmetry, it requires each layer to transmit two distinct types of signals at different times. Moreover, TP often suffers from convergence instability.
Extensions such as Difference Target Propagation (DTP) (Lee et al., 2015) and SDTP (Bartunov et al., 2018) improved stability and performance but offered little progress towards greater biological plausibility, and Biologically-plausible Reward Propagation (BRP) (Zhang et al., 2021), which replaces floating-point interlayer signals in TP with spike-based signals, has not demonstrated reliable generalization across tasks.
Other methods, such as Dendritic Localized Learning (DLL) (Lv et al., 2025), employ local error signals for weight updates, yet still rely on transmitting signed floating-point values and generally perform poorly across benchmarks. Counter-Current Learning (CCL) (Kao & Hariharan, 2024) faces the same limitation of floating-point communication and further lacks demonstrated effectiveness in sequential regression tasks.

Other biologically inspired mechanisms, like Hebbian learning (Donald, 1949) and spike-timing-dependent plasticity (STDP) (Song et al., 2000), adjust synaptic strength according to correlations in neuronal activity, while burst-dependent synaptic plasticity(Payeur et al., 2021) regulates synaptic plasticity by high-frequency bursts of spikes.
Although fully consistent with biological observations, these rules struggle to integrate supervised learning signals, and STDP additionally requires precise temporal resolution.
Energy-based learning approaches (LeCun et al., 2006), e.g., Boltzmann machines (Ackley et al., 1985), Hopfield networks (Hopfield, 1984), and contrastive learning frameworks (Hinton, 2002), instead optimize an energy function.
However, minimizing energy does not always correspond to reducing task-specific loss, limiting their utility for general supervised learning.

In contrast, our approach, which is inspired by the brain’s bidirectional interplay between perception and recall, exhibits improved biological plausibility while achieving stable convergence and superior performance on various benchmarks and tasks.

## 3 Preliminary

### 3.1 Spiking Neurons

We adopt the leaky integrate-and-fire (LIF) neuron model (Maass, 1997) in our spiking neural networks (SNNs). The dynamics of the LIF neuron are formulated in discrete time as follows:

$$ $\displaystyle U[t]$ $\displaystyle=H[t](1-S[t])+U_{\text{reset}}S[t],$ (1) $\displaystyle H[t]$ $\displaystyle=U[t-1]+\frac{1}{\tau}\big(I[t]-(U[t-1]-U_{\text{reset}})\big),$ (2) $\displaystyle S[t]$ $\displaystyle=\Theta(H[t]-U_{\text{thr}}),$ (3) $$

where $I[t]$ represents the input current at time step $t$. Here, $H[t]$ and $U[t]$ denote the membrane potential before and after the trigger of a spike $S[t]$, respectively. The parameter $\tau$ is the membrane time constant, while $U_{\text{thr}}$ and $U_{\text{reset}}$ specify the firing threshold and reset potential. A spike is generated when the pre-spike potential $H[t]$ exceeds the threshold $U_{\text{thr}}$, in which case the neuron fires ($S[t]=1$) and the membrane potential is reset to $U_{\text{reset}}$. Additional preliminaries are given in the appendix [B](https://arxiv.org/html/2509.20284v1#A2).

### 3.2 Biological Plausibility Criteria

In this work, we mainly build upon three biological plausibility criteria established by Lv et al. (2025): C1. Asymmetric synaptic weights for feedforward and feedback pathways; C2. Local synaptic plasticity based solely on locally available information without global error signals; C3. Non-dual-phase training that eliminates sequential forward–backward dependencies. We will also introduce two additional criteria in Section [4](https://arxiv.org/html/2509.20284v1#S4), further extending the framework of biologically plausible learning.

## 4 Method

### 4.1 Design Principles

Inspired by the brain’s bidirectional processes of perception and recall,
we propose Bidirectional Spike-Based Distillation (BSD), which frames learning as a transformation
between stimulus encoding and concept encoding through spiking representations.
The feedforward pathway maps sensory inputs to concepts for perception and decision-making,
while the backward pathway reconstructs inputs from concepts, thereby supporting feedforward learning during training.

Before introducing BSD in detail,
we extend the three biological plausibility criteria of Lv et al. (2025) with two additional principles.
Designed to satisfy all five criteria, BSD ensures a biologically grounded foundation for learning.
The two additional criteria we propose are:

##### C4. Model of Neurons.

Neurons in conventional artificial neural networks produce continuous activations, which are typically interpreted as approximations of average firing rates (Maass, 1997).
This stands in contrast to the brain’s actual processing, where biological neurons communicate using discrete action potentials (0-1 spikes) (Gerstner et al., 2014).

##### C5. Unsigned error signaling.

The learning mechanism in most artificial neural networks relies on signed error signals to propagate directional gradient information for weight adjustments (Rumelhart et al., 1986).
This stands in contrast with neurophysiological findings, which indicate that neurons encode unsigned reward prediction errors, firing in response to surprising outcomes irrespective of their positive or negative valence (Hayden et al., 2011).

To facilitate the understanding of the BSD algorithm, we illustrate both the network architecture and the neuronal learning mechanism in Figure [1](https://arxiv.org/html/2509.20284v1#S4.F1), using an $L$-layer MLP configuration.
The complete form of the global algorithm is detailed in Algorithm [1](https://arxiv.org/html/2509.20284v1#alg1).

Figure: Figure 1: (a) Overview of the bidirectional spike-based distillation framework. (b) Illustration of feature alignment using the properties of pyramidal neurons, which receive feedforward and feedback signals through their basal and apical dendrites, respectively. (c) Performance comparison of neural networks trained with BSD, dendritic localized learning (DLL), a recently proposed biologically plausible learning algorithm, and backpropagation. The experimental results show that BSD achieves performance comparable to backpropagation.
Refer to caption: x1.png

### 4.2 Model Architecture

##### Neuronal Dynamics.

To highlight the biological foundations of BSD, we first describe the neuron model employed in BSD.
We follow Sacramento et al. (2018) and utilize pyramidal neurons with a three-compartment structure: soma, apical dendrites (carrying backward learning signals), and basal dendrites (receiving feedforward inputs).
To satisfy criterion C4 (model of neurons), we use spiking neurons that emit discrete pulses instead of continuous activation values.
For each neuron, the membrane potentials arriving at the soma from the basal and apical compartments are denoted $v$ and $\hat{v}$, respectively.
The apical potential $\hat{v}$ acts as a supervisory signal guiding synaptic plasticity on the basal dendrites, while the basal potential $v$ drives spike generation through $s=\mathcal{SN}(v)$, where $\mathcal{SN}(\cdot)$ denotes the spiking operation that integrates input voltage and produces spike outputs.

##### Network Architecture.

The network comprises $L$ layers of pyramidal neurons, where each layer contains an equal number of two distinct types of neurons.
Type 1 neurons receive synaptic inputs from lower-layer neurons and constitute the feedforward pathway, implementing a stimuli-to-decision process that transforms
sensory inputs into conceptual representations. Conversely, Type 2 neurons receive inputs from higher-layer neurons and form the backward pathway, which attempts to
reconstruct sensory features from conceptual encodings to assist feedforward learning. During training, the original input $\mathbf{x}$ is delivered to
bottom-layer Type 1 neurons, while the learning target is first encoded into a spike train $\hat{\mathbf{s}}$ and provided to top-layer Type 2 neurons. The two
pathways are jointly optimized through mutual distillation of their spiking feature representations, enabling bidirectional information flow that mirrors the
brain’s perception-recall architecture.
To satisfy criterion C1 (asymmetric synaptic weights), we employ independent synaptic weight matrices $\mathbf{W}$ and $\bm{\Theta}$ for the feedforward and backward pathways, respectively. The feedforward path is described by:

$$ $\mathbf{v}_{1}=\mathbf{x};\quad\mathbf{v}_{i}=\hat{\mathbf{v}}_{i}^{\prime}=\mathbf{W}_{i-1}\mathbf{s}_{i-1},\quad\mathbf{s}_{i}=\mathcal{SN}(\mathbf{v}_{i}),\quad i=2,3,\ldots,L,$ (4) $$

where $\mathbf{v}_{i}^{\prime}$ denotes the somatic membrane potential of Type 1 neurons in layer $i$ arising from basal dendritic integration, $\hat{\mathbf{v}}_{i}$
denotes the somatic membrane potential of Type 2 neurons in layer $i$ arising from apical dendritic integration. $\mathbf{W}_{i}$ is the synaptic weight for Type 1 neurons in layer $i$, and $\mathbf{s}_{i}$ denotes the spike train that Type 1 neurons in layer $i$ output.

The backward pathway is described by:

$$ $\mathbf{v}_{L}^{\prime}=\hat{\mathbf{s}};\quad\mathbf{v}_{i}^{\prime}=\hat{\mathbf{v}}_{i}=\bm{\Theta}_{i}\mathbf{s}_{i+1}^{\prime},\quad\mathbf{s}_{i}^{\prime}=\mathcal{SN}(\mathbf{v}_{i}^{\prime}),\quad i=1,2,\ldots,L-1,$ (5) $$

where $\mathbf{v}_{i}^{\prime}$ denotes the somatic membrane potential of Type 2 neurons in layer $i$ arising from basal dendritic integration, $\hat{\mathbf{v}}_{i}$
denotes the somatic membrane potential of Type 1 neurons in layer $i$ arising from apical dendritic integration. $\bm{\Theta}_{i}$ is the synaptic weight for Type 2 neurons in layer $i$, and $\mathbf{s}_{i}^{\prime}$ denotes the spike train that Type 2 neurons in layer $i$ output.

### 4.3 Training Procedure

Motivated by the neuronal least-action principle (Senn et al., 2024), which postulates that pyramidal neurons minimize somato-dendritic mismatch errors through
voltage dynamics, and to satisfy criterion C2 (local error computation), our BSD algorithm introduces local loss functions for individual neurons to
align basal-received voltage $v$ with apical-received voltage $\hat{v}$. Type 1 and Type 2 neurons respectively receive bottom-up sensory inputs $\mathbf{x}$ and top-down target signals $\hat{\mathbf{s}}$, which in classification tasks correspond to distinct modalities. Thus, the alignment between $v$ and $\hat{v}$ can be viewed as aligning cross-modal embeddings.
Inspired by contrastive learning in multimodal representation learning and to satisfy C5 (Unsigned Error Signal), we employ the Relaxed Contrastive (ReCo) loss (Lin et al., 2023), which is an unsigned loss function.
In our design, error computation is localized within each neuron, and no error signals are propagated across the network.
Specifically, for Type 1 neurons in a given layer $i$ (for $i=2,\dots,L-1$), let $\mathbf{v}_{i,k}$ and $\hat{\mathbf{v}}_{i,k}$ denote the basal and apical membrane voltage for the $k$-th sample in a batch, respectively. By stacking these vectors across the batch dimension $B$, we construct matrices $\mathbf{V}_{i}\in\mathbb{R}^{B\times D_{i}}$ and $\hat{\mathbf{V}}_{i}\in\mathbb{R}^{B\times D_{i}}$, where $D_{i}$ is the number of neurons in layer $i$. The layer-specific affinity matrix $\mathbf{C}_{i}\in\mathbb{R}^{B\times B}$ is then defined by its elements:

$$ $[\mathbf{C}_{i}]_{kj}=\frac{\mathbf{v}_{i,k}\cdot\hat{\mathbf{v}}_{i,j}}{\|\mathbf{v}_{i,k}\|\|\hat{\mathbf{v}}_{i,j}\|}\,,$ (6) $$

The local loss for Type 1 neurons in layer $i$ (for $i=1,\dots,L-1$), denoted $\mathcal{L}_{i}$, is defined as:

$$ $\mathcal{L}_{i}=\sum_{k=1}^{B}(1-[\mathbf{C}_{i}]_{kk})^{2}+\lambda\sum_{k=1}^{B}\sum_{j\neq k}\left(\max(0,[\mathbf{C}_{i}]_{kj})\right)^{2},$ (7) $$

where $\lambda$ is a hyperparameter that controls the penalty strength for suppressing spurious correlations between non-corresponding voltage pairs.
The local loss for Type 2 neurons, $\mathcal{L}^{\prime}_{i}$, is defined analogously, with its formulation provided in Appendix [D](https://arxiv.org/html/2509.20284v1#A4).
Compared with InfoNCE loss commonly used in contrastive learning scenarios, ReCo loss avoids penalizing negatively correlated voltage pairs between $\mathbf{V}_{i}$ and $\hat{\mathbf{V}}_{i}$, thereby introducing enhanced flexibility and representational richness to learned embeddings while preserving alignment objectives.
Considering criterion C2 (local synaptic plasticity without global error signals), we employ detach() operations to truncate the computational graph between layers, ensuring that each neuron’s loss only propagates learning signals to synaptic weights connected to its dendrites. As a result, the gradient of the local loss $\mathcal{L}_{i}$ with respect to the feedforward weights $\mathbf{W}_{i-1}$ (for $i=2,\dots,L-1$) is given by:

$$ $\displaystyle\frac{\partial\mathcal{L}_{i}}{\partial\mathbf{W}_{i-1}}=\sum_{k=1}^{B}\Bigg[$ $\displaystyle-2(1-[\mathbf{C}_{i}]_{kk})\frac{1}{\|\mathbf{v}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}_{i,k}}{\|\hat{\mathbf{v}}_{i,k}\|}-[\mathbf{C}_{i}]_{kk}\frac{\mathbf{v}_{i,k}}{\|\mathbf{v}_{i,k}\|}\right)$ (8) $\displaystyle\quad+\sum_{j\neq k}2\lambda\max(0,[\mathbf{C}_{i}]_{kj})\frac{1}{\|\mathbf{v}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}_{i,j}}{\|\hat{\mathbf{v}}_{i,j}\|}-[\mathbf{C}_{i}]_{kj}\frac{\mathbf{v}_{i,k}}{\|\mathbf{v}_{i,k}\|}\right)\Bigg](\mathbf{s}_{i-1,k})^{T},$ $$

where $B$ denotes the batch size, $\mathbf{s}_{i-1,k}$ represents the spike output vector of Type 1 neurons from layer $i-1$ for the $k$-th sample, and $\mathcal{L}_{i}$ denotes the local loss for neurons in layer $i$.
The gradient update formula for $\bm{\Theta}$ takes an analogous form, and the detailed gradient derivations for both $\mathbf{W}$ and $\mathbf{\Theta}$ are presented in Appendix [D](https://arxiv.org/html/2509.20284v1#A4). The total loss for Type 1 neurons is defined as:

$$ $\mathcal{L}_{\text{total}}=\sum_{i=1}^{L-1}\mathcal{L}_{i}+\mathcal{L}_{\text{top}},$ (9) $$

where the top-layer loss is defined as $\mathcal{L}_{\text{top}}=\sum_{k=1}^{B}\mathcal{L}_{\text{CE}}(\mathbf{v}_{L,k},\hat{\mathbf{v}}_{L,k})$, where $\mathcal{L}_{\text{CE}}$ represents the cross-entropy loss. The total loss for Type 2 neurons, $\mathcal{L^{\prime}}_{\text{total}}$, is defined symmetrically.
As all information
processing can be separated within a single neuronal compartment, the feedforward and backward distillation processes do not require strict temporal separation and
can learn simultaneously, thereby satisfying C3 (non-two-stage training). For classification tasks, during inference, the predicted label for
input $\mathbf{x}$ is determined by computing cosine similarity between the ensemble of output spikes $\mathbf{s}_{L}$ and the spike trains corresponding to all candidate labels. The detailed training and inference procedures for RNN architectures are presented in Appendix [C](https://arxiv.org/html/2509.20284v1#A3).

### 4.4 Learning for generation tasks

We employ an autoencoder architecture for generation tasks, where both bottom-up and top-down inputs are images $\mathbf{x}$.
To capture fine-grained edge details while reducing noise artifacts, we apply Fast Fourier Transform (FFT) decomposition to the input images
as well as to all basal and apical membrane voltages.
This frequency-domain representation separates low- and high-frequency components, enabling adaptive loss computation.
Within each layer, the regularization parameter $\lambda$ is adjusted according to frequency content:
larger values are assigned to high-frequency components to suppress spurious correlations and preserve edge fidelity,
whereas smaller values are used for low-frequency components to avoid noise amplification and maintain structural coherence.
At the top layer, mean squared error is employed to compute the loss between basal and apical voltages.

## 5 Experiments

In this section, we first present the experimental settings and implementation details. We then evaluate our proposed BSD algorithm on image classification tasks,
comparing it against other biologically plausible learning algorithms. We assess BSD-trained RNNs on sequential regression tasks and evaluate BSD-trained autoencoders on image generation tasks. Finally,
we conduct ablation studies and analyze the network’s convergence properties and performance.

### 5.1 Experimental Settings

Image Classification.
We evaluate our BSD algorithm on widely used benchmarks including MNIST, FashionMNIST, SVHN, CIFAR-10, and CIFAR-100, using classification accuracy as the metric.

Text Character Prediction.
We conduct next-character prediction experiments with BSD-trained RNNs on the Harry Potter text corpus (Plath et al., 2019), reporting prediction accuracy.

Time-Series Forecasting.
BSD-trained RNNs are evaluated on three widely used real-world multivariate time-series forecasting datasets: Electricity (Lai et al., 2018), METR-LA (Li et al., 2018), and PEMS-BAY (Li et al., 2018), with performance primarily measured by mean squared error (MSE).

Image Generation.
BSD-trained autoencoders are applied to generation tasks on MNIST, FashionMNIST, and CIFAR-10, with generation quality assessed using Fréchet Inception Distance (FID) (Heusel et al., 2017).
Additional results and visualizations are provided in Appendix [F](https://arxiv.org/html/2509.20284v1#A6).

### 5.2 Implementation Details

To ensure fair comparison, identical network architectures are used across all learning algorithms for each task.
For MLPs, CNNs, RNNs, and autoencoders, the same configurations are applied when comparing methods on a given dataset.
Detailed dataset descriptions, architectural specifications, hyperparameter settings, and evaluation metrics are provided in Appendix [E](https://arxiv.org/html/2509.20284v1#A5).

**Table 1: Comparison of various learning algorithms in terms of biological plausibility criteria and image classification performance. Our proposed BSD algorithm satisfies all five criteria of biological plausibility (C1–C5) while achieving performance comparable to backpropagation. “C1, C2, C3, C4, C5” refer to the criteria defined in Section [4.1](https://arxiv.org/html/2509.20284v1#S4.SS1). Results are averaged over four random seeds.**
| Method | C1 | C2 | C3 | C4 | C5 | Model | MNIST | FashionMNIST | SVHN | CIFAR-10 | CIFAR100 | Avg. |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Backpropagation<br>on ANNs | ✗ | ✗ | ✗ | ✗ | ✗ | MLPs | $\bf 98.77\%$$\pm 0.33\%$ | $\bf 89.59\%$$\pm 0.14\%$ | $\bf 61.65\%$$\pm 0.42\%$ | $\bf 57.65\%$$\pm 0.08\%$ | $\bf 27.92\%$$\pm 0.17\%$ | $\bf 67.12\%$ |
|  |  | CNNs | $\bf 99.56\%$$\pm 0.14\%$ | $\bf 92.68\%$$\pm 0.42\%$ | $\bf 94.36\%$$\pm 0.73\%$ | $\bf 87.12\%$$\pm 1.76\%$ | $\bf 57.75\%$$\pm 0.35\%$ | $\bf 86.29\%$ |  |  |  |  |
| Backpropagation<br>on SNNs | ✗ | ✗ | ✗ | ✓ | ✗ | MLPs | $98.57\%$$\pm 0.24\%$ | $88.87\%$$\pm 0.65\%$ | $61.28\%$$\pm 0.52\%$ | $49.95\%$$\pm 0.53\%$ | $23.32\%$$\pm 0.87\%$ | $64.40\%$ |
|  |  | CNNs | $99.25\%$$\pm 0.02\%$ | $92.48\%$$\pm 0.48\%$ | $94.13\%$$\pm 0.02\%$ | $87.02\%$$\pm 0.09\%$ | $57.21\%$$\pm 0.34\%$ | $86.02\%$ |  |  |  |  |
| Predictive Coding | ✓ | ✓ | ✓ | ✗ | ✗ | MLPs | $98.42\%$$\pm 0.13\%$ | $88.72\%$$\pm 0.65\%$ | $59.05\%$$\pm 0.45\%$ | $47.34\%$$\pm 0.24\%$ | $19.72\%$$\pm 0.32\%$ | $62.65\%$ |
|  |  | CNNs | $99.41\%$$\pm 0.40\%$ | $92.03\%$$\pm 0.70\%$ | $94.53\%$$\pm 1.54\%$ | $72.94\%$$\pm 0.32\%$ | $53.08\%$$\pm 0.43\%$ | $82.40\%$ |  |  |  |  |
| CCL | ✓ | ✓ | ✓ | ✗ | ✗ | MLPs | $98.13\%$$\pm 0.10\%$ | $88.58\%$$\pm 0.29\%$ | $60.98\%$$\pm 0.23\%$ | $52.73\%$$\pm 0.59\%$ | $21.76\%$$\pm 0.22\%$ | $64.44\%$ |
|  |  | CNNs | $96.30\%$$\pm 0.05\%$ | $83.70\%$$\pm 0.57\%$ | $88.78\%$$\pm 0.37\%$ | $82.94\%$$\pm 0.53\%$ | $56.29\%$$\pm 0.25\%$ | $81.60\%$ |  |  |  |  |
| DLL | ✓ | ✓ | ✓ | ✗ | ✗ | MLPs | $97.57\%$$\pm 0.40\%$ | $87.50\%$$\pm 0.43\%$ | $56.60\%$$\pm 0.12\%$ | $45.87\%$$\pm 0.10\%$ | $18.24\%$$\pm 0.18\%$ | $61.16\%$ |
|  |  | CNNs | $98.87\%$$\pm 0.30\%$ | $90.88\%$$\pm 0.40\%$ | $85.81\%$$\pm 0.17\%$ | $70.89\%$$\pm 0.58\%$ | $38.60\%$$\pm 0.21\%$ | $77.01\%$ |  |  |  |  |
| R-STDP | ✓ | ✓ | ✓ | ✓ | ✓ | MLPs | $77.18\%$$\pm 0.17\%$ | $70.03\%$$\pm 0.28\%$ | $41.76\%$$\pm 0.46\%$ | $22.68\%$$\pm 0.30\%$ | $1.33\%$$\pm 0.15\%$ | $42.58\%$ |
|  |  | CNNs | $91.67\%$$\pm 0.04\%$ | $74.29\%$$\pm 0.30\%$ | $50.02\%$$\pm 0.32\%$ | $33.19\%$$\pm 0.38\%$ | $1.49\%$$\pm 0.22\%$ | $50.10\%$ |  |  |  |  |
|  |  |  |  |  |  | MLPs | $\bf 95.62\%$$\pm 0.09\%$ | $\bf 86.39\%$$\pm 0.13\%$ | $\bf 60.40\%$$\pm 0.18\%$ | $\bf 48.90\%$$\pm 0.54\%$ | $\bf 22.10\%$$\pm 0.25\%$ | $\bf 62.68\%$ |
| BSD (Ours) | ✓ | ✓ | ✓ | ✓ | ✓ | CNNs | $\bf 99.44\%$$\pm 0.03\%$ | $\bf 91.05\%$$\pm 0.20\%$ | $\bf 90.81\%$$\pm 0.11\%$ | $\bf 84.13\%$$\pm 0.34\%$ | $\bf 53.48\%$$\pm 0.22\%$ | $\bf 83.78\%$ |

### 5.3 Image Classification

We benchmark the performance of backpropagation on Artificial Neural Networks (ANNs) and Spiking Neural Networks (SNNs),
Predictive Coding, Reward-modulated Spike-Timing-Dependent Plasticity (R-STDP) (Izhikevich, 2007),
Counter-Current Learning (CCL),
Dendritic Localized Learning (DLL), and our BSD algorithm on image classification tasks.
Comprehensive results are presented in Table [1](https://arxiv.org/html/2509.20284v1#S5.T1), with additional implementation details provided in Appendix [E.2](https://arxiv.org/html/2509.20284v1#A5.SS2).

Spiking neuron outputs degrade performance compared to continuous activations.
When trained with backpropagation, Spiking Neural Networks (SNNs) consistently exhibit inferior performance compared to Artificial Neural Networks (ANNs) across all evaluated datasets and network architectures.
This performance degradation highlights the disadvantage of using spiking neurons, which emit binary 0–1 spikes, as opposed to neurons that produce continuous-valued activations.

Our proposed BSD algorithm reconciles biological plausibility with competitive performance.
BSD satisfies all five criteria of biological plausibility (C1–C5) while achieving performance comparable to backpropagation,
with stable convergence across diverse datasets.
In particular, on challenging datasets such as SVHN, CIFAR-10, and CIFAR-100, and on more complex architectures including CNNs,
BSD consistently delivers robust performance.
Taken together, these findings underscore the feasibility of attaining biological plausibility without sacrificing task performance.

**Table 2: Comparison of different learning algorithms for RNN training on text character prediction and time-series forecasting tasks. Our proposed BSD algorithm achieves performance comparable to backpropagation. $\uparrow$ ($\downarrow$) denotes higher (lower) values indicate better performance. All results are averaged across $4$ random seeds. The best results and the results of BSD are presented in bold.**
| Method | Harry Potter | Electricity | Metr-la | Pems-bay |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Pred. Acc. $\uparrow$ | MSE $\downarrow$ | MAE $\downarrow$ | MSE $\downarrow$ | MAE $\downarrow$ | MSE $\downarrow$ | MAE $\downarrow$ |  |
| Backpropagation on ANNs | $\bf 51.9\%$$\pm 1.0\%$ | $0.175$$\pm 0.007$ | $0.324$$\pm 0.007$ | $0.131$$\pm 0.004$ | $0.214$$\pm 0.005$ | $\bf 0.164$$\pm 0.001$ | $\bf 0.190$$\pm 0.002$ |
| Backpropagation on SNNs | $27.8\%$$\pm 0.9\%$ | $0.169$$\pm 0.018$ | $0.316$$\pm 0.021$ | $0.154$$\pm 0.014$ | $0.243$$\pm 0.022$ | $0.166$$\pm 0.005$ | $0.201$$\pm 0.002$ |
| Predictive Coding | $38.8\%$$\pm 1.8\%$ | $\bf 0.162$$\pm 0.019$ | $\bf 0.312$$\pm 0.018$ | $0.141$$\pm 0.001$ | $0.228$$\pm 0.005$ | $0.178$$\pm 0.004$ | $0.202$$\pm 0.003$ |
| DLL | $33.7\%$$\pm 0.6\%$ | $0.172$$\pm 0.018$ | $0.321$$\pm 0.013$ | $0.155$$\pm 0.005$ | $0.264$$\pm 0.001$ | $0.178$$\pm 0.005$ | $0.224$$\pm 0.004$ |
| BSD (Ours) | $\bf 41.8\%$$\pm 0.1\%$ | $\bf 0.165$$\pm 0.018$ | $\bf 0.314$$\pm 0.018$ | $\bf 0.125$$\pm 0.005$ | $\bf 0.197$$\pm 0.005$ | $\bf 0.174$$\pm 0.007$ | $\bf 0.206$$\pm 0.013$ |

### 5.4 Sequential Regression Tasks

We evaluate BSD on four representative sequential regression tasks, comparing its performance against backpropagation on ANNs and SNNs,
Predictive Coding, and Dendritic Localized Learning (DLL).
Comprehensive results are reported in Table [2](https://arxiv.org/html/2509.20284v1#S5.T2).

For time-series forecasting, we utilize the Electricity, Metr-la, and Pems-bay datasets.
Predictive Coding satisfies only the second biological plausibility criterion, whereas DLL meets the first three.
In contrast, BSD satisfies all five criteria while achieving performance comparable to backpropagation,
with RNNs trained under BSD converging reliably across all forecasting tasks.

For text character prediction, experiments are conducted on the Harry Potter corpus.
As shown in Table [2](https://arxiv.org/html/2509.20284v1#S5.T2), BSD again converges successfully, outperforming both Predictive Coding and DLL.

Together, these results demonstrate that BSD effectively captures temporal dependencies in sequential regression tasks
while remaining consistent with biologically plausible learning principles.

Figure: Figure 2: Results on image generation tasks using autoencoders trained with backpropagation and BSD. GT denotes the ground truth, BP indicates backpropagation, and BSD refers to our proposed method. The reconstruction quality demonstrates that BSD attains performance comparable to backpropagation on generation tasks, while preserving biological plausibility
Refer to caption: graph/ae_result.png

### 5.5 Image Generation

Figure [2](https://arxiv.org/html/2509.20284v1#S5.F2) presents a comparison of image reconstruction on CIFAR-10 across three approaches: BP-trained autoencoders, the Fully Spiking Variational Autoencoder (FSVAE) (Kamata et al., 2022), and BSD-trained autoencoders.
Generation quality is evaluated using the Fréchet Inception Distance (FID) (Heusel et al., 2017), a widely adopted metric for assessing the quality of generative models.

**Table 3: Performance comparison of different algorithms on image generation tasks. $\downarrow$ denotes lower values indicate better performance.**
| Dataset | Model | FID $\downarrow$ |
| --- | --- | --- |
| CIFAR-10 | ANN-BP | $\mathbf{127.34}$ |
| FSVAE | $175.5$ |  |
| BSD (Ours) | $\mathbf{168.12}$ |  |
| MNIST | ANN-BP | $\mathbf{49.56}$ |
| FSVAE | $97.06$ |  |
| BSD (Ours) | $\mathbf{72.39}$ |  |

The visual results suggest that BSD adapts robustly to generative tasks and produces reconstruction quality closely comparable to that achieved with backpropagation.
Table [3](https://arxiv.org/html/2509.20284v1#S5.T3) reports FID scores on CIFAR-10 and MNIST, indicating that BSD attains competitive performance across both datasets and thereby further demonstrating its suitability for generative modeling while maintaining biological plausibility.
Additional visualizations and results, including experiments on FashionMNIST, are presented in Appendix [F.1](https://arxiv.org/html/2509.20284v1#A6.SS1).

### 5.6 Ablation Study

As mentioned in Section [4](https://arxiv.org/html/2509.20284v1#S4), inspired by contrastive learning, we adopt ReCo loss as our layer-wise loss
function. Here, we delve deeper into the impact of layer-wise loss function selection, analyzing how BSD performs when
using MSE or InfoNCE (van den Oord et al., 2018), a loss commonly used in contrastive learning scenarios, for aligning
intraneuronal voltages. We term the method using MSE as layer-wise loss ”BSD-MSE” and the method using InfoNCE as
”BSD-InfoNCE,” and conduct ablation experiments on both image classification and sequential regression tasks.

**Table 4: Ablation study of loss functions on image classification tasks.**
| Method | Model | MNIST | FashionMNIST | SVHN | CIFAR-10 | CIFAR-100 |
| --- | --- | --- | --- | --- | --- | --- |
| BSD-MSE | MLPs | $12.51\%$ | $13.72\%$ | $19.11\%$ | $11.49\%$ | $1.39\%$ |
|  | CNNs | $21.10\%$ | $29.31\%$ | $19.46\%$ | $16.93\%$ | $1.58\%$ |
| BSD-InfoNCE | MLPs | $94.56\%$ | $85.71\%$ | $57.33\%$ | $43.77\%$ | $19.25\%$ |
|  | CNNs | $98.77\%$ | $88.97\%$ | $83.27\%$ | $72.38\%$ | $38.06\%$ |
|  | MLPs | $\bf 95.62\%$ | $\bf 86.39\%$ | $\bf 60.40\%$ | $\bf 48.90\%$ | $\bf 22.10\%$ |
| BSD | CNNs | $\bf 99.44\%$ | $\bf 91.05\%$ | $\bf 90.81\%$ | $\bf 84.13\%$ | $\bf 53.48\%$ |

Table [4](https://arxiv.org/html/2509.20284v1#S5.T4) compares the performance of BSD, BSD-MSE, and BSD-InfoNCE on image classification tasks.
BSD-MSE yields markedly inferior results: MLPs attain only 19.11% accuracy on SVHN and fail to converge on other datasets,
while CNNs exhibit substantial performance gaps across all tasks and do not converge on CIFAR-100.
BSD-InfoNCE converges reliably on all datasets, but BSD with ReCo loss consistently achieves higher accuracy.
On complex benchmarks such as CIFAR-100, BSD surpasses BSD-InfoNCE by more than 15 accuracy points.
Additional ablation experiments for text character prediction, time-series forecasting, and image generation tasks are provided in Appendix [F](https://arxiv.org/html/2509.20284v1#A6).

To conclude, employing MSE loss for intraneuronal voltage alignment leads to convergence difficulties, indicating that it is ill-suited for BSD training.
Moreover, compared to InfoNCE, ReCo loss offers a distinct advantage by not penalizing unpaired samples that are already orthogonal or negatively correlated,
which allows for more flexible alignment and richer feature representations.

Figure: (a)
Refer to caption: x2.png

### 5.7 Training Analysis

In this section, we analyze the convergence behavior of BSD-trained models and the properties of Type 1 and Type 2 neurons during training.
Since BSD applies ReCo loss to all layers except the top one and relies on spikes for inter-neuronal communication,
both the penalty weight $\lambda$ in ReCo loss and the neuronal firing threshold are critical factors influencing its convergence.

Figure [3](https://arxiv.org/html/2509.20284v1#S5.F3)(a) shows the effect of the penalty weight $\lambda$ on BSD-trained CNNs evaluated on the SVHN dataset.
Performance drops markedly when $\lambda=0$, underscoring the role of $\lambda$ in promoting separation among sample representations
and thereby enlarging the representational space.
We further observe that smaller values of $\lambda$ accelerate convergence, while the best final performance is achieved at $\lambda=0.6$.
Figure [3](https://arxiv.org/html/2509.20284v1#S5.F3)(b) illustrates the relationship among neuronal firing threshold, spike sparsity, and network performance.
The network attains its best performance at a firing threshold of 0.6, suggesting that thresholds that are either too low or too high impair effective learning.
Figure [3](https://arxiv.org/html/2509.20284v1#S5.F3)(c) shows the Hamming similarity between the spike trains of Type 1 and Type 2 neurons within the same layer.
The similarity rises quickly during training and exceeds 0.85 by epoch 20, demonstrating that the feedforward (stimuli-to-decision) and backward (concept-to-stimuli) pathways successfully achieve
mutual alignment of their spiking feature representations through bidirectional distillation, thereby validating the effectiveness of our joint training framework.

For completeness, we provide $t$-SNE visualizations of representations on BSD-trained CNNs in Appendix [F.5](https://arxiv.org/html/2509.20284v1#A6.SS5)
and examine the degree of alignment between weights $\mathbf{W}$ and $\bm{\Theta}$ in Appendix [F.4](https://arxiv.org/html/2509.20284v1#A6.SS4).

## 6 Conclusion

Human learning and cognition emerge from bidirectional processes that integrate bottom-up sensory perception with top-down memory recall.
In this framework, the feedforward network supports perception and decision-making by transforming sensory stimuli into conceptual representations, while the feedback network facilitates memory recall by reconstructing stimuli from semantic concepts.
Inspired by this principle, we propose a novel bidirectional learning framework in which the feedforward and feedback networks are jointly trained by distilling their hidden feature representations from one another. This approach leverages the properties of pyramidal neurons, which receive feedforward (perception) and feedback (learning) signals through their basal and apical dendrites, respectively.
Extensive experiments across diverse network architectures and tasks show that the resulting learning algorithm achieves performance comparable to error backpropagation while yielding stronger adherence to biological fidelity.
Specifically, BSD achieves competitive performance across tasks: within 3% of backpropagation on MNIST (99.44% vs
99.56%) and CIFAR-10 (84.13% vs 87.12%), and superior MSE on time-series forecasting tasks like Electricity (0.165 vs
0.175), demonstrating the effectiveness of our approach. The limitations and future directions are discussed in Appendix [G](https://arxiv.org/html/2509.20284v1#A7).

## Ethics Statement

This research presents a biologically plausible learning algorithm and does not involve human subjects or sensitive data.
All experiments were conducted on publicly available datasets that are widely used in machine learning research.
The proposed BSD algorithm is a general-purpose learning method without inherent bias, unethical practices, or discriminatory applications.
The authors declare no conflicts of interest or competing financial interests related to this work.
The research methodology adheres to well-established standard practices in machine learning and computational science,
with no foreseeable harmful implications or misuse potential.

## Reproducibility Statement

The authors have made extensive efforts to ensure the reproducibility of the empirical results reported in this paper. First, detailed dataset characteristics are
documented in Appendix [E.1](https://arxiv.org/html/2509.20284v1#A5.SS1). Second, comprehensive implementation details, covering network architectures, hyperparameter settings, training procedures, and evaluation metric explanations, are provided in Appendix [E.2](https://arxiv.org/html/2509.20284v1#A5.SS2).
Finally, the authors have included the source code for the proposed training algorithm as supplementary material and promised to release it publicly on GitHub upon acceptance.

## References

- Ackley et al. (1985)
David H. Ackley, Geoffrey E. Hinton, and Terrence J. Sejnowski.
A Learning Algorithm for Boltzmann Machines.
*Cognitive Science*, 9:147–169, 1985.
doi: 10.1207/s15516709cog0901˙7.
- Albers et al. (2013)
Alexandra M Albers, Peter Kok, Ivan Toni, H Chris Dijkerman, and Floris P de Lange.
Shared representations for working memory and mental imagery in early visual cortex.
*Current Biology*, 23(15):1427–1431, 2013.
- Bartunov et al. (2018)
Sergey Bartunov, Adam Santoro, Blake A Richards, Luke Marris, Geoffrey E Hinton, and Timothy Lillicrap.
Assessing the scalability of biologically-motivated deep learning algorithms and architectures.
*Advances in Neural Information Processing Systems*, 31, 2018.
- Bengio (2014)
Yoshua Bengio.
How auto-encoders could provide credit assignment in deep networks via target propagation.
*arXiv preprint arXiv:1407.7906*, 2014.
- Bonetti et al. (2024)
Leonardo Bonetti, Gemma Fernández-Rubio, Francesca Carlomagno, Mathias Dietz, Dimitrios Pantazis, Peter Vuust, and Morten L Kringelbach.
Spatiotemporal brain hierarchies of auditory memory recognition and predictive coding.
*Nature Communications*, 15(1):4313, 2024.
- Caucheteux et al. (2023)
Charlotte Caucheteux, Alexandre Gramfort, and Jean-Rémi King.
Evidence of a predictive coding hierarchy in the human brain listening to speech.
*Nature Human Behaviour*, 7(3):430–441, 2023.
- Crick (1989)
Francis Crick.
The recent excitement about neural networks.
*Nature*, 337(6203):129–132, 1989.
- Cubuk et al. (2020)
Ekin Dogus Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V. Le.
Randaugment: Practical automated data augmentation with a reduced search space.
In *Advances in Neural Information Processing Systems*, volume 33, 2020.
- Donald (1949)
Hebb Donald.
The organization of behavior.
*New York 1952 Donald The Organization of Behaviour 1952*, 1949.
- Fang et al. (2021)
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian.
Incorporating learnable membrane time constant to enhance learning of spiking neural networks.
In *Proceedings of the IEEE/CVF international conference on computer vision*, pp. 2661–2671, 2021.
- Gerstner et al. (2014)
Wulfram Gerstner, Werner M Kistler, Richard Naud, and Liam Paninski.
*Neuronal dynamics: from single neurons to networks and models of cognition*.
Cambridge University Press, 2014.
- Guerguiev et al. (2017)
Jordan Guerguiev, Timothy P Lillicrap, and Blake A Richards.
Towards deep learning with segregated dendrites.
*eLife*, 6:e22566, 2017.
- Hayden et al. (2011)
Benjamin Y Hayden, Sarah R Heilbronner, John M Pearson, and Michael L Platt.
Surprise signals in anterior cingulate cortex: neuronal encoding of unsigned reward prediction errors driving adjustment in behavior.
*Journal of Neuroscience*, 31(11):4178–4187, 2011.
- Hazan & Ezra Tsur (2021)
Avi Hazan and Elishai Ezra Tsur.
Neuromorphic analog implementation of neural engineering framework-inspired spiking neuron for high-dimensional representation.
*Frontiers in Neuroscience*, 15:627221, 2021.
- Heusel et al. (2017)
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.
Gans trained by a two time-scale update rule converge to a local nash equilibrium.
In *Advances in neural information processing systems*, pp. 6626–6637, 2017.
- Hinton (2022)
Geoffrey Hinton.
The forward-forward algorithm: Some preliminary investigations.
*arXiv preprint arXiv:2212.13345*, 2(3):5, 2022.
- Hinton (2002)
Geoffrey E. Hinton.
Training Products of Experts by Minimizing Contrastive Divergence.
*Neural Computation*, 14(8):1771–1800, 2002.
doi: 10.1162/089976602760128018.
- Hopfield (1984)
John J Hopfield.
Neurons with graded response have collective computational properties like those of two-state neurons.
*Proceedings of the national academy of sciences*, 81(10):3088–3092, 1984.
- Izhikevich (2007)
Eugene M Izhikevich.
Solving the distal reward problem through linkage of stdp and dopamine signaling.
*Cerebral Cortex*, 17(10):2443–2452, 2007.
- Jiao et al. (2022)
Hou Jiao et al.
The new generation brain-inspired sparse learning: A comprehensive survey.
*Information Fusion*, 89:35–57, 2022.
- Kamata et al. (2022)
Hiromichi Kamata, Yusuke Mukuta, and Tatsuya Harada.
Fully spiking variational autoencoder.
In *Proceedings of the AAAI conference on artificial intelligence*, volume 36, pp. 7059–7067, 2022.
- Kao & Hariharan (2024)
Chia-Hsiang Kao and Bharath Hariharan.
Counter-current learning: A biologically plausible dual network approach for deep learning.
*Advances in Neural Information Processing Systems*, 37:70905–70925, 2024.
- Kosslyn et al. (1993)
Stephen M Kosslyn, Nathaniel M Alpert, William L Thompson, Vera Maljkovic, Steven B Weise, Christopher F Chabris, Susan E Hamilton, Scott L Rauch, and Fabio S Buonanno.
Activation of human primary visual cortex during visual recall: a magnetic resonance imaging study.
*Proceedings of the National Academy of Sciences*, 90(24):11802–11805, 1993.
- Lai et al. (2018)
Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu.
Modeling long-and short-term temporal patterns with deep neural networks.
In *The 41st international ACM SIGIR conference on research & development in information retrieval*, pp. 95–104, 2018.
- LeCun et al. (2006)
Yann LeCun, Sumit Chopra, Raia Hadsell, Marc’Aurelio Ranzato, and Fu Jie Huang.
A Tutorial on Energy-Based Learning.
In Gökhan Bakir, Thomas Hofmann, Bernhard Schölkopf, Alexander Smola, and Ben Taskar (eds.), *Predicting Structured Data*. MIT Press, 2006.
An extended tutorial introducing energy-based models and their relation to Hopfield networks and Boltzmann machines.
- LeCun et al. (2015)
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton.
Deep learning.
*Nature*, 521(7553):436–444, 2015.
- Lee et al. (2015)
Dong-Hyun Lee, Saizheng Zhang, Asja Fischer, and Yoshua Bengio.
Difference target propagation.
In *Joint european conference on machine learning and knowledge discovery in databases*, pp. 498–515. Springer, 2015.
- Li et al. (2024)
X Li et al.
A review of learning in biologically plausible spiking neural networks.
*Frontiers in Neuroscience*, 2024.
- Li et al. (2018)
Yaguang Li, Rose Yu, Cyrus Shahabi, and Yan Liu.
Diffusion convolutional recurrent neural network: Data-driven traffic forecasting.
In *International conference on learning representations*, 2018.
- Lillicrap et al. (2020)
Timothy P Lillicrap, Adam Santoro, Luke Marris, Colin J Akerman, and Geoffrey Hinton.
Backpropagation and the brain.
*Nature Reviews Neuroscience*, 21(6):335–346, 2020.
- Lin et al. (2023)
Zudi Lin, Erhan Bas, Kunwar Yashraj Singh, Gurumurthy Swaminathan, and Rahul Bhotika.
Relaxing contrastiveness in multimodal representation learning.
In *Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision*, pp. 2227–2236, 2023.
- Lv et al. (2025)
Changze Lv, Jingwen Xu, Yiyang Lu, Xiaohua Wang, Zhenghua Wang, Zhibo Xu, Di Yu, Xin Du, Xiaoqing Zheng, and Xuanjing Huang.
Dendritic localized learning: Toward biologically plausible algorithm.
In *Forty-second International Conference on Machine Learning*, 2025.
- Maass (1997)
Wolfgang Maass.
Networks of spiking neurons: the third generation of neural network models.
*Neural Networks*, 10(9):1659–1671, 1997.
- Payeur et al. (2021)
Alexandre Payeur, Jordan Guerguiev, Friedemann Zenke, Blake A Richards, and Richard Naud.
Burst-dependent synaptic plasticity can coordinate learning in hierarchical circuits.
*Nature Neuroscience*, 24:1010–1019, 2021.
- Pfeiffer & Pfeil (2018)
Michael Pfeiffer and Thomas Pfeil.
Deep learning with spiking neurons: Opportunities and challenges.
*Frontiers in neuroscience*, 12:409662, 2018.
- Plath et al. (2019)
James Plath, Gail Sinclair, and Kirk Curnutt.
*The 100 Greatest Literary Characters*.
Rowman & Littlefield, 2019.
- Rumelhart et al. (1986)
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams.
Learning representations by back-propagating errors.
*Nature*, 323(6088):533–536, 1986.
- Sacramento et al. (2018)
João Sacramento, Rui Ponte Costa, Yoshua Bengio, and Walter Senn.
Dendritic cortical microcircuits approximate the backpropagation algorithm.
*Advances in Neural Information Processing Systems*, 31, 2018.
- Schmidgall et al. (2024)
Samuel Schmidgall, Rojin Ziaei, Jascha Achterberg, Louis Kirsch, S Hajiseyedrazi, and Jason Eshraghian.
Brain-inspired learning in artificial neural networks: a review.
*APL Machine Learning*, 2(2), 2024.
- Senn et al. (2024)
Walter Senn, Dominik Dold, Akos F Kungl, Benjamin Ellenberger, Jakob Jordan, Yoshua Bengio, João Sacramento, and Mihai A Petrovici.
A neuronal least-action principle for real-time learning in cortical circuits.
*eLife*, 12:RP89674, 2024.
- Song et al. (2000)
Sen Song, Kenneth D Miller, and Larry F Abbott.
Competitive hebbian learning through spike-timing-dependent synaptic plasticity.
*Nature neuroscience*, 3(9):919–926, 2000.
- Stork (1989)
Stork.
Is backpropagation biologically plausible.
In *International 1989 Joint Conference on Neural Networks*, pp. 241–246. IEEE, 1989.
- van den Oord et al. (2018)
Aaron van den Oord, Yazhe Li, and Oriol Vinyals.
Representation learning with contrastive predictive coding.
*arXiv preprint arXiv:1807.03748*, 2018.
- Wu et al. (2021)
Zhenzhi Wu, Hehui Zhang, Yihan Lin, Guoqi Li, Meng Wang, and Ye Tang.
Liaf-net: Leaky integrate and analog fire network for lightweight and efficient spatiotemporal information processing.
*IEEE Transactions on Neural Networks and Learning Systems*, 33(11):6249–6262, 2021.
- Zhang et al. (2021)
Tielin Zhang, Shuncheng Jia, Xiang Cheng, and Bo Xu.
Tuning convolutional spiking neural network with biologically plausible reward propagation.
*IEEE Transactions on Neural Networks and Learning Systems*, 33(12):7621–7631, 2021.

## Appendix A Global Algorithm of BSD

Figure: Algorithm 1 Spike Bidirectional Distillation Algorithm

## Appendix B Additional Preliminaries

Spiking neurons can be categorized into two types based on their signal propagation mechanisms. One category, referred to as spike-based neurons, conveys information through spike trains. Discrete Spiking Neural Networks (SNNs) integrate spike inputs at each time step and generate a binary spike output when the integrated value exceeds a threshold. Continuous analog input signals are typically encoded (e.g., rate encoding, time encoding, or $\Delta$-encoding) into spike sequences. Representative models in this category include IF, PLIF(Fang et al., 2021), and LIF(Maass, 1997). Discrete SNNs are compatible with existing digital design processes, allowing the grouping and virtualization of large numbers of neurons on computational cores to achieve parallel architectures. These systems exhibit high scalability and efficient trainability, benefiting from sparse, event-driven computation with low power consumption during the inference phase. However, the discrete nature of spike signals limits their information capacity relative to traditional neuron models, which transmit continuous floating-point values directly(Pfeiffer & Pfeil, 2018).

The second type of spiking neuron architecture uses similar temporal equations as spiking neurons but propagates analog signals, referred to as analog-based neurons, such as LIAF(Wu et al., 2021). These signals can encode more information and exhibit lower response latency in certain tasks. However, hardware implementations of such neurons require analog circuits and because computations cannot be skipped during inference, the system exhibits higher power consumption (Hazan & Ezra Tsur, 2021).

In Section [3.1](https://arxiv.org/html/2509.20284v1#S3.SS1), we provide the time dynamics equation for LIF. The equation [1](https://arxiv.org/html/2509.20284v1#S3.E1) and equation [3](https://arxiv.org/html/2509.20284v1#S3.E3) for different types of spiking neurons are similar, while equation [2](https://arxiv.org/html/2509.20284v1#S3.E2) exhibits multiple variations. For example, the equation for IF is defined as follows:

$$ $\displaystyle H[t]$ $\displaystyle=U[t-1]+I[t]$ (10) $$

The primary distinction between IF and LIF lies in the inclusion of a voltage leakage mechanism in LIF, which more closely resembles the behavior of biological neurons. Other variants, such as PLIF, consider the parameter $\tau$ in equation [2](https://arxiv.org/html/2509.20284v1#S3.E2) to be learnable, while LIAF replaces $\Theta$ in equation [3](https://arxiv.org/html/2509.20284v1#S3.E3) with ReLU, thereby enabling the transmission of continuous floating-point values.

## Appendix C The Model Architecture of RNNs

The network comprises one layer of auto-regressive pyramidal neurons, which contains a pair of two distinct types of neurons.
Type 1 neurons receive synaptic inputs on their basal dendrites from axons of lower-layer neurons, while Type 2 neurons receive basal dendritic inputs from axons of higher-layer neurons. $\mathbf{x}_{1},\mathbf{x}_{2},\dots,\mathbf{x}_{N}$ represent the input sequence where $N$ is the sequence length.
The training objective is to align the output $\mathbf{o}_{i}$ of each timestep Type 1 neurons with $\hat{\mathbf{y}}_{i}$ at each timestep $i$. During learning, the original input $\mathbf{x}_{i}$ is delivered to the basal dendrites of Type 1 neurons at timestep $i$, while the target $\hat{\mathbf{y}}_{i}$ is provided to the basal dendrites of Type 2 neurons.

We employ synaptic weight matrices $\mathbf{W}_{\text{ih}}$,$\mathbf{W}_{\text{hh}}$ and $\mathbf{W}_{\text{ho}}$ in the feedforward pathway corresponding to the input to the hidden layer weight, the spike of the previous time to the hidden layer weight and the hidden layer to the output weight respectively. Similar to the feedforward pathway, $\bm{\Theta}_{\text{oh}}$,$\bm{\Theta}_{\text{hh}}$ and $\bm{\Theta}_{\text{hi}}$ in the backward pathways corresponding to the output target to the hidden layer weight, the spike of the previous time to the hidden layer weight and the hidden layer to the reconstructed input weight respectively.

The feedforward process is described by:

$$ $\quad\mathbf{h}_{i}=\hat{\mathbf{h}}^{\prime}_{i}=\mathbf{W}_{\text{ih}}\mathbf{x}_{i}+\mathbf{W}_{\text{hh}}\mathbf{s}_{i-1},\quad\mathbf{o}_{i}=\mathbf{W}_{\text{ho}}\mathbf{h}_{i},\quad\mathbf{s}_{i}=\mathcal{SN}(\mathbf{h}_{i}),\quad i=2,3,\ldots,N,$ (11) $$

where $\mathbf{h}_{i}$ denotes the somatic membrane potential of Type 1 neurons at timestep $i$ arising from basal dendritic integration, $\hat{\mathbf{h}}^{\prime}_{i}$
denotes the somatic membrane potential of Type 2 neurons at timestep $i$ arising from apical dendritic integration. $\mathbf{s}_{i}$ denotes the spike train that Type 1 neurons at timestep $i$ output and $\mathbf{s}_{1}=0$.

The backward process is governed by:

$$ $\quad\mathbf{h}^{\prime}_{i}=\hat{\mathbf{h}}_{i}=\bm{\Theta}_{\text{oh}}\hat{\mathbf{y}}_{i}+\bm{\Theta}_{\text{hh}}\mathbf{s}^{\prime}_{i+1},\quad\hat{\mathbf{x}}_{i}=\bm{\Theta}_{\text{hi}}\hat{\mathbf{h}}_{i},\quad\mathbf{s}^{\prime}_{i}=\mathcal{SN}(\hat{\mathbf{h}}_{i}),\quad i=1,2,\ldots,N-1,$ (12) $$

where $\mathbf{h}^{\prime}_{i}$ denotes the somatic membrane potential of Type 2 neurons at timestep $i$ arising from basal dendritic integration, $\hat{\mathbf{h}}_{i}$
denotes the somatic membrane potential of Type 1 neurons at timestep $i$ arising from apical dendritic integration. $\bm{\Theta}_{i}$ is the synaptic weight for Type 2 neurons in layer $i$, and $\mathbf{s}_{i}^{\prime}$ denotes the spike train that Type 2 neurons at timestep $i$ output and $\mathbf{s}^{\prime}_{N}=0$.

## Appendix D Gradients Derivation for BSD

This section details the derivation of the gradients for the local loss functions with respect to the feedforward weights $\mathbf{W}$ and backward weights $\bm{\Theta}$.

### D.1 Gradient with respect to Feedforward Weights 𝐖 i − 1 \mathbf{W}_{i-1} (Type 1 Neurons)

For Type 1 neurons in layer $i$ (for $i=2,\dots,L-1$), the local loss $\mathcal{L}_{i}$ is defined as:

$$ $\mathcal{L}_{i}=\sum_{k=1}^{B}(1-[\mathbf{C}_{i}]_{kk})^{2}+\lambda\sum_{k=1}^{B}\sum_{j\neq k}\left(\max(0,[\mathbf{C}_{i}]_{kj})\right)^{2},$ (13) $$

where $[\mathbf{C}_{i}]_{kj}$ is the cosine similarity between $\mathbf{v}_{i,k}=\mathbf{W}_{i-1}\mathbf{s}_{i-1,k}$ and $\hat{\mathbf{v}}_{i,j}=\mathbf{v}^{\prime}_{i,j}$.

The gradient with respect to $\mathbf{W}_{i-1}$ is found via the chain rule. The resulting gradient is computed as a sum of outer products over the batch:

$$ $\frac{\partial\mathcal{L}_{i}}{\partial\mathbf{W}_{i-1}}=\sum_{k=1}^{B}\left(\frac{\partial\mathcal{L}_{i}}{\partial\mathbf{v}_{i,k}}\right)(\mathbf{s}_{i-1,k})^{T}$ (14) $$

The gradient with respect to the basal voltage $\mathbf{v}_{i,k}$ is:

$$ $\frac{\partial\mathcal{L}_{i}}{\partial\mathbf{v}_{i,k}}=-2(1-[\mathbf{C}_{i}]_{kk})\frac{\partial[\mathbf{C}_{i}]_{kk}}{\partial\mathbf{v}_{i,k}}+\sum_{j\neq k}2\lambda\max(0,[\mathbf{C}_{i}]_{kj})\frac{\partial[\mathbf{C}_{i}]_{kj}}{\partial\mathbf{v}_{i,k}}$ (15) $$

The derivative of the cosine similarity term is:

$$ $\frac{\partial[\mathbf{C}_{i}]_{kj}}{\partial\mathbf{v}_{i,k}}=\frac{1}{\|\mathbf{v}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}_{i,j}}{\|\hat{\mathbf{v}}_{i,j}\|}-[\mathbf{C}_{i}]_{kj}\frac{\mathbf{v}_{i,k}}{\|\mathbf{v}_{i,k}\|}\right)$ (16) $$

Combining these terms yields the final gradient for $\mathbf{W}_{i-1}$:

$$ $\displaystyle\frac{\partial\mathcal{L}_{i}}{\partial\mathbf{W}_{i-1}}=\sum_{k=1}^{B}\Bigg[$ $\displaystyle-2(1-[\mathbf{C}_{i}]_{kk})\frac{1}{\|\mathbf{v}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}_{i,k}}{\|\hat{\mathbf{v}}_{i,k}\|}-[\mathbf{C}_{i}]_{kk}\frac{\mathbf{v}_{i,k}}{\|\mathbf{v}_{i,k}\|}\right)$ (17) $\displaystyle\quad+\sum_{j\neq k}2\lambda\max(0,[\mathbf{C}_{i}]_{kj})\frac{1}{\|\mathbf{v}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}_{i,j}}{\|\hat{\mathbf{v}}_{i,j}\|}-[\mathbf{C}_{i}]_{kj}\frac{\mathbf{v}_{i,k}}{\|\mathbf{v}_{i,k}\|}\right)\Bigg](\mathbf{s}_{i-1,k})^{T}$ $$

### D.2 Gradient with respect to Backward Weights 𝚯 i \bm{\Theta}_{i} (Type 2 Neurons)

For Type 2 neurons in layer $i$ (for $i=2,\dots,L-1$), the local loss $\mathcal{L}^{\prime}_{i}$ is defined symmetrically:

$$ $\mathcal{L}^{\prime}_{i}=\sum_{k=1}^{B}(1-[\mathbf{C}^{\prime}_{i}]_{kk})^{2}+\lambda\sum_{k=1}^{B}\sum_{j\neq k}\left(\max(0,[\mathbf{C}^{\prime}_{i}]_{kj})\right)^{2},$ (18) $$

where $[\mathbf{C}^{\prime}_{i}]_{kj}$ is the cosine similarity between $\mathbf{v}^{\prime}_{i,k}=\bm{\Theta}_{i}\mathbf{s}^{\prime}_{i+1,k}$ and $\hat{\mathbf{v}}^{\prime}_{i,j}=\mathbf{v}_{i,j}$.

The derivation for $\bm{\Theta}_{i}$ follows the same structure, yielding a sum of outer products over the batch:

$$ $\frac{\partial\mathcal{L}^{\prime}_{i}}{\partial\mathbf{\Theta}_{i}}=\sum_{k=1}^{B}\left(\frac{\partial\mathcal{L}^{\prime}_{i}}{\partial\mathbf{v}^{\prime}_{i,k}}\right)(\mathbf{s}^{\prime}_{i+1,k})^{T}$ (19) $$

The gradient with respect to $\mathbf{v}^{\prime}_{i,k}$ is:

$$ $\frac{\partial\mathcal{L}^{\prime}_{i}}{\partial\mathbf{v}^{\prime}_{i,k}}=-2(1-[\mathbf{C}^{\prime}_{i}]_{kk})\frac{\partial[\mathbf{C}^{\prime}_{i}]_{kk}}{\partial\mathbf{v}^{\prime}_{i,k}}+\sum_{j\neq k}2\lambda\max(0,[\mathbf{C}^{\prime}_{i}]_{kj})\frac{\partial[\mathbf{C}^{\prime}_{i}]_{kj}}{\partial\mathbf{v}^{\prime}_{i,k}},$ (20) $$

where the derivative of the cosine similarity term is:

$$ $\frac{\partial[\mathbf{C}^{\prime}_{i}]_{kj}}{\partial\mathbf{v}^{\prime}_{i,k}}=\frac{1}{\|\mathbf{v}^{\prime}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}^{\prime}_{i,j}}{\|\hat{\mathbf{v}}^{\prime}_{i,j}\|}-[\mathbf{C}^{\prime}_{i}]_{kj}\frac{\mathbf{v}^{\prime}_{i,k}}{\|\mathbf{v}^{\prime}_{i,k}\|}\right)$ (21) $$

This gives the final gradient for $\bm{\Theta}_{i}$:

$$ $\displaystyle\frac{\partial\mathcal{L}^{\prime}_{i}}{\partial\bm{\Theta}_{i}}=\sum_{k=1}^{B}\Bigg[$ $\displaystyle-2(1-[\mathbf{C}^{\prime}_{i}]_{kk})\frac{1}{\|\mathbf{v}^{\prime}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}^{\prime}_{i,k}}{\|\hat{\mathbf{v}}^{\prime}_{i,k}\|}-[\mathbf{C}^{\prime}_{i}]_{kk}\frac{\mathbf{v}^{\prime}_{i,k}}{\|\mathbf{v}^{\prime}_{i,k}\|}\right)$ (22) $\displaystyle\quad+\sum_{j\neq k}2\lambda\max(0,[\mathbf{C}^{\prime}_{i}]_{kj})\frac{1}{\|\mathbf{v}^{\prime}_{i,k}\|}\left(\frac{\hat{\mathbf{v}}^{\prime}_{i,j}}{\|\hat{\mathbf{v}}^{\prime}_{i,j}\|}-[\mathbf{C}^{\prime}_{i}]_{kj}\frac{\mathbf{v}^{\prime}_{i,k}}{\|\mathbf{v}^{\prime}_{i,k}\|}\right)\Bigg](\mathbf{s}^{\prime}_{i+1,k})^{T}$ $$

## Appendix E Experimental Settings

### E.1 Statistics of Datasets

We evaluate our proposed Bidirectional Spike-based Distillation (BSD) algorithm across a diverse range of tasks. The datasets employed in our experiments are detailed below, categorized by task.

##### Image Classification.

For the image classification tasks, we conducted experiments on five widely-used benchmarks to assess the model’s performance in visual recognition.

- •
MNIST. The Modified National Institute of Standards and Technology (MNIST) dataset is a cornerstone benchmark comprising 60,000 training and 10,000 testing images of handwritten digits (0-9). Each image is a $28\times 28$ pixel grayscale representation.
- •
FashionMNIST. As a more challenging drop-in replacement for MNIST, the FashionMNIST dataset contains 60,000 training and 10,000 testing examples of clothing items and accessories across 10 classes, each being a $28\times 28$ grayscale image.
- •
Street View House Numbers (SVHN). The SVHN dataset consists of $32\times 32$ color images of house numbers from a real-world setting. We use the standard cropped version, which includes 73,257 digits for training and 26,032 for testing, categorized into 10 classes.
- •
CIFAR-10. The Canadian Institute for Advanced Research (CIFAR-10) dataset contains 50,000 training and 10,000 testing $32\times 32$ color images across 10 mutually exclusive object classes (e.g., airplane, automobile, bird).
- •
CIFAR-100. The CIFAR-100 dataset is a collection of 60,000 $32\times 32$ color images, designed for fine-grained object recognition tasks. It contains 100 distinct classes that are hierarchically grouped into 20 superclasses. For each class, there are 500 training images and 100 testing images.

##### Image Generation.

To evaluate the generative performance of BSD, we employed an autoencoder architecture for image reconstruction on the MNIST, FashionMNIST, and CIFAR-10 datasets. The quality of the generated images was quantitatively assessed using the Fréchet Inception Distance (FID) score, which measures the similarity between the distributions of the reconstructed and original images.

##### Text Character Prediction.

For the sequential prediction task, we utilized a natural language corpus to evaluate the model’s ability to capture temporal dependencies at the character level.

- •
Harry Potter Corpus. This dataset is a text corpus derived from the Harry Potter book series (Plath et al., 2019), used for next-character prediction experiments.

##### Time-Series Forecasting.

For evaluating performance on multivariate time-series forecasting, we employed three standard real-world datasets from different domains.

- •
Electricity. The Electricity dataset (Lai et al., 2018) contains hourly electricity consumption data for 321 clients from 2012 to 2014, serving as a benchmark for long-term forecasting.
- •
METR-LA. This traffic forecasting dataset (Li et al., 2018) contains traffic speed data from 207 sensors on highways in Los Angeles County, aggregated every 5 minutes over a four-month period.
- •
PEMS-BAY. Sourced from the Caltrans Performance Measurement System (PeMS), this dataset (Li et al., 2018) comprises traffic speed information from 325 sensors in the San Francisco Bay Area over six months, providing another challenging forecasting benchmark.

### E.2 Implementation Details

#### E.2.1 BSD-MLPs

##### Model Architecture.

For the feedforward path composed of Type 1 neurons, we design distinct Multi-Layer Perceptron (MLP) architectures tailored to each task, given the heterogeneity of input dimensions and class cardinalities across the five datasets.
For each layer, the synaptic weight configurations of the backward path are designed to ensure that the number of neurons in both the Type 1 feedforward and Type 2 feedback paths remain consistent across all layers
MNIST and FashionMNIST consist of single-channel $28\times 28$ grayscale images, whereas SVHN, CIFAR-10, and CIFAR-100 comprise three-channel $32\times 32$ color images.
In terms of classification objectives, MNIST, FashionMNIST, SVHN, and CIFAR-10 are 10-class tasks, while CIFAR-100 involves 100 classes.
To accommodate these differences and ensure optimal performance, we adopt dataset-specific MLP configurations.
For MNIST and FashionMNIST, we employ a six-layer MLP with layer sizes [784, 1024, 1024, 512, 256, 10], where the input dimension of 784 corresponds to the flattened $28\times 28\times 1$ images and the output dimension of 10 reflects the number of classes.
For SVHN and CIFAR-10, we adopt a wider six-layer MLP with [3072, 4096, 2048, 1024, 512, 10] neurons per layer, where the input dimension of 3072 is derived from flattening the $32\times 32\times 3$ images.
For CIFAR-100, we maintain the same six-layer depth but adapt the final output to 100 classes, yielding layer sizes of [3072, 4096, 2048, 1024, 512, 100].
For the backward pathway, class labels are converted into spike-encoded targets: given a classification task with $C$ categories, the target for a sample of class $j$ is represented by a categorical code inspired by one-hot encoding, consisting of a binary vector of length $C$ with a single active entry at the $j$-th position and all others inactive. This vector is then repeated along the temporal dimension according to the number of time steps and delivered to the basal dendrites of top-layer Type 2 neurons.
In all architectures, neuronal dynamics for membrane potential integration and spike generation are modeled using the Leaky Integrate-and-Fire (LIF) framework.
The feedback path, composed of Type 2 neurons, follows a symmetric architecture to the feedforward path composed of Type 1 neurons.

##### Training Hyperparameters.

All MLPs are optimized using AdamW with a cosine annealing learning-rate schedule.
The simulation length is fixed at $T{=}4$ time steps.
For LIF neurons, we employ the ATan surrogate function provided in the SpikingJelly framework.
To ensure that spike sparsity remains within a regime conducive to effective learning,
the firing thresholds are set to $0.2$ for Type 1 neurons and $0.1$ for Type 2 neurons.
Across all datasets, we adopt a learning rate of $1{\times}10^{-4}$, a batch size of $128$,
and apply RandAugment for data augmentation (Cubuk et al., 2020).
To stabilize optimization, gradient clipping is applied with a threshold of $0.3$.
Finally, for all non-top layers, the penalty weight in the ReCo loss is fixed at $\lambda{=}0.6$.

#### E.2.2 BSD-CNNs

##### Model Architecture.

For the feedforward path composed of Type 1 neurons, we adopt a unified CNN architecture for all tasks.
The architecture consists of five convolutional layers followed by a fully connected output layer.
Specifically, the first convolutional layer maps the input channels to 128 channels using $3\times 3$ kernels, followed by $2\times 2$ max-pooling.
The second convolutional layer maintains 128 channels with the same $3\times 3$ kernel and $2\times 2$ max-pooling.
The third and fourth convolutional layers expand the representation to 256 channels each, again using $3\times 3$ kernels with $2\times 2$ max-pooling.
The fifth convolutional layer further increases the dimensionality to 512 channels, also with $3\times 3$ kernels and $2\times 2$ max-pooling.
The feature maps are then flattened into a 512-dimensional vector, which is connected to the final output layer whose dimension is aligned with the spike-encoded target $\hat{\mathbf{s}}$.

As in the MLP case, class labels are converted into spike-coded targets and delivered to the basal dendrites of top-layer Type 2 neurons.
For a classification task with $C$ categories, the target of a sample from class $j$ is represented by a categorical spike code inspired by one-hot encoding: a binary vector of length $C$ with a single active entry corresponding to class $j$, repeated along the temporal dimension to match the simulation time steps.

To stabilize neuronal spiking activity across layers, we insert a batch-normalization layer after each convolutional operation.
This normalization constrains the membrane potentials to a comparable range across layers, thereby ensuring that spike sparsity remains consistent and that neuronal firing thresholds are properly regulated throughout the network.

Similarly to the MLP case, the feedback path, composed of Type 2 neurons, mirrors the structure of the feedforward path composed of Type 1 neurons.
In the feedback path, we use upsampling as the reverse operation of the convolutional layers used in the feedforward path,
ensuring that the number of Type 1 and Type 2 neurons in feedforward and feedback paths remains equal across all layers.

##### Training Hyperparameters.

For all tasks, we optimize our models using the AdamW optimizer and employ a cosine learning rate scheduler.
To ensure that the neuronal firing sparsity remains within an ideal range, we set the firing thresholds of both Type 1 and Type 2 neurons to 1.0.
The number of time steps is set to 4, with a warm-up period of 100 steps.
For data augmentation, we use RandAugment with a magnitude of 4 for MNIST, FashionMNIST, CIFAR-10, and CIFAR-100, while on SVHN, we apply only RandomCrop.
All images are normalized prior to processing.
The batch size is set to 128 across all datasets.
For MNIST, FashionMNIST, CIFAR-10, and CIFAR-100, the learning rates for both Type 1 and Type 2 neurons are set to $1\times 10^{-3}$, whereas for SVHN, we reduce the learning rate to $1\times 10^{-4}$.
For the layer-wise ReCo loss applied to all layers except the top layer, we set the penalty weight $\lambda$ to 0.6, consistent with the configuration used in MLPs.

#### E.2.3 BSD-RNNs

##### Model Architecture.

For all sequential regression datasets, we employed the same RNN model architecture which is a single-layer RNN with a hidden size of 300, where the input and output dimensions corresponded to those of the respective datasets: Harry Potter(103,103), Electricity(320,1), Metr-la(206,1), Pems-bay(324,1), where the dimensions are presented in the form of (input dimensions, output dimensions).

##### Training Hyperparameters.

Similarly to the classification tasks, we utilized the AdamW optimizer along with a cosine learning rate scheduler across all sequential regression datasets. To identify optimal performance in different datasets, we maintained identical activation thresholds for both Type 1 and Type 2 neurons and fixed the number of timesteps at 4 for all tasks. However, we varied the activation threshold according to each dataset: 1.0 for the Harry Potter and Metr-la datasets, 0.8 for Electricity, and 0.5 for Pems-bay. All weights were initialized with PyTorch’s default weight initialization. The batch size was set to 128 for the Electricity dataset and 64 for the remaining sequential regression datasets, with the sequence length fixed to 32 across all datasets. The learning rate was set to 0.001 for the Harry Potter, Electricity and METR-LA datasets, and 0.00015 for PEMS-Bay. Due to the larger dataset size of Harry Potter, training was limited to 10 epochs, whereas the other datasets were trained for 100 epochs. To stabilize training, gradient clipping with an upper bound of 1 was employed for all sequential regression datasets, and batch normalization was applied to the hidden layer in PEMS-Bay to normalize each batch to zero mean and unit variance.

##### Metric.

For time-series forecasting datasets (Electricity, METR-LA, and PEMS-Bay), the mean squared error (MSE) and the mean absolute error (MAE) were used as performance metrics. The formulas are as follows:

$$ $\text{MSE}=\frac{1}{B\times T\times D}\sum_{b=1}^{B}\sum_{t=1}^{T}\sum_{d=1}^{D}\left(\hat{y}_{b,t,d}-y_{b,t,d}\right)^{2},$ (23) $$

$$ $\text{MAE}=\frac{1}{B\times T\times D}\sum_{b=1}^{B}\sum_{t=1}^{T}\sum_{d=1}^{D}\left|\hat{y}_{b,t,d}-y_{b,t,d}\right|,$ (24) $$

where $B,T,D$ represents the batch size, sequence length, and output dimensions, respectively. While $y_{b,t,d}$ is the value of the $d$-th feature of the output at the $t$-th time step for the $b$-th sample in the batch and $\hat{y}_{b,t,d}$ is its target value.
We normalized each column of the raw data to zero mean and unit variance before calculating MSE and MAE, consistent with the preprocessing described in (Lv et al., 2025). We selected the model with the lowest MSE as the final recorded performance metric. For the text character prediction dataset (Harry Potter), element-wise accuracy was used as the performance metric. In preprocessing, all whitespace characters in the text were replaced with a single space, and performance was evaluated on the processed dataset.

#### E.2.4 BSD-Autoencoders

##### Model Architecture.

We address the task of image reconstruction on the MNIST, FashionMNIST, and CIFAR-10 datasets
using a convolutional autoencoder. The architecture, which remains consistent across all datasets,
is composed of a feedforward path (Type 1 neurons) and a symmetric feedback path (Type 2 neurons).
To maintain an identical neuron count between the Type 1 and Type 2 populations at each corresponding layer, the architecture employs inverse spatial operations: where one path uses a strided convolution, the other utilizes an upsampling layer, and vice versa.
To ensure stable and sparse spiking activity, each layer in both paths incorporates a batch normalization step after convolution and upsampling.

The encoder section systematically compresses the input into a latent representation through a cascade of three convolutional layers.
The first layer applies a $3\times 3$ convolution with a stride of 2, halving the input’s spatial dimensions while expanding the feature maps to 128 channels.
The second layer repeats this operation, again using a stride of 2 to halve the spatial resolution and increasing the channel depth from 128 to 256.
The final encoding layer performs a third stride-2 convolution, mapping the features to a 512-channel bottleneck representation.

Symmetrically, the decoder reconstructs the image from this bottleneck representation.
Each decoding layer first doubles the spatial dimensions of its input via a nearest-neighbor upsampling operation.
Following this, a $3\times 3$ convolution with a stride of 1 is applied to refine the features.
This two-step process is repeated three times: the first block reduces the channel count from 512 to 256, the second from 256 to 128,
and the final block restores the feature map to the original input resolution while matching the channel count of the source image.

The source image is provided as the basal input to both the bottom-layer Type 1 neurons and the top-layer Type 2 neurons.
Thus, the top-layer Type 2 neurons function as a dedicated transducer for the backward path, converting the source image into a spike-based representation.

##### Training Hyperparameters.

The model is optimized across all datasets using the AdamW optimizer, with a learning rate of $0.001$ managed by a cosine annealing schedule.
The spiking threshold for both Type 1 and Type 2 neurons is uniformly set to $0.4$, and the network is simulated for a total of 8 timesteps.
To apply differential penalties to the reconstruction error, we decompose the error in the frequency domain.
This is achieved by creating a binary mask based on the image’s Fourier transform.
A circular region in the frequency spectrum, centered at the zero frequency, is defined as the low-frequency domain, while the area outside this region constitutes the high-frequency domain.
The boundary is controlled by a freq_cutoff_ratio hyperparameter, representing the normalized radius of this low-frequency circle.
This allows us to assign distinct $\lambda$ weights for the Reconstruction-on-Construction (ReCo) loss to the low-frequency and high-frequency components.
For the MNIST and FashionMNIST datasets, the model is trained with a batch size of 32.
The freq_cutoff_ratio is set to $0.6$, with the ReCo loss $\lambda$ configured to $0.005$ for low-frequency components and $0.01$ for high-frequency components.
For the more complex CIFAR-10 dataset, the batch size is increased to 64.
The freq_cutoff_ratio is adjusted to $0.7$, and the corresponding ReCo loss weights are set to $0.005$ for the low-frequency band and increased to $0.05$ for the high-frequency band, placing a greater penalty on the reconstruction of fine-grained details.
To improve model generalization on this dataset, we also apply RandAugment for data augmentation during training.

##### Evaluation Metrics

For autoencoder-based image generation tasks, we employ Fréchet Inception Distance (FID) (Heusel et al., 2017) as our evaluation metric. FID measures the
distributional similarity between real and generated images by computing the distance between their feature representations in the Inception-v3 network’s activation
space. Specifically, the metric calculates the Fréchet distance between two multivariate Gaussian distributions fitted to the feature vectors of real and synthetic
images extracted from the final pooling layer of Inception-v3. Lower FID scores indicate higher similarity between generated and real image distributions, with a
perfect score of 0 representing identical distributions. This metric provides a reliable quantitative assessment of generation quality that correlates well with
human perceptual judgment, making it particularly suitable for evaluating the generative capabilities of our BSD-trained autoencoders.

## Appendix F Additional Experiments

### F.1 Supplementary Results for Image Generation

Figure: Figure 4: Comparison of image reconstruction on the MNIST dataset by autoencoders trained with Backpropagation (BP) and our proposed Bidirectional Spike-based Distillation (BSD).
Refer to caption: graph/mnist-result.png

Figure: Figure 5: Visual comparison of FashionMNIST reconstructions from autoencoders trained with Backpropagation (BP) and our Bidirectional Spike-based Distillation (BSD) method..
Refer to caption: graph/fmnist-result.png

Figures [4](https://arxiv.org/html/2509.20284v1#A6.F4) and [5](https://arxiv.org/html/2509.20284v1#A6.F5) present a qualitative comparison of image reconstructions on the MNIST and FashionMNIST datasets, showcasing outputs from autoencoders trained with conventional Backpropagation (BP) alongside those trained with our proposed Bidirectional Spike-based Distillation (BSD).
Table [5](https://arxiv.org/html/2509.20284v1#A6.T5) provides a complementary quantitative analysis on the FashionMNIST dataset, benchmarking the performance of BSD against BP and the Fully Spiking Variational Autoencoder (FSVAE) using the Fréchet Inception Distance (FID) metric.
While BSD-trained autoencoders demonstrate proficient generative capabilities by successfully capturing the salient global structure of the input images, compared to BP-trained ANNs, BSD-generated images tend to exhibit a loss of high-frequency detail.
This suggests that while BSD establishes a robust framework for generative modeling, exploring further refinements or architectural adaptations could be beneficial for enhancing the preservation of fine-grained textures and improving the overall sharpness of the generated images.

**Table 5: Quantitative comparison of image generation performance on the FashionMNIST dataset, evaluated using Fréchet Inception Distance (FID). $\downarrow$ denotes that lower scores indicate better performance.**
| Dataset | Model | FID $\downarrow$ |
| --- | --- | --- |
| FashionMNIST | ANN-BP | $\mathbf{29.07}$ |
| FSVAE | $90.12$ |  |
| BSD (Ours) | $\mathbf{112.97}$ |  |

**Table 6: Ablation study on text character prediction and time-series forecasting tasks.**
| Method | Harry Potter | Electricity | Metr-la | Pems-bay |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Pred. Acc. $\uparrow$ | MSE $\downarrow$ | MAE $\downarrow$ | MSE $\downarrow$ | MAE $\downarrow$ | MSE $\downarrow$ | MAE $\downarrow$ |  |
| BSD-MSE | $29.6\%$ | $0.179$ | $0.328$ | $0.160$ | $0.247$ | $0.180$ | $0.221$ |
| BSD-InfoNCE | $29.9\%$ | $0.172$ | $0.323$ | $0.159$ | $0.257$ | $0.190$ | $0.230$ |
| BSD | $\bf 41.8\%$ | $\bf 0.165$ | $\bf 0.314$ | $\bf 0.125$ | $\bf 0.197$ | $\bf 0.174$ | $\bf 0.206$ |

### F.2 Ablation study on RNNs

Table [6](https://arxiv.org/html/2509.20284v1#A6.T6) compares the performance of BSD-ReCo, BSD-MSE, and BSD-infoNCE on sequential regression tasks. In all tasks, the ReCo loss consistently demonstrated the highest performance.
In contrast to the outcomes from the ablation studies on image classification tasks, all inter-layer loss functions achieved stable convergence on the sequential regression datasets, which may be attributed to the relatively shallow depth of the RNNs.

In conclusion, for sequential regression tasks, MSE, infoNCE, and ReCo all demonstrated stable convergence, but ReCo remains the most optimal inter-layer loss, with its performance advantage being particularly evident on text character prediction datasets.

### F.3 Ablation study on autoencoders

Figure: Figure 6: Visual examples of image reconstructions on the CIFAR-10 dataset, showcasing the performance of autoencoders trained with BSD using various layer-wise loss function configurations.
Refer to caption: graph/cifar10-ablation.png

**Table 7: Ablation study on CIFAR-10 image generation, evaluating the impact of different layer-wise loss functions and frequency decomposition strategies, with performance measured by Fréchet Inception Distance (FID). $\downarrow$ denotes that lower FID values indicate better performance. BSD-primitive: BSD without adaptive frequency domain decomposition. BSD-InfoNCE: BSD employing InfoNCE loss for intra-neuronal voltage alignment. BSD-MSE: BSD utilizing Mean Squared Error (MSE) loss for intra-neuronal voltage alignment.**
| Dataset | Model | FID $\downarrow$ |
| --- | --- | --- |
| CIFAR-10 | BSD | $\mathbf{168.12}$ |
| BSD-primitive | $185.33$ |  |
| BSD-InfoNCE | $190.95$ |  |
| BSD-MSE | $177.66$ |  |

To investigate the impact of different layer-wise loss functions and frequency decomposition strategies
on image generation, we conduct an ablation study using BSD-trained autoencoders on the CIFAR-10 dataset.
Figure [6](https://arxiv.org/html/2509.20284v1#A6.F6) presents visual comparisons of image reconstructions, and Table [7](https://arxiv.org/html/2509.20284v1#A6.T7)
details the corresponding FID scores. Our ablation study includes the following configurations:
BSD: our proposed method incorporating ReCo loss and adaptive frequency decomposition;
BSD-primitive: BSD using ReCo loss but without adaptive frequency domain decomposition, applying
a uniform $\lambda$ across all frequency components; BSD-InfoNCE: BSD that substitutes ReCo loss with
InfoNCE loss for intra-neuronal voltage alignment; and BSD-MSE: BSD that replaces ReCo loss with
Mean Squared Error (MSE) loss for intra-neuronal voltage alignment.
From Table [7](https://arxiv.org/html/2509.20284v1#A6.T7), it is evident that performing frequency domain segmentation
using FFT and applying distinct $\lambda$ penalty weights for low and high-frequency regions significantly enhances
the performance of BSD-trained autoencoders. As a result, BSD achieves the best FID score of $\mathbf{168.12}$,
notably outperforming BSD-primitive (FID $185.33$). Furthermore, for image generation tasks, BSD-InfoNCE
(FID $190.95$), which employs InfoNCE for intra-neuronal voltage alignment, yields inferior results
compared to BSD (using ReCo Loss for intra-neuronal voltage alignment).
Figure [6](https://arxiv.org/html/2509.20284v1#A6.F6) shows that while BSD-MSE demonstrates an ability to capture salient
structural features and approximate pixel locations of objects, it nonetheless exhibits obvious deficiencies
in both color fidelity and detail retention. Compared to the reconstructions from BSD (which utilizes ReCo Loss
and adaptive frequency decomposition), the images produced by BSD-MSE appear significantly blurred and desaturated,
losing fine textures and distinct chromatic attributes present in the ground truth (GT) images.
These results collectively affirm the effectiveness of our proposed design, which integrates ReCo Loss as the
layer-wise loss function, performs frequency domain segmentation via FFT, and applies adaptive $\lambda$ penalty
weights for distinct low and high-frequency regions.

### F.4 Dynamics Of Synaptic Weight Alignment

Figure: Figure 7: Alignment dynamics of synaptic weights ($\mathbf{W}$ and $\bm{\Theta}$) between the feedforward and backward pathways in a BSD-trained convolutional neural network for image classification on the CIFAR-10 dataset. The figure presents the cosine similarity of weights across network layers as a function of training epochs.
Refer to caption: x5.png

In this section, we examine the alignment dynamics between the feedforward synaptic
weights ($\mathbf{W}$) and the backward synaptic weights ($\bm{\Theta}$)
within a convolutional neural network trained using the BSD algorithm.
The network was tasked with image classification on the CIFAR-10 dataset.
We measured the cosine similarity between the weights connected to the basal dendrites
of Type 1 and Type 2 neurons to quantify their alignment in each layer.
In Figure [7](https://arxiv.org/html/2509.20284v1#A6.F7), ’enc1’ through ’enc5’ denote the
five feedforward convolutional layers and their corresponding backward upsampling layers,
progressing from the shallowest to the deepest. The ’output’ label refers to the
final fully connected layer.
Figure [7](https://arxiv.org/html/2509.20284v1#A6.F7) reveals that the weight alignment for
the final fully connected layer undergoes a rapid initial increase, peaking within
the first few epochs before stabilizing at a high cosine similarity value of approximately 0.7.
In contrast, the weights of the convolutional and upsampling layers
(’enc1’ to ’enc5’) consistently exhibit negligible alignment, with their cosine
similarity values remaining close to zero throughout the entire training process.
Such alignment behavior demonstrates that the feedforward and backward
pathways do not converge toward symmetric weights under BSD.
Thereby we confirm that the BSD algorithm adheres to criterion
C1: Asymmetric synaptic weights for feedforward and feedback pathways.

### F.5 t-SNE Visualization of Learned Feature Representations in BSD-trained CNNs

Figure: Figure 8: t-SNE visualizations illustrating the evolution of learned feature representations from the final two convolutional layers of the feedforward path in BSD-trained CNNs, for an image classification task on the CIFAR-10 dataset across training epochs.
Refer to caption: x6.png

Figure [8](https://arxiv.org/html/2509.20284v1#A6.F8) shows t-distributed stochastic neighbor embedding (t-SNE)
projections of feature representations from the final two convolutional layers (Layers 4 and 5) of
the feedforward path at epochs 5, 10, 15, and 20 during BSD training on CIFAR-10. As training
proceeds, class-specific structure becomes increasingly separated; by epoch 20, Layer 5 exhibits
distinct, compact clusters for most classes. This progression indicates that BSD progressively
shapes intermediate features into more class-discriminative representations.

## Appendix G Limitations and Future Directions

### G.1 Limitations

Despite its promising performance and significant contributions, the Bidirectional Spike-Based Distillation (BSD) framework also presents opportunities for further research.
The current BSD framework has been evaluated on conventional neural network architectures such as Multi-Layer Perceptrons (MLPs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), and autoencoders.
Expanding the applicability of BSD to a broader range of network architectures, such as those that incorporate residual connections or attention mechanisms, can further enhance its capacity to address complicated tasks.

### G.2 Future directions

Future research can explore extending the BSD framework to support a wider variety of network architectures, including those that employ residual connections and attention mechanisms, to handle increasingly complex datasets and challenging tasks.

## Appendix H The Use of Large Language Models

The authors acknowledge the use of a Large Language Model (LLM) in this work.
Its role was strictly confined to polish the writing and correcting grammatical errors to enhance the overall clarity and readability of the paper.

## Peer Reviews

confidence: 2
contribution: 3
initial_rating: 8
presentation: 3
questions: See weaknesses. Moreover, could you expand the observation that "The network attains its best performance at a firing threshold of 0.6, suggesting that thresholds that are either too low or too high impair effective learning" (line 461) with a possible explanation for this phenomenon?
soundness: 3
strengths: **Biological motivation and plausibility:** The paper draws inspiration from the neuroscience principle of bidirectional processes that integrate bottom-up sensory perception with top-down memory recall. BSD implements this concept with high fidelity and satisfies five relevant criteria of biological plausibility, extending beyond the three criteria proposed by prior work.

**Comprehensive experimental validation:** The authors test the algorithm on multiple architectures (MLPs, CNNs, RNNs, autoencoders) and diverse datasets spanning image classification (MNIST, FashionMNIST, SVHN, CIFAR-10, CIFAR-100), text prediction and time-series forecasting (Harry Potter, Electricity, Metr-la, Pems-bay), and image generation tasks, demonstrating the generalizability of the approach.

**Novel algorithmic contributions:** The integration of bidirectional spiking networks trained via mutual distillation represents a unique approach to biologically plausible learning, distinguishing it from existing methods that typically compromise on some biological plausibility criteria. Moreover, BSD achieves performance comparable to backpropagation on all introduced tasks.

**Ablation study:** The paper includes an ablation experiment examining the impact of different loss functions (MSE, InfoNCE) and demonstrates that ReCo loss consistently outperforms alternatives.
summary: This paper introduces Bidirectional Spike-based Distillation (BSD), a novel biologically plausible learning algorithm for neural networks. The work addresses a fundamental challenge in neural network research: developing learning methods that satisfy biological plausibility criteria while achieving competitive performance with backpropagation. BSD is inspired by the brain's bidirectional architecture integrating bottom-up sensory perception with top-down memory recall. The algorithm jointly trains feedforward and backward spiking networks, where the feedforward pathway maps stimuli to conceptual representations (analogous to perception and decision-making), while the backward pathway reconstructs stimuli from concepts (analogous to memory recall). The authors demonstrate that BSD satisfies five biological plausibility criteria: asymmetric forward and backward weights, local synaptic plasticity, single phase learning (eliminating sequential forward-backward passes), spiking neurons, and unsigned error signals. They validate the algorithm across diverse architectures and tasks.
weaknesses: **Limited discussion of practical implications and broader impact:** The authors do not adequately address the practical implications of their work. Key questions remain unanswered:
- Will BSD be adopted by the machine learning community, and if so, for what primary applications?
- Will the code be released as an integrated Python library compatible with current frameworks (PyTorch, JAX)?
- Alternatively, will BSD primarily serve the neuroscience community for computational experiments addressing neuroscience-related problems?

**Insufficient computational performance analysis:** Performance metrics including time complexity, memory consumption, and GPU usage should be clearly stated and compared to backpropagation in the main text.

**Minor:**
- In the code, clear instructions for reproducing the experimental results are not given.
- In Section 4.2, the notation is difficult to follow and contains apparent errors. Specifically, $\hat{v}'$ is never defined, while $v'$ and $\hat{v}$ are defined but never used in equations. The same symbol $v'$ is used to define Type 1 neurons in Equation 4 and Type 2 neurons in Equation 5, and vice versa for symbol $\hat{v}$. Additionally, in Section 4.3 (line 239), matrices of membrane voltages are defined but never used in subsequent equations.
- The application of FFT decomposition to input images and membrane voltages is mentioned but not adequately explained. This technique appears important for the method's performance on generation tasks, yet it receives minimal emphasis and should be expanded at least in the appendix.
- Font sizes in figures (especially Figure 1c and Figure 3) are excessively small and difficult to read while going through the main text.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: See Weaknesses
soundness: 3
strengths: 1. BSD fulfills all five proposed biological criteria, i.e., asymmetric weights, local error signals, non-two-stage learning, spiking neuron models, and unsigned error encoding, making it more aligned with neurobiological principles than backpropagation-based methods.   

2. BSD achieves accuracy close to backpropagation on diverse benchmarks and robustly handles sequential regression and image generation, proving practicality without sacrificing biological grounding.
summary: This paper introduces Bidirectional Spike-Based Distillation (BSD), a biologically plausible learning algorithm inspired by the brain's bidirectional perception-recall architecture. BSD transforms spiking representations between stimulus encoding (feedforward path for perception) and concept encoding (feedback path for recall), satisfying five biological plausibility criteria (e.g., asymmetric weights, local plasticity). Experiments on image classification, sequential regression, and generation tasks demonstrate that BSD achieves performance comparable to error backpropagation while maintaining stronger neurobiological fidelity.
weaknesses: 1. Discrete spike-based communication reduces information capacity compared to continuous activations, leading to performance degradation in SNNs across all tasks.   

2. The description of the spike generation and information encoding lacks depth. While the LIF neuron dynamics are provided, the paper does not specify the exact strategy for converting inputs (like images) and targets into the initial spike trains (s1 and $\hat{s}$) for the feedforward and backward paths. The choice of encoding scheme (e.g., rate coding, temporal coding) can significantly impact performance, and its omission is a notable gap. Furthermore, the distillation process between the two pathways is described at a high level as feature alignment using the ReCo loss, but the precise mechanisms of how the error signals from this loss guide the updates of the distinct weight matrices (W and $\theta$) need a more detailed explanation, potentially including a mathematical derivation.

3. The selection of the Relaxed Contrastive (ReCo) loss over other alternatives is justified by its unsigned nature and performance in ablations. However, a more rigorous theoretical or empirical justification for why it is particularly suited for aligning voltage signals in spiking neurons, as opposed to other contrastive losses, would strengthen the argument. Similarly, the choice of specific hyperparameters, such as the penalty weight $\lambda$ = 0.6 and the firing thresholds for different neuron types and architectures, appears to be determined empirically. The paper would benefit from discussing the sensitivity of the results to these choices or providing a principled rationale for them.

4. The paper introduces two new biological plausibility criteria (C4 and C5) and claims BSD satisfies all five. However, the argument for satisfying C2 (local synaptic plasticity) is somewhat superficial. It states that error computation is localized using the ReCo loss, but a more detailed analysis is needed to explain how the weight updates for W and $\theta$ rely only on locally available information (e.g., pre- and post-synaptic spikes and local voltages) without any implicit non-local dependencies. A detailed diagram or step-by-step explanation of the local update rule for a single synapse would make this claim more credible.

5.  The experiments demonstrate performance on standard benchmarks but do not thoroughly test the robustness of the BSD-trained models. For instance, how do these models perform on corrupted data, out-of-distribution examples, or under varying noise conditions? Including such robustness evaluations would be crucial for claiming the practical viability of the algorithm. Furthermore, the scalability of BSD is asserted but not conclusively proven. Testing on a larger-scale, more complex dataset (e.g., ImageNet) would provide much stronger evidence for its scalability beyond relatively small datasets like CIFAR-100.

6. A key claim of the paper is that BSD is more biologically plausible. However, the experiments primarily focus on task performance metrics (accuracy, FID, MSE). There is a lack of direct analysis and quantification of the purported biological properties themselves. For example, the paper could analyze the degree of weight symmetry/alignment between the forward (W) and backward ($\theta$) pathways during training. Figure 7 shows this alignment is low for most layers, which supports adherence to C1, but a more quantitative discussion of this property across different tasks and architectures would be valuable. Similarly, the energy efficiency of the spiking models, a major claimed advantage of SNNs, is not measured or compared to analog networks.

7. In the image generation task, the qualitative results (Figure 2) and FID scores are presented. However, the FID scores for BSD are generally worse than those for BP-trained models. The paper states the quality is comparable, but a more critical discussion of the performance gap and the specific challenges BSD faces in generative tasks (e.g., possibly due to the discrete nature of spikes) would be more balanced. The visual samples in Figure 2 also lack a detailed description of what specific shortcomings (e.g., blurriness, lack of detail) are observed in the BSD-generated images compared to the BP baseline.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. The paper mentions that the number of time steps is set to 4 for MLP, CNN, and RNN tasks (e.g., Section E.2.2, E.2.3) and 8 for autoencoder-based image generation tasks (Section E.2.4), but no analysis of the impact of different time step values on performance is provided. Do you plan to supplement experiments with varying time steps (e.g., 2, 6, 10) to explore how time step adjustments affect BSD’s performance across different tasks (image classification, image generation, sequential regression)?

2. Batch normalization is applied in BSD-CNNs (after each convolutional layer, Section E.2.2) and BSD-RNNs (for the hidden layer in Pems-bay, Section E.2.3) to stabilize spiking activity, but the paper lacks quantitative and qualitative discussions on its specific impact. Could you provide experimental results comparing the convergence speed (e.g., number of epochs to reach stable performance) and final task performance (e.g., classification accuracy, FID, MSE) of BSD with and without batch normalization?

3. The BSD algorithm adopts dual forward (Type 1 neurons) and backward (Type 2 neurons) branches with independent weight matrices (W for forward, Θ for backward, Section 4.2), which may increase GPU memory consumption. Have you quantified the GPU memory usage of BSD compared to classical error backpropagation (on ANNs/SNNs) and other biologically plausible algorithms (e.g., DLL, CCL) under the same network architecture and batch size?
soundness: 3
strengths: 1. BSD is the an algorithm to simultaneously meet five core biological plausibility criteria, resolving conflicts between existing methods and neurobiological principles in aspects such as spike signal utilization and error signal type, thus providing an effective framework for biologically grounded spike-driven learning.

2. Experiments cover multiple tasks (e.g., image classification on MNIST/CIFAR-10, image generation evaluated via FID, sequential regression including text prediction and traffic forecasting) and various network architectures, verifying BSD’s strong generalization ability and task adaptability.

3. The algorithm design is closely aligned with the brain’s bidirectional mechanism of bottom-up perception and top-down recall, utilizing the basal and apical dendrites of pyramidal neurons to process feedforward and feedback signals respectively, resulting in a theoretical foundation highly consistent with neuroscientific observations.
summary: This paper proposes the Bidirectional Spike-based Distillation (BSD) algorithm. Inspired by the brain’s bidirectional perception-recall mechanism, BSD jointly trains a feedforward spiking network (for stimulus-to-concept perception and decision-making) and a backward spiking network (for concept-to-stimulus memory reconstruction), framing learning as a transformation between stimulus encoding and concept encoding. Through experiments across multiple tasks (image classification, image generation, sequential regression) and network architectures (MLPs, CNNs, RNNs, autoencoders), BSD is verified to achieve performance comparable to classical error backpropagation while satisfying all five biological plausibility criteria.
weaknesses: 1. It would be beneficial to investigate the influence of different timesteps on the BSD network's performance through additional experiments.

2. Further discussions on the impact of batch normalization on the convergence and overall performance of the BSD algorithm could be included.

3. Considering the BSD algorithm's dual forward and backward branches, it presumably requires a larger amount of GPU memory, which could be a relevant consideration for practical applications.

---

confidence: 4
contribution: 4
initial_rating: 4
presentation: 2
questions: What is the effect of having spikes? In 5.3 it is noted that spikes degrade accuracy compared to ANNs. For the proposed architecture, it would seem the proposed approach would also work for identically structured non-spiking RNNs. Does this improve performance? 

How does the approach scale? All examples are relatively small/shallow  networks, whereas it is known that similar ELBO-like losses tend to not scale well. Can you comment Deeper networks?

How does ReCo scale with batch size?

Can you demonstrate why layer-wise ReCo works as well as it does, compared to e.g. non detached ReCo?
soundness: 2
strengths: The experiments are extensive and well done. The loss as described seems to be a novel application for spiking neural networks. If correct (see below), the stability of the local learning approach is amazing and would be a big step in general towards local learning.
summary: The authors present a layer-wise local learning rule for two-compartment spiking neurons where each layer tries to minimize a Relaxed Contrastive (ReCo) loss between the apical and basal activation. The network is constructed with a bottom-up discriminative and top-down generative stream, where each stream has its own classification/regression task. The authors argue that in this network, just optimizing these within-layer errors inside the spiking neurons results in competitive performance on various tasks.
weaknesses: I have a number of smaller issues, and one main one:

while the ReCo loss has been presented before, I am not aware of its use in a setting where there is only layer-wise learning. Ie in Lin et al 2023, this loss seems to be backpropagated through a deep network. It is not obvious why this loss still works when indeed the errors are not backpropagated from the output layer. As it is now, the approach seems to mimic a layer-wise ELBO loss, with some nudging of only the final layer from errors calculated from outputs. The issue is that such layer-wise ELBO losses typically collapse and need measure to learn properly. I do not see either the mathematical argument why the proposed approach should work or the ablation study that demonstrates how this works compared to not detaching. Checking the code, I am also not convinced that necessarily the detach() operation is applied correctly.  I would very much like to see convincing arguments why the described approach works, and that it works correctly. 

Smaller issues:

The text is many places is not particularly clear. Eg. where the loss is described in 4.3 (l234), it is not immediately clear that the ReCo loss is being explained. 

Figures 1 and 3 dont print well.

The term "distillation" is prominently mentioned but it is not explained what is meant by this term.  Similarly for "embeddings" 

Layer-wise learning is not necessarily local, it should be explained in detail why the learning rule is neuron-local or instead layer-local.

"predictive coding" is a closely related approach and should be discussed. It is also shown in Table 1 but not cited, so the particular predictive coding approach is unclear. 

the approach claims biological plausibility, but the ReCo contrastive loss relies on the batch to provide negative samples, which is not a quantity locally known by the neuron. 

I do not agree with all stated biological constraints: as is well noted in the literature, symmetric forward and backward weights can arise naturally provided that the same local learning is present at forward and backward site, and weight-decay is applied (easy to see). 

There is a misunderstanding of how the brain seems to employ global error signals: first, multiple neuromodulatory signals are thought to be quite a-specific, like dopamine being broadcasted widely and non-targeted. Moreover, to the degree that dopamine signals a global reward prediction error, negative errors are believed to be encoded as a decrease from a steady baseline, thus resolving the issue with error-signs.

typo l054: "spiking trains" ??

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Q1. Can you report a ImageNet-1k experiment (e.g., ResNet-18/50) with accuracy and training cost versus BP and the above baselines?

Q2. How sensitive is BSD to batch size, especially small-B/micro-batch or streaming regimes where the B×B affinity becomes ill-conditioned?

Q3. How do you reconcile C5 with a signed top-layer cross-entropy; is C5 intended only for hidden layers, or can you replace CE with an unsigned alternative and report the impact?
soundness: 2
strengths: This work presents a well-motivated learning algorithm inspired by the brain's perception-recall mechanisms. The paper clearly defines its contribution by introducing two new criteria for biological plausibility and demonstrating that the proposed BSD algorithm satisfies all five. The joint training of feedforward and backward networks via mutual distillation is an elegant approach to implement local learning. The experimental evaluation covers a diverse set of tasks and network architectures, which supports the algorithm's versatility. The results show that BSD can achieve performance competitive with backpropagation while adhering to stricter biological constraints.
summary: This paper presents Bidirectional Spike-based Distillation (BSD), a biologically motivated training recipe for spiking neural networks. BSD jointly trains a bottom-up “perception” SNN and a top-down “recall” SNN, framing learning as the alignment of basal (forward) and apical (feedback) membrane potentials. Hidden layers are optimized with a layer-local, unsigned Relaxed Contrastive (ReCo) loss that increases same-sample similarity and suppresses positive cross-sample correlations, while the top layer uses a standard supervised objective. The method is claimed to satisfy five criteria for biological plausibility (asymmetric weights, layer-local plasticity, non-dual-phase training, spiking neurons, and unsigned signals at hidden layers). Experiments on image classification, sequence regression, and simple autoencoding show performance close to BP on modest benchmarks with MLP/CNN/RNN backbones. Conceptually, the novelty appears moderate: BSD primarily assembles known ideas from local/decoupled learning and feedback/target-prop-style alignment into an all-spiking, two-pathway realization. Moreover, the advertised “top-down memory recall” is instantiated as label-driven feedback reconstruction rather than an addressable memory or retrieval mechanism. Finally, the work lacks validation on modern architectures and large-scale datasets and does not compare against strong local-learning baselines for SNNs, so the contribution is best viewed as an effective integrated recipe rather than a fundamentally new paradigm.
weaknesses: The paper positions layer-local losses and decoupled feedback as distinctive, yet these ideas substantially overlap with established families (local errors/greedy layerwise, synthetic gradients, feedback alignment, target/representation propagation; in SNNs, DECOLLE and e-prop), and the batch-correlation alignment mirrors redundancy-reduction/self-correlation objectives (Barlow Twins/VICReg), so the conceptual novelty appears moderate without head-to-head baselines. 

Claim–implementation gaps further temper the contribution: the “top-down memory recall” component is effectively label-driven feedback reconstruction (no addressable memory, retrieval/gating, replay, or partial-cue/label-free recall). 

Scalability is unclear because ReCo forms a B×B cosine matrix per layer (O(B²) compute/memory) with no memory profiling or scalable approximations, leaving open questions about large-batch, streaming/micro-batch, and deeper/wider models. 

External validity is limited to small datasets and shallow backbones (MNIST/SVHN/CIFAR; 6-layer MLP/5-layer CNN), with no results on modern architectures (ResNet) or large-scale data (ImageNet), so it is difficult to judge whether the gains persist at realistic scale or against strong local-learning baselines for SNNs.

---

References 

Deep Supervised Learning Using Local Errors, https://pmc.ncbi.nlm.nih.gov/articles/PMC6127296/

Greedy Layerwise Learning Can Scale to ImageNet, https://proceedings.mlr.press/v97/belilovsky19a/belilovsky19a.pdf

Decoupled Neural Interfaces using Synthetic Gradients, https://arxiv.org/abs/1608.05343

Random synaptic feedback weights support error backpropagation for deep learning, https://www.nature.com/articles/ncomms13276

Training Neural Networks with Local Error Signals, https://www.gatsby.ucl.ac.uk/~pel/tnlectures/papers/local2019.pdf

Difference Target Propagation, https://arxiv.org/abs/1412.7525

Biologically Motivated Algorithms for Propagating Local Target Representations (LRA-E), https://ojs.aaai.org/index.php/AAAI/article/view/4389/4267

Synaptic Plasticity Dynamics for Deep Continuous Local Learning (DECOLLE), https://www.frontiersin.org/articles/10.3389/fnins.2020.00424/full

A solution to the learning dilemma for recurrent networks of spiking neurons (e-prop), https://www.nature.com/articles/s41467-020-17236-y

Barlow Twins: Self-Supervised Learning via Redundancy Reduction, https://proceedings.mlr.press/v139/zbontar21a.html

VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning, https://arxiv.org/abs/2105.04906


Decision: Accept (Poster)