SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zihang Shao $^{1}$  Xuanye Fang $^{1}$  Yaxin Li $^{1}$  Chaoran Feng $^{1}$  Jiangrong Shen $^{2}$  Qi Xu $^{1,2*}$
$^{1}$ School of Artificial Intelligence, Dalian University of Technology
$^{2}$ College of Computer Science and Technology, Zhejiang University
Spiking neural networks (SNNs) have undergone continuous development and extensive research over the decades to improve biological plausibility while optimizing energy efficiency. However, traditional deep SNN training methods have some limitations, and they rely on strategies such as pre-training and fine-tuning, indirect encoding and reconstruction, and approximate gradients. These strategies lack complete training models and lack biocompatibility. To overcome these limitations, we propose a novel learning method named Deep Spiking Neural Networks with Joint Excitatory Inhibition Loop Iterative Learning (EICIL). Inspired by biological neuron signal transmission, this method integrates excitatory and inhibitory behaviors in neurons, organically combining these two behavioral modes into one framework. EICIL significantly improves the biomimicry and adaptability of spiking neuron models and expands the representation space of spiking neurons. Extensive experiments based on EICIL and traditional learning methods show that EICIL outperforms traditional methods on various datasets such as CIFAR10 and CIFAR100, demonstrating the key role of learning methods that integrate both behaviors during training.
In recent years, the spiking neural network (SNN) has garnered significant attention and research in the fields of neuroscience and artificial intelligence (AI) [33, 26, 7]. As a computational model that simulates the functioning principles of biological neural systems, SNN exhibits high biological plausibility and energy efficiency, making it a crucial approach for neural information processing[17, 20, 14]. However, traditional deep SNN training methods face certain limitations, such as reliance on pre-training and fine-tuning strategies, indirect encoding, and approximate gradients. These limitations lead to the training of SNNs leaning more towards deep learning-based methods [12, 17, 20, 3] and away from the principles of brain heuristics.
In order to enhance the biological plausibility and adaptability of SNN, and overcome these limitations, we propose a novel learning and training method called Excitatory-Inhibitory Cooperative Iterative Learning (EICIL). The core idea of EICIL is to simulate the excitatory and inhibitory behaviors of biological neurons and seamlessly integrate them into the training process of SNN. In this way, EICIL can better emulate the characteristics of neural signal transmission and expand the representation space of SNN.
* Corresponding author: xuqi@dlut.edu.cn
In particular, EICIL uses an iterative training framework that emphasizes and optimizes the interaction between excitatory and inhibitory mechanisms. During the training process, EICIL combines excitatory and inhibitory mechanisms in both temporal and spatial dimensions, and enables collaborative training of the same network model structure. Additionally, we gradually adjust the ratio between excitatory and inhibitory mechanisms, aiming to approximate the ideal ratio of approximately 8:2[18]. This allows us to achieve better performance and effectiveness in SNN learning.
To validate the effectiveness of the EICIL method, we conducted extensive experiments and compared it with traditional deep SNN training methods. The experimental results demonstrate significant performance improvements of EICIL across multiple datasets, such as MNIST[5], CIFAR10 [19] and CIFAR100[20]. This further confirms the potential and advantages of the EICIL method in enhancing SNN learning capabilities. The main contributions in this paper are summarized as follows:
- This paper introduces a joint learning method based on excitatory and inhibitory mechanisms, namely the EICIL method. ELCIL provides a comprehensive and effective learning framework for deep SNN training and overcomes the limitations of traditional approaches. By integrating excitatory and inhibitory mechanisms in a cohesive manner, EICIL can better mimic the functional principles of the neural system, thereby enhancing the biological plausibility of SNN.
- The proposed method combines excitatory and inhibitory mechanisms for training. In addition, this paper introduces a multi-layer STDP (Spike-Timing-Dependent Plasticity) method as the inhibitory mechanism. At the same time, considering the poor training effect of the multi-layer STDP, we also modify and adjust the multi-layer STDP method. And keep it biologically interpretable and optimize its training results.
- EICIL has shown the ability to enhance the biological interpretability and adaptability of SNN, and expand its representation space. And it has achieved significant experimental results. This research is of great significance for further advancing the development of spiking neural networks in both theoretical and practical aspects.
2.1 Learning of Spiking Neural Networks
In recent years, a number of learning methods have been developed to train deep SNNs with excellent performance, which can be categorized as ANN-to-SNN conversion (ANN2SNN) and end-to-end backpropagation methods. ANN2SNN pre-trains a source ANN and then converts it to an SNN by replacing the artificial neurons with spiking neurons. The main idea is to use the firing rate of the spiking neuron or average postsynaptic potential to approximate the ReLU activation of the artificial neuron. Although some advanced transformation methods achieve high accuracy on large-scale datasets in trained artificial neural networks with ResNet and VGG structures[6, 8, 16, 22], they ignore the rich temporal dynamics and require many time steps to achieve precoding precision. In contrast, backpropagation methods train SNNs directly by unfolding the network over time steps and then computing surrogate gradients in the spatial and temporal domains, which draws on the idea of training recurrent neural networks (RNNs) via temporal backpropagation [26, 31]. Since the gradient of the spikes with respect to the membrane potential is not differentiable, surrogate gradients are proposed by approximating the gradient with a smoothing function [25]. The backpropagation method with gradient surrogate can be applied to both static and neuromorphic datasets and requires much fewer time steps than the ANN2SNN method. Another backpropagation method is the time-based [32], which directly calculates the gradient from the spike firing time to the membrane potential without expanding the network over the time step. Inspired by the residual connections of Resnet and VGG network models, our EICIL improves the connection structure of residual networks to increase the representation space of SNN network models and improve the performance of SNNs.
2.2 Biological interpretability of Spiking Neural Networks
The Surrogate Gradient Method(GS)[1, 27] and Spike-Timing-Dependent Plasticity(STDP)[4, 29] are two concepts commonly used in neuroscience to describe the synchronization behavior between neurons and synaptic plasticity, which can be used for weight learning in convolutional and fully
connected layers in SNNs. However, GS is too simplified for the synchronization relationship between neurons, but synchronization in the brain is relatively complex, and there exist different forms such as partial synchronization and phase synchronization, so there are limitations in explaining and modeling the real synchronization phenomenon in the brain [24, 28, 30, 23]. Although STDP has great significance in describing synaptic plasticity, STDP ignores other forms of synaptic plasticity [2], such as long-range synaptic enhancement (LTP) and long-range synaptic inhibition (LTD). In order to make up for the shortcomings of the above methods, based on the idea of spike/no-spike classification[10, 13, 15] and combining the characteristics of STDP and GS, we developed an iterative training method of STDP and GS, aiming to improve the expression ability of spiking neurons and SNN.
3.1 Traditional IF
Integrate Fire(IF)[11]is based on the principle of integration and firing of neurons for information transfer. Neurons receive input spike sequences from Presynaptic neurons and integrate the input signals by weighting and summing the input spike sequences. The weights used in the integration process can be adjusted according to the specific task and network structure. When the integrated potential reaches the threshold of the neuron, the neuron fires a spike. At this point, the integrating potential resets to the baseline level, and the neuron undergoes a period of absolute underdrive. It does not respond to any input spikes during this time. The IF model is calculated as Eq. (4)
$$
V (t) = \sum_ {i} w _ {i} \cdot S \left(t - t _ {i}\right) \tag {1}
$$
where  $V(t)$  denotes the integrated potential of the neuron at time  $t$ ,  $w_{i}$  denotes the weight of the  $i$ th input spike, and  $S(t - t_i)$  is the firing function that represents the effect of the input spike at the moment of time  $t - t_i$ .
The condition for determining whether a neuron issues an output spike is
$$
V (t) \geq V _ {\text {t h r e s h o l d}} \tag {2}
$$
If the integrated potential  $V(t)$  exceeds the threshold  $V_{\mathrm{threshold}}$ , the neuron will fire a spike.
3.2 Excitatory and inhibitory in neural systems
In the biological nervous system, signal transmission models mainly include two aspects: synaptic transmission and intra-neuronal electrical signal transmission. In terms of synaptic transmission, there are mainly two types: excitatory transmission and inhibitory transmission. Excitatory and inhibitory neurons are usually combined to produce complex behaviors and functions. For example, in the cerebral cortex, there is a widespread neural network between excitatory and inhibitory neurons, which can support higher functions such as perception, cognition, and decision-making. In this type of network, excitatory and inhibitory neurons are usually mixed in a certain proportion. For example, in the visual cortex, about 80 percent of neurons are excitatory neurons, and 20 percent are inhibitory neurons. This proportion may vary in different brain regions and different biological species. In this combination, excitatory neurons can activate adjacent neurons by generating spiking signals, while inhibitory neurons can suppress adjacent neurons by generating opposite spiking signals. This interaction between excitatory and inhibitory can generate complex dynamic behaviors and support information processing and control functions in the nervous system.
4.1 Surrogate Gradient Method
The Surrogate Gradient Method is a commonly used technique in training spiking neural networks (SNNs) that replaces continuous functions in gradient descent with surrogate gradient functions to generate neural spiking output. In the backpropagation algorithm of SNNs, the gradient of each neuron is discrete in time, but traditional gradient descent algorithms require continuous gradient
values to update weights, which necessitates converting discrete gradient values into continuous functions. GS achieves the use of discrete gradients by replacing these continuous functions with spike functions[1]. In this study, we select GS as the excitatory signal transmission method and choose sigmoid function as the gradient surrogate function. At the same time, we use the most basic neuron model to generate spikes. Below is the description of the sigmoid gradient approximation function: sigmoid function itself is defined as:
$$
\operatorname {s i g m o i d} (x) = \frac {1}{1 + e ^ {- x}} \tag {3}
$$
$$
s i g m o i d \_ g r a d (x) = s i g m o i d (x) \cdot (1 - s i g m o i d (x)) \tag {4}
$$
4.2 Multi-STDP
STDP is a synaptic plasticity rule based on spiking neurons, which simulates the process of synapses continuously strengthening or weakening with the interaction between neurons. When a neuronal synapse receives a series of spikes, there will be plastic changes due to the difference in the timing of the firing of the spikes between the pre- and post-synaptic neurons. Specifically, if the pre-synaptic neuron spikes before the post-synaptic neuron, the strength of the synapse increases, while if the pre-synaptic neuron spikes after the post-synaptic neuron, the strength of the synapse decreases. Let  $w_{ij}(t)$  be the weight of the synapse from neuron  $i$  to neuron  $j$  at time  $t$ , and  $\Delta t = t_j - t_i$  be the time difference between the spikes of neurons  $i$  and  $j$ . The STDP rule can be expressed as Eq. (5)
$$
\left\{ \begin{array}{l l} \Delta w _ {i j} (t) = \eta_ {+} \cdot f _ {+} (\Delta t) \cdot \left(1 - w _ {i j} (t)\right), & \Delta t > 0 \\ \Delta w _ {i j} (t) = - \eta_ {-} \cdot f _ {-} (\Delta t) \cdot w _ {i j} (t), & \Delta t <   0 \\ \Delta w _ {i j} (t) = 0, & \Delta t = 0 \end{array} \right. \tag {5}
$$
Where,  $\eta_{+}$  and  $\eta_{-}$  represent the learning rates for synaptic potentiation and depression, respectively, and  $f_{+}$  and  $f_{-}$  denote the weight update functions for positive and negative spike timing differences. The commonly used weight change function is exponential.
$$
\left\{ \begin{array}{l} f _ {+} (\Delta t) = e ^ {- \Delta t / \tau_ {+}} \\ f _ {-} (\Delta t) = e ^ {\Delta t / \tau_ {-}} \end{array} \right. \tag {6}
$$
Where,  $\tau_{+}$  and  $\tau_{-}$  are the time constants for synaptic potentiation and depression, respectively.
Although for many years, due to factors such as the dynamic nature of neurons, complex interneuronal connections, and training time and computational costs, multi-layer STDP models have been rarely used by researchers[21]. However, in terms of lateral inhibition, the STDP rule can make the lateral inhibition effect of a neuron more significant. Therefore, we adopt a multi-layer STDP model as the inhibitory mechanism of the EICIL structure.
In a multi-layer STDP model, the STDP method is utilized to perform temporal logic processing of spike neuron propagation. In this model, the STDP rule is employed for weight adjustments in the linear layer, and convolutional layer.
The STDP rule adjusts connection weights based on timing differences in firing spikes between neurons. When the timing of the spikes between neurons is consistent, the connection weight is strengthened. When the timing difference is large, the connection weight is weakened. So the strength of connections between neurons is adjusted according to their activity patterns. In this way, the connection weights between neurons in the multi-layer STDP model will be adjusted according to their temporal relationship, thereby achieving multi-layer propagation. This timing-dependent plasticity mechanism allows the neural network to adaptively adjust the connection strength during the learning process to suit the statistical characteristics of the input data and task requirements.
In the initial experiments, we found that the original multi-layer STDP method performed extremely poorly in training across three network structures. It even failed to reach an accuracy of 50 percent on the CIFAR10 dataset. Upon analyzing the multi-layer STDP model and its code, we discovered that the activation of neurons and synaptic weight updates only occurred during the forward propagation process. This limitation meant that the gradient information from backpropagation could not directly propagate to earlier layers, thereby restricting the model's learning capability and training effectiveness. Additionally, the inability to effectively transmit gradient information to earlier layers resulted in either diminishing gradients (gradient vanishing) or exponentially increasing gradients
(gradient explosion). This instability during the training process made it challenging to achieve good convergence. Furthermore, the model lacked the ability to simultaneously adjust weights across the entire network to minimize the overall loss function. Therefore, after careful consideration, we introduced the STDP-BW method by incorporating the backpropagation technique into the STDP model. The integration of the STDP model with backpropagation presents a comprehensive and enhanced framework for training multi-layer networks. This approach combines the biological principles of synaptic plasticity with the optimization capabilities of backpropagation. It not only takes into account the biological principles of synaptic plasticity but also leverages the optimization power of backpropagation. This integration provides a more comprehensive and advanced framework for training multi-layer networks.
4.3 The combination of excitability and inhibition
4.3.1 Approach I: Heterogeneous Learning
We combine excitatory and inhibitory mechanisms by using them to compute different types of network layers within the same network structure. The excitatory mechanism is represented by the gradient replacement method, and the inhibitory mechanism is represented by the STDP method. The neural models for the gradient replacement method and the STDP method are denoted by  $F(\cdot)$  and  $G(\cdot)$ , respectively. For a neuron  $j$ , its membrane potential  $V_{j}(t)$  at time  $t$  changes according to:
$$
\frac {d V _ {j} (t)}{d t} = \sum_ {i = 1} ^ {N} w _ {i j} F \left(s _ {j} (t - \Delta) - \theta_ {j}\right) + \sum_ {i = 1} ^ {N} w _ {i j} G \left(s _ {j} (t - \Delta) - s _ {i} (t - \Delta) - \theta_ {i j}\right) \tag {7}
$$
where  $w_{ij}$  represents the connection weight between neurons  $i$  and  $j$ ,  $\theta_j$  represents the threshold of neuron  $j$ ,  $s_j(t)$  represents the signal indicating whether neuron  $j$  has fired at time  $t$ , and  $\Delta$  represents the time delay of spiking propagation. This equation expresses the rate of change in the membrane potential of a neuron in response to input from other neurons, representing its excitatory or inhibitory nature.
In this study, we propose a novel approach combining STDP models, backpropagation, etc. to train different layers of a neural network. We utilize the STDP model to train the last linear layer in the network to dynamically update the weights according to the spiking neural network. For the convolutional layer and other layers, we employ a gradient-based method to optimize the weights. This framework facilitates the integration of excitatory and inhibitory training by applying STDP-based weight adjustments across the network after each training round.
Furthermore, we conducted additional experiments to explore various training configurations. First, we exclusively trained the convolutional layer using the STDP model while utilizing the gradient-based method to train the remaining layers. This allowed us to evaluate the individual impact of STDP training on the convolutional layer. Additionally, we investigated a hybrid approach where the convolutional layer was trained using both STDP and backpropagation, while the other layers were trained using the gradient-based method. This multiple-STDP-based inhibitory training regime enabled iterative fine-tuning of the neural network's weight parameters, creating a dynamic interplay between excitatory and inhibitory processes. We refer to this method as STDP-BW-GS.
4.3.2 Approach II: Cycle Iteration Learning
The second method involves training different epochs of the same network using both excitatory and inhibitory mechanisms within the network's structural framework. This approach entails training the complete network with different methods at distinct time steps, effectively encompassing a single training session within each time interval.
In terms of biological interpretability, excitatory and inhibitory mechanisms exhibit alternating and interacting patterns within neurons, employing diverse mechanisms to influence signal propagation. In the biological neural system, signal transmission between neurons is subject to temporal dynamics, including time delays and other temporal information. The integration and analysis of these temporally modulated signals contribute to processing the input information before transmitting it for further analysis in the brain. Motivated by this, we adopt distinct methods to train neurons during different epochs. Each epoch represents a training cycle within the network model, utilizing different time steps as intervals to train the network using excitatory and inhibitory mechanisms. We
leverage the current optimal results obtained from the excitatory mechanism to train the inhibitory mechanism, and reciprocally, use the current optimal results from the inhibitory mechanism to train the excitatory mechanism. This iterative process facilitates the network model's adaptation to both excitatory and inhibitory mechanisms, culminating in the attainment of optimal model parameters through synergistic training.
In this case, the network's connection weights at each time step can be represented as  $W_{t}$ , where  $t$  denotes the time step. At each time step  $t$ , the following update rules using gradient replacement and STDP can be employed as Eq. (8) and Eq. (9):
$$
W _ {t + 1} = W _ {t} - \eta \frac {\partial W _ {t}}{\partial L} \tag {8}
$$
$$
\Delta W _ {i j} (t) = \sum_ {k \in S ^ {+}} f _ {+} \left(t - t _ {k}\right) - \sum_ {k \in S ^ {-}} f _ {-} \left(t - t _ {k}\right) \tag {9}
$$
Where,  $L$  represents the network's loss function,  $\eta$  denotes the learning rate, and  $i$  and  $j$  represent the indices of the neurons.  $S^{+}$  and  $S^{-}$  correspond to the excitatory and inhibitory spiking sequences received by neuron  $j$  from neuron  $i$ , respectively, and  $f_{+}$  and  $f_{-}$  represent the spatiotemporal weight functions for excitatory and inhibitory spiking. The updated connection weights can be obtained as Eq. (10)
$$
W _ {t + 1} = W _ {t} + \eta \Delta W _ {t} \tag {10}
$$
This approach can preserve the biological realism of SNN models to a certain extent while harnessing the powerful capabilities of deep learning.
Simultaneously, considering that the excitatory and inhibitory mechanisms in the visual neural network of the human brain do not exhibit a 1:1 ratio, we introduce a proportional adjustment of training durations for these mechanisms by controlling the ratio of time steps. Specifically, we introduce a parameter  $\mathrm{K}$  to modulate the relative durations of training for the excitatory and inhibitory mechanisms. When  $\mathrm{K}$  is a positive value, it indicates that the inhibitory mechanism accounts for a proportion of  $1 / \mathrm{K}$ . Conversely, when  $\mathrm{K}$  is a negative value, it indicates that the excitatory mechanism accounts for a proportion of -1/K. By tuning the value of  $\mathrm{K}$ , we seek to determine the optimal ratio. Here, we refer to the iteration using the GS method and STDP-BW method as GSI, and the iteration using the GS method and STDP-BW-GS method as GSGI.
4.4 Network model based on excitatory inhibition