Training Deep Normalization-Free Spiking Neural Networks with Lateral Inhibition
Authors: Peiyu Liu
Jianhao Ding
Zhaofei Yu
Venue: ICLR 2026 Poster (2026)
Keywords: Spiking Neural Networks
Normalization
Excitation-Inhibition Balance
Lateral Inhibition
Primary area: applications to neuroscience & cognitive science
TL;DR: We propose a SNN learning framework that enables the effective training of deep SNNs without explicit normalization.

Abstract: Spiking Neural Networks (SNNs) have garnered significant attention as a central paradigm in neuromorphic computing, owing to their energy efficiency and biological plausibility. However, training deep SNNs has critically depended on explicit normalization schemes, leading to a trade-off between performance and biological realism. To resolve this conflict, we propose a normalization-free learning framework that incorporates lateral inhibition inspired by cortical circuits. Our framework replaces the traditional feedforward SNN layer with distinct excitatory (E) and inhibitory (I) neuronal populations that capture the key features of the cortical E-I interaction. The E-I circuit dynamically regulates neuronal activity through subtractive and divisive inhibition, which respectively control the excitability and gain of neurons. To stabilize end-to-end training of the biologically constrained SNNs, we propose two key techniques: E-I Init and E-I Prop. E-I Init is a dynamic parameter initialization scheme that balances excitatory and inhibitory inputs while performing gain control. E-I Prop decouples the backpropagation of the circuit from the forward pass, regulating gradient flow. Experiments across multiple datasets and network architectures demonstrate that our framework enables stable training of deep normalization-free SNNs with biological realism, achieving competitive performance. Therefore, our work not only provides a solution to training deep SNNs but also serves as a computational platform for further exploring the functions of E-I interaction in large-scale cortical computation. Code is available at https://github.com/vwOvOwv/DeepEISNN.

Published as a conference paper at ICLR 2026
TRAINING DEEP NORMALIZATION-FRE: SPIKING NEURAL NETWORKS WITH LATERAL INHIBITION
Peiyu Liu 12, Jianhao Ding' & Zhaofei Yu!
School of Computer Science, Peking University
2 Institute for Artificial Intelligence, Peking University
ABSTRACT
Spiking Neural Networks (SNNs) have garnered significant attention as cen- tral paradigm in neuromorphic computing, owing to their energy efficiency and biological plausibility. However, training deep SNNs has critically depended on explicit normalization schemes. leading to a trade-off between performance and biological realism. To resolve this conflict, we propose normalization free learn- ing framework that incorporates lateral inhibition inspired by cortical circuits. Our framework replaces the traditional feedforward SNN layer with distinct excita- tory (E) and inhibitory (I) neuronal populations that capture the key features of the cortical E-I interaction. The E-I circuit dynamically regulates neuronal ac- tivity through subtractive and divisive inhibition, which respectively control the excitability and gain of neurons. To stabilize end-to-end training of the biologi- cally constrained SNNs, we propose two key techniques: E-I Init and E-I Prop. E-I Init is a dynamic parameter initialization scheme that balances excitatory and inhibitory inputs while performing gain control. E-1 Prop decouples the backprop- agation of the circuit from the forward pass, regulating gradient flow. Experiments across multiple datasets and network architectures demonstrate that our frame- work enables stable training of deep normalization-fiee SNNs with biological re- alism, achieving competitive performance. Therefore, our work not only provides a solution to training deep SNNs but also serves as a computational platform for further exploring the functions of E-I interaction in large-scale cortical computa- tion. Code is available at htthtttss//gtthhieee com/ http:///OOOww/DeeeEE..
INTRODUCTION
Inspired by computational principles of biological neurons, Spiking Neural Networks (SNNs) stand at the intersection of artificial intelligence and neuroscience (Maass 1997 Ghosh-Dastidar & Adeli,
2009 They not only enable highly energy-efficient computation neuromorphic hardware (Roy et al. 2019 Xiao et al. (2025) but also provide models for understanding cortical computation across multiple scales (Kumarasinghe et al. 2021 Koresak-Gorza al.) 2022] N'dri et al. (024). This
duality places SNNs at the heart of the emerging field of NeuroA1 (Sadeh & Clopath 2025). fos- tering a synergy between artificial intelligence and neuroscience. While deeper understanding of the brain's 's computational principles inspires novel SNN architectures (Fang et al. 2021b) Pan jet al. (202)), advances deep learning have concurrently enabled the training of large-scale, high- performance SNNs (Wu et al., 2018 Fang et al.) 202 la Bu et al. (022).. Some of these well-trained models, in turn, can serve as in silico platforms for investigating multi-scale cortical computation that is difficult to access through wet-lab experiments (Bellec et al. 2018).
Despite the promising synergy, realizing the potential of SNNs as an ideal platform for exploring both machine and biological intelligence is hindered by trade-off between computational perfor- mance and biological plausibility. Many learning algorithms achieve high performance by adopting backpropagation based techniques, treating spiking neurons recurrent units unrolled over time (Neftci et al. 2019 Fang et al. 2021a). While this strategy yields models whose performance is comparable their Artificial Neural Network (ANN) counterparts, it reduces SNNs mere deep
"Corresponding author; yuzfl 2pp..e.nkued

---

Published as a conference paper at ICLR 2026
Batch BatchNorm
Batch Norm
1ayer
Layer
Layer ++1
Layer
Layer
Layer (
Excitatory Neuron
Inhibitory Neuron
Unconstrained Neuron
Feedforward Excitation
Lateral Inhibition
Unconstrained Connection
Figure 1:
The proposed feedforward E-I circuit (left), compared with normalization-cqqipped archi- tecture (right, BN as example). Neurons in layer - 1 and + are not shown.
learning artifacts, sacrificing basic biological properties. As result, these models often ignore fun- damental principles in neuroscience such as E-I dynamics, which are crucial in gain control (Gold- wyn et al. 2018 Del Rosario al., 20055) neural oscillation (Buzsáki & Draguhn 2004), selective attention (Zhang et al.) (014), etc. Consequently, deriving meaningful insights for neuroscience from these biologically unfaithful models becomes challenge.
However, approaches that prioritize biological realism also face the challenge of unstable training. Biologically plausible learning rules such as spike-timing-dependenl plasticity (STDP) (Gerstner et al. 1996: Bi & Poo. 1998). often struggle with the instability during training and thus can only be applied shallow SNNs (Habenschuss
al. 2012 Beyeler et 2013). the field of deep learning, this training instability issue is partly overcome by explicit normalization schemes, most notably batch normalization (BN) (Ioffe Szegedy 2015). While such normalization schemes are powerful tools to accelerate and stabilize training, they explicitly collect statistics from inputs, which has no known biological analogue, making it implausible for brain-inspired models. This widens the gap between high performance and biological plausibility, highlighting the need for biologically grounded alternative.
Here, we address the challenge of training deep biologically plausible SNNs by introducing lateral inhibition, canonical interaction mechanism between excitatory and inhibitory neurons in cortex. E-I circuit, as shown in Figure
We propose normalization free learning framework based Our framework presents a brain-inspired alternative to standard normalization schemes, bridging the gap between perfoperformance deep learning and biologically plausible neural computation. The main contributions of our work are summarized as follows:
We incorporate a canonical E-I circuit, composed of distinct excitatory and inhibitory neu- populations, into deep SNNs to enable normalization-fiee training.
2. We introduce dynamic initialization scheme to ensure effective learning from the very beginning of training.
3. We integrate adaptive stabilization of divisive inhibition and straight-through estimator (STE) into the framework. These techniques prove to be essential for stable learning of deep SNNs with the circuit.
4. Experiments demonstrate that the framework achieves competitive performance across dif- ferent datasets and architectures, indicating the viability of our brain-inspired learning al- gorithm.
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
2.1 NORMALIZATION IN SNNS
Recent methods for training deep SNNS can be broadly categorized into two approaches. ANN-to-
SNN conversion (ANN2SNN) (Rueckauer et al.
2017:
Sengupta et al. 2019 Han et al. 2020: Han
& Roy, 2020; Ding et al., 2021:
Stöckl & Maass
2021
Bu al. 2022] Zhao et al.] (0255 and direct
end-to-end training (Lee et al.
2016 et al.
2019 Li et al. 2021 Fang et al. 2202aa Guo
Normalization is critical in both methods. Many ANN2SNN methods merge normalization parameters of ANNs into synaptic weights of spiking
al.) 2022b Xiao al.) 2022
Zhu et 2024).
neurons (Sengupta et al. 2019; Han al., 2020 Bu all 2022). In contrast, training SNNs from scratch usually directly adopts normalization schemes developed for ANNs, especially BN (Loffe & Szegedy, (213)). There are also BN-derived normalization schemes designed for SNNS, like NeuNorm (Wu et al. 2019). BNTT (Kim & Panda (021). tdBN (Zheng et al., (02T), TEBN (Duan al. 2022), and TAB (Jiang et al. (024). However, these strategies still inherit the biological implausibility of normalization schemes due to their dependence on statistics collected from batch
inputs throughout the training. Therefore, the need for fully brain-inspired normalization alternatives remains.
2.2 NEURAL NETWORKS WITH SEPARATE EXCITATORY AND INHIBITORY UNITS
The interaction between excitatory and inhibitory neurons has been a key topic in neuroscience (Haider et al. 2006; Ahmadian & Miller 2021 Cohen Kadosh 2025). ) Historically, computational models of these circuits have been confined to shallow networks, often focusing on the dynamics of few interacting populations/neurons to explain basic principles (Somers et al. 1995 Wilson & Cowan 1972 Carandini & Heeger 2012). One of the reasons for this limitation is that training deep networks with the E-I circuit proves to be significant challenge. This has left the whole picture of E-I dynamics largely unexplored. A remarkable step towards deep E-I networks was taken by Cornford al. (202I). By developing techniques for parameter initialization, they demonstrated that ANNs with separate excitatory and inhibitory neuron populations could be effectively trained. However, the model is built fully with the rectified linear units (ReLU) and thus does not capture the temporal properties of the circuit. While SNNs can address temporal processing, such initialization techniques cannot be directly applied to SNNs with the E-I circuit.
PRELIMINARIES
3.1 EXCITATION AND INHIBITION IN CORTEX
A fundamental principle in the cortex is the functional segregation of neurons into distinct excita- tory and inhibitory populations (Barranca et al. 2022 This circuit-level architecture implies that a given neuron typically exerts a uniform influence (either depolarization or hyperpolarization) on all its postsynaptic targets. When translating this principle to artificial neural networks, all outgoing synaptic weights from a given neuron should share the same sign. This makes standard initialization techniques like Xavier (Glorot & Bengio 2010) and Kaiming (He al., 2015) initialization inappli- cable. as they sample weights from zero-centered distributions that assign both positive and negative weights each neuron.
3.2 NEURON MODELS
Excitatory neurons. We model excitatory neurons with the widely adopted leaky-integrate-and- fire (LIF) model (Gerstner al. 2014). For given layer I with ng superscript [[] to denote the 1-th layer), the sub-threshold dynamics of the membrane potential
!!] excitatory neurons (we use
ug (t) E T are described by the following equation:
du g (t) - up (1) WF, + (L),
(1)
dt
where To and ME,ret are the membrane time constant and resting potential of all excitatory neurons, respectively. I!l (1) E T represents the input currents the excitatory neurons time assuming
3

---

Published as a conference paper at ICLR 2026
unit membrane resistance. For discrete- teterette-meeeeee simulation, approximate Equation using the first- order Euler method with a time step At 1. By setting Wg,test to 0 and omitting decay input currents, obtain the following iterative update rule:
M t + 1| =
1 тв
[
(2)
excitatory neuron emits a spike when its membrane potential exceeds a firing threshold 0p (which is set to I for all excitatory neurons). To model the subsequent reset, we employ soft reset mech- anism where the potential a firing neuron is reduced by 0g. This leads to full dynamics for excitatory neurons in layer
_ (1- TE
-1-11)11
(u 1 (u
PD stll) 1 [t],
E
+11==
(3)
where the spikes are generated by the Heaviside step function H, i.., sg [t] H
0g)
To distinguish this process from the dynamics of inhibitory neurons in layer I, we encapsulate into an operator F This operator takes the input currents at the current time step its arguments and produces the corresponding output spikes.
- [+]
F [t: [t]. TE, TE),TE,00 0p)
(4)
Inhibitory neurons. Many inhibitory neurons, especially parvalbumin (PV+) neurons, are known to be fast-spiking (FS), characterized by a much smaller membrane time constant πi compared to that of excitatory pyramidal neurons (Hu al.] 2014 Prince et al. (021). In our discrete-time simulation, the time step A/- = is chosen to be on a similar scale as the time constant excitatory neurons (e.g., TE 2), which implies π < Al since 7 < TE- Under this condition, the dynamics of inhibitory neurons can reach a steady state almost instantaneously within a single time step. This allows us apply an approximation to LIF model by treating πI as negligible, which leads to
- 0u-(utttt) U1.rest) +11118
Here we use notations similar to those in the excitatory neuron model, and the subscript I denotes inhibitory neurons. By setting rerrest 0 we find the membrane potential of inhibitory neuron is determined purely by its input currents at time t:
u M [t [
Since this potential remains constant throughout duration At, the neurons can fire max (o. (. [t) /0 times we apply soft reset and fixed firing threshold 0.. Finally, by setting 01 1. can directly model the total spike outputs of inhibitory neurons at time f as
! [ = max (o. (0.1%(t)) ~ max 10.
s
Similar to the excitatory neuron model, we encapsulate this process into an operator F!"): [ - =
(1")tt) ss max (o. 11)
(8)
See Appendix D.1 for detailed derivation.
METHODS
4.1 E-I CIRCUIT IN SNNS
The model is constructed according to the canonical E-I circuit shown in Figure 1 Fach layer [1]
I comprises ng excitatory neurons and n²) inhibitory neurons, where n /n/ 4 according to
biological evidence (Markram et al. 2004). Computation performed by this circuit at each time step
t is demonstrated in Figure
4

---

Published as a conference paper at ICLR 2026
[[c]
Inaibawry Netrons
Neunma
E-J Prop
M!ll
b2
Element-wiss Addition
Trainable Panameters
Excitatory Ouipuls
Element wise Mulkiplicaiin
Inhibitory Ouputs/Urpputs
Element-wiee Division
Integrated Currents
Matrix Multiplication
Figure 2: An overview of the proposed framework. E-I Init enables effective learning from the very beginning through a dynamic parameter initialization scheme. E-I Prop then ensures stable end-to- end training by regulating the forward and backward passes. For the sake of brevity, our analysis in the main text focuses on a fully connected architecture. Extension of our method to convolutional neural networks (CNNs) is detailed in Appendix E.I
Excitatory projections. First, excitatory population in layer undergoes dynamics described by Equation to Equation and emits spikes $ [t] E {0, 1}d, where is the dimension of input spikes. These input spikes induce two excitatory currents into both the excitatory and inhibitory populations of layer I:
I beld
w [ [t]. E
(9)
4 w R-
w [[].
(10)
Here, subscript AB denotes projections from population B to population A. will EB
E R'
_ xd and
IE will! E RR xd are corresponding synaptic weight matrices, which are constrained to non-
negative during training due to E-1 segregation.
Lateral inhibition. Following the fast-spiking approximation Section 3.2 the activity of in- hibitory neurons is modeled by F!!!
'"u = Fl (H
(11)
This inhibitory signal then laterally regulates the excitatory population. Motivated by the biophysical distinction between dendritic and somatic inhibition, we decompose this regulation into subtractive inhibition for E-I balance and divisive inhibition for gain control.
El. [d]
"
w E [t.
(12)
1 ELai = will
g 11)
(13)
where will] EI R xnf") is the weight matrix for inhibitory-to-exiittte to-exitator projections, gi E Re is a trainable parameter modulating the strength of divisive inhibition, and @ denotes the Hadamard product.
[1]
5

---

Published as a conference paper at ICLR 2026
Input integration and spiking. Finally, the excitatory population integrates the excitatory currents with both forms of inhibition to compute the total input currents.
Bl.ubll
[]
!1 SE
EE
1
(14)
El.
where BE by R are trainable parameters, and the division is performed element- wise. Taking the integrated currents as input currents, excitatory neurons emit spikes to the next layer,
Sp [ F
[1 up [t], TE, OF).
(15)
4.2 E-I INIT: DYNAMIC PARAMETER INITIALIZATION
As discussed in Section 3.1 standard zero-centered initialization schemes like Xavier (Glorot & Bengio 2010) and Kaiming He et al., 2015) are incompatible with the strict sign constraints of F-1 segregation. Some naive initializations under this constraint lead pathological network activity, making the training of deep architectures infeasible (see Section 5.2). Therefore, we propose E-I Init, an initialization scheme designed for deep SNNs with the E-I circuit. Its design guided by two primary objectives: (1) establishing an initial E-I balance to prevent neurons from silencing or saturating, and (2) setting an appropriate initial gain to ensure stable signal propagation.
E-I balance via subtractive inhibition. A key goal of our initialization scheme is to ensure that neu- rons operate responsive regime. We achieve this by setting the expected subtractive inhibitory currents to approximately balance the expected excitatory currents, which is defined as
E
EE,
E
El.ub,
(16)
for each excitatory neuron i in layer and results in near-zero expected net input, preventing rons from being saturated or silent at initialization. To implement this under the constraint of E-I segregation, we draw inspiration from Cornford al. (202T) and leverage the exponential distribu- tion for weight initialization. Specifically, we draw the excitatory weights wil! EE and wil IE from an exponential distribution with rate parameter Al9, The inhibitory weights will] are deterministically set to 1/m1" to uniformly distribute the inhibitory signals. Here will] E R denotes elements of AB
w AB. Assuming that presynaptic neurons fire independently with an average probability of p (i.e., the spike from each neuron at any time step is independent and identically distributed (ii.d.) Bernoulli trial with probability p), the expected excitatory input to excitatory neuron in layer is
FR.1 = dpl
will EE
(17)
where d is the dimension of input spikes. Similarly, the expected subtractive inhibitory currents are
E
Elaub.s
[1]
1
dpE
w
E
wi[
EI
(18)
Therefore, by setting E balance defined by Equation 16
wil
w[[
1/x0 and EI = 1/m we arrive at the desired
EE
iE
(see Appendix [D.2 for detailed derivation).
Gain control via divisive inhibition. Our second objective is establishing stable signal propaga- tion by setting an appropriate initial gain for excitatory neurons. In our model, gain is primarily modulated by the divisive inhibitory currents. Inspired by normalization techniques, our strategy is modulating divisive inhibition through gi such that the expected value of this divisive inhibitory currents approximates the standard deviation of the excitatory inputs. This can be formulated as
E
[]] dii,div.i]
std
(1
BE
(19)
for each excitatory neuron i in layer In this way, the divisive operation effectively scales the final input, leading the excitatory neurons to a responsive regime at initialization. Similar to Equation 18 for inhibitory neuron i in layer 1,
dpE
! gi
Biv = nof dpl
R g
W TE
W FI
(20)
A

---

Published as a conference paper at ICLR 2026
By assuming that the spike from each at any time step is i..d. Bernoulli trial with probability p, the standard deviation of excitatory input currents of neuron can formulated as
dp(2 p) A
sld
(21)
EE,
see Appendix D.3 for details. Therefore, by setting each element gi to V normalization effect initialization, making the effective training of deep SNNs possible.
11] 2-P dp we achieve a
Initialization of other parameters. Finally, we initialize A d(2-pl by setting the condition std
I-p
v (p)) (see Appendix D.4) gF and by initialized as vector 1 and 0, EEE
respectively.
Dynamic firing probability estimation. Since the initialization depends the averaged firing probability p. we use the first batch in training set to compute point estimations p and other statistics, leading dynamic initialization regime (see Appendix E.2 for implementation details).
4.3 E-I PROP: STABILIZING END-TO-END TRAINING
While E-I Init provides stable initial state, the interplay between divisive inhibition and discrete input spikes induces training instabilities. These instabilities arise mainly from two sources: first, near-zero divisive currents in the forward pass trigger numerical explosions; and second, dispropor- tionately large gradients in the lateral inhibitory pathway destabilize training. overcome these issues, propose E-I Prop, toolkit that decouples the backpropagation of the E-I circuit from the forward pass, regulating gradient flow. The forward stability is ensured by adaptive divisive inhibi- tion, while the backward stability is achieved through a straight-through estimator (STE) combined with gradient scaling.
1.0
0.1
0.2
0.5
1.0
0.0
0.1
0.0
0.2
0.5
Forward (with adaptive stabilization)
Backward (with STE)
Figure 3: Mechanism of adaptive stabilization and STE. Forward (bottom-up): The adaptive sta- bilization handles numerical instability by dynamically replacing zero elements in the denominator with the smallest positive value in the sample, preserving maximal dynamic range. Backward (top- down): The STE allows gradients to bypass the replacement operation, treating as an identity function.
Adaptive stabilization of divisive inhibition. To prevent division-by-zero error in the forward pass, common technique is adding a small constant to the denominator. However, fixed is ill-suited for our network because the divisive inhibitory currents are designed to provide a suitable dynamic range and perform gain control. A pre-defined that too small may fail prevent numerical instability if the denominator collapses towards zero, while one that is too large will artificially suppress the dynamic range by dominating the denominator. As shown in F here we propose an adaptive stabilization method. Instead of using a static constant, our approach sets dynamic, input-dependent lower bound for the denominator. Specifically, for each sample within batch, we identify any zero values in IgLdiv- Then, these zero values are replaced by the smallest positive value found within the same sample, which proves to be necessary for effective training (see Section 5.2 for details).

---

Published as a conference paper at ICLR 2026
Straight-through estimator (STE). Since the replacement operation in adaptive stabilization is non-differentiable, misdirects the gradient flow and destabilizes learning. To address this, we employ STE, common technique for handling non-differentiable operations in neural networks (Bengio et al. 2013). In the forward pass, we perform the adaptive stabilization as described ahove. In the backward pass, approximate the derivative of this non-differentiable operation with an identity function (see Appendix E.3 for implementation). This approach decouples the forward-pass requirement for numerical stability from the backward-pass requirement for a clean gradient path, ensuring that the network can learn robustly.
Gradient scaling, To ensure stable training, it is crucial balance the influence of the feedforward excitatory and lateral inhibitory pathways on parameter updates. Both theoretical and empirical analyses reveal that gradients for the lateral inhibitory weight, w are disproportionately larger than those for other synaptic weights (see Appendix D.5 and F.I). To counteract this gradient amplification, scale the gradients of W [!] input spikes. This effectively balances the update magnitudes between the two pathways.
El by factor of 1/d, where d is the dimension of the
As illustrated in Figure 2 our method provides completely normalization-fre learning framework composed of E-I Init and E-I Prop, enabling stable and effective end-to-end training from scratch.
5
EXPERIMENTS
5.1 PERFORMANCE ON CLASSIFICATION TASKS
We evaluate our framework on both static and event-based datasets, with results summarized in Table [ Our ResNet-18 model (He et al. 2016 Fang al. 2021a) achieves 92.05=0.119% top-1 accuracy on CIFAR-10 Krizhevsky] 2009), ) surpassing all normalization-fre baselines. Notably, on the more challenging TinyImageNe (Le & Yang. 2015), the model attains 50.29% accuracy, demon- strating scalability to large-scale datasets. Furthermore, to validate temporal processing capabilities, test on neuromorphic datasets where our method achieves 94.86.10.86% on DVS-Gesture (Amir iet al. (017)) and 77. 7..66%100488% on CIFARIO-DVS (Li al. (017). outperforming several BN-based methods. Collectively, these results across diverse datasets confirm that our E-I circuit, empowered by E-I Init and E-I Prop, serves as robust and scalable alternative to explicit normalization in deep SNNs.
5.2 ABLATION STUDY
Results of ablation study are summarized in Table 2 confirms that each component in our method is indispensable for stable and
training. Replacing E-I constraint (on sign of weights) or E-I Init by applying a standard/clamped Kaiming initialization (He al. 2015) leads to either a complete training failure or a significant accuracy drop. proving its necessity for establishing a proper E-I balance and gain control through E-I segregation and E-I init. Furthermore, the common c-stabilization fails across all tested values, confirming that our adaptive stabilization mechanism is crucial for maintaining both numerical stability and gain control. Finally, removing gradient scaling on WF causes training collapse, which validates its role in stabilizing learning dynamics. These results demonstrate that E-I constraint, E-I Init, and E-I Prop for effective training of deep E-I SNNs.
5.3 BIMODAL REPRESENTATION LEARNED BY THE E-I CIRCUIT
Visualization of the integrated currents distribution reveals that our framework leverages normalization-lik effect at initialization but ultimately learns more sophisticated representation. As shown in Figure 4 E-I Init successfully produces stable, zero-centered Gaussian-like distribu- tions at the beginning of training. However, after training, some of the distributions evolve into a distinct bimodal shape, in contrast to the Gaussian-like outputs of SNNs with vanilla BN (see Appendix
This emergent bimodality can be considered as a mixture of two Gaussian- like distributions, with one distribution centered at the negative regime. This indicates that the E-I circuit is not equivalent to simple normalization, but rather dynamic separation of neurons into activated and suppressed populations, confirming its role as strategy distinct from standard normalization.
8

---

Published as a conference paper at ICLR 2026
Table Comparison with other E-I constrained, normalization free, and BN-qquippe methods.
Normalization
Method
Architecture
steps
Top-I Accuracy (%)
DAP (Mccelli
2025
7-layer CNN
3
57.52
SRI
Liin et sie 20ca
VGG-9
20
87.62
B. BNNN
(Karimah at 2025
VGG-8
64
87.73
EICIL
ResNes-18
90.34
IM-Loss Gio 21 40224
CIFARNei
90.90
VGG-16
Vanilla BN
VGG-19
94,11 + 0.15
ResNet-18
95.37 + 0.13
BNTT Kim & Panda
2021
VGG-9
20
90.30
NeuNorm Wu et al
2019
CIFARNei
12
90,53
X
VGG-9
92.81
CIFAR-10
TEBN Duan al.
2022
ResNet-19
94,70
IBNN Zheng al.
2021
ResNet-19
92.92
TAB Diang al
2024
ResNet-19
VGG-9
94.76
93.41
FDI Rossbroich et
2022
6-layer CNN
50
65.60
DANN Cornford et al
2021
VGG 19
VGU-16
88.54 d: 0.38
88.28 1. 0.27
EICIL (Shao et
2023
E-1 Net
89.43
BackEISNN Zhao et al
2022
S-layer CNN
20
90099
VGG-16
90.80 =: 0.16
DeepEISNN (Ours)
VGG-19
90.93 0.33
ResNet-18
SRI Ding
2025
VGG-11
20
54,94
X
EICIL Staa eta
ResNet-18
63.47
BNIT Kim Panda
2021
VGG-11
50
66.60
TEBN
VGG-11
CIFAR-100
TAB Diang
2024
VGG-11
75.89
EICIL (Shao 2023
Net
53.86
DeepEISNN (Ours)
VGG-16
VGG-19
64.95 + 1.14
64.31 + 0.41
IM-I Gno 2022
ResNet-19 19
10
72.60
NeuNorm Wu
2019
7-yyeer CNN
40
60.50
BNTT Kim & Pando
2021
7-layer CNN
20
63,20
IdBN Zheng et al
2021
ReaNet-19
10
67,80
CIEARIO-DVS
TEBN Duan al.
Jayee CNN
10
75.10
TAB Tiang el al.
2024
7-layer CNN
76.70
DeepEISNN (Ours)
VGG-8
VGG-11
10
77.66 : 0.48
FDI (Rossboich et al
2022
6-layer CNN
500
86,70
DVS-Gesture
DeepEISNN (Ours)
VGG-8
16
94.86 ::
IlnyImageNet
/
DeepEISNN (Ours)
ResNet-18
$0.29
Results marked with an asterisk are reproduced (averaged over independent runs). results are existing literature.
Table 2: Ablation study individual components of the method CIFAR-10 with VGG-8.
Ablation Setting
E-1 Adap, Stab.
Grad. Scale
Ablation Details
Top-1 Accuracy (%)
Cons.
Init
V
Proposed Method (Ours)
87.03
x
Kaiming Init on Wae and Wie (clamped (o. 4 +oo)) 85.61
Fixed c 10-8
Failed converge
V Fixed e
Failed converge
Fixed 6
Collapsed (Epach 22)
10
Collapsed (Epoch 5)
X No scaling on Wia
Collapsed (Epoch 10)
X
Kaiming Init on Was and Wis (no sign constraint)
Failed to converge

---

Published as a conference paper at ICLR 2026
F-1 Layer #1
Layer #3
H-1 Layer #5
0.6
0.3
0.30
Density
0.4
Density
Density
0.2
0.15
0.2
0.1
0.0
3
3
0.00
3
0
3
0.0
4
Value
Value
Value
Before truining
stops
0.45
Emc stops
stops
0.30
Density
0.16
Density
0.30
Density
0.15
0.08
0.15
0.00
0.00
0.00
-3
Value
Value
Value
After training
Figure 4: Distributions of the integrated input currents in the first, third and fifth layers of our model before and after training.
CONCLUSION AND DISCUSSION
In this work, we address the critical challenge in training deep normalization-free SNNs by intro- ducing biologically inspired E-I segregation and lateral inhibition. Through fine-grained initial- ization scheme E-I Init, and a toolkit of stabilization techniques E-I Prop, we enable the stable end-to-end training of deep normalization-fre SNNs that capture features of canonical E-I circuit in the cortex. Our experiments demonstrate that the framework not only achieves competitive per- formance on multiple datasets but also learns a sophisticated mechanism of activity regulation that functionally distinct from standard normalization. We show that the E-I circuit starts an ini- tial normalization-Il like state, and ultimately learns an activity regulation mechanism distinct from explicit normalizations like BN. Therefore, our work provides both practical solution for build- ing powerful, normalization- free SNNs and a compelling computational model for exploring how canonical cortical circuits perform complex and large-scale computation, further bridging the gap between deep learning and neuroscience.
While our framework successfully eliminates the need for normalization during training and in- ference, the deployment of the E-I circuit on digital neuromorphic hardware requires approxima- tion techniques, such as bit-shifts or look-up tables. While there are obstacles for digital chips, emerging analog mixed-signal platforms like DYNAP-SE2 (Richter et al. 2024) have already supported such operations. Therefore, our framework offers direct algorithmic compatibility with next-generation analog or mixed-signal neuromorphic hardware.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work supported by STI 2030-Major Projects 20252D0217101, the National Natural Science Foundation of China (62422601, U24B20140), Beijing Municipal Science and Technology Program (Z241 (20000000422)004 Beijing Nova Program (20230484362, 20240484703), National Key Labora- tory for Multimedia Information Processing, and Beijing Key Laboratory of Brain-inspired Spiking Large Models.
REFERENCES
Yashar Ahmadian and Kenneth Miller. What is the dynamical regime cerebral cortex? Neuron, 109(21):3373-339) 2021.
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, Jeff Kusnitz, Michael Debole, Steve Esser, Tobi Delbruck, Myron Flickner, and Dharmendra Modha. A low power, fully event-based gesture recognition system. In Proceedings of the IEEE/CVF Interna- fional Conference on Computer Vision, pp. 7388-7397 2017.
Victor J. Barranca, Asha Bhuiyan, Max Sundgren, and Fangzhou Xing. Functional implications of Dale's law in balanced neuronal network dynamics and decision making. Frontiers in Neuro- science, 16:801847, 2022.
Guillaume Bellec, Darjan Salaj, Anand Subramoney, Robert Legenstein, and Wolfgang Maass. Long short-term memory and learning-to-lee le-rn in networks of spiking neurons. In Advances in Neural Information Processing Systems, pp. 795-805, 2018.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 130..3333.. 2013.
Michael Beyeler, Nikil D. Dutt, and Jeffrey L. Krichmar. Categorization and decision-making in neurobiologically plausible spiking network using a STDP-like learning rule. Neural Network, 48:109-124, 2013.
Guo-qiang Bi and Mu-ming Poo. Synaptic modifications in cultured hippocampal neurons: depen- dence on spike timing, synaptic strength, and postsynaptic cell type. Journal of Neuroscience, 18 (24):00444-1042, 1998.
Tong Bu, Wei Fang. Jianhao Ding, PengLin Dai, Zhaofei Yu, and Tiejun Huang. Optimal ANN-SNN conversion for high-accuracy and ultra-low-latency spiking neural networks. In International Conference on Learning Representations, 2022.
György Buzsáki and Andreas Draguhn. Neuronal oscillations cortical networks. Science, 304: 1926-1929, 2004.
Matteo Carandini and David Heeger. Normalization as canonical neural computation. Nature Reviews Neuroscience, 13:51 62, 2012.
Roi Cohen Kadosh. Rethinking excitation/inhibition balance in the human brain. Nature Reviews Neuroscience, 26(8):451-45) 2025.
Jonathan Cornford, Damjan Kalajdzievski, Marco Leite, Amélie Lamarquette, Dimitri Michael Kullmann, and Blake Aaron Richards. Learning to live with Dale's principle: ANNs with separate excitatory and inhibitory units. In International Conference on Learning Representations, 2021.
Joseph Del Rosario, Stefano Coletta, Soon Ho Kim, Zach Mobille, Kayla Peelman, Brice Williams, Alan J. Otsuki, Alejandra Del Castillo Valerio, Kendell Worden, Lou T. Blanpain, Lyndah Lovell, Hannah Choi, and Bilal Haider. Lateral inhibition vl controls neural and perceptual contrast sensitivity. Nature Neuroscience, 28:836-844 847, 2025.
Jianhao Ding, Zhaofei Yu, Yonghong Tian, and Tiejun Huang. Optimal ANN-SNN conversion for fast and accurate inference in deep spiking neural networks. In International Joint Conference on Artificial Intelligence, pp. 2328 228-233,.2222 2021.

---

Published as a conference paper at ICLR 2026
Jianhao Ding, Jiyuan Zhang, Tiejun Huang, Jian K. Liu, and Zhaofei Yu. Assisting training of deep spiking neural networks with parameter initialization. IEEE Transactions on Neural Networks and Learning Systems, 36(8):15015-15028 2025.
Chaoteng Duan, Jianhao Ding, Shiyan Chen, Zhaofei Yu, and Tiejun Huang. Temporal effective batch normalization in spiking neural networks. In Advances in Neural Information Processing Systems, volume 35, pp. 34377-34390, 2022.
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. In Advances in Neural Information Processing Systems, volume 34, pp. 21056-21069, 2021a.
Wei Fang, Zhaofei Yu, Yanqi Chen, Timothée Masquelier, Tiejun Huang, and Yonghong Tian. In- corporating learnable membrane time constant to enhance learning of spiking neural networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2641-2651, 2021b.
Wulfram Gerstner, Richard Kempter, J. Leo van Hemmen, and Hermann Wagner. A neuronal learn- ing rule for sub-millisecond temporal coding. Nature, 383:76-78, 1996.
Wulfram Gerstner, Werner M. Kistler, Richard Naud, and Liam Paninski. Neuronal Dynamics: From Single Neurons to Networks and Models of Cognition. Cambridge University Press, 2014.
S. Sthosh-astiddat and H. Adeli. Spiking neural networks. International Journal of Neural Systems, 19(04):295-308 2009.
Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS, volume 9, pp. 249-256, 2010.
Joshua H. Goldwyn, Bradley R. Slabe, Joseph B. Travers, and David Terman. Gain control with A-type potassium current: IA as a switch between divisive and subtractive inhibition. PLOS Computational Biology, 14:1-23. 2018.
Yufei Guo, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Yinglei Wang, Xuhui Huang, and Zhe Ma. IM-Loss: Information maximization loss for spiking neural networks. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 156-166, 2022a.
Yufei Guo, Xinyi Tong, Yuanpei Chen, Liwen Zhang, Xiaode Liu, Zhe Ma, and Xuhui Huang. RecDis-SNN: Rectifying membrane potential distribution for directly training spiking neural net- works. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 326- 335, 2022b.
Stefan Habenschuss, Johannes Bill, and Bernhard Nessler. Homeostatic plasticity in Bayesian spik- ing networks expectation maximization with posterior constraints. In Advances in Neural Information Processing Systems, volume 25, 2012.
Bilal Haider, Alvaro Duque, Andrea R. Hasenstaub, and David A. McCormick. Neocortical network activity In Vivo is generated through a dynamic balance of excitation and inhibition. Journal of Neuroscience, 26(17):4535-455 2006.
Bing Han and Kaushik Roy. Deep spiking neural network: Energy efficiency through time based coding. In European Conference on Computer Vision, pp. 388-404. Springer, 2020.
Bing Han, Gopalakrishnan Srinivasan, and Kaushik Roy. RMP-SNN Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pp. 13555-13564, 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Delving deep into rectifiers: Surpassing human-leve performance on ImageNet classification. In Proceedings of the IEEE/CVF Interna- fional Conference on Computer Vision, pp. 1034-1034, 2015.
12

---

Published as a conference paper at ICLR 2026
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 770- 778. 2016.
Hua Hu, Jian Gan, and Peter Jonas. Fast-spiking, parvalbumin" GABAergic interneurons: From cellular design microcircuit function. Science, ( 255263, 2014.
Sergey Loffe and Christian Szegedy. Batch normalization: accelerating deep network training by reducing internal covariate shift. In Proceedings of the International Conference on Machine Learming, pp. 448 456, 2015.
Haiyan Jiang, Vincent Zoonekynd, Giulia De Masi, Bin Gu, and Huan Xiong. TAB: Temporal accumulated batch normalization in spiking neural networks. International Conference on Learning Representations, 2024.
Hasna Nur Karimah, Chankyu Lee, and Yeongkyo Seo. Batchnorm-free binarized deep spiking neural network for lightweight machine learning model. Electronics, 14(8):1602, 2025.
Youngeun Kim and Priyadarshini Panda. Revisiting batch normalization for training low-latency deep spiking neural networks from scratch. Frontiers Neuroxcience, 15:773954-773954, 2021.
Agnes Soresak-Gorzo Michael G. Müller, Andreas Baumbach, Luziwei Leng, Oliver J. Breitwieser, Sacha J. van Albada, Walter Senn, Karlheinz Meier, Robert Legenstein, and Mihai A. Petrovici. Cortical oscillations support sampling-bass computations in spiking neural networks. PLOS Computational Biology, 18: 18:1-11,22 2022.
Alex Krizhevsky. Learning multiple layers of features from tiny images, 2009.
Kaushalya Kumarasinghe, Nikola Kasabov, and Denise Taylor. Brain-inspired spiking neural net- works for decoding and understanding muscle activity and kinematics from electroencephalogra- phy signals during hand movements. Scientific Reports, 11(1):2486, 2021.
Ya Le and Xuan S. Yang. Tiny imagenet visual recognition challenge, 2015.
Jun Haeng Lee, Tobi Delbruck, and Michael Pfeiffer. Training deep spiking neural networks using backpropagation. Frontiers in Neuroscience, 10:508, 2016.
Hongmin Li, Hanchao Liu, Xiangyang Ji, Guoqi Li, and Luping Shi. Cifarl0-dvs: An event-stream dataset for object classification. Frontiers in Neuroscience, 2017.
Yuhang Li, Yufei Guo, Shanghang Zhang, Shikuang Deng, Yongqing Hai, and Shi Gu. Differen- tiable Spike: Rethinking gradient-descent for training spiking neural networks. In Advances in Neural Information Processing Systems, volume 34, pp. 23426-23439, 2021.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models. Neural Networks, 10(9):1559-1671, 1997.
Henry Markram, Maria Toledo-Rodriguee Yun Wang. Anirudh Gupta, Gilad Silberberg, and Caizhi Wu. Interneurons of the neocortical inhibitory system. Nature Reviews Neuroscience, 5(10): 793-807, 2004.
Aurora Micheli, Olaf Booij. Jan Gemert, and Nergis Tömen. Deep activity propagation via weight initialization spiking neural networks. In Neuro Inspired Computational Elements (NICE), pp. 1-9, 2025.
Antony W. N d'dri, William Gebhardt, Céline Teulière, Fleur Zeldenrust, Rajesh P. N. Rao, Jochen Triesch, and Alexander Ororbia. Predictive coding with spiking neural networks: survey, 2024.
Emre O. Neftei, Hesham Mostafa, and Friedemann Zenke. Surrogate gradient learning in spiking neural networks: bringing the power of gradient- maddeett-based optimization to spiking neural networks. IEEE Signal Processing Magazine, 36:51-63, 2019.
13

---

Published as a conference paper at ICLR 2026
Yuqi Pan, Yupeng Feng, Jinghao Zhuang. Siyu Ding, Zehao Liu, Bohan Sun, Yuhong Chou, Han Xu, Xuerui Qiu, Anlin Deng, Anjie Hu, Peng Zhou, Man Yao, Jibin Wu, Jian Yang, Guoliang Sun, Bo Xu, and Guoqi Li. SpikingBrain technical report: spiking brain-inspine large models. arXiv preprint arXiv:2509.0522 05276, 2025.
Luke Y. Prince, Matthew M. Tran, Dorian Grey, Lydia Saad, Helen Chasiotis, Jechyun Kwag, Michael M. Kohl, and Blake A. Richards. Neocortical inhibitory interneuron subtypes differ- entially attuned to synchrony- and rate-coded information. Communications Biology, 4(1):935, 2021.
Ole Richter, Chenxi Wu, Adrian M Whatley, German Köstinger, Carsten Nielsen, Ning Qiao, and Giacomo Indiveri. DYNAP-SE2: a scalable multi-core dynamic neuromorphic asynchronous spiking neural network processor. Neuromorphic Computing and Engineering, 4(1):014003, 2024.
Julian Rossbroich, Julia Gygax, and Friedemann Zenke. Fluctuation-driven initialization for spiking neural network training. Neuromorphic Computing and Engineering, 2(4):044016, 2022.
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshiní Panda. Towards spike-based machine intelligence with neuromorphic computing. Nature, 575:607-617, 2019.
Bodo Rueckauer, Allia-Alexandra Lungu, Yuhuang Hu, Michael Pfeiffer, and Shih-Chii Liu. Con- version of continuous-valued deep networks efficient event-driven networks for image classifi- cation. Frontiers in Neuroscience, 11:682, 2017.
Sadra Sadeh and Claudia Clopath. The emergence of NeuroAI: bridging neuroscience and artificial intelligence. Nature Reviews Neuroscience, 26:583-584, 2025.
Abhronil Sengupta, Yuting Ye, Robert Wang, Chiao Liu, and Kaushik Roy. Going deeper in spiking neural networks: VGG and residual architectures. Frontiers in Neuroscience, 13:95, 2019.
Zihang Shao, Xuanye Fang, Yaxin Li, Chaoran Feng, Jiangrong Shen, and Qi Xu. EICIL: Joint excitatory inhibitory cycle iteration learning for deep spiking neural networks. In Advances in Neural Information Processing Systems, volume 36, pp. 32117-32128. 2023.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
DC Somers, SB Nelson, and M Sur. An emergent model of orientation selectivity in cat visual cortical simple cells. Journal Neuroscience, 15(8))5448-544 1995.
Christoph Stöckl and Wolfgang Maass. Optimized spiking neurons can classify images with high accuracy through temporal coding with two spikes. Nature Machine Intelligence, 3(3)2200-28, 2021.
Hugh R. Wilson and Jack D. Cowan. Excitatory and inhibitory interactions in localized populations of model neurons. Biophysical Journal, 12(1):1244, 1972.
Yujie Wu, Lei Deng, Guoqi Li. Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for
training high-performance spiking neural networks. Frontiers in Neuroscience, 12:331, 201 8.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: faster, larger, better. In AAAI Conference on Artificial Intelligence, 2019.
Mingqing Xiao, Qingyan Meng, Zongpeng Zhang, Di He, and Zhouchen Lin. Online training through time for spiking neural networks. In Advances in Neural Information Processing Systems, volume 35, pp. 20717-20730, 2022.
Yu Xiao, Yize Liu, Bihua Zhang, Peng Chen, Huaze Zhu, Enhui He, Jiayi Zhao, Wenju Huo, Xi- aofei Jin, Xumeng Zhang, et al. Bio-plausible reconfigurable spiking neuron for neuromorphic computing. Science Advances, 11())eadr66333 2025.
Siyu Zhang, Min Xu, Tsukasa Kamigaki, Johnny Phong Hoang Do, Wei-Cheng Chang, Sean Jenvay, Kazunari Miyamichi, Liqun Luo, and Yang Dan. Long-range and local circuits for top-down modulation of visual cortex processing. Science, pp. 660-665, 2014.
14

---

Published as a conference paper at ICLR 2026
Dongcheng Zhao, Yi Zeng, and Yang Li. BackEISNN: A deep spiking neural network with adaptive self-feedback and balanced excitatory-inhiiitrr neurons. Neural Networks, 154:68-77 2022.
Lusen Zhao, Zihan Huang, Jianhao Ding, and Zhaofei Yu. TTFSFormer: A TTFS-based lossless conversion of spiking transformer. In Proceedings of the International Conference Machine Learning, 2025.
Hanle Zheng. Yujie Wu, Lei Deng, Yifan Hu, and Guoqi Li. Going deeper with directly- larger spiking neural networks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 11062-11070, 2021.
Yaoyu Zhu, Jianhao Ding, Tiejun Huang, Xiaodong Xie, and Zhaofei Yu. Online stabilization of spiking neural networks. In International Conference on Learning Representations, 2024.
15

---

Published as a conference paper at ICLR 2026
A LLM USAGE
We utilize LLMs to assist in proofreading and enhancing the clarity and readability the manuscript. The core scientific contributions, experimental design, and data analysis conducted entirely by the authors.
B REPRODUCIBILITY STATEMENT
All essential details regarding datasets, model architectures, and training hyperparameters are pro- vided in Section 5 and Appendix F.6 ensure reproducibility. The source code for our framework has been made publicly available at https:: /nttps://githhb com/v//dOwww//eep2I...
C ETHICS STATEMENT
This work is foundational research focused on learning algorithms for brain-inspired neural net- works and exclusively uses standard public datasets. We do not foresee any direct negative societal impacts or ethical concerns arising from this research.
D DETAILED MATHEMATICAL DERIVATIONS
This section provides detailed derivations for mathematical formulations introduced in the main text, using same notations.
D.1 DYNAMIC'S OF THE FAST-SPIKING INHIBITORY NEURONS
In this section, we provide a formal derivation justifying the modelling of inhibitory neurons as stateless ReLU-like units in discrete-time simulation. We demonstrate that this approximation mathematically rigorous under the condition π < At. The sub-threshold dynamics a LIF inhibitory neuron governed by the following differential equation:
du 14 (t) = - (uff (t) rest + 1 (t),
(22)
dt
where is the membrane time constant, Me.res is the resting potential, and I (() is the input rents. Assuming that f1,rest = and the input currents remain constant at In over the duration of discrete time step At (starting from time t), the analytical solution for the membrane potential at
time + At is
(t + At) uL (t)e + Io (1 e (
(23)
In our simulation setup, excitatory neurons typically have a time constant TE = 2, which is on the same order of magnitude as the simulation time step At
In contrast, inhibitory neurons in our model represent biological fast-spiking interneurons (e.g.. PV+ neurons), which are characterized by significantly smaller time constants compared to excitatory pyramidal neurons (Hu et al., 2014 Prince et al. 2021). This biological property implies the condition π & Tg. and consequently,
Ti < At. Under this condition, the ratio At becomes very large, causing the exponential decay 77
factor to approach zero:
lim 7-
7 0.
(24)
Substituting this limit into the update rule, the term (t)(()) vanishes, and the equation simplifies to:
ui (t+ lo-
(25)
This result indicates that the membrane potential reaches steady state determined entirely by the input currents almost instantaneously within a single time step. Consequently. the inhibitory neurons effectively converge stateless units that do not carry temporal information across time steps.
Furthermore, the simulation time step At much longer than the intrinsic dynamics of the inhibitory neurons, which allows them to fire multiple times within single step when the input is strong.
16

---

Published as a conference paper at ICLR 2026
applying firing threshold of and soft-reset mechanism, the total spike count at time t + 1 is given by
of 1t+ max (o. u (t (t+1))]
mmx [o) max (0, In).
(26)
This derivation justifies the ReLU-like approximation for inhibitory neurons in the main text. Im- portantly, since the transient response is completed within single step A,, there is no accumulation of approximation error over time. The validity of this model is strictly determined by the timescale separation 7I
At, making it reasonable approximation.
D.2 INITIALIZATION OF W EE W | TE:
W El
Our primary goal is to achieve pero-mean expected net input for each excitatory neuron in layer 9 and Equation I0 we have
7 (Equation 16) at initialization. From Equation
E =
will
| BE, E.j
[t]. i
..,,...,.. .
(27)
==1
0 IE
[1] =
Σ 1=1
w
1- 1E,1, E.
[t, 1,2,...,17
(28)
EE and W IE. respectively. Other notations are consistent with those in the main text. Then similarly, from Equation II and Equation 12 obtain
Here W EE and will iē,y denote the (i,5) element of w
we
n
12
sl.sub,
(t]
=
BL.
ELss
18
(29)
n
will
Bi.Ú
IB,jk E.
1- [1] = 1, 1,2....,..
(30)
k=1
Note that 1 [1] F (1) as max (o, (o.TEE,)1) (4) = [4] since the elements of w and [1-]]
IE
sg are non-negative. Therefore, by assuming $
|1-1| [t] Lid Bern(p) and elements of weights are
i..d. at initialization, we obtain Equation 17 and Equation
18
D.3 INITIALIZATION OF gi [4]
gi gain factor modulating the strength of divisive inhibition. As mentioned in the main text, [T
[ divii
initialize it by setting the condition E
std
FF.,
Similar to the derivation of [
su. [1] in Appendix D.2
dpl
gi !!]
E
1 Bi.div
m gi
dpI
[4]
E
will IE
E
w E1
(31)
A
Below we derive the standard deviation of I neuron is
F The variance of the excitatory input currents to
Var
( (TEE.)
Var
w 2]
11-1]
(32)
where w!ll] "EE.i denotes the (i. j) element of W EE- Assuming the terms will E.j are independent for each j. the variance of the sum is the sum of the variances,
[t-1]
Var (will (i
Var
EE,
=
(33)
==1
17

---

Published as a conference paper at ICLR 2026
By further assuming that the weights w EE,i and input signals $E.j are independently distributed, can simplify this to
|1-1)
[ EEE, d- Var (W W -
Var
(34)
d
E (sg U- = Var (w!)) + Var (sg -
E²
w
(35)
EE
As established in the main text, we model the input spikes as an i.id. Bernoulli distribution with p. IE
parameter p. Thus, E B " [[-1]]
(s L
p, and Var [1 [ -
p(1 p). The weights
will EE drawn from exponential distribution with rate Al1, for which E EE 1/x44 and
Var (wll
1/ Substituting these into Equation 35 yields:
1
Var EE,)) = d (111)2 + p(1 - p)
(36)
(Al1)2)
d
(211)2 (p + ppl - p))
(37)
dp(2 - p)
(38)
(Al1)2
Therefore, the standard deviation of I!!l EE, is:
v dp(2 - p) A
std (LEE.)) V Var L
(39)
D.4 SELECTION OF RATE PARAMETER A!
To ensure stable signal propagation initialization, set the standard deviation of the input cur- rents to match that of the input spikes, i.e., std (IER.)) std (sk
Vp 1 p). Using the
result from Equation 39 we have
dp(2 - p) AM
Vp(1 p),
(40)
1 /dpl2 - pp)
d(2 - p) 1 p
(41)
vp(l p)
D.5 BACKPROPAGATION THE E-I CIRCUIT
In this section, we provide detailed backpropagation derivation for the E-I circuit to theoretically justify the necessity of the gradient scaling on W FI.
We first re- re-write the forward pass formulation at time step for layer 1:
[4]
[
gg
ER
+
b
(42)
EL.div
where the currents are defined as:
4
El suu [t] = w ||| [
Ei [t],
(43)
[
El.
[4] will !||
(.) El (g)
(44)
s [t] ~ w || - --"
Sg [t].
(45)
Note that here we use the linear approximation for derived in Appendix D.I
Let [t] = _!
the enror signal backpropagated from subsequent layers at time t. The gradients are accumulated over T time steps. Below we provide the derivative of loss w.r.t. each trainable parameter in layer of the E-I circuit.
18

---

Published as a conference paper at ICLR 2026
1. will EE
Oc
Σ
T
8
BB E1, div [t]
(s [t]
T
(46)
aw 11
EE
2. W EI
W contributes both the subtractive and divisive pathways. Therefore, its gradient can be de- composed into subtractive and divisive components that correspond to the two types of inhibitory currents.
OC ow
[t)
R BF
s
(t)
[t]
El.div
Component
+
8 [
+
BF
halanced [t]
gi
[4]
T
(47)
Fi.div [t]
O Fl.div
Divisive Component
where 1
balanoed (4)
11 Ee(4
Bl. [4]
w TE.
The error propagates back through the inhibitory neurons.
OC
= Σ (1)
T
it
[t]
(48)
ow
where the error term dn combines gradients from both w Ei pathways. Therefore,
of 2
[
w
T
à 10
B
+
gi 11
w ||| Fl
8 [
gg
balanced
[t]
(49)
EI.div
[t]
EL.div [t]
El.
[1] This parameter modulates the inhibitory signals before they are projected by w for the divisive pathway. Therefore, the error signal propagates back through the divisive branch of w!ll EI
gi
El
Oc
W
[ El
5
g
balanced [
(50)
0g
R
[t]
EI.div
5. gg
0c
T
balancedlt]
0g
11
=
8 [t]
EI div [t]
(51)
1=1
6. b
7c
8
(52)
Ib E
19

---

Published as a conference paper at ICLR 2026
E IMPLEMENTATION DETAILS
E. CNNS WITH THE E-I CIRCUIT
In experiments, mainly CNNs like VGG and ResNet, where all convolutional layers and the fully connected classifier are implemented with the proposed E-I circuit (for output layer, we do not apply divisive inhibition to better stabilize the output logits). Here clarify the implementation of the convolution layer used in our experiments. Specifically, WEE and WIE are standard K K point-wise convolution kernel. In
convolution kernels, while the lateral connection WEI is this context, the input dimension d is defined as = Cin x K x K, where Cin is the number of channels of excitatory input, and the population sizes nE and ni the number of excitatory and inhibitory output channels, respectively. This configuration allows inhibitory neurons to regulate excitatory neurons densely across the channel dimension while preserving the spatial structure.
E.2 DYNAMIC INITIALIZATION
While theoretical analysis under Bernoulli assumption provides basic ideas of E-I Init, find it more effective and stable during training we estimate statistics from training data at initialization, rather than formulating them with Bernoulli distribution parameter p and manually p. Similar to the theoretical analysis under the Bernoulli assumption above, we have
d
E
[1 8B
-1--))²
+ Var
11-11)
A
(53)
[1-11)
Var
ss
[t-1) 1)
[1-1]
E
E
+ Var 8
|1] gi
(54)
dI
R-1]]
Therefore, by computing mean, second raw moment, and sample-wisc variance only once at initial- ization, the model performs self-regulated E-I balance and gain control, stabilizing training without explicit normalization. Algorithm summarizes the implementation of the proposed E-I init.
Algorithm E-I Init
Require: Input X of shape (B....) (first batch), input dimension d, inhibitory neuron count 74,- Ensure: Initialized parameters WEE, WIE, WEI, gt, ga, bg- procedure EI-INIT(X, d, m²)
>> Step 1: Estimate input statisties from the first batch X
2: mean X.mean()
3: var t X.var vaa m).nn
4: moment <- (X2) meean()
Step 2: Calculate the rate parameter based on statistics D exp scale 1/A
5: exp scale [momeett+
var
6: gain
/mooent- +var mean
> Initial value for gi
vd
> Step 3: Initialize trainable parameters
7:
WEB ~ EXPONENTIAL(SCalE exp_scall scale)
8:
WIE EXPONENTIAL (scale exalsc
9:
WEI f-
10:
gi +- gain I
gE <-
12:
bg +-
13:
end procedure
20

---

Published as a conference paper at ICLR 2026
E.3 ADAPTIVE STABILIZATION WITH STE
demonstrates the full procedure of our proposed adaptive stabilization of divisive inhi- bition mechanism, including the backward pass with STE.
Algorithm 2
Algorithm/ E-I Prop
Require: Input X of shape (B....), where B is the batch size.
Ensure: Output Xout with zeros adaptively replaced.
1: procedure
2: if contains no zero values then
3: return X
4: end if
Step 1: Replace zeros with second minimum b Create boolean mask for all zero locations
5: M
- (X = 0)
6:
Xamp t X
X
7:
Xemp M
Temporarily replace zeros with infinity
8:
for each sample from 1 to do
9:
8, min (0 [i])
Find the smallest positive value of the original sample
10:
S +++ S2
11:
end for
12:
Xfwd f where M, S, X)
Replace zeros with the smallest positive value of the sample Step 2: Construct the final output with STE STE via the detach trick
13:
Xout + detach = (X detach(X))
return Xout
15:
end procedure
F SUPPLEMENTARY RESULTS AND EXPERIMENT DETAILS
F.I EMPIRICAL ANALYSIS OF GRADIENT FLOW AND SCALING ROBUSTNESS
Gradient Magnitude at the Beginning of Training
0.012
Wee- We 9u 9:
Average Gradient Magnitude
0.010
0.008
0.006
0.004
0.002
0.000
9.
9e
9:
9t
War
9)
Wa
Wer
9e
Parameters
Figure 5: Empirical analysis of gradient norms at initialization for convolutional layers in VGG-8. 8, without gradient scaling. The gradients for WF (red) are orders of magnitude larger than those for WEE, WIE. and gain parameters (blue), consistent with the theoretical analysis.
Figure 5 visualizes the magnitudes of gradient norms for all trainable parameters across the convolu- tional layers of VGG-8 the first training iteration (without gradient scaling). Consistent with
21

---

Published as a conference paper at ICLR 2026
theoretical derivation in Appendix D.5 the gradient norms for WEI disproportionately larger than those of other parameters. This imbalance stems primarily from the divisive operation, which introduces term proportional to 1/ ( [TELaviit I EL (tl) in the gradient, leading to quadratic
! amplification when the denominator is small. While the gradients for w and gi also contain this term, they are implicitly dampened since the gradients backpropagate through w an averaging filter due its deterministic initialization of 1/ml In contrast, the gradient for W lacks such averaging mechanism and is instead directly proportional to the inhibitory activity Si
EI which acts as
Consequently, the gradient magnitude of w El is driven by the input spikes st [d] w [t- SE [t].
scaling linearly with the input dimension d.
To counteract this amplification, we choose the scaling factor as 1/d. To verify the robustness of scaling factor choice, sensitivity analysis is conducted VGG-8 (CIFAR-10) by varying the scaling factor from 1/vd to 1 1 (see Table 3). The result demonstrates that method is stable across broad range of scaling factors (e.g.. 1/vd and 1/d yield comparable performance), whereas removing the scaling immediately leads to collapse.
Table 3: Sensitivity analysis of gradient scaling factor with VGG-8 on CIFAR-10.
Scaling Factor
Top-1 Accuracy (%)
No Sealing
Collapsed
1 1/vd 1/d (Default) 1/dl
86.87
86.88
86.33
F.2
COMPARISON WITH E-I ANNS
83
DANN JANN)
DeepEISVN
92
90
09
86
85-
VGG-8
VGG-11
VCG-16
VGG.19
Figure 6: Comparison between method and DANN CIFAR-10. Error bars denote the stan- dard deviation over multiple independent runs. Statistical significance between the two methods is indicated by asterisks (+* p < 0.01, *** p 0.001).
Comparison with DANN (Cornford et al., 2021) highlights the advantage of our method. Figure shows that our method consistently and significantly outperforms DANN across all tested VGG architectures. Statistical analysis confirms that these improvements are statistically significant (p < 0.01 for VGG-8 and p 0.001 for deeper models). Notably, the advantage of our method increases as the network depth increases. The performance gap widens from 0.87% on VGG-8 to 2.65% on VGG-19. This trend strongly suggests that proposed mechanisms, E-I Init and E-I Prop,
22

---

Published as a conference paper at ICLR 2026
more effective at preserving stable signal propagation and facilitating effective learning in very deep architectures.
F.3
DISTRIBUTIONS OF BN OUTPUTS
BN #1
BN 43
BN #S
Lime
time staps
0 45
0.6
Density
Density
Density
05
0.2
-10
10
-3
0
-2
Value
Value
Value
Figure 7: Distributions the outputs in the first, third and fifth BN layers after training.
Figure demonstrates output distributions of BN layers after training, which are all Gaussian-like and zero-centered.
F.4
VISUALIZATION OF E-I INTERACTION
Input Image
IgE
1
gl,sub
EE El.
IFI,iiv
Intt
Output of BN
Figure 8: Comparison between feature maps of the first E-I circuit layer in our model and the feature map of the first BN layer SNN with vanilla BN after training.
visual comparison of the feature maps suggests that our E-I circuit and vanilla BN forces network to focus different feature representations. As shown in Figure & our E-I circuit produces a feature map where activations are concentrated along the object's contours, indicating a learned focus on feature edges and boundaries. In contrast, standard BN layer may preserve dense spatial output of
23

---

Published as a conference paper at ICLR 2026
its preceding convolution, normalizing the representation of the feature's overall shape and texture rather than isolating its boundaries.
F.5 COMPUTATIONAL OVERHEAD ANALYSIS
To quantify the computational cost associated with ensuring biological fidelity, we measure the training time (per epoch) and peak GPU memory usage on single NVIDIA GeForce RTX 4090. We compare our DeepEISNN with SNNs with vanilla BN across various architectures. The results are summarized in Table
Table 4: Computational overhead (T = 4, batch size=128, single GPU).
Arch.
Metric
SNN (Baseline)
DeepEISNN (Ours)
Factor
Time/Epoch
7.3s
18.1s
2.48x
VGG-8
Memory
2394 MB
2684 MB
1.12x
Time/Epoch
9.5s
23.1s
2.43x
VGG-11
Memory
2392 MB
2886 MB
1.21x
Time/Epoch
16.9s
42.2s
2.50x
VGG-16
Memory
3544 MB
5072 MB
1.43x
Time/Epoch
19.4s
47.6s
2.45x
VGG-19
Memory
3734 MB
5402 MB
1.45x
Time/Fpoch
38. 88.8s
90.0s
2.32x
ResNet-18
Memory
5918 MB
9302 MB
1.57x
As shown in Table our method introduces a computational overhead of approximately 2.3x ~ 2.5x in training time and 1.1x 1.6x in GPU memory usage. This increase is expected and necessary trade-off for the E-1 circuit with a 4:1 excitatory that utilize single synaptic weight matrix per layer, our framework explicitly models three distinct synaptic projections (WEE, WIE, WEI) and maintains an additional inhibitory population. Impor- tantly, this overhead scales linearly with network size, ensuring tractability for deep architectures. Despite the increased per-epoch cost, our method demonstrates robust convergence comparable to BN-equipped baselines (as evidenced by the competitive accuracy in Table D.. thereby enabling normalization-free learning using biologically grounded mechanisms.
ratio. Unlike standard SNNs
F.6 EXPERIMENT DETAILS
Code is implemented using the PyTorch framework and run NVIDIA GeForee RTX 4090 GPUs.
Network architectures. We employ standard backbones, including VGG- (GG-8/11/16// 1/16/19 (Simonyan & Zisserman 2015) and ResNet-18 (He et al. 2016, Fang et al. 2021a). In these architectures, the standard convolutional blocks (Cony-BN-LIF) and linear classifiers are replaced by our proposed E-I circuit. To construct a lightweight classifier, we apply global average pooling (GAP) before the linear readout layer. VGG-8 is utilized primarily for ablation studies on CIFAR-10, while deeper models (VGG-16/19, ResNet-18) are employed for SOTA comparisons and large-scale benchmarks.
Data preprocessing. Our method is validated on multiple datasets, including CIFAR- 10/1 00 (Kizhevsky 2009). CIFARIO-DVS (Li al. 2017), DVS-Gesture (Amir et al.) (017). and Tinylm- ageNet (Le & Yang 2015), using their standard train and validation splits. We apply distinct data augmentation strategies on different datasets. Specifically, for CIFAR-10 and CIFAR-100, we em- ploy random cropping with a size of 32 x 32 (padding of 4 pixels), random horizontal flipping, and cutout. For Imayem, images are downsampled to 32 x 32, and augmentations include random resized cropping, random horizontal flipping, color jittering, and cutout. Regarding neuromorphic datasets, both CIFARIO-DY. and DVS- WVS-esture are resized to spatial resolution of 48 x 48. The training pipeline for these went-based datasets includes random resized cropping and random hori- zontal flipping. Additionally, we apply random temporal deletion specifically for the DVS-Gesture dataset enhance temporal robustness.
24

---

Published as a conference paper at ICLR 2026
Global configuration. All models are trained for 300 epochs. The optimization is performed ing SGD with a momentum of 0.9 and a weight decay 0.0005, together with a cosine annealing learning rate scheduler combined with initial linear warm-up. The ratio of excitatory to inhibitory neurons is fixed at 4:1 across all layers. No dropout is applied. We use the standard cross-entropy loss. The final prediction is obtained by averaging the output logits of the classifier across all sim- ulation time steps before computing the loss. For performance evaluation, we report the best top-1 accuracy achieved on the validation set throughout the training process.
Task-specific configuration. To accommodate the varying complexities and temporal dynamics of different datasets and network architectures, we finetune some hyperparameters for each specific task. The detailed task-specific configurations are summarized in Table 5
Table 5: Task-specific hyperparameter configurations.
Dataset
Architecture
Batch Size
Time Steps
Peak LR
Warm-up Epochs
VGG-8/11
128
4
0.002
10
CIFAR-10
VGG-16/19, ResNet-18
128
0.001
CIFAR-100
VGG-16/19
128
4
0.001
30
TinylmageNet
ResNet-18
128
0.003
CIFARIO-DVS
VGG-8/11
32
10
0.001
10
DVS-Gesture
VGG-8
32
0.001
25


Reviews:
[{"review_id": "JBiGigbLUf", "reviewer": "Reviewer_L1V9", "summary": "This paper revisits the important role of lateral inhibition in the cortical circuit and develops a normalization-free learning framework. The authors also developed a useful dynamic initialization scheme to ensure effective learning. The proposed framework supports the effective training of deep networks and shows competitive results on multiple benchmarks. Overall, I find this paper is quite interesting and the main idea is easy to follow.", "questions": "The authors incorporate Dale’s law into their models. I fully understand their consideration from a biological perspective. Beyond this, I wonder if the authors observed any other functional implications or accuracy gains by following this biological law?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The work addresses the critical challenge of training deep SNNs without biologically implausible normalization schemes like Batch Normalization. The proposed framework, inspired by lateral inhibition in cortical circuits, is a significant step towards bridging the gap between high-performance machine learning and biological realism.\n\n- The paper is well-written, and the core concepts are presented in a way that is easy to understand. The motivation for replacing standard normalization with a brain-inspired alternative is clearly articulated.", "weaknesses": "- While the performance has been validated on static image classification tasks, I think it would be better to further examine the performance on more extensive neuromorphic datasets for a broader impact.\n- It would be great to discuss the potential deployment on existing neuromorphic hardware.\n- Table 1 lacks statistical significance analysis.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gWqW1X8MBM", "reviewer": "Reviewer_RXYj", "summary": "To resolve the performance-biological realism trade-off in deep Spiking Neural Networks, this work proposes a normalization-free framework incorporating lateral inhibition.", "questions": "1. Could you please provide further explanation of the statement: \"In our discrete-time simulation, the time step ∆t = 1 is chosen to be on a similar scale as the time constant of excitatory neurons (e.g., τE = 2), which implies τI ≪ ∆t. Under this condition, the dynamics of inhibitory neurons can reach a steady state almost instantaneously within a single time step\"?\n\n2. Should the weight matrices W mentioned in the text be understood as fully connected? This point requires clarification.\n\n3. Is the variable 'g' in Equation 13 a floating-point parameter? Does the formulation involve a direct multiplication between a floating-point parameter and a floating-point weight matrix?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 2, "contribution": 4, "strengths": "1. It presents a novel and practical solution to a core challenge in neuromorphic computing by replacing artificial normalization with bio-plausible E-I circuits, enabling both high performance and biological fidelity.\n\n2. The work makes substantial contributions to brain-inspired AI by rigorously implementing Dale's law and cortical microcircuit principles, providing a valuable platform for computational neuroscience exploration.\n\n3. The framework achieves competitive accuracy on benchmark datasets while demonstrating sophisticated neural regulation through emergent bimodal activity patterns, offering a biologically grounded alternative to conventional normalization.", "weaknesses": "The presentation of the core methodologies, particularly the derivation of E-I Init and the custom backward pass of E-I Prop, is excessively condensed. A more structured, step-by-step exposition is needed, as the current description lacks clarity and logical progression, consequently hindering both readability and reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yqfUv8nAaR", "reviewer": "Reviewer_p9Mj", "summary": "This paper proposes a normalization-free biologically inspired method to train deep SNNs. The authors argue that most SNNs rely on non-biological normalization layers (BatchNorm, LayerNorm, etc.) to maintain activation stability, which limits biological plausibility. To overcome this, the authors propose an Excitatory-Inhibitory (E-I) architecture, including E-I init and E-I Prop, that enforces Dale’s law, meaning each neuron has a fixed excitatory or inhibitory role throughout training.", "questions": "1. How does your method perform without enforcing Dale’s law, i.e., allowing all weights to freely take positive/negative values or other cases? \n2. Could you cite recent neuroscience literature justifying Dale’s law as a valid and necessary constraint for modern models? Otherwise using a seemingly outdated theory is not sufficient for this paper.\n3. What are the computational overheads (training time, memory costs, etc.) compared to normalization-based baselines? The convergence is essential for training deep SNNs.\n4. Have you tested on event-based neuromorphic datasets or other dataset to validate real-world relevance?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The aim to find a biologically plausible stabilization mechanism is well motivated. The integration of Dale’s law into deep SNNs is technically challenging and conceptually significant.\n2. E-I Init and E-I Prop are technically well described and appear easy to integrate.\n3. The paper could inspire further exploration into lateral inhibition and homeostatic balance as normalization-free learning tools.", "weaknesses": "1. Experiments are restricted to CIFAR-10/100; it remains unclear if the framework scales to large-scale or event-based datasets (e.g., ImageNet, DVS-Gesture).\n2. The experiments do not include ablations isolating the impact of Dale’s-law enforcement itself. It is unclear whether improvements arise from E-I Prop (adaptive inhibition) or from Dale’s constraint.\n3. The paper overclaims biological fidelity while omitting discussions of modern findings that contradict the core assumptions.\n4. The training costs are not claimed. The overhead of maintaining separate excitatory and inhibitory populations and additional inhibitory weight updates is not quantified in runtime or memory terms.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "5OFiIVHigi", "reviewer": "Reviewer_s6FP", "summary": "The paper proposes a biologically-inspired framework to enable the stable training of deep Spiking Neural Networks (SNNs) without explicit normalization layers. This is achieved by introducing a lateral inhibition circuit composed of distinct excitatory and inhibitory populations as a substitute for standard normalization. To this end, the authors have developed a comprehensive suite of techniques, including a principled initialization scheme (E-I Init) and a stabilization method for propagation (E-I Prop).", "questions": "1.Does the proposed method alter the standard forward-pass rules of SNNs? Can the operations within the E-I circuit, particularly the divisive inhibition, be efficiently deployed on existing neuromorphic hardware platforms?\n\n2.A core motivation of this work is the biological implausibility of BN. However, during inference, BN parameters can be folded into the weights of the preceding convolutional or fully-connected layers, meaning no standalone BN operation exists in the final network. Could you please elaborate on why BN is still considered a critical barrier to the biological plausibility of SNNs in this context?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "This work addresses a critical challenge in the field of neuromorphic computing that training deep SNNs while preserving biological plausibility with a novel and well-conceived solution. The proposed E-I dynamic regulation mechanism is compelling, and the thorough ablation studies convincingly demonstrate the framework's effectiveness and the necessity of its individual components.", "weaknesses": "1.The simplification of the inhibitory neuron model to a static, ReLU-like unit undermines the network's temporal dynamics. While computationally convenient, its validity is questionable, and it may introduce significant errors, especially when the number of timesteps (T) is small.\n\n2.The 1/d gradient scaling for the inhibitory weights W_EI appears to be an ad-hoc choice. The paper lacks a theoretical analysis or extensive empirical validation explaining why this specific scaling factor is optimal, which may raise concerns about its generalizability.\n\n3.The paper is missing a direct performance comparison with state-of-the-art (SOTA) SNNs that use Batch Normalization (BN) within the same architectures. This weakens the claim of achieving \"competitive performance.\"\n\n4.All experiments are conducted on static image classification tasks. To truly validate the framework's temporal processing capabilities, it should be evaluated on event-based datasets (e.g., DVS-Gesture) or other sequential tasks where SNNs are expected to excel.\n\n5.As depicted in Figure 2, the model incorporates both subtractive and divisive inhibition. However, the manuscript (e.g., around Equation 14) fails to provide a clear mathematical formula describing how these two operations are combined. This ambiguity severely hinders the method's reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)