Biologically Inspired Dynamic Thresholds for Spiking Neural Networks
Authors: Jianchuan Ding
Bo Dong
Felix Heide
Yufei Ding
Yunduo Zhou
Baocai Yin
Xin Yang
Venue: NeurIPS 2022 Accept (2022)
Keywords: Spiking Neural Networks
dynamic threshold
robot obstacle avoidance
robot continuous control

Abstract: The dynamic membrane potential threshold, as one of the essential properties of a biological neuron, is a spontaneous regulation mechanism that maintains neuronal homeostasis, i.e., the constant overall spiking firing rate of a neuron. As such, the neuron firing rate is regulated by a dynamic spiking threshold, which has been extensively studied in biology. Existing work in the machine learning community does not employ bioinspired spiking threshold schemes. This work aims at bridging this gap by introducing a novel bioinspired dynamic energy-temporal threshold (BDETT) scheme for spiking neural networks (SNNs). The proposed BDETT scheme mirrors two bioplausible observations: a dynamic threshold has 1) a positive correlation with the average membrane potential and 2) a negative correlation with the preceding rate of depolarization. We validate the effectiveness of the proposed BDETT on robot obstacle avoidance and continuous control tasks under both normal conditions and various degraded conditions, including noisy observations, weights, and dynamic environments. We find that the BDETT outperforms existing static and heuristic threshold approaches by significant margins in all tested conditions, and we confirm that the proposed bioinspired dynamic threshold scheme offers homeostasis to SNNs in complex real-world tasks.

Biologically Inspired Dynamic Thresholds for Spiking Neural Networks
Jianchuan Ding' Bo Dong Felix Heide²
Yufci Ding Yunduo Zhou² Baocai Yinè Xin Yang
Dalian University of Technology, Department of Computer Science 2Princeton University, Department of Computer Science University of California-Sanaa Barbara, Department of Computer Science
Abstract
The dynamic membrane potential threshold, as one of the essential properties of a biological neuron, is spontaneous regulation mechanism that maintains neuronal homeostasis, i.e., the constant overall spiking firing rate of neuron. As such, the neuron firing rate is regulated by dynamic spiking threshold, which has been extensively studied in biology. Existing work in the machine learning commu- nity does not employ bioinspired spiking threshold schemes. This work aims bridging this gap by introducing a novel bioinspired dynamic energy-temporal threshold (BDETT) scheme for spiking neural networks (SNNs). The proposed BDETT scheme mirrors two bioplausible observations: a dynamic threshold has I) a positive correlation with the average membrane potential and 2) a negative correlation with the preceding rate of depolarization. We validate the effectiveness the proposed BDETT on robot obstacle avoidance and continuous control tasks under both normal conditions and various degraded conditions, including noisy observations, weights, and dynamic environments. We find that the BDETT out- performs existing static and heuristic threshold approaches by significant margins in all tested conditions, and we confirm that the proposed bioinspired dynamic threshold scheme offers homeostasis to SNNs in complex real-world tasks.
Introduction
spiking neural network (SNN) is a bioinspired neural network. Each spiking neuron mathe- matical model abstracted from the properties of a biological neuron. Spiking neurons communicate with each other through spike trains, mimicking the information transfer process of biological neu- rons 2 2 3 Similar to how biological action potentials are all-or-none impulses, the spikes of SNNs are commonly binary voltage pulses. Leveraging this binary representation. specifically designed neuromorphic hardware [ (] e.g.. TrueNorth [7] and Loihi [8] can run SNNs at extremely low power levels; they are 75 times more energy-e energy-efficient than their deep neural network counterparts on low-power GPU platforms [9] As such, recently, SNNs have rapidly emerged as effective models for robotic control tasks, especially in mobile robots that demand low power consumption [10 11
However, existing SNNs suffer from poor generalizability. unlike their biological counterparts. Biologically, neuron leverages a spontaneous regulation mechanism maintain neuronal home- ostasis 12]-the stable overall spiking firing rate or excitability within a network [ robustly adapt to different external conditions and offer strong generalization. A dynamic threshold, one type of regulatory mechanism, plays an essential role in maintaining neuronal homeostasis by reg- ulating the action potential firing rate; such thresholds are widely observed in different nervous systems [14] 15 16 17
18
19
20
22 23 This threshold can be regarded as an adaptation to 21
membrane potentials at short timescales
16 and it influences how the received signals of a neuron
are encoded into spike.
*Corresponding author xinyangddlut.edu. edu. bo. bo.ddon@princetonncco edu
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

Even though different dynamic threshold schemes have been observed and extensively studied in neuroscience, only a handful of existing works investigate bioinspired dynamic threshold rules to improve the generalization of SNNs. Hao et al. [24] proposed dynamic threshold method that relies on a heuristic dynamic scaling factor to gradually slow the growth of threshold. Conversely, instead of controlling threshold growth, Shaban et al. [25] leveraged double exponential functions to manage the threshold decay. Kim et al. (26) used a predefined target firing count to adjust their threshold but did not define the optimal target firing count. No existing work has demonstrated that bioinspired dynamic threshold scheme can achieve homeostasis in real-world tasks. More importantl, the existing work only validates the proposed dynamic threshold rules under ideal normal conditions without testing generalization to degraded conditions, which we argue is essential to validate whether homeostasis is achieved or not.
The direct use of bioplausible models in SNNs remains challenging, as most of these models are based on single cells in the nervous system and contain many optimized constants. In this work, we lift this limitation and introduce novel dynamic energy- energy-temporal threshold (BDETT) scheme for SNNs; the scheme comprises two components: a dynamic energy threshold and dynamic temporal threshold schema. The two components reflect the following two biological observations: in vivo, the dynamic threshold exhibits positive correlation with the average membrane potential and negative correlation with the preceding rate of depolarization (i.e., the excitatory status) [[6]. The dynamic energy threshold is inspired by biological predictive model which can predict the occurrences of spikes based on the previous membrane potential in the inferior colliculus of a barn owl [66]. The proposed dynamic temporal threshold component is inspired by the fact that a monocxponential function can effectively present negative correlation [7 [2]. Notably, we provide an analysis of the original biological models and propose layerwise statistical cues for SNNs to replace the constants in the two original biological models.
We integrate the proposed BDETT into two widely used SNN models: a spike response model (SRM) (27 and a leaky integrate-and- (LIF) model (28]. The effectiveness of BDETT is validated with these two SNN models for autonomous robotic obstacle avoidance, continuous control and image classification tasks under normal and various degraded conditions, e.g., dynamic obstacles, noisy inputs, and weight uncertainty. Extensive experimental results validate that the SNNs equipped with the proposed BDETT offer the strongest gencralization across all tested scenarios. More importantly, we quantitatively validate that BDETT can significantly increase the homeostasis of the host SNN for robotic control tasks. This is the first work to demonstrate that dynamic threshold schemes can offer bioplausible homeostasis to SNNs in robotic real-world tasks under normal and degraded conditions, dramatically enhancing the generalizability and adaptability of the host SNNs.
In particular, we make the following contributions in this work:
We introduce a bioinspired dynamic threshold scheme for SNNs that increases their general- izability.
We devise a method that uses layerwise statistical cues of SNNs to set the parameters of our bioinspired threshold method.
We validate that the proposed threshold scheme achieves bioplausible homeostasis, dramati- cally enhancing the generalizability across tasks, including obstacle avoidance and robotic control, and in normal and degraded conditions.
Scope We propose novel approach to setting the parameters of our threshold scheme using layerwise statistical cues of an SNN. Although this is essential for the proposed method to be effective, implementing these statitical blocks directly in neuromorphic hardware may require extra engineering efforts, which is out of the scope of this work.
Background and Related Work
2.1 Spiking Neural Networks (SNNs)
Various models for spiking neurons have been described to mathematically describe the properties of a nervous neuron. Typically, three conditions are considered by these models: resting, depolarization, and hyperpolarization. When a neuron is resting, it maintains a constant membrane potential. The change in membrane potential can be either decrease or an increase relative to the resting potential. An increase in the membrane potential is called depolarization, which enhances the ability of a cell to generate an action potential; it is excitatory. In contrast, hyperpolarization describes reduction in
2

---

the membrane potential, which makes the associated cell less likely to generate an action potential, and, as such, is inhibitory. All inputs and outputs of a spiking neuron model are sequences of spikes. A sequence of spikes is called a spike train and is defined s(t) Ex t(n), where FF represents the set of times at which the individual spikes occur (29). Typical spiking neuron models set the resting potential as However, existing models achieve depolarization and hyperpolarization in substantially different ways. In the following, we briefly review two commonly used models: the spike response model (SRM) [27] and leaky integrate-ane and---ireee fire (LIF) model (28). More details about these two models are provided in Supplementary Note 1.
Spike Response Model (SRM) An SRM first converts an incoming spike train sc(1) into a spike response signal as (s (t), where s(.) is a spike response kernel. Then, the generated spike response signal is scaled by synaptic weight wt. Depolarization is achieved by summing all the scaled spike response signals: EE st) When incoming spike trains trigger a spike s(t), the SRM models hyperpolarization by defining a refractory potential as (C (t), where ((-) is refractory kernel.
Leaky Integrate-and-Finn (LIF) An LIF model is a simplified variant of an SRM. This scheme directly processes incoming spike trains and ignores the spike response kernel. Hyperpolarization is achieved by simplified step decay function, fa(s(t)) = D for s(t) 0; for s(t) =.
2.2 Spiking Neural Networks for Robot Control
Biological neural circuits have impressive ability to avoid obstacles robustly in complex dynam- ical environments, e.g., as in dragonfly flight trajectories. Inspired by this observation, recently, researchers have explored SNNs for obstacle avoidance [30 31 33], For example, Tang et al. B3
devised an SNN to mimic a meurophysiologicall plausible connectome of the brain's 's navigational
system without assuming all-to-all connectivity. Following the path, Tang et al. [9] proposed spiking deep deterministic policy gradient (SDDPG) method to train a LIF-based spiking actor- network (SAN) for mapless navigation. They show that SNNs can robustly control robot in mapping tasks while being able to explore an unknown environment. SNNs have also been proposed for continuous robot control tasks. Patel el al. 34 proposed to combine SNNs with Deep Q-network algorithm, improving the robustness to occlusion in the input image. Tang et al. (35) proposed a population-coded spiking actor network (PopSAN) to solve high-dimensional continuous control problems, trained using deep reinforcement learning algorithms. Recently, modern neuromorphic hardware has made it possible deploy SNNs on neuromorphic processors in ultra-low power Compared to existing convolutional deep policy networks 39 on the mobile-GPUS such as the Nvidia Jetson TX2. SAN and PopSAN on Loihi neuromorphic processor consume 75 and 140 times less energy per inferences, respectively. All SNN-based models discussed above only consider static spiking thresholds. More importantly, experiments show that they suffer from poor generalization and fail in realistic degraded conditions. In this work, we use both SAN and PopSAN as testbeds and baseline methods validate the effectiveness of the proposed bioinspired dynamic threshold scheme, BDETT.
envelopes 9 36 37 38
3 Bioinspired Dynamic Energy-Temper Threshold (BDETT)
Motivated by the behavior of spiking threshold dynamics in biological nervous systems, propose a model with dynamic thresholds that exhibit positive and negative correlations with the average membrane potential and the preceding rate of depolarization, respectively. To achieve this behavior in the proposed scheme, given the i-th neuron in the 1-th layer at timestamp + 1, we define a dynamic threshold ( + 1) as
e 1) = (E)())) + (t
where (t) is the dynamic energy threshold (DET) of the neuron for ensuring a positive correlation, 1) is the dynamic temporal threshold (DTT), which ensures a negative correlation; see Figure Dh. Note that each neuron has different dynamic threshold at timestamp + 1 based on the proposed DET and DTT, which we describe below.
and ( (t
Dynamic Energy Threshold (DET) Positive correlations between dynamic thresholds and average membrane potentials have been observed in several areas of diverse biological nervous systems, such as the visual cortex and auditory midbrain [17] 21 22). With sufficient voltage measurements at spike onsets, one can fit a model to directly predict the voltage of a threshold [40]. However, the fitted biological model is only meaningful to specific nervous system, and stimulus or measurement
uncertainty can significantly impact the model accuracy. Fontaine el al. [66] proposed a biological predictive approach to assess the occurrence of spikes based on the previous membrane potential; this
3

---

40
C
(L06
6.0
(6X(
BDET7
1/1
1/0+9
Membrane potential (mV)
1.5
(0) (0)
+
(=V)
3.0
(4)
(1)())..... (1))))
OFF (1(+++1)
6 e/tl c(0
[1 + 1) of
Preceding rate of depularization (mVVss)
Figure 1: An illustration of the proposed BDETT scheme. We demonstrate the intuitive idea of our BDETT scheme for the i-th neuron in the (-th layer at timestamp + from the perspective of an SRM-based SNN model; (t) and (t) are the dynamic threshold and postsynaptic membrane potential of the i-th neuron in the (-th layer at timestamp t, respectively. b & c. Two example DET and DTT graphs, respectively.
method does not rely on voltage measurements at spike onsets. Even though the model was based on a barn owl's inferior colliculus, it exhibits great generality in terms of threshold variability statistics with other nervous systems (e.g., cortical neurons) [16]. The proposed dynamic energy threshold is inspired by this biological predictive model but includes several changes that are critical for the model to be effective in SNNs. For the i-th neuron in the (-th layer at timestamp t, we define
vico vi.
(1) = v(0((() - ViL (1)) + V/(() + 1n(1 + V.L. (1) m()(()) 0.2(max(1v min()((1)) for i = 1,,,..., n2,
W
(2)
(3)
(t))) (I) (.)(m)x(e((()) min(O((()) for i 1,2, n²
where (!()) is the neuron postsynaptic membrane potential timestamp 4; fa is the mean operator; n' is the total number of neurons in the 1-th layer; and v) and ve are two hyperparameters, which are set empirically. Figure Th shows two example graphs for Eq. 2 17 controls the shallow slope, and + n defines the slope of The steep part.
&
Intuitively, (t) and Vil, (t) define a critical region. When the membrane potential 2t (t) is smaller than V (t). the function has a shallower slope, and the threshold value is dominated by (t) In the opposite case, the energy threshold has higher rate of increase to inhibit a high spiking firing rate. In the biological predictive model proposed by Fontaine et al. [16.. V (t) and (t))), are the constants to be optimized during the model fitting process. However, we find that directly adopting these two fitted constants in an SNN does not result in generalization; see section 4.4] To tackle this challenge, we leverage the statistical cues of SNN layers adjust these two important parameters, defined in Eqs. and 4 Specifically, we model m (4) as the mean of the membrane potentials of the neurons in the layer The mean value is shifted by a bias, (.2(max((f(()) the range of the potentials; see Eq. The motivation behind this formulation is that aim to couple the DET and the potentials of all other neurons in the same layer. Furthermore, we leverage the bias term to adjust the DET sensitivity to the layerwise potential range. Here, (t())) is modeled based on threshold potentials (i.e., (t))) instead of membrane potentials; 4 We note that the performance of the proposed BDETT is not sensitive to the constant value 0.2; see Supplementary Note for details.
min(v, (t))). which is based on
similar insights, where
Dynamic Temporal Threshold (DTT) We propose DTT scheme to address the observed negative correlation between the spiking threshold and the preceding rate of depolarization. Azouz et al.
17 discovered that a monoexponential function y a + be VVC can effectively capture the negative correlation of a biological neuron, where V dVm dt: and C' are parameters to optimize. The authors applied this function to 42 cortical neurons and found significant correlations 92% of the trials (17. We propose a variant of this mechanism. In particular, we replace the constant a with an exponential decay function, and we base the decay rate on the mean of the dynamic thresholds of all neurons in the 1-th layer at the previous timestamp I; is set to 1. Additionally, empirically set the delay constant C. Mathematically, for the i-th neuron
decay constant; and a,

---

in the 1-th layer, the DTT at timestamp t + 1 is defined as
T 1) +
(5)
(m for i = 1,2,...,.t' n'.
(6)
Figure Th shows two example graphs for Eq. 5 These plots highlight that higher depolarization
(i.e., of +1++) 1) (t) > 0) leads to a lower temporal threshold, while higher hyperpolarization (i.e.
(f(1+ - (1) < 0) significantly increases the temporal threshold. We propose modeling a similar to how Vn (t) is modeled in the DET, that is, by coupling the DTT value and the layerwise dynamic thresholds at the previous timestamp (i.e., (t)). The delay constant C' adjusts the sensitivity of the DTT to changes in the temporal potential of a neuron. As shown in Figure[p.. lower C' value results in a substantially faster drop in the DTT value (i.e., the black curve) than that provided by a higher C' value (i.e., the blue curve).
Interaction of DET and DTT A critical difference between DET and DTT lies in the drivers of the two threshold schemes. DET leverages the magnitude of the membrane potential to estimate a threshold, while the DTT based on the preceding rate of depolarization. Therefore, they may be counteracting or helping each other to achieve an optimal threshold. One example is that when noise causes low potential fluctuations, the overall threshold should increase to suppress the noise. this case, the DET increases as the noise increases the membrane potential. However, DTT remains at relatively constant threshold (i.e.. a + 1) as the preceding rate of depolarization caused by the noise is close to O. When neuron experiences a fast membrane potential drop, e-g., during the relative refractory period, we expect the overall threshold to increase. In this scenario, even though DET decreases with the reduced membrane potential, DTT increases faster. Hence, the proposed method increases the overall threshold in this case. Please see Supplementary Note 10 for details.
Experiments
We assess the effectiveness of BDETT on three different tasks: robot obstacle avoidance, robotic continuous control and image classification. In the robot obstacle avoidance task, robot aims to reach a randomly chosen destination without touching any obstacle within 1000 steps, counted as a "pass". For this task, we assess methods by measuring success rate (SR), the percentage of successful passes out of 200 trials. As continuous control tasks, we evaluate the HalfChectah-v3 and Ant-v3 control outputs (see Figure (3a) from the OpenAl gym 4T.. In these two continuous control tasks, an agent relies on learned SNN-based control policy to decide the next action based on the current observation (i.e., state), and each action associated with reward; see Figure Bp. We assess control policies with the total sum of the rewards. Note that the Ant-v3 control task is more challenging than HalfCheetah-v3, with significantly large state and action spaces. Top-I classification accuracy is used to assess image classification.
For the robotic control tasks, in addition to evaluating the control output, i.e., SR and total reward, we also measure the homeostasis of the host SNNs. In particular, we use three statistical metries, FR.. FR and FRed to quantify the homeostasis of an SNN; these metries are based on the neuron firing rate. FR,, is the mean neuron firing rate of an SNN across all P trials; FR std is the average of standard deviations, and each of them is the standard deviation of the neuron firing rates of an SNN during single trial; FR* std denotes the standard deviation of the P standard deviations. FR represents the standard deviation across all P trials, while FR denotes the mean of these standard deviations. Details on these three metrics can be found in Supplementary Note 2.
variants of the spiking actor network (SAN) g9 as host SNN. The original SAN uses LIF as its neuron model, but it resets the membrane potentials of all neurons to zero for each robot state. The resting operation is contradictory to the leaky function of LIF. Therefore, we modify the SAN by removing the resting operation, which dubbed SAN-NR. To validate the effectiveness of the proposed BDETT, we integrate it into both LiF-based and SRM-based SAN-NR models and compare them with their original static threshold and two heuristic dynamic threshold schemes, DT1 24 and DT2 (6).. See Supplementary Note 2 for details on the DTI and DT2 schemes. We set the batch size to 256 and the learning rate to 0.00001 for both the actor and critic networks during the training process. In addition, we use the following hyperparameter settings for the proposed BDETT: 7 0.01 and vb = 4.0 for the DET and C = 3.0 for the DTT. For estimating homeostasis, we set P 200. See Supplementary Notes 4 for further training details.
Experimental Setup For robot obstacle avoidance tasks, the
5

---

For robot continuous control, we adopt the population SAN (PopSAN) BS as our baseline model; modified version of SAN [9] with a specifically designed encoder and decoder for accommodating high-dimensional control tasks. Note that PopSAN does not rest the membrane potentials as the encoder leverages soft-reset IF neurons. Hence, PopSAN the counterpart of the SAN-NR used in the obstacle avoidance tasks. We integrate BDETT into both LIF- and SRM-based PopSAN models and compare them with their original static threshold schemes and the two heuristic dynamic schemes, DTI and DT2. Following the evaluation settings of PopSAN [35], we train ten models corresponding to ten random seeds, and the best-performing model is used for our assessment conducted under different degraded conditions. In partieular, the best- best-performing model is evaluated ten times under each experimental condition, and the mean reward of the ten evaluations represents the model performance. Each evaluation consists of ten episodes, and each episode lasts for a maximum of 1000 execution steps. Hence, the P value used for estimating homeostasis is set to 100, i.e., 10 episodes x 10 evaluations. PopSAN and its variants
trained by using the twin-delayed deep deterministic policy gradient off-policy algorithm [42.. The hyperparameter settings of BDETT is the same as the ones used for obstacle avoidance tasks, except the & for the DET is set to 6.0. Following the training protocol of PoPSAN (35], we set the batch size to 100 and the learning rate to 0. 0.0001 for both the actor and critic networks. The reward discount factor is set 0.99, and the maximum length of the replay buffer is set to million. See Supplementary Notes 5 for training details.
For the SRM-based baseline methods, the spike response kernel and refractory kernel of the SRM are adopted from [27 29), and they are defined as =(t) = tel and ((t) = (t) respectively. For all tasks, each dimension of robot state is encoded into a spike train with T timesteps. All experimental results are obtained with 7 = 5. For a demonstration of the generalization provided by the BDETT, we provide the experimental results obtained with T 25 in Supplementary Notes 4, 5, 6 for the obstacle avoidance, HalfCheetah-v3, and Ant-v3 tasks, respectively.
4.1 Robot Obstacle Avoidance with BDETT
We evaluate the proposed method for robot obstacle avoidance tasks with one standard condition, ie.. static obstacles, and three specifically designed adverse conditions: dynamic obstacles, degraded inputs, and weight uncertainty. For the dynamic obstacle experiments, we introduce 11 dynamically moving cylinders in a static testing environment, and each repeatedly wanders between two points; see Figure Zh. The wandering distance and speed are designed to provide sufficient space and time to allow possible passes. The robot utilizes a Robo Peak light detection and ranging (RPLIDAR) system as its sensing device to detect obstacles, offering a field of view of 180 degrees with 18 range measurements, as shown in Figure P
In our degraded input scenario, we disturb the obtained range measurements in three different ways: "0.2": We set the range of the 3rd, 9th, and 15th lasers to 0.2 m, always reporting obstacles even when none oceur; "6.0": This similar to the "0.2" setting, but we set the three lasers' ranges to 6.0 m, which is the average visible range in the test environment and means that the three lasers cannot perceive any obstacles; "GN": We add Gaussian noise [3] to each of the 18 range measurements. The three proposed degraded input settings are illustrated in Figure 2
In the weight uncertainty experiments, as illustrated in Figure 24, the learned synaptic weights of the host SNNs are also disturbed in three different ways. "8-bit Loihi weight": Neuromorphic hardware (e.g.. Loihi) achieves computing efficiency by sacrificing the weight precision. Therefore, when deploying an SNN on neuromorphic hardware, one needs to scale and round up the learned floating peating-pointtooe point synaptic weights to low -ereciccn weights. "GN weight": We add Gaussian noise, (0, 0.05). to all synaptic weights. "30% zero weight": Among the synaptic weights between every two adjacent layers, we randomly set 30% of them to 0. To reduce the impact of the randomness introduced in the "GN weight" and "30% zero weight" experiments, we report the average success rates (SRs) and standard deviation of 5-round tests.
Success Rate The SRs of the competing LIF- and SRM-based approaches across all experimental settings are reported in Figure 2 and Table [ For the "GN weight" and "30% zero weight" experiments, the standard deviations of the 5-round SRs are also reported. The proposed BDETT achieves the highest SRs in all experiments, demonstrating its effectiveness. Notably, under dynamic obstacle conditions. the BDETT outperforms the runners-up by significant margins (9% versus the LIF and 12% versus the SRM). Under degraded input conditions, the BDETT yields at least 10% more successful passes than other competing methods. In the weight uncertainty experiments, our BDETT increases the SRs of the baseline SAN-NR model by at least 10.5%, 24.6%, and 15.6% under "8-bit Loihi weight", "GN weight", and "30% zero weight" settings, respectively. We observe

---

that our BDETT can help the robots effectively avoid both static and dynamic obstacles under all three adverse conditions; see Supplementary Tables 2, 3, and for details.
Cbrsck Awaklanee Tack
Clletacle Aveidlace
UbW
Deguoded Brquas
Wazght LrconRER
Sedd Dyeariic Chamada
Kn
CN Weight
Norral LUICT
Dirxneed Loct
Zoe Woight
BUETT
OCI
afti
AFF
AFF
OFR.
LIF
SRM
wzight Proposed method for robot obstacle avoidance. a. The static and dynamic testing environments of the obstacle avoidance tasks. b. The control loop of a robot. c. The three specifically designed degraded input conditions. d. A demonstration of the three weight uncertainty experimental settings, e. The SRs of obstacle avoidance under all experimental settings. 'SO' and 'DO' indicate the testing environments with static and dynamic obstacles, respectively; '30%' and 'GNW" denote the "30% zero weight" and "GN weight" conditions. f & g show the LIF- and 3RM-based SNNs' homeostasis changes with respect to the base condition (i.e., DO) in terms of three metrics. e-h use the same color codes as shown in f.
6.0
GN weight yourd))
30% Zero weight sounds;
Figure 2:
Table 1: Quantitative performance of obstacle avoidance under degraded conditions. Here, o is the standard deviations of the 5-round SRs.
LIF
SRM
LIF
SRM
LIF
SRM
Type
Name
SR1
SR1
Type
Name
Type
Name
SRT
SR1
SAN
78.5%
68%
SAN
71%
70%
SAN
71.5%
57%
SAN
NR
80%:
59%
SAN NR
70%
61.5%
SAN NR
72%
65.5%
0.2
24
64%
6.0
24
GN
DT11
58%
DT2
26
78%
53.5%
DT2 26
61.5%
55%
DT2 26
71.5%
61.5%
BDETT
90%
79.5% %
BDETT
84.5%
83%
BDETT
84.5%
82,5%
78.3%
77%
51.3%
(a-6.8)
0% (a-0)
59.3%
(a-10.6)
0% (ar-0)
30%
8-hit SAN
NR
79.5%
76.5%
GN
SAN NR
52.5%
(m17.11
37.2%
(m--7.6 7.6)
SAN NR
61.6%
(or 7.5)
46.5% (cr 12.4)
Loihi
1311
24
70%
67%
weight rounds)
DT1 24
54.6% (7-7.9) 44.9% (0-1 1.4)
weight
Zero
1Y11
24
41.2%
(o-7.7) 44.3%
(1-11.7)
weight
DT2
26
78.5%
67.5%
DT2 26
73.2%
7.47 43.6% (a (.4)
DT2 26
55.6%
(a 9.3)
49.1%
10.8)
BDETT
90% 88.5%
BDETT
87.7% (o-3.3) 61.8% (o-2.9)
rounds)
BDETT
77.2% (o-3.6) 65.2% (o-2.7)
Homeostatic Evaluation When an SNN is in homeostasis, all neurons are expected to have similar and sparse firing patterns under different conditions [4 45). Therefore, when transferring from one condition to another, the SNNs with stronger homcostasis are expected to induce fewer changes in all three metrics. The changes induced in all successful trials involving the LIF- and SRM- host SNNs under different experimental settings are illustrated in Figures 2 and g, respectively. The changes (i.e., in AFR... AFRT and AFR are estimated with respect to the corresponding homeostasis achieved the dynamic obstacle experiments, i.e., under the base condition. The proposed BDETT scheme yields minimal changes in all three metrics when transferring from the base condition to all other experimental settings, except for the AFR estimated based on the SRade 8-bit Loihi weight experiment. The figures highlight that the proposed BDETT significantly improves on the baseline SAN-NR model, as evidenced by the remarkable drops in these three statistical metrics. For example, as shown in the "6.0" section of the AFR,, in Figure 2 our dynamic threshold scheme reduces the AFR,, from 0.043 to 0.001. In the "0.2" section of the AFR; in Figure Plp. the AFREd is decreased to 1.7% of its original value (from 0.0058 to 0.0001). We also witness that the DTI and DT2 schemes significantly weaken the baseline model's homeostasis, as shown in Figure 2 in the "0.2" section of the AFRM and the "6.0" section of the AFR
The goal of homeostasis to enhance the host SNN generalization. Therefore, we expect SNNs with stronger homeostasis (i.e., smaller AFR,. AFR and AFR values) to outperform those with weaker homeostasis. Our experimental results confirm this, validating that the strong homeostasis provided by our BDETT can improve the generalization capabilities of SNNs to different degraded
7

---

d
Rewsrrd 3
Continuous Control Task
Action
Besg
Hnilt Deett betth
Ant
Hoet SINN
RV
rOW BV
State
Degraded Inputs
Reward of Ané-
KI
33%
RU
30%
Weight Uncertainsy
S-bel
SAA
Random Joint Position Joint Velocity
Random
GN
32 bit--8 bit
ON Weight
30% Zero Weight
SBM
PaySAX
DT1
D12
RDETT
Bltto 11-4
g
Hail Haetahh
ROOETT 680344
Ant
CUATE
AFR
OPK
2FRC
AFR
AFR
DFR,,
LIF
SRM
Random
Bandom,
Figure 3: Proposed method for continuous robot control. a. The control loops of HalfCheetah-v3 and Ant-v3, b. Examples of the three specifically designed degraded input conditions, where the red dots and curved arrows indicate the disturbed joint positions and velocities, respectively. c. The three specifically designed weight uncertainty conditions. d & The rewards of the HalfCheetah- v3 and Ant-v3 tasks across all experimental conditions, respectively. 'Base' indicates the normal base condition; 'RP' and 'RV' denote 'Random joint position' and 'Random joint velocity'. f & g- The LIF- and SRM-based SNNs' homeostasis changes with respect to the 'Base' condition in the HalfCheetah-v3 v3 tasks. h & The LIF- and ERM-based SNNs' homeostasis changes with respect to the 'Base' condition in the Ant-v3 tasks, d-i use the same color codes shown in e.
conditions. We argue that this a highly desired capability not only for mobile robotics but also for broader machine learning. See Supplementary Note 4 for more experimental results and analysis.
4.2 Continuous Robot Control with BDETT
For the HalfCheetah-v'' v3 and Ant-v3 tasks, similar to the robot obstacle avoidance tasks, we evaluate on standard and two specifically designed degraded inputs and weight uncertainty adverse conditions to demonstrate the strong generalization enabled by our BDETT. In this context, for the degraded input conditions,
disturb the observations of these two control tasks in three ways. "Random joint position": For each episode, one of the joint positions is randomly selected, and its original position replaced by random number sampled from Gaussian distribution (0, 0.1). "Random joint velocity": We randomly select one of the joint velocities in each episode and change its observed velocity random number sampled from Gaussian distribution (0, 10.0). "GN": In each episode, we add Gaussian noise sampled from the distribution /O 1.0) to each dimension of state; see Figure 3p. The weight uncertainty conditions of the control tasks are the same as those used in the robot obstacle avoidance tasks, as illustrated in Figure
Rewards As shown in Figures B]. and Table 2 under all experimental settings, the proposed BDETT offers the host SNNs the highest rewards, significantly improving upon the rewards of the baseline PopSAN model by least 438 (i.e., the SRM-basedd PopSAN model under the "GN" setting) for the HalfCheetah-v3 tasks and 213 (i.e., the LIF-based PopSAN model under the "Random joint velocity" setting) for the Ant-v3 tasks. Notably, under weight uncertainty conditions with HalfCheetah-v2 agent, even with low low-precision 8-bit weights, the proposed BDETT helps the SRM- based host SNN achieve higher reward than that obtained with high-precision floating-poin weights (1 (11767 vs. 1 11268); see Supplementary Tables 7 and 9. With an Ant-v3 agent, the proposed BDFTT helps both the LIF- and SRM-based host SNNs achieve higher rewards, even with low-precision weights, i.e., 5570 5526 and 5648 5643, respectively. See Supplementary Notes 5 and 6

---

Table 2: Quantitative performance of continuous robot control tasks under degraded conditions. For each cell, we report the estimated rewards for both HalfCheetah-v3 and Ant-v3 tasks in the format of HaliCheetah--</Antt--3
LIF
SRM
LIF
SRM
LIF
SRM
Type
Name
Rewardt
Rewardf
Type
Name
Reward
Reward
Type
Name
Rewardf
Reward
PopSAN
7832/2503
7120/3004
PopSAN
7020/2890 6576/2372
PopSAN
2440977
3457/1031
Random
Random
joint
DT1
24
3923/1435
6830/1333
joint
DI1
24
3187/2628
3836/2508
GN
DI1 24
2790922
2210958
position
DT2
26
3750/1280
3230/1330
velocity
DT2
20
3395/1579
3081/1025
DT2 26
1994/560
2307/583
BDETT
8465/3339
7883/3450
BDFTT
8302/3103
7116/2984
BDFTT
3909/1269
3895/1559
bib
PopSAN
10728/5347
10802/5285
PopSAN
46401/37
3583/467
30%
PopSAN
5020/287
3233/372
weight
Loihi
DT2
24
26
6026/5004
4372/3122
6569/4889
4629/3463
weight
GN
DT1 24
D12
26
1334/-265
4483/221
2028/-173
4128/-57
weight
Zero
DT1 24
D12 26
2721/-548
1995/1247
3056/-203
3503/1450
BDETT
10N21/5700
11767/5648
BDETT
6928/2782
8381/1658
BDETT
6551/2931
33863346
for additional experimental results and analysis related to the HalfCheetah-v3 and Ant-v3 tasks, respectively.
Homeostatic Evaluation In Figures -i,, we show the changes induced in these three metrics when shifting from normal conditions (i.e., the base conditions) to all other experimental settings. The proposed BDETT offers the strongest homeostasis to the host SNNs among all competing approaches for both the HalfCheetah-v3 and Ant-v3 control tasks. In particular, for the HalfCheetah-v3 control task, as shown in the "30% zero weight" section of the AFRm. m. in Figure 3r our dynamic threshold scheme reduces the AFR, of the baseline PopSAN model from 0.069 to 0.006. In the "GN weight" section of the AFR; std in Figure 3r. the proposed BDETT decreases the AFRE of the -RMed PopSAN to 8.3% of its original value (from 0.0012 to 0.0001); see Supplementary Table 10 for details. For the Ant-v3 control task, as shown in the "GN weight" section of the AFR,,. in Figure 3h. our dynamic threshold scheme reduces the AFR,,. of the LIF-based baseline model from 0.041 to 0.003. In the "Random joint position" section of the AFR.,a in Figure 3h, the AFR of the LIF-based baseline model is decreased to 10% of its original value (from 0.0010 to 1; see Supplementary Table 15 for details. As in the obstacle avoidance tasks, the DTI and DT2 schemes significantly decrease the homeostasis of both the LIF- and SRM-based baseline models in both continuous control tasks. Some extreme cases are shown in the "Random joint velocity" section of the AFR
std in Figure 3 and the "GN weight" section of the AFR,, m in Figure 3
These experimental results obtained for the two continuous control tasks support the observations obtained in the obstacle avoidance tasks. More importantly. we witness that the strong homeostasis provided by our BDETT improves generalization to severely degraded conditions.
4.3 Image Classification with BDETT
We assess the proposed SNN- learning method on image classification as a relevant vision task. In particular, to measure the generalization of the proposed BDETTscheme, we conduct additional image classification experiments under normal and degraded conditions. To this end, we simulate degraded inputs similar to the robotic control tasks; see the weight uncertainty degraded settings illustrated in Figure 4t. addition, we test on degradations that are tailored to classification from
two adversarial attack methods; the fast gradient sign method (FGSM) 46) and projected gradient descent (PGD) [47] ; see Figure
Specifically, we train the SCNN model 48 on the MNIST dataset [49 as our baseline model. Each pixel of an MNIST image is encoded into 30 Poisson spikes as inputs SCNN for training and testing. As shown in Figure and Table 33 directly applying the proposed approach without any changes to image classification in degraded conditions compares favorably across all experimental settings and in terms of generalization. For stronger degradations, the Top-1 classification accuracy of both the baseline and our approach decreases, but the proposed method is less affected. Note that the SCNN model contains CNN layers, blocking us from estimating homeostasis.
4.4 BDETT without Statistical Parameter Adjustment
We found it essential to replace the constants in the two biological models we base our approach on with layerwise statistical cues. Here, we report the performance of the BDETT with the original constants of the fitted biological models, demonstrating the effectiveness of the proposed layerwise statistical parameter settings. In particular, we first use the corresponding constants of the fitted
adaptive threshold model [16] and replaced the VL (!) and V3 (4), i.e., Eq. 3 Eq. 4 with 3 and 7, respectively. These two constants are obtained by shifting the originally fitted constants -67 and -63 by 70 to compensate for the difference of the rest potentials; - 70 m in the original model but 0 mV

---

Image Clestretion Task
Accurncy of Classification
a
5 6
2 3
4
5
6
8
7
L.L.
37%
FISM
506
1 2
5
5
6
b
7
?
7
Degioded lopea
d
LL
PG
PED
106
1 2
3
5
6
9
Wesht Encoaary
GNI55)
0
/
2
3
4
5
6
8
9
8
3
5
9
LIP
32ha Exc
Negg
Tlankekk
Figure 4: Proposed method for image classification. a. The examples of MNIST dataset. b. The forward pass of SCNN model [88.. C. Examples of the two adversarial samples as degraded input conditions. d. The three specifically designed weight uncertainty conditions. e. The Top-1 accuracy(AC) of image classification under all condition settings. '20%' and '30%' denote the "20% zero weight" and "30% zero weight" condition, respectively; 'GN(x)' denotes the Gaussian noise, (O z). in "GN Weight" settings.
Table 3: Quantitative performance of image classification tasks in Top-1 accuracy (AC). For FGSM, we set e = 0.2. For PGD, we set = 0.01 and run 20 iterations. GN(x) indicate Gasussian noise, I).
LIF
SRM
LIF
SRM
LIF
SRM
LIF
SRM
Type Name
ACT
AC1
Type Name
AC1
AC1
Type Name
AC1
AC1
Type Name
ACT
AC1
SCNN
99.42%
99.13%
SCNN
66.33%
56.85%
SCNN
84.31%
67.53%
SCNN
9..86%
98.05%
8-hit
Normal
24
99.40%
99.05%
FGSM
43.70%
43. 19%
PGD
24
77.32%
61.82%
Loihi
DT1
98.75%
99%
DT2
26
98.24%
98.138
DT2
26
36.48%
37.90%
DT2
26
78.25%
60.44%
weight
DT2 26
97.17%
96.68%
BDETT
99.45%
99.15%
BDETT
69.14%
57.01%
BDETT
85.74%
68.06%
BDETT
98.86%
98.22%
SCNN
81.98%
78.24%
SCNN
39.84%
32.
SCNN
90.52%
95. 10%
SCNN 84.37%
89.75%
20%
30%
GN
DTI
24
56. 19%
54.38%
GN
DTI
X
39.92%
41.13%
Zero
DTI
X4
87.20%
93.58%
Zero
DTT1 24
80.25%
83.09%
0.3)
DT2 26
26.23% (0, 0.5)
26
14.35%
9.81%
weight
D12 26
79.43%
83.00%
weight
1Y12 26
66.77%
19%
BDETT
85.09P% 78.68%
BDETT
47.74% 46.34%
BDETT
96.37% 96.59%
BDETT
90.68% 91.02%
for LIF and SRM models. Furthermore, we use the original fitted parameters in our DTT, and Eq. becomes (1 = 1.0 + 10e -((((++))=((00)
For obstacle avoidance, with the originally fitted constants, the base policy cannot produce any successful pass even under the standard testing condition; SR drops from 92.5% to 0%. For the HalfCheetah-v3 and Ant-v3 tasks, with the originally fitted constants, the rewards achieved by LIF-based policy dropped from 11064 to -35 and 5276 to 99, respectively. Note that an untrained BDETT-based policy achieves - 124 and -73 rewards for these two continuous control tasks. Image classification tasks follow the same pattern; that is, with the originally fitted constants, the Top-1 accuracy achieved by a LIF-based policy dropped from 99.45 99.45% to 9.80%. These experimental results validate that the proposed statistical cues are essential to the proposed method.
Conclusion
This work introduces a novel biologically inspired BDETT scheme to SNNs that significantly im- proves generalization, and such, fills gap between biological research and machine learning. Dynamic threshold behavior plays an essential role in maintaining a neuronal homeostasis bio- logical nervous systems. Motivated by this observation, we propose dynamic threshold scheme to achieve homeostasis in artificial SNNs. We assess the proposed approach in real-world tasks under normal and severely degraded conditions validate its generalization capabilities. We find that the proposed dynamic threshold achieves strong homeostasis along with generalization to diverse degraded conditions. This finding is step toward employing bioplausible SNNs in real-world applications. As future work, we plan to implement the proposed scheme on neuromorphic hardware to broadly deploy BDETT in future robotic platforms.
Acknowledgements
This work was supported in part by National Key Research and Development Program of China (2022ZD0210500, 2018 A0100000,,, 2021ZD01 12400), the National Natural Science Foundation of China under Grant 11970067/U21120499 A20411/ (9UU1999114 14, and the Innovation Technology Funding of Dalian (20201266XX036. Felix Heide was supported by an NSF CAREER Award (2047359), Sony Young Faculty Award, Project X Innovation Award, and an Amazon Science Research Award.
10

---

References
[1] Manuel Valero, Ipshita Zutshi, Euisik Yoon, and György Buzsáki. Probing subthreshold dynamics of hippocampal neurons by pulsed optogenctics. Science, 375(6580)5570-574 2022.
[2] Ai Nakashima, Naoki Ihara, Mayo Shigeta. Hiroshi Kiyonari, Yuji Ikegaya, and Haruki Takeuchi. Structured spike series specify gene expression patterns for olfactory circuit formation. Science, 365(6448)caaww000) 2019.
[3] Cheng yu T. Li, Mu ming Poo, and Yang Dan. Burst spiking of single cortical neuron modifies global brain state. Science, 324(5927):643 646, 2009.
[4] Paul A. Mcrolla, John V. Arthur, Rodrigo Alvarez-Icaze Andrew S. Cassidy, Jun Sawada, Filipp Akopyan, Bryan L. Jackson, Nabil Imam, Chen Guo, Yutaka Nakamura, Bernard Brezzo, Ivan Vo, Steven K. Esser, Rathinakumar Appuswamy, Brian Taba, Amon Amir, Myron D. Flickner, William P. Risk, Rajit Manohar, and Dharmendra Modha. million spiking neuron integrated circuit with scalable communication network and interface. Science, (1566119):6688677, 2014.
[5] Youbui Zhang, Peng Qu, Yu Ji. Weihao Zhang, Guangrong Gao, Guanrui Wang, Sen Song, Guoqi Li, Wenguang Chen, Weimin Zheng, Feng Chen, Jing Pei, Rong Zhao, Mingguo Zhao, and Luping Shi. system hierarchy for brain-inspired computing. Nature, (86(7789))378-384 2020.
[6] J. Feldmann, N. Youngblood, C. D. Wright, H. Bhaskaran, and W. H. P. Pernice. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature, 569(7755):008 224,, 2019.
[7] Michael V. DeBole. Brian Taba. Arnon Amir. Filipp Akopyan, Alexander Andreopoulos. William P. Risk. Jeff Kusnitz, Carlos Ortega Otero, Tapan K. Nayak. Rathinakumar Appuswamy. Peter J. Carlson, Andrew S. Cassidy, Pallab Datta, Steven K. Esser, Guillaume J. Garreau, Kevin L.. Holland, Scott Lekuch, Michael Mastro, Jeff McKinstry, Carmelo di Nolfo, Brent Paulovicks, Jun Sawada, Kai Schleupen, Benjamin G. Shaw, Jennifer L. Klamo, Myron D. Flickner, John V. Arthur, and Dharmendra S. Modha. Trucnorth: Accelerating from zero to 64 million neurons in 10 years. Computer, 52(5):20-29, 2019.
[8] Mike Davies, Narayan Srinivasa, Tsung-Han Lin, Gautham Chinya, Yongqiang Cao, Sri Harsha Choday, Georgios Dimou, Prasad Joshi, Nabil Imam, Shweta Jain, Yuyun Liao, Chan Lin, Andrew Lines, Ruokun Liu, Deepak Mathaikutty, Steven McCoy, Amab Paul, Jonathan Tse, Guruguhanathan Venkatara- manan, Yi-Hsin Weng, Andreas Wild, Yoonseok Yang. and Hong Wang. Loihi: A neuromorphie manycore processor with on-chip learning. IEEE Micro, 38(1):82-99, 2018.
[9] Guangzhi Tang, Neelesh Kumar, and Konstantinos P Michmizos. Reinforcement co-learning of deep and spiking neural networks for energy-efficient mapless navigation with neuromorphic hardware. arXiv preprint arXiv:2003.0111 2020.
[10] Travis DeWf Spiking neural networks take control. Science Robotics, 6(58):eahk3268, 2021.
[11] Ignacio Abadía, Francisco Naveros, Eduardo Ros, Richard R. Carrillo, and Niceto R. Luque. curebellar-based solution to the nondeterministic time delay problem in robotic control. Science Robotics, 6(58)cebb22756 2021.
[12] Kevin B. Koronowski and Paolo Sassone-Corsi. Communicating clocks shape circadian homeostasis. Science, 37 (6530))aaa 2021.
[13] Gina G Turrigiano and Sacha Nelson. Homeostatic plasticity the developing nervous system. Nature reviews neuroscience, 5(2):97-077, 2004.
[14] Wei Zhang and David Linden. The other side of the engram: experience-driven changes in neuronal intrinsic excitability. Nature Reviews Neuroscience, 4(1 1)8885-900, 2003.
[15] Leon N Cooper and Mark F. Bear. The bem theory of synapsc modification at 30: interaction of theory with experiment. Nature Reviews Neuroscience, 13(11):798-810 2012.
[16] Bertrand Fontaine, José Luis Peña, and Romain Brette. Spike-threshold adaptation predicted by membrane potential dynamics in vivo. PLOS Computational Biology, 10(4), 2014.
[17] Rony Azouz and Charles M. Gray. Dynamic spike threshold reveals a mechanism for synaptic coincidence detection in cortical neurons in vivo. Proceedings of the National Academy of Sciences of the United States of America, 97(14): 10-8115, 2000.
[18] Luk Chong Yeung. Harel Z Shouval, Brian S Blais, and Leon N Cooper. Synaptic homeostasis and input selectivity follow from a malcium-dependens plasticity model. Proceedings of the National Academy of Sciences, 101(41):1493.- -911444444-48888881111 2004.
11

---

[19] Qian-Quan Sun. Experience-ddeprddent intrinsic plasticity in interneurons of barrel cortex layer iv. Journal of Neurophysiology. 102(5):2955-2973. 2009.
[20] Karine Pozo and Yukiko Goda. Unraveling mechanisms of homcostatic synaptic plasticity. Neuron, 66(3):337-351, 2010.
[21] Jose Luis Pena and Masakazu Konishi. From postsynaptic potentials to spikes in the genesis of auditory spatial receptive fields. Journal of Neuroscience, 22(13):5652-5658 2002.
[22] Rony Azouz and Charles M Gray. Adaptive coincidence detection and dynamic gain control visual cortical neurons in vivo. Neuron, 37(3):513- (1--533,1--23 2003.
[23] Guo Zhang, Ke Yu, Tao Wang. Ting-Ting Chen, Wang-Ding Yuan, Fan Yang, Zei Le, Shi-Qi Guo, Ying- Yu Xue, Song-An Chen. Zhe Yang. Feng Liu, Elizabeth C. Cropper, Klaudiusz R. Weiss, and Jian Jing. Synaptic mechanisms for motor variability in feedforward network. Science Advances, 6(25):eab4485. 2020.
[24] Yunzhe Hao, Xuhui Huang, Meng Dong, and Bo Xu. biologically plausible supervised learning method for spiking neural networks using the symmetric stdp rule. Neural Networks, 121:387-395, 2020.
[25] Ahmed Shaban, Sai Sukruth Bezugam, and Manan Suri. An adaptive threshold neuron for recurrent spiking neural networks with nanodevice hardware implementation. Nature Communications, 12( (11444234, 2021.
[26] Tacyoon Kim, Suman Hu, Jacwook Kim, Joon Young Kwak, Jongkil Park, Suyoun Lee, Inho Kim, Jong- Keuk Park, and YconJoo Jeong. Spiking neural network (smn) with memristor synapses having non-linea weight update. Frontiers computational neuroscience, 15:22, 2021.
[27] Wulfram Gerstner. Time structure of the activity in neural network models. Physical Review E, 51(1):738- 758, 1995.
[28] Wulfram Gerstner and Werner M. Kistler. Spiking Neuron Models: Single Neurons, Populations. Plasticity. 2002.
[29] Sumit Bam Shrestha and Garrick Orchard. Slayer: Spike layer error reassignment in time. In Advances in Neural Information Processing Systems, volume 31, pages 1412-1421, 2018.
[30] Moritz B Milde, Hermann Blum, Alexander Dietmüller, Dora Sumislawska, Jörg Conradt, Giacomo Indiveri, and Yulia Sandamirskaya. Obstacle avoidance and target acquisition for robot navigation using a mixed signal analog/digital neuromorphic processing system. Frontiers in neurorobotics, 11:28, 2017.
[31] Amarnath Mahadevuni and Peng Li. Navigating mobile robots to target in near shortest time using reinforcement learning with spiking neural networks. In 2017 International Joint Conference on Neural Networks (IJCN),, pages 2243 2250. IEEE, 2017.
[32] Zhenshan Bing, Ivan Baumann, Zhuangyi Jiang, Kai Huang, Caixia Cai, and Alois Knoll. Supervised learning in san via reward-moot modullatd spple-eeninggdeeeedentt plasticity for a target reaching vehicle. Frontiers in newrorobotics, 13:18, 2019.
[33] Guangzhi Tang and Konstantinos P Michmizos. Gridbot: an autonomous robot controlled by a spiking neural network mimicking the brain's 's navigational system. In Proceedings of the International Conference Neuromorphic Systems, pages 1-8, 2018.
[34] Devdhar Patel, Hananel Hazan, Daniel J Saunders, Hava T Siegelmann, and Robert Kozma. Improved robustness of reinforcement learning policies upon conversion spiking neuronal network platforms applied to atari breakout game. Neural Networks, 120:108 115, 2019.
[35] Guangzhi Tang, Neelesh Kumar, Raymond Yoo, and Konstantinos P Michmizos. Deep reinforcement learn- ing with population-coded spiking neural network for continuous control. arXiv preprint 20100200000090 2020.
[36] Guangzhi Tang, Arpit Shah, and Konstantinos P Michmizos. Spiking neural network on neuromorphic hardware for energy-efficient unidimensional slam. arXiv preprint arXiv: 1903.02544, 2019.
[37] Kate D. Fischl, Kaitlin Fair, Wei-Yt Yu Tsai, Jack Sampson, and Andreas Andreou. Path planning on the truenorth neurosynaptic system. In 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1-4, 2017.
[38] Hermann Blum, Alexander Dietmüller, Moritz Milde, Jörg Conradt, Giacomo Indiveri, and Yulia San- damirskaya, A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor. Robotics Science and Systems, RSS
12

---

[39] Timothy P Lillicrap, Jonathan Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Silver, and Daan Wierstra. Continuous control with deep reinforcement learning. arXiv preprint (tt1::5500099777 2015.
[40] Anton V. Chizhov, Elena Yu. Smirnova, K. Kh. Kim, and Aleksey V. Zaitsev. A simple markov model of sodium channels with dynamic threshold. J. Comput. Neurosci., 37(1):181-191, 2014.
[41] Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01 2016.
[42] Scott Fujimoto, Herke Hoof, and David Meger. Addressing function approximation error actor-critic methods. In International Conference on Machine Learning, pages 1587-1596. PMLR, 2018.
[43] Jinyoung Choi, Kyungsik Park, Minsu Kim, and Sangok Seok. Deep reinforcement learning navigation in complex and crowded environment with a limited field of view. In 2019 International Conference on Robotics and Automation (ICRA), pages 5993-69669 6000. IEEE, 2019.
[44] Andreea Lazar, Gordon Pipa. and Jochen Triesch. 2007 special issue: Fading memory and time series prediction in recurrent networks with different forms of plasticity. Neural Networks, 20(3):11--32,, 2007.
[45] Friedemann Zenke, Guillaume Hennequin, and Wulfram Gerstnr. Synaptic plasticity in neural networks needs homeostasis with fast rate detector. PLOS Computational Biology, 9(11), 2013.
[46] lan Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and hamessing adversarial examples. arXiv preprint arXiv:1412.6572 2014.
[47] Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: 06083, 2017.
[48] Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, and Luping Shi. Spatio-temporal backpropagation for training high-performance spiking neural networks. Frontiers Neuroscience, 12:331, 2018.
[49] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(1 96(11):2278-233 1998.
Checklist
1. For all authors...
(a) Do main claims made the abstract and introduction accurately reflect the paper's contribu- tions and scope? [Yes]
(b) Did you deseribe the limitations of your work? [Yes] The limitations are pointed out at the end
of introduction.
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs all theoretical results? [N/A]
3. If you experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)? [No] We will release the code upon acceptance.
(b) Did you specify all the training details (c.g., data splits, hyperparameters, how they were chosen)?
Yes] See section 4.1, Supplementary note 4 and 5.
(c) Did you report error bars (e.g.. with respect to the random seed after running experiments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, cloud provider)? [Yes]] See supplementary note 4 and 5.
4. If you are using existing assets (c.g., code, data, models) or curating/relcasngg new assets...
(a) your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material URL? [N/A]
13

---

(d) Did you discuss whether and how consent was obtained from people whose data you're re us- ing/curating? [N/A]
(c) Did you discuss whether the data you using/curating contains personally identifiahle informa- tion offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants total amount spent on participant compensation? [N/A]
14


Reviews:
[{"review_id": "rYoxqz3OcOZ", "reviewer": "Reviewer_wR8D", "summary": "In this work, the authors aim at bridging this gap by introducing a novel bioinspired dynamic energy-temporal threshold (BDETT) scheme for spiking neural networks (SNNs). Meanwhile, the authors propose a BDETT scheme mirrors two bioplausible observations: a dynamic threshold has 1) a positive correlation with the average membrane potential and 2) a negative correlation with the preceding rate of depolarization. \n--------------------------------------------------------------------------------------------------\nThe author's revisions address all my concerns and I would further recommend this manuscript.", "questions": "1. Since the authors claim in the methods section that the two dynamic threshold mechanisms can help each other achieve optimal settings. Thus, more sufficient evidence needs to be provided to support this viewpoint.", "limitations": "The authors emphasize in the introduction section that it still requires hardware engineering efforts.", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n1. The authors designed a bio-inspired dynamic threshold mechanism to enhance the generalization of SNNs.\n2. The new dynamic energy-temporal threshold mechanism reflects the two biological observations.\n3. The parameter setting of the dynamic threshold mechanism can use layerwise control of statistical information.\n\nWeaknesses:\n1. The interaction of DET and DTT does not seem to be supported by ablation experiments.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "rUU-BLQBRJn", "reviewer": "Reviewer_qTH8", "summary": "The authors study spiking neural networks (SNNs) with a biologically-motivated dynamic spiking threshold.  The underlying hypothesis is that having a dynamic threshold allows the network to fire similarly with a wide array of stimuli / external conditions, allowing the network to potentially generalize across such conditions better.  They show that the networks increase robots' performance under degraded conditions, and increase homeostasis per metrics they provide, such as the mean and standard deviation of the firing rates across trials.", "questions": "1- What is the motivation for choosing the 3 statistics outlined for homeostatsis? While the mean and standard deviation of the firing rates are 2 natural statistics to measure, why they (and the 3rd) should be the ones to choose for measuring homeostasis is unclear. Are there drawbacks in deciding on these particular ones (e.g., perhaps in choosing these, the networks perform worse wrt other candidate metrics?) ?\n\n2 - Do the shaded regions in Figs 2e and 3d represent SDs, SEMs, or something different?\n\n2- L234: \"soft-reset\" is unclear", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Originality:**\n\nTo my knowledge, dynamic thresholds have not been studied in the context of ANNs.\n\n**Quality:**\n\nThe authors have provided biologically motivated models and intuitions for why these models might be beneficial in ANNs.  They have then provided highly applied use cases to test the performance of their models.  However, while the performance of the networks is illustrated, no statistical tests are undertaken.  The reader is thus left to wonder as to whether which, if any, of the performance improvements are statistically significant.  Moreover, while the authors have shown both an increase in homeostasis with their metrics and an increase in performance in 2 robotic tests, the reader is left with only this correspondence.  Presumably there are ways in which generalization can occur without improved homeostasis, and ways to increase homeostasis too far, resulting in decreased learning performance?  Yet these possibilities are not raised in the present work.\n\n**Clarity:**\n\nOverall, the motivation, intuitions, experimental setups, and analyses are all clear. However, several aspects remain unclear to me.  Please see questions, below.  Also, it is only made clear in the appendix that the robotic experimental setup is indeed a virtual one, from what I have been able to see.  The reader's understanding would be facilitated if this information were included in the main text and in Fig 2's caption.  \n\n\n**Significance:**\n\nBiological neurons are known, as pointed out by the authors, to have dynamic thresholds, though the function of such thresholds is, as yet, uncertain. This study represents a welcome introduction of these known dynamics into ANNs, where such functionality can be queried in the context of feedforward networks, and the authors provide an existence proof that in certain contexts, such dynamics might contribute to increased generalization performance.  However, without stronger statistical analysis and, especially, a substantively deeper exploration of how homeostasis and generalization are linked beyond example cases, it is unclear how robust the authors' findings are, and whether the hypothesized link from increased homeostasis --> increased generalization performance is causal, and under what circumstances if so.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9yAmUPBtf72", "reviewer": "Reviewer_T4kg", "summary": "This paper introduces a new bioinspired dynamic energy-temporal threshold (BDETT) scheme for spiking neural networks (SNNs), and \nvalidates the strong homeostasis along with its generalization to diverse degraded conditions. Experiments are conducted on the robot obstacle avoidance and continuous control tasks. The result shows that BDETT outperforms existing static and heuristic threshold approaches.  The supplemental video is impressive. \n", "questions": "See the weakness part. ", "limitations": "None. ", "rating": 7, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strenghts:\n1. Introduces a bioinspired dynamic threshold scheme for SNNs that increases their generalizability.\n2, Designs a new method that uses layerwise statistical cues of SNNs to set the parameters of our bioinspired threshold method.\n3. The proposed threshold scheme achieves bioplausible homeostasis, dramatically enhancing the generalizability across tasks, including obstacle avoidance and robotic control, and in normal and degraded conditions. \n\n\nWeakness:\n1. The paper combines two dynamic thresholds that exhibit positive and negative correlations with the average membrane potential. It would be better to explain the reason and the related mathematical formulation in detail.  Plus, the author compares BDETT with four variants of the spiking actor-network (SAN), are there any other recent SNNs to compare with?\n2. The experiments are restricted to the robot and control tasks. It would be better to include more tasks such as tasks in computer vision. 3. The supplementary material has a heavy overlap with the main body part.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "7zfws4EhJ27", "reviewer": "Reviewer_r3Sy", "summary": "The authors propose a dynamic spiking threshold function that consists of DET and DTT. DET is determined by the distributions of the thresholds and membrane potentials over the neurons in a given layer at a given timestep. DET is reconfigured s.t. the larger the average potential over the in-layer neurons, the larger the threshold, so that it avoids too high firing rate over the neurons as well as it induces competition among the neurons. DTT effectively realizes a refractory period s.t. it raises the spiking threshold when the potential decreases. The effect of the proposed spiking threshold (BDETT) applied to SNN-based policy functions for reinforcement learning on two tasks, i.e., robot obstacle avoidance and robotic continuous control. The results highlight a better performance for the use of BDETT in terms of success rate and homeostasis than the baseline methods.", "questions": "Q1. Applicability of BDETT to other domains. It is required to apply the proposed method to different application domains, e.g., computer vision. The goal is two-fold: (i) the impact of BDETT on SNNs can be highlighted, and (ii) true baseline performances are available, particularly, for vision domain, there exist tons of previous publications reporting their official performances. Can the authors evaluate the impact of BDETT on other application domains?\n\nQ2. Complexity analysis. Can the authors evaluate the computation and space complexities of the proposed method and compare with previous works? \n", "limitations": "The authors did not address the limitations of the present work. But, I believe that the current result limits the impact of the present work to merely two simple tasks with unreliable baseline methods.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "$\\textbf{Strengths}$:\n\nS1. BDETT highlights a better performance than the baseline methods at least for a given set of parameters. \nS2. As the authors highlight, BDETT has bio-plausible grounds and is novel.\n\n$\\textbf{Weaknesses}$:\n\nW1. Weakness of baseline. the baseline performance is questionable given that the performance of BDETT is not compared with the performance of any previously published works. I am convinced that BDETT outperforms the previous methods that the authors addressed for a given set of parameters. However, I am not convinced if the proposed baseline is the ground-truth or close to the ground-truth.\n\nW2. Wrong citations. I found that Refs. 24 and 26 are irrelevant to the baseline methods DT1 and DT2. \n\nW3. Lack of in-depth analysis of BDETT. I am slightly tired of the authors’ emphasis on the bio-plausibility of BDETT. I agree on the point that bio-plausibility of a newly proposed method is good. But, such bio-plausibility does not justify the proposed method. The authors took a biological notion and recreated the notion largely, so that I do not think the fidelity of BDETT to biological notions is very high. Instead, I would like to suggest the authors to systematically address the advantages of BDETT over the previous methods from an engineering viewpoint rather that bio-plausibility.\n\nW4. Additional computation and space complexity. The evaluation and memorization of BDETT for all neurons cause additional computation and space complexity, which should have been addressed in detail.\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept