ICML 2025

Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks
for Full-Range Timestep Deployment
Chengting Yu 1 2 † Xiaochen Zhao 2 † Lei Liu 2 Shu Yang 2 Gaoang Wang 2 Erping Li 1 2 Aili Wang 1 2 Abstract
Spiking Neural Networks (SNNs) are emerging
as a brain-inspired alternative to traditional Arti-
ficial Neural Networks (ANNs), prized for their
potential energy efficiency on neuromorphic hard-
ware. Despite this, SNNs often suffer from accu-
racy degradation compared to ANNs and face
deployment challenges due to fixed inference
timesteps, which require retraining for adjust-
ments, limiting operational flexibility.
To ad-
dress these issues, our work considers the spatio-
temporal property inherent in SNNs, and pro-
poses a novel distillation framework for deep
SNNs that optimizes performance across full-
range timesteps without specific retraining, en-
hancing both efficacy and deployment adaptabil-
ity. We provide both theoretical analysis and em-
pirical validations to illustrate that training guaran-
tees the convergence of all implicit models across
full-range timesteps.
Experimental results on
CIFAR-10, CIFAR-100, CIFAR10-DVS, and Im-
ageNet demonstrate state-of-the-art performance
among distillation-based SNNs training methods.
Our code is available at https://github.com/Intelli-
Chip-Lab/snn temporal decoupling distillation.
1. Introduction
Spiking Neural Networks (SNNs) are modeled after biologi-
cal neural systems and feature spiking neurons that replicate
the dynamics of biological neurons (Maass, 1997; Roy et al.,
2019). In contrast to Artificial Neural Networks (ANNs),
which utilize continuous data forms, SNNs employ a spike-
coding approach, using discrete binary spike trains for data
transmission (Panzeri & Schultz, 2001). This binary sig-
naling significantly reduces the multiply-accumulate op-
† Equal contribution.
1College of Information Science and
Electronic Engineering, Zhejiang University 2ZJU-UIUC Insti-
tute, Zhejiang Univ...