Bayesian Transferability Assessment for Spiking Neural Networks
Authors: Haiqing Hao
Wenhui Wang
Venue: Accepted by TMLR ()

Abstract: Brain-inspired spiking neural networks (SNNs) attract broad interest in neuromorphic computing but suffer the problem of being difficult to optimize. Concurrently, pre-trained models (PTMs) have become a foundation for developing and applying artificial intelligence. Therefore, it is expected that pre-trained SNNs can alleviate the optimization difficulty of training from scratch. However, with a lot of PTMs available in the model hubs, effectively selecting the most appropriate PTM for a given task remains a significant challenge, often necessitating exhaustive fine-tuning and grid-searching. While several solutions to this challenge have been proposed for the mainstream artificial neural network (ANNs), aimed at developing efficient methods to assess the transferability of PTMs on target tasks, the realm of SNNs remains unexplored. The currently most used transferability assessment method for ANNs predicts transferability in a Bayesian perspective. Feature maps extracted by the PTM backbone on the target task are used to calculate the maximum model evidence as the indicator of transferability. However, ANNs and SNNs differ in architecture, rendering the existing Bayesian method incompatible with SNNs. To solve this problem, this paper introduces a novel approach to using the feature maps averaged over the time domain to calculate maximum evidence. Our proposed $\textbf{M}$aximum $\textbf{E}$vidence method with $\textbf{A}$veraged $\textbf{F}$eatures (MEAF) demonstrates effectiveness for SNNs. Additionally, the current algorithm calculates maximum evidence in an iterative way. To accelerate the selection of PTMs, an approximation method is proposed to avoid iteration in the calculation of maximum evidence, significantly reducing time consumption. It is shown through experiment that the proposed MEAF method is effective for the transferability assessment of SNNs. MEAF outperforms information theory-based assessment methods such as LEEP and NCE, which can directly adapt to SNNs on neuromorphic datasets, underscoring its potential to streamline PTM selection and application in the realm of SNNs.

Published in Transactions on Machine Learning Research (04/2025)
Bayesian Transferability Assessment for Spiking Neural Net- works
Haiqing Hao
Department of Precision Instrument Tsinghua Universily
Wenhui Wang
wwch Wisingguaaaa edu.cr
Department Precision Instrument.
Tsinghua University
Reviewed on OpenReview: https: oppereviee. net/ forum? id= GaUttgMMMMe
Abstract
Brain-inspired spiking neural networks (SNNs) attract broad interest in neuromorphic com- puting but suffer the problem of being difficult to optimize. Concurrently, pre-trained mod- els (PTMs) have become foundation for developing and applying artificial intelligence. Therefore, it is expected that pre-trained SNNs can alleviate the optimization difficulty of training from scratch. However, with a lot of PTMs available the model hubs, effectively selecting the most appropriate PTM for a given task remains significant challenge, often necessitating exhaustive fine-tuning and grid-searching. While several solutions to this chal- lenge have been proposed for the mainstream artificial neural network (ANNs), aimed at developing efficient methods to assess the transfcrability of PTMs on target tasks, the realm of SNNs remains unexplored. The currently most used transferability assessment method for ANNs predicts transferability in a Bayesian perspective. Feature maps extracted by the PTM backbone on the target task are used to calculate the maximum model evidence as the indicator of transferability. However, ANNs and SNNs differ in architecture, rendering the existing Bayesian method incompatible with SNNs. To solve this problem, this paper introduces a novel approach to using the feature maps averaged over the time domain to calculate maximum evidence. Our proposed Maximum Evidence method with Averaged Features (MEAF) demonstrates effectiveness for SNNs. Additionally, the current algorithm calculates maximum evidence an itcrative way. To accelcrate the selection of PTMs, an approximation method is proposed to avoid ileration in the calculation maximum evi- dence, significantly reducing time consumption. proposed MEAF method is effective for the transferability assessment of SNNs. MEAF out- performs information theory-based assessment methods such LEEP and NCE, which can directly adapt to SNNs on neuromorphic datasets, underscoring its potential to streamline PTM selection and application in the realm of SNNs.
is shown through experiment that the
1
Introduction
Spiking neural networks (SNNs), known the third generation of neural networks (Masss 1997 have obtained broad attention in the field of neuromorphic computing (Roy al. 2019). Inspired by the way biological neurons communicate with spikes in the brain, neurons in SNNs encode information into sequential binary spikes, offering potential way lowards energy-effciiet machine intelligence (Dampfhoffer el al. 2022) Based on its distinctive mechanism, SNNs have achieved outstanding performance in robotics (Bing et al. Debat et al.] (021), event-based vision (Zhu et al. 2022 Hagenaars et al. 2021), and other fields (Poi He et al. (0244). However, SNN models are much harder to optimize compared ANNs (Wu 2019 Wang et. al. 2020 because of the non-differentiable nature of spiking signals.
2019
et al.
2019:
et al.

---

Published in Transactions on Machine Learning Research (04/2025)
Pre-trained models (PTMs) have fueled the development and applications of artificial intelligence (Jiang et al. 2022 In computer vision, neural networks pre-trained on ImageNet (Deng et al. 2009) exhibit
improved performance or convergence efficiency (Yosinski et al. 2014 He et al.) 2019 Kornblith et al, 2019), and recent advances in vision-language pre-training provide large-scale visual encoders capable of even zero-shot generalization on downstream tasks (Radlord et al.
2021 Jia el al. 2021 Meanwhile,
language models sclf-supervised pre-trained on a large-scale unlabeled corpus demonstrate in-context learning
capabilities (Devlin et al.) 2018 Brown et al.) 2020 Radford et al. 2019 Du et al. 2022). The pre-training and adaptation paradigm has become foundational methodology for the implementation of large neural network models. With lots of PTMs available, model hubs such as TorchVision (maintainers & contributors, 2016) and Hugging Face Transformers (Wolf et al. 2020) have attracted widespread interest. Meanwhile, pre-trained SNNs have been shown to exhibit higher training efficiency and performance on downst ream tasks, significantly reducing the time required to optimize the SNN and improving generalization [ee et al., 2018 Lin et al. 2022). It is expected that pre-trained SNNs can alleviate the difficulty of optimization when training SNNs from scratch.
However, when using PTMs in model hub, selecting the most suitable PTM for a given task without exhaustive fine-tuning and grid-scarching (You et al., 2021) remains a challenge. Several model hub schedul- ing methods have been proposed to address this challenge for the mainstream artificial neural network (ANN) (Tran et al. 2019 Nguyen et al. 2020: You et al. 2021 They focus on designing efficient methods to assess the transferability of PTMs on a target dataset and then select the highest-ranked candidate. To
the best of our knowledge, however, there is no research focusing on the scheduling of SNN PTMs. Therefore, to bridge the gap, it is meaningful to explore model hub scheduling methods for SNN PTMs.
One problem is that not all existing transferability assessment methods for ANNs can be applied to SNNs directly. Current methods use the pseudo-output (Tran et al. 2019 Nguyen et al., 2020) or the feature map Yoou et al.] 2021 2022) generated by the PTMMs to assess the transferability on the target task. The pscudo-output approaches, which are information theory-based, including LEEP (Nguyen et al. 2020 and NCE Tran el al. 2019), can be applied dirretlly to SNN. However, the feature-based approach log YYo et al.] 2021), which more effective on ANNs, cannot be adopted to SNN directly. In this logME approach, feature maps extracted by the PTM backbone are used to calculate the maximum model cvidence as indicator of transferability from Bayesian perspective. Differences between ANNs and SNNs in architecture render the feature-based hots incompatible with SNNs from two aspects. i) The feature map extracted by the SNN backbone has an additional temporal dimension, which cannot adapt to the current model evidence calculation framework. fi) SNNs generate predictions based on the fcature with a nonlincar transform, while the maximum model evidence method for ANNs supposes a linear hepothesis.
Taking these two factors into consideration, this paper introduces a Bayesian method of PTM transferability for SNNs inspired by the counterpart for ANNs. To match the feature map of SNN with the current calculation framework of maximun model evidence, propose to use feature maps averaged through the time domain for the assessment of transfcrability. Thcoretical analysis shows that the nonlinear transform of the SNN layer can be estimated by a linear transform with the proposed averaged features, thus satis satisfying the linear assumption of the maximum model evidence method. Moreover, the calculation of maximum model evidence requires MacKay's algorithm (MacKay 1999) to maximize the model evidence iteratively. To further accelcrate the selection of PTMs, we propose an approximated maximum model evidence method to avoid iteration while giving similar results as logME. Experiment results show that. the proposed Maximum Evidence method with Aveaged Features (MEAF) is effective for SNNs and outperforms information theory- based NCE and LEEP on neuromorphic datascts, and achieves comparable results on static datascts. The effectiveness of the approximated maximum model evidence method is also validated. Our contributions are as follows:
1. We identify the gaps of current Bayesian transferability assessment methods between ANNs and SNNs, and propose the Maximum Evidence method with A Averaged Features (MEAF) for the assessment of transferability of pre-trained SNNs.
2. To further accelerate the selection of PTMs, we provide an approximated maximum model evidence method, which avoids iteration while giving effective results.
2

---

Published in Transactions on Machine Learning Research (04/2025)
3. We validate our proposed method through experiments, and confirm that our Bayesian- method outperforms information theory-based methods like NCE and LEEP for SNNs neuromorphic datasets, and achieves comparable results on static datasets.
2 Related works
2.1 Bayesian model selection
Bayesian model selection methods use model evidence (or marginal likelihood) among different hypotheses the criterion for selection (Wasserman 2000). It is believed that model evidence implicitly incorporates the Occam Razor Principle, which balances the complexity and the representation capability of given model (Knuth et al.; 2015). Gull uses model evidence in image reconstruction to determine critical prior hyperparameter (Gull 1989). Inspired by Gull's work, model evidence is used in the field of machine learning by MacKay to determine the degrec of polynomials in interpolation problems (MacKay 1992ah. It is also used to gel the optimal regularization coefficient fication {MacKay (192b) problems, even in multi-layer backpropagation neural network models, for better generalization performance (Wolpert, 1992). An algorithm to automatically determine the regularization co- efficient in linear regression problems is proposed by MacKay by maximizing the model evidence iteratively (MacKay 1999).
the training of regression (MacKay 1992c) and classi-
Evidence-based Bayesian method provides practical framework to automatically tune the hyperparameters of deep neural networks without the use of validation data. However, a main obstacle lies in the difficulty of estimating the model evidence when the model is deep, as conventional methods proposed by MacKay are only suitable for basic linear regression problems. The Hessian matrix of loss with respect to the trainable parameters is required to calculate the model evidence, which is impossible to efficiently calculate for large- scale neural networks. To solve this problem, scalable Laplace approximation is used to calculate model evidence for deep neural networks (Daxberger et al. 2021 Immer et al. 2021). However, this process is still Lime-consuning and suffers the problem of inaccurate estimation KKnstnne el al. 2019
2.2 Transferability assessment
Assessment of the transferability of PTMMs starts from measuring the similarity between the source task where the models are pre-trained and the target task to transfer. Several distance-based metrics are proposed to measure the difference between source and target tasks, which gives error bounds of the transferred model in theory (Ben- David & Schuller 2003 Ben-David et al. 2006 Mansour et al. 2009). While it is obvious that models trained on more challenging datasets have better performance when transferred to simple tasks, the opposite is not truc. Therefore, distance, which is a symmetry metric, is not an idcal measure of transferability (Nguyen et al. 2020).
As an improvement, some new metrics other than distance have been proposed. Tran al. (2019) uses conditional entropy (NCE) to estimate the test set accuracy after transfer by calculating the conditional entropy of the target task relative to the source task. Conditional entropy is calculated based on the probability distributions to generate source and target tasks. It is asymmetrical and can reflect the difficult relationship between tasks, achieving good results in experiments.
Contrary to the above methods, Nguyen et al. (2020) no longer uses the relation between tasks as the basis for evaluation, but instcad directly inputs the target tasks into the pro-trained model, and quantitatively evaluates the transferability through the output of the pre-trained model. They calculate the expected probability of the real result obtained from the pre-trained model based the joint distribution between the results obtained from the pro-trained model and the real result of the target task, as a basis for evaluation.
Model evidence-based Bayesian methods are also used in transferability assessment. Kim et al. (0016) uses model evidence to select pre-trained convolutional neural network (CNN) models for best feature extraction. You et al. 2001 2022) use model evidence to predict the transfcrability of diffcrent PTMs on given target datasct. In this approach, features extracted by the PTMs instead of predicted results are used as the basis for transferability evaluation. Compared to prediction results, the features extracted from the middle
3

---

Published in Transactions on Machine Learning Research (04/2025)
layer more universal, this method can be applied not only to classification problems but also to other problems such as regression.
3 Methodology
3.1
Background
We consider supervised learning task on observed data D = {(),,4)) with inputs and targets & E IR SNN model Ma predicts & for each input Ei, where 0 E RD represents the trainable weights of the model. The model evidence of M on task D is defined as the probability the model generates the observed data, which is calculated as
MDIIM = | (D, MD)00
For different candidate models, the best one can be chosen by maximizing the model evidence in the Bayesian perspective.
The model evidence can be used to automatically determine the regularization coefficients in ridge regression tasks. Given feature vectors f., 1,... ,N, the model predicts f, 1,.., ,N using linear transforma- tion. Ridge regression assumes that the parameters follow Gaussian prior distribution 0 ~ (0, -I), where a controls the strength of regularization. Furthermore, it assumes that the targets are conditionally distributed according to a Gaussian likelihood t, ~ ~N( f,,8-'), where 3-1 represents the noisc variance. For determination of regularization coefficients, linear regression models with different regularization coef- ficients a, B are viewed as different candidate models, and the evidence is calculated as a function of o, B like
Evidence(a, a, 3) = p(D(a, 3)
=/0 320 p(D)0,B)p(0(a)de
(2)
2 a exp (-2 Fm t1| 2
"m
TnTm m (det A)-
where E N denotes inputs and & E R.N denotes largets. A = al + FF m = 3A-1FTA
For convenience calculation, the logarithm of equation
2 is more often used as
N 3) log 3 log 2 + log
N
D
0 Fm - t|| m m 2
1 log det A.
(3)
2
2
2
We denote = ΣD 3a, a+ where is the in singular value of FTF.
By iteratively updating a, with MacKay's algorithm
N 7
(4)
m T m
Fm t
model evidence can be maximized with a, 3 optimized simultaneously.
Lincar probe is usually used as a method of transfer learning, especially when the pro-trained backbone is large (Radford et al.) 2021) as the retraining of large models sometimes prohibitive. In such setting, pre-trained backbones results based on the representations. This process is linear transform, and the model evidence can be calculated analytically.
used to extract representations, and fully connected layer is used to predict
Therefore, to assess and compare the transferability of different models on target task D, for each pre-trained backbone M,, evidence (a, (8) can be calculated and maximized with MacKays's algorithm, resulting the logarithm of maximum evidence La(a", (5)) Eventually, transferability can be compared in terms of C(a" ,8*), and candidate models can be ranked.
4

---

Published in Transactions on Machine Learning Research (04/2025)
3.2 Adapting maximum evidence method to SNN with averaged features
Architectural differences between SNNs and ANNs impede the direct application of existing feature-based transferability assessment methods for ANNs to SNNs. SNN uses multiple time steps of forward pass to generate results. Therefore, the feature map extracted by an SNN backbone is a two-dimensional matrix instead a single-dimeniional vector. In addition, SNN models generate predictions based on the feature map with a non-linear transform in classification tasks.
We consider the most used leaky-integral and fire (LIF) spiking neuron model. One layer of LIF neurons in the SNN is given by
dl)(1) (t) + (t), dt
T
(t) = 1, (t) 2 Uch.
(() = O, if wf(() < Wth:
where n is the layer number of the multilayer SNN, t is time and T > is a time constant. Each neuron R and output spike e {0,1). The membrane
in the last layer depicted by its membrane potential potential uf (t) will be reset to uo (usually un 0) if (t) 1. The spiking frequency is usually used the prediction of logit of the target category. This nonlinear transform makes existing ANN transferability assessment methods incompatible with SNNs.
We propose to use features averaged through time steps for the evaluation of the transferability of SNN PTMs. Consider a pre-trained SNN model with T time steps, for data (æx, E D, the feature map extracted
by the model is f(1) E IRP, 1, 1,... "T. We denote averaged feature as f = Σ [(1 ER) To get the
output prediction ł, the feature map ft) is processed by equation 5
m1() = e + u(l 1) - - -((-1))) 0 O().
(6)
o(t) = s(u(t) - u(n),
(7)
1 T
f = T
Σ o(t).
(8)
t=1
where e ~ denotes the docay of membrane potential in a time step At, time constant 7 describes the speed to decay, s()) is the function describe the spiking behavior of spiking neuron, giving when input is non-negative and 0 otherwise, and uih is the threshold to spike.
In classification tasks where the target t E {0, 1}, cach predicted result i should tend towards its target t. ,T Therefore, the membrane potential in each time
Suppose that t = i = 1, then o(t) = 1 for all t = 1, 1,... ,. step is a linear transform of the feature of this time step as
u(t) = f())
(9)
From equation 7 it is clear that 0T f()(() = =a(()= 2 Weh, Vt = 1.....T T. Therefore, we can get that
1 T F0)
1 Σ TT
T
T Uth 4th-
8¹ f -
f(t)
(10)
T
T t=1
t=1
Conversely, when t= f 0, we have f < ugh- To keep the main text concise, the derivation for this case is provided in Appendix A.2.3. This shows that the SNN transform of single layer can be approximated by a lincar classification problem, and then transformed into a lincar problem on average features.
3.3 Accelerating convergence with approximated maximum evidence
MacKay's algorithm iteratively updates the regularization coefficients and maximizes the model evidence. However, the convergence of MacKay's algorithm is not guaranteed YYuu al. 2022), and the iteration
5

---

Published in Transactions on Machine Learning Research (04/2025)
number might be large in some cases. To reduce the time consumption and thus further improve the efficiency of model hub scheduling, here we propose alternative method that avoids the iteration while giving an approximated result of maximum model evidence with MacKay's algorithm (hereinafter referred to as MacKay's 's method). We name our proposed method approzimated mazimun model evidence method (hereinafter referred to as the upprozimaled melhod).
The starting point of designing the approximated version is to make sure that our approximated results share the same properties MacKay's method. To achieve this, we state three properties of MacKay's method. These properties describe the invariance of MacKay's 's method with some operation on the input feature map. We design our approximated method adhering to these properties.
Property (invariance with stacking). Consider input data F E RN and targets t E RN. Suppose that MacKay's is algorillmm converyes to xx and 3 and the logarillm of marimized model evidence is Li, then when input data is stacked times as [F...., F] E xNxxD, MacKay's 's method converges to qa" and 8", and the logarithm of mazimized model evidence C equals exactly to LT.
Property 2 (invariance with padding zero). Consider input data F E ERRx and targets t R Suppose that MacKay's algorithm converges to a* and 8 and the logarithm of mazimized model evidence is Li, then when imput data is padded with zeros as F,, E NRNx MacKay's 's method converges to a' and 8", and the logarithm of mazimized model evidence L3 equals exactly to Li.
Property 3 (invariance with scalar multiplication). Consider inpul data F E R.NxD and targets t E IRN. Suppose that MacKay's algorithm converges to a' and 3 and the logarithm of marimized model evidence is Li, then when impul data is multipplied by a scalar q > O as qF NRD MacKay's 's method converges to a²a" and B*, and the logarithm of mazimized model evidence Ls equals exactly to Li.
The proof of property 1 and property 2 can be found in reference You et al. (2022). The proof of property given in appendix A.2.1.
The approximated maximum model evidence method is given below.
Consider input feature F E RNDD and labels t E IR.. Let
A F
(11)
N
m + (AI
F FT',
(12)
N
Bo Fm
(13)
tll + X |m]
Og + XB0-
(14)
Then the approximate model evidence can be calculated by equation 3 with an and Sg- Theorem 1. The model evidence calculated by our method satisfies properties
and
Proof. See appendix A.2.2.
The experimental results section 4.4 show that our method gives similar results to MacKay's method when the number of samples N is larger than the dimension of features D. Therefore, the approximated result can serve an approximated maximum model evidence under such circumstances. We have summarized the procedure of MEAF with MacKay's method and our approximated method in Algorithm !
4 Results
4.1
Experimental setup
Datasets
The experiment is conducted on both neuromorphic datasets and static datasets. Neuromorphic datasets vision datasets recorded with event cameras (aallego et al. 2020), which the commonly

---

Published in Transactions on Machine Learning Research (04/2025)
Algorithm Algorithm of maximun evidence method with averaged feature
1: Input: Feature map of an SNN PTM target dataset (F(() RX
J1=0 labels t E R
2: Output: Score of transferability L
3: Step 1: calculate averaged feature through time F == =- F(t)
4: Step 2 (MacKay's method):
a 1, B =
repeat
8:
A +- al F
9: m + A-
10: a m= B 5 Fm
11:
X+
12: until | X| <
13: Step (approximated method):
14: A + ||]
15: + (AI+ F"F
FTt
N
16: Fm 1
17: a A
18: A +
19: Step 3: calculate score transferability C by equation 3 with A, m, a, 3
used benchmark for SNNs (He et al., 2020). As SNNs can also apply to conventional frame-based images (static datasets), we also validate our method on these datasets. For experiments on neuromorphic datasets, use ES-ImageNee dataset (Lin et al. 2021) for pre-training and DVS128 Gesture (DVS128), CIFAR10- DVS (C10-DVS), N-Caltoch101 (N-C101) and N-MNIST for fine-tuning, and for that on static datasets, the SNN models are pre-trained on ImageNet (Deng et al. 2009 and fine-tuned on CIFAR10 (C10), CIFAR100 (C100), Caltech101 (C101) and MNIST.
Pre-train SNN models We pre-train SNN models as the foundation of our experiment. We use spiking version of vanilla ResNet, spiking version of multi-layer perception (MLP), spike-element-wise (SEW) ResNet (Fang et al. (221)) and attention spiking neural networks (Yao et al. 2023) as candidate PTMs. Ten different kinds of SNN models are used different kinds of PTMs on neuromorphie datasets, and eight different kinds of SNN models used as PTMs static datasets. The details of the PTMs given in appendix A.1. For experiments on neuromorphic datasets, models are pre-trained with Adam optimizer (Kingma & Ba 2014) with momentum 0.9 and 0.999 for 10 epochs with 8 RTX3090 GPUs. We use a step learning rate scheduler with an initial learning rate of 0.01, step size 3, and = 0.3. For experiments on static datasets, we use PTMs from Fang et al. 2021
Fine-tune pre-trained models The fine-tuning of pre-trained SNN models are as the settings in logME (You et al. (021). The parameters of the pre-trained neural network backbone are frozen during fine- tuning, and a spiking FC layer re-trained as the classifier of the PTMs. On neuromorphic datasets, each model is re-trained for 200 epochs. The arper-parameters. learning rate, and weight decay, are determined by grid-searching from le to 1e 3 and 1e - 5 to le - 8 respectively. On static datasets, each model is re-trained for 100 epochs. The same hyper-parameters are determined by grid-scarching from le 1 to le 3 and le - 5 to le 7 respectively. The test set accuracy of the best model on the validation set. is used as the ground truth of transferability.
Assessment transferability For each PTM, the transferability is assessed with MEAF, LEEP, and NCE We calculate the logarithm of maximum model evidence on the train set. Note that ranking models with the maximum model evidence does not require the involvement of the validation test set. This is also
"ode available https://githui/ http:://gthhbb.com/laabe aaap:////mmmammmmmmmmmmma.m..

---

Published in Transactions on Machine Learning Research (04/2025)
advantage of the assessment method over the fine-tuning methods. Same You et al 2021 the Kendall correlation coefficient is used as a metric to evaluate the validity of our assessed score of transferability. Given N candidate models, {T', < i S. } is the list of their ground truth transferability and {St, 1 S i J is the list of predicted scores. Kendall's coefficient т gives a quantitative illustration of how the ranking of S. can stand for the ranking of T. that when S: > Sj, then the probability that T, > T3 is _ (Fagin) ct al. (2003). We also provide Pearson correlation coefficients between the test accuracy and predicted scores, which is used in LEEP (Nguyen al., 2020) and NCE (Tran et al.) 2019) as the metric. This metric could better illustrate how the predicted scores are linearly correlated with the ground truth transferability.
4.2 Transferability assessment neuromorphic datasets
We demonstrate the effectiveness of the maximum evidence method with averaged features on neuromorphic datasets. The results of Kendall correlation coefficients are shown in Figure [ It is shown that our MEAF method gives the best prediction result on all datasets, compared to NCE and LEEP. The Kendall coefii- cients are above 0.78 on all datasets, meaning that the predicted scores can represent the ranking of real transferability with a probability of more than 89%. It is shown that LEEP also gives effective predictions SNNs but with relatively lower Kendall coefficient. Note that N-MNIST, NCE gives predictions with correlation coefficients nearly equal to zero.
The Pearson correlation coefficients are shown in Table I which shows that MEAF gives predicted scores more linearly correlated with the ground truth transferability. Neuromorphic datasets have an additional temporal dimension, which results a more complicated feature map extracted by the SNNs (He et al. 2020 Samadzadeh et al., 2023: Zhang el al. 2023 Therefore, making assessments of transferability based on the feature map could leverage more information than based on the outputs, which may downgrade the performance LEEP and NCE on such datasets.
Table 1: Pearson correlation coefficients of NCE, LEEP and MEAF on neuromorphic datasets
DATASET C10-DVS
DVS128
N-MNIST N-C101
NCE
0.67
0.08
0.11
0.63
LEEP
0.76
0.31
0.24
0.57
MEAF
0.99
0.91
0.97
0.96
4.3 Transferability assessment on static datasets
Our experimental results of Kendall correlation coefficients demonstrate the effectiveness of the MEAF method on static datasets, as illustrated in Figure 2 The Kendall coefficients shows that the MEAF approach achieves comparable performance with NCE and LEEP on CIFAR10 and CIFAR100 datasets. Notably, MEAF outperforming both NCE and LEEP by a margin of 0.5 on the MNIST dataset. While MEAF shows comparable performance to NCE on the Caltech101 dataset, it exhibits a marginal difference of 0.14 compared to LEEP, When compared with the results obtained from neuromorphic datasets, the Kendall coefficients of MEAF in this experiment show relatively lower values. However, is noteworthy that all coefficients remain above 0.5 across all datasets, indicating that the predicted scores can effectively represent the ranking of real transferability with probability of more than 0.75. We suspect that this is because the transferabilities (represented by the test set accuracy) of the PTMs are much closer in this experiment. The Pearson cocfficients are given in Table 2 MEAF shows better lincar corrclation with the ground truth transferabilily compared with LEEP and NCE on CIFARIO, CIFAR100 and MNIST datasets, and achieves the second performance on the Caltech101 dataset behind LEEP. Overall, the experiment results on the static datasets show that MEAF gives comparable results as LEEP and NCE, Static datascts do not have a temporal dimension, so they could not leverage the temporal dynamics of SNNs. SNNs act like ANNs on the static datasets, which could explain that L.EPP and NCE perform well here.
8

---

Published in Transactions on Machine Learning Research (04/2025)
(a) CIFAR10-DVS (0.96)
(b) DVS122 28 Gesture (0.78)
(c) N-MNIST (0.87)
(d) N- Caltech1111 (0.87)
0.1
Maximum Evidence
0.00
0.125
1.10
0.0
-0.05
0.175
-0.1
1.06
-0,200
-0100
1.04
0.2
0.4
0.3
0.4
0.5
0.8
0.4
0.6
0.8
0.2
0.4
0.6
(a) CIFAR10-DVS (0.56)
(b) DVS128 Gesture (0.2)
(c) N-MNIST (0.2)
(d) N-Caltech 101 (0.51)
-2.15
2.1
-3.6
-2.20
LEEP
-3.8
-2.25
-2.3
-2.2
-4.0
-2.30
-2.3
-4.2
0.2
0.4
0.3
0.4
0.5
0.6
0.4
0.6
0.8
0.2
0.4
0.6
(a) 1SSVSS (0.6)
(b) DVS128 Gesture (0.16)
(c) N-MNIST (-0.02)
(d) N-Caltech 101 (0.29)
2.0
-3.00
-2.15
-2.1
-2.1
-3,25
NCE
-2.20
2.2
-2.2
-3.50
-2.25
-3.75
-2.3
-2.3
0.2
0.4
0.3
0.4
0.5
0.6
0.4
0.8
0.2
0.4
0.6
Spiking MLP-12
SEW-ResNel-18 SEW-ResNet-34
SEW-ResNet-50 1EW-ResNet-10
BeW-ResNet-15 152
Spiking ResNet-34 Spiking ResNet-50
Attention SNN
Spiking ResNet-18
Figure Result of assessed transferability scores (y-axis) vs. test set accuracy (x-axis) on 4 neuromorphic datasets. The first row gives the result of the MEAF, The second and third row shows the results of LEEP and NCE respectively. Kendall correlation coefficients are labeled in the title of cach subplot after the dataset name.
Table 2: Pearson correlation coefficients NCE, LEEP and MEAF on static datasets
DATASET
C10
C100
C101
MNIST
NCE
0.89
0.87
0.62
0.37
LEEP
0.90
0.90
0.80
0.38
MEAF
0.95
0.91
0.64
0.83
4.4 Validation of approximated maximum evidence
We first show that the approximated maximum evidence mothod gives consistent results with MacKay's method when the number of samples is large. In this experiment, we generate toy data for linear regression and add Gaussian noise to the data with different SNRs from -10 dB to 10 dB. The dimension of the features (D) is fixed at 100 and the number of samples (N) increases from 100 to 3,000. The results are illustrated in Figure 3 showing that the approximated maximum evidence converges to MacKay's results as N tends to infinity.
We then validate that the approximated maximum evidence effective model hub schoduling. Figure f shows that with the approximated maximum evidence, the Kendall coefficients are above 0.85 on CIFAR10- DVS, N-MNIST, and N-Caltech101 datasets, which is not lower than the results of MacKay's method. On
9

---

Published in Transactions on Machine Learning Research (04/2025)
(a) CIFAR10 (0.86)
(b) CIFAR 100 (0.71)
(c) Caltech101 (0.5)
(d) MNIST (0.79)
Log Maximum Evidence
1.02
0.60
0.20
1.00
0.15
0.50
0.98
1.25
0.96
0.40
0,800
0,825
0.850
0.60
0.65
0.88
0.90
0.92
0.96
0.97
0.98
(a) CIFAR10 (0.86)
(b) CIFAR 100 (0.71)
(c) Caltech101 (0.64)
(d) MNISTT (0.29)
-3.1
-1.0
-1.4
3.2
LEEP
-1.1
3.3
-1.9
-1.5
-1.2
-3.4
-1.6
-3.5
-1.3
0.800
0,825
0.850
0.60
0.65
0.88
0.90
0.92
0.96
0.97
0.98
(a) CIFAR10 (0.86)
(b) CIFAR100 (0.71)
(c) 01 (0.5)
(d) MNIST (0.29)
-2.8
1.30
-1.8
-2.9
-1.9
NCE
-1.40
-0.80
-1.45
-3.1
-2.1
0.800
0.825
0.850
0.60
0.65
0.88
0.90
0.92
0.96
0.97
0.98
SEW-ResNet-18
SEW-ResNet-50 SEW-ResNet-001
SEW-ResNet- Spiking ResNet-18
Spiking ResNel-34
SEW-ResNet-34
Spiking ResNet-50
Figure 2: Result of assessed transferability scores (y-axis) vs. test set accuracy (x-axis) on 4 static datasets. The first row gives the result of the MEAF. The second and third row shows the results of LEEP and NCE respectively. Kendall correlation coefficients are labeled in the title of each subplot after the dataset name.
the DVS128 Gesture dataset, the Kendall coefficient is 0.56, which is still higher than LEEP (0.2) and NCE (0.16). We believe that the drop coefficient on DVS128 Gesture datasct because the DVS128 Gesture a small dataset, with only 1176 samples in the training set (Amir et al. 2017). This contradicts our assumption of the number of samples being much greater than the dimension of features. As shown in Figure D when N x D, the result of approximated method differs a lot from the results of MacKay's method. The results static datasets (Figure 5) give the Kendall coefficient not lower than the results by MacKay's method CIFAR10 and CIFAR100 datasets. The result on the Caltech101 dataset gives Kendall coefficient of 0.43, which is 0.07 lower than the result of MacKays's method (0.5). For the MNIST dataset, the Kendall coefficient is 0.64, 0.15 lower than the result of MacKay's method (0.79), but still comparable to the results given by LEEP (0.29) and NCE (0.29). We believe this might be due the fact that the test set accuracy of the PTMs on MNIST overly concentrated, with values distributed between approximatcly 0.96 and 0.98. As result, the transferability assessment results are also very similar, making the impact of errors more significant. This might explain the noticeable decline in the correlation coefficient on this dataset. The Pearson correlation coefficients are given in Table 3. It is shown that the approximated method gives similar results with MacKay's method on most of the datasets.
Here we give the average number of iterations required for MacKay's method in Table The stopping critcrion for the convergence of MacKay's algorithm is set such that the relative crror between two successive a/3 is less than 0.01. This is a relatively lenient stopping criterion; employing a stricter criterion would result in higher number of iterations. Note that the approximated method does not need iteration
10

---

Published in Transactions on Machine Learning Research (04/2025)
3
MacKay's Algorithm
3
MacKay's Algorithm
Ours
Ours
2
score
score
0
0
100
300
100
100
-1
-1
-10
-5
10
-10
-5
10
SNR (dB)
SNR (dB)
MacKay's Algorithm
3
MacKay's Algorithm
Ours
Ours
2
2
score
score
1
N = 1000 0
N = 3000
D = 100
D 100
-1
-10
-5
SNR (dB)
SNR (dB)
Figure 3: Consistency between the results of MacKay's algorithm and our approximated result on toy data.
Dimension of feature (D) fixed as 100 and the number of samples (N) is 100, 3,000, 1,000, and 3,000 from (a) to (d). The abscissa represents the SNR of the loy data, and the ordinate represents the scores of transferability given by MacKay's method and approximated method.
Approdmaatd Maximum Evidence
(a) CIFAR10-DVS (0.96)
(b) DVS 128 Gesture (0.56)
(c) N-MNIST (0.87)
(d) N-Caltech101 (0.87)
1.10
-0.12
-0.04
0.05
4
-0.14
0.06
0.00
1.08
-0.16
0.08
0.05
1.06
-0.18
0.10
-0.10
-0.20
-0.12
0.16
1.04
0.2
0.4
0.4
0.6
0.6
0.2
0.4
0.6
Spiking MLP.12
SEW-ResNet-1e 3EW-ResNet-34
SEW-ResNet-50 1EW-ResNet-101
SEW-ResNet-1e Spiking ResNet-
Spiking ResNet-34 Spiking ResNet-50
Attention
Figure Result of assessed transferability scores (y-axis) vs. test set accuracy (x-axis) on 4 neuromorphic datasets. The abscissa represents the test accuracy on the corresponding dataset, and the ordinate represents the scores of transferability given by the approximated maximum evidence method. Kendall correlation coefficients labeled in the title of each subplot after the dataset name.
procedure. Therefore, could accelerate the assessment of transferability from about 1x to 7x on these datasets.
We summarized the Kendall coefficients of NCE, LEEP, MEAF (with MacKay's method), MEAF (with
A Approximated method) all the datasets in Table 5
11

---

Published in Transactions on Machine Learning Research (04/2025)
Approximate Maximum Evidence
(a) CIFAR10 (0.86)
(b) CIFAR100 (0.71)
(c) Caltech101 (0.43)
(d) MNIST (0.64)
1.02
0.200
1.28
0.50
0.175
1.00
1.26
0.150
0.45
0.125
0.98
1.24
0.40
0.100
0.800
0.825
0.850
0.88
0.90
0.92
0.97
0.98
SEW-ResNee-18
SEW-ResNet-50 SEW-ResNet-101
SEW-ResNet-152 Spiking ResNet-18
Spiking ResNet-34 Spiking ResNet-50
SEW-ResNet-34
Figure 5: Result of assessed transfcrability scores (y-axis) vs. test sct accuracy (x-axis) 4 static datasets. The abscissa represents the test set accuracy on the corresponding dataset, and the ordinate represents the scores of transferability given by the approximated maximum evidence method. Kendall correlation coefficients are labeled in the title of each subplot after the dataset name.
Table 3: Pearson coefficients of MEAF with MacKay's method and approximated method
Dataset
C10-DVS
DVS128
N-MNIST
N-C101
C10
C100
C101
MNIST
MEAF (+MacKay) 0.99
0.91
0.97
0.96
0.95
0.91
0.64
MEAF (+Appox)
0.99
0.87
0.97
0.98
0.96
0.91
0.63
5 Discussion
This paper introduces novel approach to assessing the transferability of SNN PTMs. We highlight the challenges of directly applying the cxisting Bayesian method to SNN transferability assessment and propose the MEAF method to address these issues. Additionally, an approximated maximum evidence method is introduced reduce the computational time of model hub scheduling. We hope this work can help SNN PTMs be more widely and efficiently used.
Our work is limited in the diversity of SNN PTMs. There are currently no large-scale SNN model hubs, and our computational resources for pro-training are restricted. A more extensive collection of SNN PTMs would enhance the accuracy and generalizability of our conclusions.
Acknowledgments
This work supported by the National Key Research and Development Program of China (Grant No. 20212D0200300).
References
Arnon Amir, Brian Taba, David Berg, Timothy Melano, Jeffrey McKinstry, Carmelo Di Nolfo, Tapan Nayak, Alexander Andreopoulos, Guillaume Garreau, Marcela Mendoza, et al. low power, fully event- based ges- ture recognition system. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7243 7243-7252, 2017.
Shai Ben-David and Reba Schuller. Exploiting task relatedness for multiple task learning. In Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings, pp. 567 580. Springer, 2003.
Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. Analysis of representations for domain adaptation. Advances in neural information processing systems, 19, 2006.
12

---

Published in Transactions on Machine Learning Research (04/2025)
Table 4: Iteration number used for MacKay's method
MODEL
C10-DVS DVS128
N-MNIST
N-C101
C10
C100
C101
MNISTT
Spiking MLP-12
4.9
6.8
Attention SNN
6.6
8.3
3.7
SEW ResNet-18
6.0
5.2
2.3
7.9
2.0
3.2
4.3
1.1
SEW ResNet-34
5.6
5.2
3.3
7.6
2.0
3.0
4.0
1.4
SEW ResNet-50
4.5
6.3
4.0
7.1
3.1
4.4
6.5
2.0
SEW ResNct-101
3.6
6.6
3.5
6.3
3.2
4.3
6.3
2.1
SEW ResNet-152
4.1
6.8
3.7
5.3
3.1
4.2
6.3
2.0
Spiking ResNet-18
6.0
5.3
3.1
8.0
2.1
3.4
4.6
1.1
Spiking ResNet-34
5.0
5.3
2.9
7.0
2.1
3.4
4.8
0.7
Spiking ResNet-50
3.5
6.9
2.9
6.2
3.2
5.0
7.2
2.3
Average
5.0
6.2
3.2
7.6
2.6
3.9
5.5
1.6
Table 5: Kendall coefficients of NCE, LEEP and MEAF
DATASET
C10-DVS
DVS128
N-MNIST
N-C101
C10
C100
C101
MNIST
AVERAGE
NCE
0,60
0.16
-0.02
0.29
0.86
0.71
0.50
0.29
0.42
LEEP
0,56
0.20
0.20
0.51
0.86
0.71
0.64
0.29
0.50
MFAF(MacKay
0.96
0.78
0.87
0.87
0.86
0.71
0.5
0.79
0.79
MEAF(Approxx
0.96
0.56
0.87
0.87
0.86
0.71
0.43
0.64
0.74
Zhenshan Bing, Zhuangyi Jiang, Long Cheng, Caixia Cai, Kai Huang, and Alois Knoll. End end learning of multi- layer- SNN based on R-STDP for target tracking snake-like robot. In 2019 International Conference on Robotics and Automation (ICRA), pp. 9645 9651. IEEE, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neclakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in newral information processing systems, 33:1877 1901, 2020.
Manon Dampfhoffer, Thomas Mesquida, Alexandre Valentian, and Lorena Anghel. Are smns really energy-efficient than anns? in-depth hardware-aware study. IEEE Transactions Emerying Topics in Computational Intelligence, 2022.
Erik Daxberger, Agustinus Kristiadi, Alexander Immerr, Runa Eschenhagen, Matthias Bauer, and Philipp Hennig. Laplace redux-effortless Bayesian deep learning. Advances in Neural Information Processing Systems, 34:20089 20103, 2021.
Guillaume Debat, Tushar Chauhan, Benoit R Cottereau, Timothée Masquelier, Michel Paindavoine, and Robin Baures. Event-based trajectory prediction using spiking neural networks. Frontiers in computational neuroscience, 15:658764, 2021.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. 2009 IEEE conference on computer vision and pattern recognition, pp. 248 255. leee, 2009.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training deep bidirectional transformers for language understanding. arXiv preprint arXi::1810.0488 04800, 2018.
Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. Glm: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Compulational Linguistics (Volume 1: Long Popers), pp. 320 335, 2022.
Ronald Fagin, Ravi Kumar, and Dakshinamurthi Sivakumar. Comparing top lists. SIAM Journal on discrele mathematics, 1341)1134-160, 2003.
13

---

Published in Transactions on Machine Learning Research (04/2025)
Wei Fang, Zhaofei Yu, Yanqi Chen, Tiejun Huang, Timothée Masquelier, and Yonghong Tian. Deep residual learning in spiking neural networks. Advances in Neural Information Processing Systems, 01-20056-21000 2021.
Guillermo Gallego, Tobi Delbrüick, Garrick Orchard, Chiara Bartolozzi, Brian Taba, Andrea Censi, Stefan Leutenegger, Andrew J Davison, Jörg Conradt, Kostas Daniilidis, et al. Event-based vision: A survey, IEEE transactions on pattern analysis and machine intelligence, 44(1):154-180, 2020.
Stephen Gull. Developments in maximun entropy data analysis. In Mazimum Entropy and Bayesion Methods: Cambridge, England, 1988, pp. 53-71 Springer, 1989.
Jesse Hagenaars, Federico Paredes- Valléss-Vallés, and Guido De Croon. Self-supervised learning of event-based optical flow with spiking neural networks. Advonces in Neural Information Processing Systems, 34:7167- 7179, 2021.
Kaiming He, Ross Girshick, and Piotr Dollár. Rethinking imagenet pre-training, In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4918-4927, 2019.
Weihua He, YuJie Wu, Lei Deng, Guoqi Li, Haoyu Wang, Yang Tian, Wei Ding, Wenhui Wang, and Yuan Xic. Comparing SNNs and RNNs on ncuromorphic vision datasets: Similaritics and differences. Neural Networks, 132:108 120, 2020.
Weihua He, Junwen Zhu, Yongxiang Feng, Fei Liang, Kaichao You, Huichao Chai, Zhipeng Sui, Haiqing Hao, Guoqi Li, Jingjing Zhao, et al. Neuromorphic--nabled video-activated cell sorting. Nature Comma- nications, (5(1)21079,, 2024.
Alexander Immer, Matthias Bauer, Vincent Fortuin, Gunnar Rätsch, and Mohammad Emtiyaz Khan. Scal- able marginal likelihood estimation for model selection in deep learning. In International Conference on Machine Learning, pp. 4563 4573. PMLR, 2021.
Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International conference on machine learning, pp. 4904-49044 4916. PMLR, 2021.
Junguang Jiang, Yang Shu, Jianmin Wang, and Mingsheng Long. Transferability in deep learning: A survey. arXiv preprint arXiv:2201.0520 058667, 2022.
Yong- Beok Kim, Taewoong Jang, Bohyung Han, and Seungjin Choi. Learning to select pre-trained deep representations with Bayesian evidence framework. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5318-5326, 2016.
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ar preprint arXiv:1412. 6980, 2014.
Kevin H Knuth, Michael Habeck, Nabin K Malakar, Asim M Mubeen, and Ben Placek. Bayesian evidence and model selection. Digital Signal Processing, 47:50-63 2015.
Simon Kornblith, Jonathon Shlens, and Quoe V Le. Do better imagenet models transfer better? In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 2661-2671, 2019.
Frederik Kunstner, Philipp Hennig, and Lukas Balles. Limitations of the empirical Fisher approximation for natural gradient descent. Advances in neural information processing systems, 32, 2019.
Chankyu Lee, Priyadarshini Panda, Gopalakrishnan Srinivasan, and Kaushik Roy. Training deep spiking convolutional neural networks with STDP-basce based unsupervised pre-training followed by supervised fine- tuning. Frontiers in neuroscience, 12:435, 2018.
Yihan Lin, Wei Ding, Shaohua Qiang, Lei Deng, and Guoqi Li. Es-imagenet: A million event-stream classification dataset. for spiking neural networks. Frontiers in neuroscience, 15:1546, 2021.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Yihan Lin, Yifan Hu, Shijie Ma, Dongjie Yu, and Guoqi Li. Rethinking pretraining as bridge from ANNs to SNNs. IEEE Transactions on Neural Networks and Learning Systems, 2022.
Wolfgang Maass. Networks of spiking neurons: the third generation of neural network models, Neural networks, 10(9):1699-167, 1997.
David JC MacKay. Bayesian interpolation. Neural compulation, 4(3):415-444 1992a.
David JC MacKay. The evidence framework applied to classification networks. Neural computation, 4(5): 720-736, 1992b.
David JC MacKay. A practical bayesian framework for backpropagation networks. Neural computation, (3):448 472, 1992c.
David JC MacKay. Comparison of approximate methods for handling hyperparameters. Neural computation, 11(5):1035-1068, 1999.
TorchVision maintainers and contributors. Torchvision: Pytorch's computer vision library. https://githee com/pytorcc/visiinn 2016.
Yishay Mansour, Mchryar Mohri, and Afshin Rostamizadch. Domain adaptation: Learning bounds and algorithms. Xiv preprint arXiv:0902.3430, 2009.
Cuong Nguyen, Tal Hassner, Matthias Seeger, and Cedric Archambeau. Leep: A new measure to evaluate transferability learned representations. In International Conference on Machine Learning, pp. 7294- 7305. PMLR, 2020.
Jing Pei, Lei Deng, Sen Song, Mingguo Zhao, Youhui Zhang, Shuang Wu, Guanrui Wang, Zhe Zou, Zhenzhi Wu, Wei Hc, et al. Towards artificial general intelligence with hybrid tianjic chip architecture. Nature, 572(7767):106-11 111, 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, llya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, pp. 8748 8763. PMLR, 2021.
Kaushik Roy, Akhilesh Jaiswal, and Priyadarshini Panda. Towards spike-basod machine intelligence with neuromorphic computing. Nature, 575(7784):6077611) 617, 2019.
Ali Samadzadeh, Fatemeh Sadat Tabatabaci Far, Ali Javadi, Ahmad Nickabadi, and Morteza Haghir Chehreghani. Convolutional spiking neural networks for spatio-temporal feature extraction. Neural Pro- cessing Letters, 6579)6997,, 2023.
Anh T Tran, Cuong V Nguyen, and Tal Hassner. Transferability and hardness of supervised classification tasks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1395-1405, 2019.
Xiangwen Wang, Xianghong Lin, and Xiaochao Dang. Supervised learning in spiking neural networks: A review of algorithms and evaluations. Neural Networks, 125:258-28 2020.
Larry Wasserman. Bayesian model selection and model averaging. Journal of mathematical psychology, 44 (1):92-107, 2000.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier- ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush. Transformers: State-of-the-art natural language processing.
15

---

Published in Transactions on Machine Learning Research (04/2025)
In Proceedings of the 2020 Conference Empirical Methods in Natural Longuage Processing: System Demonstrations, pp- 38 38-45, Online, October 2020. Association for Computational Linguistics. URL
https://www.aclle acwweb org//anthooogy/2020 emnlp-demos 6
David Wolpert. On the use of evidence in neural networks. Advances in neural information processing systems, 5, 1992.
Yujie Wu, Lei Deng, Guoqi Li, Jun Zhu, Yuan Xie, and Luping Shi. Direct training for spiking neural networks: Faster, larger, better. In Proceedings of the AAAI conference artificial intelligence, volume 33, pp. 1311-1318, 2019.
Man Yao, Guangshe Zhao, Hengyu Zhang, Yifan Hu, Lei Deng, Yonghong Tian, Bo Xu, and Guoqi Li. Attention spiking neural networks. IEEE transactions on pattern analysis and machine intelligence, 45 (8):9393 9410, 2023.
Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? Advances in neural information processing systems, 27, 2014.
Kaichao You, Yong Liu, Jianmin Wang, and Mingsheng Long. Logme: Practical asscssment of pre-trained models for transfer learning. In International Conference on Machine Learning, pp. 12113-121444 PMLR, 2021.
Kaichao You, Yong Liu, Ziyang Zhang, Jianmin Wang, Michael Jordan, and Mingsheng Long. Ranking and tuning pre-trained models: new paradigm for exploiting model hubs. The Journal of Machine Learning Research, (23())9400-9446)) 2022.
Yuan Zhang, Jian Cao, Jue Chen, Wenyu Sun, and Yuan Wang. Razor smm efficient spiking neural network with temporal embeddings. In International Conference on Artificial Neural Networks, pp. Springer, 2023.
Lin Zhu, Xiao Wang, Yi Chang, Jianing Li, Tiejun Huang, and Yonghong Tian. Event-based video recon- struction via potential-assisted spiking neural network. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3594-3604, 2022.
A Appendix
A.1 Details of pre-training SNNs
We use spiking version of MLP, spiking version of vanilla ResNet, SEW ResNet Fang et al. 2021 and Atlention spiking neural network Yao et al. 2023) as candidate PTMs. On neuromorphic datasets, the accuracy on validate datasets is shown in Table 6 We use the same model architectures as in Fang et al. p2021). To adapt the neuromorphic datasct,
modified the input layer of the model, shown in Table 7 not notified in the table. The number of time steps on static datasets
The unchanged syper-parameet is 4 to be consistent with Fang et al. (2021) and 8 on neuromorphic datasets for better leverageing the temporal dimension of SNNs.
A.2 Proofs
A.2.1 Proof of Property
Proof. The logarithm of maximum evidence is calculated as equation 3
N 1n + Ina"
D
N
gr
Fm - t|| a'
1
C = 2 2
In
2m
m m
det A,
(15)
2
2
2
2
where A = I F F 3 A A
16

---

Published in Transactions on Machine Learning Research (04/2025)
Table 6: Pre-trained SNN models on ES-ImageNett dataset
MODEL
TOP ACC. TOP 5 ACC.
Spiking MLP-12
0.0180
0.0615
Attention SNN
0.1297
0.6710
SEW ResNet-18
0.3443
0.5840
SEW ResNet-34
0.3507
0.5922
SEW ResNet-50
0.3140
0.5470
SEW ResNet-101
0.3031
0.5347
SEW ResNet-152
0.2615
0.4956
Spiking ResNet-18 18
0.3156
0.5486
Spiking ResNet-33
0.2376
0.4475
Spiking ResNet-50
0.1661
0.3568
Table 7: Modification of the input layer of the models on neuromorphic datasets ORIGINAL (STATIC)
NEUROMORPHIC
Conv2d(in_chuuueees
Batch normalization
Batch normalization
LJF SioxPoool2((eemme_siessss
[ygeool2((temme_ssbee sime=3) LIF
When the input data F is multiplied by q as F qF, let ā be q²a" and B be 8r, we have
A qp a 1 3 FF OF g2 A,
(16)
1
m = A FT m, q
(17)
det 2D det A.
(18)
Cg is the maximum of logarithm of model evidence and B. Therefore we have
5 2 C2(0, 3)
N In 3-
D
3
+
ln
In 2T
Fm
t
m
m
In det À
2
2
2
2
2
2
N
N
2
q²a"
(1
T
(1 m
1n
+
In
Inqa"
In 2m
(qF)
m
t
m
In det (g²A)
2
2
2
N
D
N
8"
a*
1
In
ln
2T
Fm
t|
m
In det A
2
2
Ct.
(19)
Similarly we have Li 2 Lz. Therefore Ci Czz.
A.2.2 Proof of Theorem
Proof. Consider input data F E NDD and targets E R.N. Let the approximated logarithm of maximum model evidence be CAA The singular value decomposition (SVD) of = UEVT where U E N and V DXD are orthogonal matrix, 01 2 .... 20 0 are singular values.
We first prove the case of property When the input data F is stacked times as F....,F F.. From the proof of property in You al. (2022), the SVD of F is UEV U [ ONZ,0Nx()11D1 VT where
17

---

Published in Transactions on Machine Learning Research (04/2025)
1
VI
v9
E RaDxoD product each element of Q
Rox0 with V.
Here Q is orthogonal matrix with first column
1 1
v which can be obtained through Gram-
1
σ1 voo increases / times, where Ø1 denotes the first singular value of F. N is unchanged in this case. Therefore, x= = qd. By equation 12
Schmidt orthogonalization. The norm stacked matrix
[F
F2
-1 F V (1 +"EE
m = (AI [AI+ FTF)
E-UF
((1-0))
V
vi
VQET
U
a(ttoo)
0 (111))D
(20)
+ ΣT
ETUT
m
q
!v "
m
Therefore we have Bu = fo. āτ
XBu = gag. From equation the approximated logarithm F and t is
N
ạD
N
3u
0 2
CØ
In 30 +
In an
In 2π
Fm
t
- T
m
In det A
2
2
2
2
N
In Bo
+
QD
In 90t
In 2nT
B0
Fm
E11²
a0
m
2
2
2
Tq(a0
)
1 In det
4(00 + (0o
2
qa0
N
D
(q- 1)D
9D
N
80 2
In Bo
+
In
+
In an +
In
In
2m
Fm
t||
m T m
In det A
2
2
2
2
2
2
2
1
In aD a0
11D1)D
2
= -41-
(21)
Then prove the case of property 2
When the input data F is padded with zeros F F, O. Its SVD
is Σ, O
V1 : where V1 is an orthogonal matrix. The norm [F] = 5 keeps the same
and N is unchanged well. Therefore, Ā is unchanged for F.
18

---

Published in Transactions on Machine Learning Research (04/2025)
From the proof of property
in You al. 2022).
we have
V
Σ
Ut
m
V1
Atop
(
0
(22)
(AI+ Σ) EU
0 (g-1))x 1)Dx
0 (g-
Dx1
Therefore we have 30 = 80, 00
AB0= = on-
From equation
3
C
N In 30 + In āg
qD N
2
In
30
Fm
0gg m
2
m
2
1
In det A
(ao + Buo
N In 30 +
qD
In 00
N
In
30
Fm
1|2
a0
m
m
Int
(ao + BooD)
2
2
2
a
N
In 90 +
D
In 0g +
1)D
In orp
N
In 2π
Bu
Fm t1| ag m
1
In A
1
In in
(g-1)D
2
2
2
2
2
2
LA
(23)
We prove the case if property When the input data F is multiplied by scalar > 0 as F qF. [1 = ō 1 increases times and N is unchanged. Therefore, increases x² times for F.
From the proof of property Ä A m equation 3
1 m. Therefore we have 3o Bu. 2 q²a. From a
N
D
N
80
t 2
0o m
In
90
+
In āg
In 2T
Fm
m
In det À
2
2
2
N
D
N
3o
2
8²00
T
1
In 90
+
In 2 00
In 2m
(qF)
t
m
m
In det (q²A)
2
2
N
In 8o +
D
In 0g
N
In 2m
80
Fm
||(
Og
m
1
In det A
2
2
2
2
2
2
(24)
A.2.3 Proof of case t t=f=0
For t = f 0, we have o(t) = 0, T and thus u(t) = c- -1 u(( - 1) + 0T ft). Note that u(0) = 0, it
can be conducted that.
m( = 0T f(1) <
Weh
To
(25)
u(To) r
Σ
(To -
1) at 5(1)
< Uth: 1 5 To ST
1=1
By summing u, (To) from 1 to T - 1, we have
T-1 Σ
T-To
- )) < S 1)uch.
(TTb) TT
f(To)
Σ
(26)
===1
T==1
t=0
19

---

Published in Transactions on Machine Learning Research (04/2025)
This is equivalent to
TT Σ
f(Tb) (1-
s(1-0 -0 0T-11 1)usk
(27)
Note that we have
T
(T) = TT
(T (T-))) + f(To)
S uen.
(28)
By adding Equation 27 and 28 together, we have 0
(Iro- s(To))
S -T'- -1e--1e--77 -))=
weh. which is
0 T 5( <
)
1
T -1---1--4)
0
Uth S uth-
(29)
7
A.3
Validation of approximated maximum evidence
(a) CIFAR10-DVS (1.0)
(b) DVS128 Gesture (0.78)
(c) N-MNIST (1.0)
(d) N-Caltech101 (0.91)
1.10
Logerithm Approximated
-0.04
0.05
Maximum Evidence
-0.06
0.00
1.08
-0.16
-.08
1.06
-0.18
0.10
-0.10
-0.20
0.12
-0.15
1.04
-0.20
-0.15
-0.10
-0.05
0.00
-0.1
0.0
0.1
1.04
1.05
1.08
1.10
Logorithm Maximum Evidence
Logorithm of Maximum Evidence
Logerithm Maximum Evidence
Logorithm Maximum Evidence
(a) CIFAR10 (1.0)
(b) CIFAR100 (1.0)
(c) Caltech101 (0.93)
(d) MNIST (0.86)
Logorithm of Approximaie aaded
Maximum Evidence
0.200
1.28
0.50
0.175
1.00
1.26
0.150
0.45
1.24
0.100
1.22
0.96
0.10
0.15
0.20
0.96
0.98
1.00
1.02
1.25
1,30
0.4
0.5
0.6 Evidencr
Logorithm Maximum Evidence
Lagorithm
Lagerithm
Logorithm
Spiking MLP-12
SEW-RosNet-34
SEW-ResNet-155 Spiking ResNet-18
Spiking ResNet-34
Attention SNN
SEW-ResNet-50
Spiking ResNel-50
SEW-ResNet-18
SEW-ResNet-101
Figure 6: Results scores given by approximated method (y-axis) vs. MacKay's method (x-axis). Kendall correlation coefficients are labeled in the title of each subplot after the dataset. name.
We demonstrated a strong correlation between the approximated maximum evidence and those obtained using MacKay's algorithm (Figure G). is worth noting that the correlation is relatively low on the DVS128Gesture dataset. We believe this because the DVS128 Gesture is a very small dataset, with only 1176 samples in the training (Amir al.] 2017). which contradicts our assumption of the number of samples being much greater than the dimension of features (as shown in Figure 3
A.4 Study on error range
Here we discuss the error range of accuracy in the experiments. Due to computational resource limitations, we only conducted experiments single typical dataset, DVS128 Gesture. Each pre-trained
20

---

Published in Transactions on Machine Learning Research (04/2025)
model was fine-tuned on this dataset with 8 different random seeds and grid searched with learning rates ranging from le-1 to le-3 and weight decay ranging from le-5 to 1e-8. Each model is fine-tuned 200 epoches. Based on the results, we calculated the mean and standard deviation (std) of the test set accuracy on this datasct, which are presented in Table g Additionally, to better illustrate the impact of the error range in test set accuracy on transferability assessment, we also present the corresponding results in Figure 7
Table Error range of lest set accuracy on DVS128 Gesture dalaset
MODEL
NCE
LEEP
MEAF
TEST ACC. (STD)
Spiking MLP-12
-2.10835
-2.25007
-0.11412
0.306 (0.009)
Attention SNN
-2.06354
-2.36722
-0.00881
0.615 (0.022)
SEW ResNet--8
-2.16645
-2.18893
0.00596
0.656 (0.012)
SEW ResNet-31
-2.00966
-2.07468
0.00053
0.635 (0.008)
SEW ReaNet-50
-2.26797
-2.26722
-0.03956
0.622 (0.014)
SEW ResNet-101
-2.17705
-2.18028
-0.02884
0.639 (0.017)
SEW ResNet-112
-2.14735
-2.16500
-0.05603
0.565 (0.013)
Spiking ResNet-18
-2.28786
-2.27276
-0.05153
0.623 (0.012)
Spiking ResNet-34
-.233009
-2.26244
-0.07663
0.480 (0.015)
Spiking ResNet-50
-2.18648
-2.29238
-0.09107
0.497 (0.012)
(a) DVS128 Gesture (0.16)
DVS128 Gesture (0.2)
(c) DVS128 Gesture (0.78)
0.6
0.6
0.6
Test accuracy
0.5
0.5
0.5
0.4
0.4
-2.3
-2.2
-2.1
-2.0
-2.3
-2.2
-2.1
-0.100
-0.075 -0.050 -0.025 0.000
LEEP
Log maximum evidence Spiking ResNet-34 Spiking ResNet-50
Spiking MLP-12
34W-ResNet-34
SEW-ResNet-152 Spiking ResNet-18
Attention SNN
SEW-ResNet-50
SEW-ResNet-18
SEW-ResNe:-101
Figure 7: Results of test set accuracy vs. assessment score (x-axis) with error bars. Kendall correlation coefficients are labeled in the title of each subplot after the dataset name.
A.5 Ablation study
Here investigate the effectiveness of averaging the features. We selected three alternative ways for constructing feature vectors of shape N x D from SNN feature maps of shape N x T x D, where N is the dataset size, T is the number of time steps and N is the dimension of feature of each time step. We use these feature vectors to assess transferability with MacKay's method and compare the results with those from MEAF, The three methods we selected are:
1. LAST: Selecting the feature vector from the last time step,
MAX: Applying max-pooling along the time dimension to the feature vectors,
3. PCA: Using PCA to reduce the SNN feature map from shape N x x x NoNNN D.
and the results presented in Table
21

---

Published in Transactions on Machine Learning Research (04/2025)
The MEAF method performs optimally on most datasets, achieving the highest average Kendall correlation coefficient. On the CIFARIO-DVS and N-Caltech101 datasets, it also achieves the second-best results, with gap of less than 0.04 from the optimal results. This demonstrates the effectiveness of average feature con- struction. Other methods to aggregate feature from multi-time steps, such as using a convolution layer along the time dimension may have better performance: however, these involve additional learnable parameters that need to be trained or set, making them more complex compared to the current approaches. We leave this potential direction for future research.
Table 9: Kendall cocfficient of transferability calculated by alternative ways to construct feature vector and MEAF
DATASET
LAST
MAX
PCA
MEAF
DVS128
0.69
0.73
0.69
0.78
C10-DVS
0.91
0.91
1.00
0.96
N-C101
0.91
0.87
0.69
0.87
N-MNIST
0.87
0.78
0.73
0.87
C10
0.79
0.79
0.86
0.86
C100
0.71
0.71
0.71
0.71
MNIST
0.71
0.71
0.71
0.79
C101
0.43
0.43
0.50
0.50
AVERAGE
0.75
0.74
0.74
0.79
22


Reviews:
[{"review_id": "tey2Mup4Wm", "reviewer": "Reviewer_6bD1", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths\n- Presents a method to estimate the transferability of SNNs, which is a relatively unexplored area.\n- Comprehensive datasets for evaluation, including both static and spike-based vision datasets; as well as a toy dataset to evaluate the approximation and the original iterative methods of maximum evidence.\n- Overall, the paper is well-written and easy to follow.\n\nWeaknesses\n- Only ResNet-based models are used in the analysis, which is noted in the limitation paragraph of the Discussion. However, I encourage the authors to add other SNN architectures, even simple models like SNN-based MLP, to the mix to show the generalizability of the method. \n- Please see the Requested Changes section for minor suggestions.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper aims to assess the transferability of pre-trained spiking neural networks (SNNs), which many existing methods for deep learning models cannot be applied due to the difference in rate-based and spike-based architectures. The authors adapted an existing maximum evidence assessment method to SNNs by averaging the temporal dimension of the feature maps of the spike-based model. The authors evaluated their proposed method against two baseline methods on a range of static and spiking vision datasets. Moreover, the authors proposed an approximation approach to the iterative process used in the maximum evidence method. They showed that their method can achieve similar performance while reducing the overall computation time.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "Very minor comments and suggestions\n- Can the authors clarify the x-axis of Figures 1, 2, 4 and 5 is the test accuracy? I don’t think they are currently labelled.\n- I suggest adding a table in the main text that summarizes all the Kendall correlation coefficient values from Figures 1, 2, 4 and 5 so that the reader can easily compare all the methods in different dataset configurations.\n- I believe Table 3 in Appendix A.3 is an important contribution of this work, which shows the time saved with the approximation method against MacKay’s method. I suggest adding the table, or at least referencing the table, in the main text.\n- Some typos here and there, e.g. page 5: “equation equation 7”; page 6: “equation equation 3”; page 7: I believe it should be “we use PTMs from Fang et al., 2021.” (\\citep vs \\citet).", "broader_impact_concerns": "I do not believe this work requires a broader impact statement.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FfMTW6cSSw", "reviewer": "Reviewer_6Yoo", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths**:\n- well-motivated problem,\n- intuitive and pragmatic solution,\n- good experimental results, including strong evidence that the proposed approximation method is sufficiently accurate.\n\n\n**Weaknesses**:\n- a number of typos and mistakes,\n- no errorbars on any experimental results (I imagine this is due to limited computational budget rendering multiple SNN PTM training runs impractical),\n- limited details in theoretical proofs, potentially a mistake in one of them too.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Spiking neural networks (SNNs) are in general hard for practitioners to train, so it would be desirable to have a selection of pre-trained SNN models from which to choose in order to perform the relatively easier procedure of transfer learning. When offered a selection of pre-trained models (PTMs), practitioners need a cheap way to assess which PTM is best for them, and while there are cheap and powerful ways to perform this assessment for standard PTMs, many of the developed methods are not applicable to SNN PTMs. The authors provide a new way to conduct such assessment of SNN PTMs for transfer learning tasks. The method builds upon previous work that approximates the (maximum log) marginal likelihood of a linear model mapping from the PTM-obtained features to their respective targets/labels. The main technical challenge that the authors overcome is the difficulty in assuming a linear model from SNN outputs (which, for a given input, is a 2D matrix corresponding to a sequence of representations) to targets---i.e. \"how should we perform aggregation in the time-dimension such that a linear model can be assumed?\". The authors propose averaging the 2D feature matrices in their time-dimension, and present an efficient way to approximate the maximum log marginal likelihood.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. In the paragraph between equations 1 and 2, the authors refer to the linear model's likelihood as the \"prior distribution of the observed data\"---I find this very peculiar. I have not heard the term \"prior distribution of the observed data\" before, but logically (to me anyways) the only distribution that makes any sense to correspond to this term would be $p(\\mathcal{D})$---the marginal likelihood.\n2. Equation 5 needs clarification regarding the use of $n$ to denote the network layer index. It took me a few reads to figure this out.\n3. In the paragraph below equation 10, $\\mathbf{f}$ seems to be missing an appropriate superscript (i.e. is it $\\bar{\\mathbf{f}}$ or $\\mathbf{f}^{(t)}$?). Also the converse case of $t_i=0$ is not obvious to me; it would be nice to have more detail on this.\n4. Between equations 3 and 4 the authors use the word “donate” when I believe it should be “denote”. The same goes for the paragraph before equation 6.\n5. “Properties 1” should be “Property 1”.\n6. Section 3 refers to Appendix B for proof of property 3---there is no appendix B, only subappendices of appendix A.\n7. In appendix A.2.1 equation 19, I believe the equality between lines 2 and 3 might not be correct. Using equations 16, 17, and 18, I find the second line of equation 19 to be equal to $\\mathcal{L}_1^* + \\frac{\\beta^*}{2}m^Tm\\Bigl(1-\\frac{1}{q^2}\\Bigr)$, i.e. I cannot see how the $\\frac{\\beta^*}{2}\\tilde{m}^T\\tilde{m}$ term disappears. I could of course be wrong. Some more details on how this equality holds would be appreciated.\n8. In appendix A.2.2 in the paragraph before equation 21, the authors refer to stacking $\\mathbf{F}$ (the feature matrix) $q$ times in a row to reach [F, 0]. I imagine this is a copy-paste typo from the paragraph above.\n9. In appendix A.2.2, the proof of theorem 1 looks right so long as the proof in A.2.1 is assumed to be correct. However, it requires heavy referral to [You et al., 2022], which makes it harder to parse on its own and harder to assess the correctness of. Some more details would be appreciated.", "broader_impact_concerns": "No concerns as I do not believe a Broader Impact Statement is required.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "H9BhvIdt6C", "reviewer": "Reviewer_HHBr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n1. The paper is largely well written and was easy to follow, although there is scope for more clarity.\n2. The problem of transferability assessment for SNNs is indeed untackled and may require some thinking from the community, especially as pretraining large foundational models seems to be becoming a norm.\n3. The experimental results, despite their limitations, provide initial validation of the proposed method and demonstrate its potential effectiveness.\n\n**Weaknesses:**\n1. **Claims:**\n    - The primary motivation that the paper states is that SNN models are much harder to optimize than ANN due to non-differentiability of the spike. However, the primary motivation for pretraining + finetuning in ANNs is to alleviate the issue of training data scarcity. In the context of SNNs, the non-differentiability issue will still remain to some degree during finetuning even after picking the right pretrained model. The paper should state explicitly how optimization issues are resolved starting from pretrained model, as opposed to starting from scratch.\n    - The limited availability of large pretrained models for SNNs reduces the practical impact and interest of the proposed solution. Given the fact that commonly used pretrained backbones do not differ significantly in test performance, it is difficult to think of a scenario where one would run the proposed algorithm before picking which backbone to use.\n\n2. **Clarity:** While the paper is mostly clear, it could benefit from having an Algorithm that describes the process.\n\n3. **Method Critique:** My main critique of the method is that the authors propose one approach for approximating linear probing- use time-averaged feature maps from the SNN. They provide a proof of why this is a linear transform. However, there are many other potential solutions that have not been discussed:\n    - Using features from the last time step\n    - Applying a linear dim-reduction technique such as PCA on the feature maps from all the layers\n    - Adding a temporal convolution layer to get weighted time-average of feature maps\n    - Pooling operation on time\n\n4. **Approximated Method Clarity:** The approximated method (for model evidence) is not clear to me. Why do equations 11-14 accelerate the convergence? The last paragraph of Section 3 says: “Moreover, with more reasonable prior values, MacKay’s method can be effectively accelerated”. What is “more reasonable” in this sense? How is this acceleration different from approximated method?\n\n5. **Choice of Baselines:** The choice of baselines isn’t motivated. Why are only information-theoretic methods used for comparison? Have these been used previously for SNN applications? Also, the baselines seem to be doing pretty bad in most of the experiments. Are these the right methods to calculate transferability?\n\n6. **Experiments and Results:** Several points require clarification:\n    - Pretrained models are all ResNets. The paper would benefit from adding attention-based architectures such as transformers [1].\n    - From Figure 2, it looks like if the test accuracy of finetuning pretrained model is high, then transferability measure loses its effectiveness. This can be a big issue with large pretrained models that can have high test performance on target datasets.\n    - While the paper shows # of iterations for Mackay in appendix, there is no comparison with # of iterations with approximated method. How much time is being saved because of using the approximated method? Is this a function of dataset?\n    - Looks like the approximated method has highly varied performance w.r.t. computing transferability. For some datasets, it is very close to Mackay but for some others- DVS Gesture and MNIST, there is a big drop. The paper states that it could be due to DVS Gesture being very small w.r.t. the number of features (this line should be moved to the main section). Then how realistic is the assumption of N>>D? Finetuning is best used in limited training data regime, then why would the approximated method be useful there? Also, why does MNIST show a drop?", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes an approach for model selection for spiking neural networks (SNNs). The approach is based on a commonly used technique in ANNs where features from pretrained networks are passed to a fully connected layer for prediction (linear probing). The model evidence can then be calculated analytically using Mackay’s algorithm The paper proposes the use of time-averaged feature maps from the pretrained SNNs to approximate the linear probing process of ANNs. It also proposes a method to accelerate the convergence of Mackay’s algorithm using an approximation. Through multiple experiments, the paper shows that the proposed approach can compute the transferability of the pre-trained SNN effectively.", "claims_and_evidence_yn": "No", "claims_explanation": "", "audience_yn": "No", "audience_explanation": "", "requested_changes": "See weaknesses", "broader_impact_concerns": "None", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper is generally well written but I still have a few minor comments to improve the quality of the paper. I encourage the authors to improve this, although the decision of the paper does not depend on these changes.

1. The matrix vector notation is mixed in Eq. 2-4, e.g., t is a bold-faced vector while $f_i$, $\theta$, $m$ etc. are not bold-faced despite being vectors.

2. I felt that it is better to write “invariance of Mackay’s method” rather than “invariant” (same thing for all the properties).

3. Font size in figures is too small. In general, the font size in the figures should be only slightly smaller than the main text. Please increase marker size. Use rid lines, so that it is easier to see the differences. Font in Table 1 and 2 can also be fixed easily (eg, use abbreviation for the headings).

4. Reference formatting can be improved, for example, "r-stpd" is "R-Stpd" in the original paper, "ssn" should be "SSN", and in Kunster et al. 2019, Fisher does not have a capitalized F. Also, "bayesian" should be "Bayesian". I also find it amusing that in one of the reference my name is written wrong (it is "Mohammad Emtiyaz Khan", not "Khan Mohammad Emtiyaz") but perhaps this is propagated through Google Scholar, so I don't blame the authors :) 

5. I think that the citation for Kunstner et al. is not appropriate. This paper is for Fisher matrix, not on model evidence, from what I remember. So it might be better to remove it from this particular location.

These are only minor changes and will improve the formatting of the paper.
Decision: Accept (poster)