TAS-GNN: Topology-Aware Spiking Graph Neural Networks for Graph Classification

Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-15 05:54:18

Abstract: The recent integration of spiking neurons into graph neural networks has been gaining much attraction due to its superior energy efficiency. 
Especially because the irregular connection among graph nodes fits the nature of the spiking neural networks, spiking graph neural networks are considered strong alternatives to vanilla graph neural networks. 
However, there is still a large performance gap for graph tasks between the spiking neural networks and artificial neural networks. 
The gaps are especially large when they are adapted to graph classification tasks, where none of the nodes in the testset graphs are connected to the training set graphs.
We diagnose the problem as the existence of neurons under starvation, caused by the irregular connections among the nodes and the neurons. To alleviate the problem, we propose TAS-GNN.
Based on a set of observations on spiking neurons on graph classification tasks, we devise several techniques to utilize more neurons to deliver meaningful information to the connected neurons. 
Experiments on diverse datasets show up to 27.20% improvement, demonstrating the effectiveness of the TAS-GNN.

Main paper: 
 introduction: 
 Graph neural networks (GNNs) are types of popular neural networks to learn the representations from graphs, which comprise multiple nodes and edges between them. Because of their flexibility to model any kind of connection existing in nature, it has various applications ranging from drug discovery [6, 47, 9] , social influence prediction [39, 2] , traffic forecasting [3, 7] , and recommendation systems [38, 15, 61] . One known challenge of GNNs is their sparse memory and computational pattern. Because many messages are passed between randomly connected nodes, there is a significant inefficiency in processing them with conventional systems [53, 58, 57, 19] .To address the inefficiency, spiking neural networks (SNNs) are considered strong alternatives.Inspired by the way biological behavior of brains, SNNs process information by communicating binary spikes between the neurons. Because SNNs utilize intermittently occurring spikes, they have superior energy efficiency, especially for the domain of GNNs [1] .Although the spiking graph neural network (SGNN) has been recently studied by many researchers [32, 64, 48] , we find that its performance experiences a huge drop when adapted to graph classification, compared to that of the conventional GNNs implemented with artificial neural networks (ANNs).Upon closer analysis of the performance degradation, we identify spike frequency deviation of the neurons within the model. In our investigation, many neurons experience starvation, which do not emit any spike during the inference. This leads to severe information loss, due to being unable to deliver signals to the subsequent neurons.Submitted to 38th Conference on Neural Information Processing Systems (NeurIPS 2024). Do not distribute.Such a problem was less exposed in previous spiking GNNs. This is because the testset nodes are available during the training time (transductive learning [27] ) or they are part of the training graph (inductive learning [21] ). In such settings, the model could be trained to mitigate the performance drop. However, in graph classification tasks, the graphs are independent of each other, and the testset comprises multiple unseen graphs, aggravating the problem.Fortunately, our further analysis reveals that such phenomena are related to the topology of the input graphs. We discover that a strong pattern exists among the neurons in the GNN, where 1) neurons in a node have similar behaviors, 2) each feature causes different behaviors, and 3) neurons in high-degree nodes tend to emit more spikes.Motivated by the observations, we propose to group the neurons according to the degree of the node (topology-aware group-adaptive neurons). The neurons in each group adapt the threshold together to steer the firing rate toward ideal rates. To further mitigate the initial value sensitivity problem, we further propose to learn the initial values.We evaluate TAS-GNN over multiple GNN models and datasets. Experiments reveal that the proposed TAS-GNN achieves superior performance over the baselines, setting a new state-of-the-art method for graph classification. Our contributions are summarized as the following:• We identify starvation problem of spiking neurons in GNNs for graph classification tasks.• We observe the spike frequency patterns have a strong correlation with the graph topology.• Based on the observations, we propose topology-aware group-adaptive neurons, which dynamically adjusts the threshold together with the other neurons in the group to address the spike frequency deviations.• We propose techniques to reduce the initial value sensitivity caused by the topology-aware group-adaptive neurons.• We evaluate TAS-GNN on several public datasets and achieve superior performance over existing techniques. 
spiking neural networks and spike training: 
 Spiking neural networks (SNNs) are third-generation neural network designs that mimic the human biological neural systems [35] . They use spike-based communication and adopt event-driven characteristics that promote better energy efficiency than current ANNs. Similar to human neural systems, SNNs consist of spiking neurons that can model spatio-temporal dynamics of the actual biological neurons. The early forms of such neuron models are Hodgkin-Huxley neurons [23] , which accurately model the biophysical characteristics of the membrane through differential equations. However, its mathematical complexity prohibits its practical use and scalability. Instead, Leaky Integrated-and-Fire (LIF) model finds a middle ground between mathematical simplificity and biological plausibility, and is popularly adopted as the baseline architecture [23] . In the LIF neuron, the weighted sum of input spikes is accumulated over time within the neuron as membrane potential, and the output spike is generated only when the membrane potential exceeds a present threshold value. This is represented as a differential function:EQUATIONwhere V (t) denotes the membrane potential value at time t, τ a time constant of membrane, andI(t)is the input from connected synapses at time t. To make this time-varying function computationally feasible, we discretize and rewrite it iteratively for sequential simulation as follows:EQUATIONEQUATIONEQUATIONwhere β is simplified decay rate constant, V reset is the reset value and V th the threshold for the membrane potential. Note that I(t) is simplified as weighted input WX(t) which can be obtained through any operations with learnable weights including convolutional operation, self-attention, or a simple MLP. We will denote this process of forwarding through LIF neuron as SN N (•) in this paper.Direct SNN Training. The initial adoption of SNNs was through ANN-SNN conversion, primarily due to their remarkable potential for reducing energy consumption. Various studies have aimed to address the accuracy degradation that occurs during the conversion from ANNS to SNNs [22, 41, 24, 42] .The spike generation by the step function in Equation ( 4) interfered with direct training without modifying the functions. To bypass the step function, which is non-differentiable and thus unsuitable for backpropagation, several approaches have been proposed [43, 5, 13, 14, 8, 51, 10] . Recent research has demonstrated that directly training SNNs can yield competitive results by addressing the challenges posed by non-differentiability. Our work focuses on directly training graph neural networks (GNNs) with SNNs and exploring a different domain, such as ANN-SNN conversion methods, which do not focus on using backpropagation concepts directly.Graph neural networks (GNNs) take graph-represented data as input, which consist of nodes and their connected edges G = (V, E), with node features X ∈ R |V |×F and optionally edge featuresE ∈ R |E|×D .The common GNN architectures follow a message passing paradigm [20] , which learns node or edge representations through aggregating information from its neighboring nodes and updating the node features iteratively. Thus a single forward of message passing layer consists of message passing, aggregation, and update: hEQUATIONj , e ij )), where l and i are indices for layer and node, respectively, and ψ(•) denote message passing function.After aggregation of neighboring features, ϕ(•) is used for feature update. For graph convolutional network [27] , the overall process can be simplified as:EQUATIONwhere the feature matrix is a concatenation of node featuresEQUATION(|V |-1) ] T which is updated through iterations of aggregation (AX) and combination (XW ). After iterative updates of X through the layers, the learned node or edge embeddings are passed through additional classification layer for node-level or edge-level predictions.In this paper we put emphasis on graph-level classification tasks where each graph is considered an individual input. Graph classification follows the same node-wise message passing framework to obtain node embeddings, but appends a readout layer to turn them into a single graph embedding:EQUATIONwhere R denotes readout function. Readout function reduces the node dimension to a single channel regardless of the input size. This is due to the inductive nature of graph classification task where the number of nodes is not known in advance. While all the other GNN layers focus on aggregating only the local features, the readout layer considers the entire graph to generate global features, and is unique to the graph classification tasks. The obtained graph embedding is passed through a classification layer for graph predictions. Graph classification tasks usually hold more difficulty than node-level classification due to its inductive nature, where inference is done on unseen graphs and thus cannot utilize any graph-specific statistics from the train set.In this paper, we adopt conventional SNN designs where LIF neurons are connected through learnabled weights, and apply is to GNN framework [64] . As mentioned in Section 2.2, each GNN layer outputs updated feature matrix X (l+1) ∈ R |V |×F . This is converted to spike representation through SNN layer:EQUATIONAfter passing the GNN layer, all of the updated h (l)i directly pass through the SNN layer, consist the feature matrix X (l) always contains spike information consistently. (b) Spike frequency visualization using each layer output. X-axis denotes feature dimension, while y-axis denotes nodes grouped and sorted by degree in descending order, top to bottom. Brighter spots denote higher frequency.Figure 1 : Analysis on spike frequency variation of GCN using IMDB-BINARY [54] dataset.The spike frequencies are associated with the corresponding nodes.Question: Does the paper discuss the limitations of the work performed by the authors?Answer: [Yes] Justification: We discuss the limitation on the Appendix. 
analysis on spike frequency variation of gnns: 
 To analyze the cause of the accuracy drop, we plot the behavior of the neurons during inference in Figure 1a , on a IMDB-BINARY dataset over five timesteps (T = 5). We create a histogram of spike counts created from each node, which is associated with 128 neurons. As depicted in the plot, it is clear that most of the neurons are under starvation. This is caused by the inputs of those neurons being insufficient to reach the threshold, and this leads to severe information loss between the layers.While unveiling the exact dynamics would require more research, we hypothesize that this is caused by the topology of the real-world graphs.To validate the hypothesis and further investigate the phenomena, we display the spike frequency heatmap of the neurons sorted by the degree of the nodes in Figure 1b . From the heatmap, we make three observations:1 (Brighter on the top and darker at the bottom) High-degree nodes tend to exhibit higher spike frequencies.The feature neurons within a node behave differently according to their positions.We believe such patterns come from the connectivity of the nodes, and the distinct role of the neurons assigned to each node. The connectivity will affect the number of receiving spikes of neurons associated with each node. It is known that most of the real-world graphs exhibit an extremely skewed distribution of degrees (i.e., power-law distribution [30] ). Due to such a characteristic, there are a few nodes with very high degrees, while a majority of nodes have low degrees. Because a GNN layer communicates signals between the neighbors, a high-degree node will likely receive a lot of spikes, while a low-degree node will receive only a few. In addition, the neurons assigned to each node are known to have different semantic functionality according to their positions, analogous to channels in convolutional neural networks or heads in large language models. For example, the input first layer of a molecular graph will have information such as its energy, x/y/z location, and atom numbers. In the intermediate layers, they represent a specific pattern sensed by the network (such as high energy + hydrogen atom), even though the exact behaviors are yet to be human-interpretable. In such a manner, the neurons in the same position are expected to behave similarly, even though they correspond to different nodes.These three observations shed light on how to close the performance gap between spiking GNNs are ANN-based GNNs. In the next section, we describe how the observations are used to build better spiking GNNs for graph classification.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?Answer: [NA] Justification: Our work does not include theoretical results.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.• All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.• Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.• Theorems and Lemmas that the proof relies upon should be properly referenced. 
overall graph classification architecture: 
 Many recent studies have tried to adapt SNN architectures into GNN tasks, however, they simply try to contact with only node classification tasks. In this work, we propose a spiking neural network specifically designed for graph classification tasks and show that it can be trained using spikes. We demonstrate the overall architecture of our graph classification model TAS-GNN in Figure 2 . For each timestep, the input graphs are first translated into spike representations through the poisson encoder, then the message passing is done in spike format. After the combination phase in the GNN layer, the node features are once again binarized into spike format through passing the SNN layer. In the last layer, we perform an extra operation of aggregation and combination on the spike features before passing the readout layer. The readout layer is essential to graph classification and is responsible for aggregating all the node embeddings in the graph into a single graph representation. A batch of graph embeddings is passed through a classification head that outputs logits for that timestep. To make the final prediction, we simply take the sum of logits from all timesteps and use softmax to obtain the class probabilities.As discussed in Section 3, GNNs suffer from a huge gap in spike frequencies between neurons. As observed, there exist some patterns (Figure 5 ) that we can utilize to address the issue. One naive way of addressing the issue is to use learnable [49] , or adaptive [4] threshold for each neuron. By adjusting the threshold, one can expect the neurons to naturally change, such that neurons under starvation will have lower thresholds to fire more often, and a few neurons with high firing rates will have higher thresholds to shift toward an ideal distribution.Unfortunately, such an idea cannot be directly applied unless all the testset nodes are available at training time (i.e., transductive task). However, such a setting would be considered a data leak for graph classification, and would also lose the advantage SNNs have on lightweight inference.Moreover, the number of nodes in a real-world dataset often ranges from at least thousands to several billions. Considering that GNNs often involve only a sub-million number of learnable parameters, storing such a large number of thresholds is considered too much overhead.To address the aforementioned issues, we propose topology-aware group adaptive neurons (TAG), which partitions the neurons by their degrees. Note that V g denotes the node group to which the node is mapped, considering degree information. S gi (t) and V gi (t) represent the output spike and membrane potential of the i-th node in group g at time t, respectively, as reformulated by Equation ( 4).We use g to represent the unique degree distribution of the training sets. When an unseen node is encountered, we apply the initial threshold, as it has not been trained at all.EQUATIONEQUATIONEQUATIONThe major advantage of this scheme is that it is straightforward to put an unseen node or an unseen graph into a group at inference. To further consider intra-node deviation, we split the group into F (number of features) neurons, which is a fixed parameter determined by the model architecture.For any unseen node, finding out its degree is trivial because visiting its neighbors is one of the fundamental requirements of graph data structures [26, 50, 36, 28] . Based on the observation 1 from Section 3 that the neuron behavior is related to the degree, this will let neurons in the group collaboratively find an adequate threshold. The proposed Group-adaptive threshold scheme effectively reduces the spike frequency variation issue. However, we find that the adaptive neurons in the proposed TAG are sensitive to their initial thresholds. As depicted in Figure 3 , the performance of the adaptive neurons can severely drop when the initial threshold value is not carefully tuned, which aligns with the findings from [4] . Moreover, manually tuning the initial thresholds individually is difficult because there are thousands of neuron groups.To address the problem, we choose to learn the two parameters: the initial threshold per group (V g th (0)) and the decay rate (β). During training, we adopted the backpropagation algorithm [51, 10, 8] to update the value of V g th (0)with the gradients at time step t=1. This is done because V g th (t) keeps updating with TAG Section 4.2 as time passes. During training, we also learn the decay rate (β) [16] , which prevents the membrane voltage of neurons in low-degree nodes from leaking faster than it accumulates. For evaluation, we use the V g th (0) values obtained during the training phase, adjusted for each group. The overall training procedure is in the Appendix.Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: We provided our codes that able to reproduce our model's result.Guidelines:• The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.• If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.• Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.• While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results. Justification: We provide our codes that are able to reproduce our full experiments.Guidelines:• The answer NA means that paper does not include experiments requiring code.• Please see the NeurIPS code and data submission guidelines (https://nips.cc/ public/guides/CodeSubmissionPolicy) for more details.• While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).• The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https://nips.cc/public/guides/CodeSubmissionPolicy) for more details.• The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.• The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.• At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).• Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
experiment settings: 
 We use a total of 5 graph datasets commonly used for benchmarking GNNs: MUTAG [9] , PRO-TEINS [6] , ENZYMES [6] , NCI1 [47] , and IMDB-Binary [54] . For the GNN layer in our architecture, we use 3 different designs, including GCN [27] , GAT [45] , and GIN [52] . The baselines include 3 works from SNN that are applicable to graph datasets: SpikingGNN [64] , SpikeNet [32] , and PGNN [16] . Since this is the first SNN design to target graph classification, we apply minor modifications to each architecture, such as appending a readout layer. Note that SpikingGNN [64] was originally proposed for GCN, but we extend it to both GAT and GIN. More details on the experiment setting are included in the Appendix.We compare TAS-GNN against prior works that adopt a spiking neural network to graph the dataset, shown in Table 1 . We also report the performance of conventional ANN for comparison. In all but 2 cases, TAS-GNN outperforms the baselines by a noticeable margin. In the cases where TAS-GNN underperforms, the gaps are less than 1.1%p, smaller than the error bounds. In the opposite cases, the improvement is up to 27.90%p, showing a great amount of improvement.An intriguing result is that TAS-GNN performs better than ANN-based GNNs in several cases.Improvements beyond the error bounds are found in MUTAG (GCN and GAT), NCI1 (GAT), and IMDB-BINARY (GCN and GAT). Note that the model architecture and the number of learnable parameters are the same in all methods. We believe this could come from the spiking neurons efficiently capturing the irregular connections over several timesteps, thereby showing an advantage over ANNs.In this section, we break down individual components of TAS-GNN and perform an ablation study, which is reported in Table 2 . Starting from baseline implementation, which does not differentiate neurons used by each node, we apply TAG to show the effect of topology-aware group-adaptive neurons. Then, we add our learnable initial threshold scheme to complete TAS-GNN. The resultsshow that TAG alone can improve the performance across all datasets and models. This means that uneven spike distribution caused by indegree variance is a general problem shared across different graph datasets, and simply grouping the nodes with similar indegree to share the same threshold helps alleviate this problem. Lastly, adding a learnable initial threshold scheme further boosts the accuracy in almost all cases, demonstrating its efficacy and stability. Figure 4 : Sensitivity study of neurons to its initial threshold. To validate our method's efficacy in alleviating the sensitivity of the initial threshold value, we perform a sensitivity study varying the values from 0.0 to 10.0. We compare our scheme against the TAG method, which also adaptively modulates the threshold during inference but does not learn it from training. Our method consistently performs indifferently to the initial threshold value, which means arduous search or tuning is unnecessary to achieve stable accuracy.On the other hand, TAG is highly sensitive to the initial threshold and shows a performance gap up to 19.68%p except for GIN architecture, which is capturing structure well.Since our scheme uses a learnable initial threshold, we also study its sensitivity for the learning rate, shown in Table 3 . TAS-GNN performs best around η = [0.005, 0.1], and starts to degrade for further increment or decrement. As denoted in the experimental setting, we use η = 0.01 as the default.In this section, we give additional analysis on TAS-GNN by studying its spike frequency distribution.In Figure 5 , we provide the same spike frequency visualization as done in Section 3, but using TAS-GNN. Unlike Figure 1 , which showed severe starvation with most nodes not generating spikes, Figure 5a reveals that most nodes fire spikes, significantly alleviating the starvation problem. This is further illustrated Figure 5b , where most neurons have non-zero spike values and, what's more, meaningfully reflect the topology of the graph. For nodes with higher degrees, the spikes are more frequent (close to 5) due to having more incoming spikes from their neighbors. For GNNs, such information is essential to capture the global topology of the graph. This shows that our design of TAS-GNN faithfully reflects such information and can successfully propagate such information using spikes. 
related works: 
 Graph Classification Graph classification requires identifying the global characteristics of each graph and is commonly applied to domains such as bioinformatics [6] , chemoinformatics [63] , or social network analysis [21, 37] . Popular examples include the molecular classification of chemical compounds, proteins, or RNAs, where identifying the graph structural information is crucial. Due to the success of GNNs, [27, 45, 52, 57] Most GNNs use a message passing paradigm [20] that only aggregates local features. Thus, to obtain global features representing the entire graph, graph pooling [56] is often used. Global pooling summarizes the entire graph into a fixed-size graph embedding, which can be done by simply averaging or taking minimum or maximum values of the node-wise embeddings. Other variations replace such simple operations with neural networks [46, 33] or integrate sorting to selectively choose which node embeddings to include [60] . More advanced techniques such as hiearchical pooling utilze hiearchical information of graphs [40, 29, 18, 11] and usually show better representation learning. [60] (a) Histogram plotting the distribution of total spikes counted over time for each node. X-axis denotes spike counts from each node, while y-axis denotes density of each bin. Spiking Neural Networks SNNs are a type of neural network where information is transmitted using spikes, similar to how biological neurons work. They use different neuron models for capturing spike signals effectively [23, 24] or adjusting parameters dynamically to compromise the accuracy [16, 49, 4, 34] . One major area of SNN research is converting traditional ANNs into SNNs by mapping ANN activation functions into spike signals [22, 41, 24, 42, 17] . Another focus is training SNNs directly using backpropagation, similar to ANNs, which involves using various techniques such as surrogate functions for backpropagation [43, 8] and adapting normalization techniques to SNNs [42, 12, 25, 62] .SNN for Graphs Previous attempts to apply SNNs to graph datasets have primarily focused on node-level classification tasks [59, 44, 64] and have not yet been extended to graph-level tasks. While [48] explored the application of spike training to Graph Attention Networks (GAT), it implemented the message passing phase after the spiking phase, which deviates from previous structures. Additionally, recent efforts have begun to integrate SNNs with other techniques for contrastive learning [31] ,particularly in dynamic graphs [55] , to adopt collaboration between GNNs and SNNs.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: We wrote experiment setting in the experiment settings including GNN layers, hyperparameter for the hidden dimension, and learning rate of the whole dataset. Also, we wrote epochs and dataset we split was used by 10 fold CV for our evaluations.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.• The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 In this paper, we explore the application of SNNs to graph neural networks for graph classification for the first time. After thoroughly analyzing the graph's uneven spike distribution, we identify that the degree of each node correlates to this phenomenon. To better accommodate such characteristics of graphs, we propose topology-aware group-adaptive neurons, which uses separate neurons for each degree group in the graph. In addition, we propose to learn the initial threshold and adaptively adjust the threshold simultaneously to reduce its sensitivity and facilitate training using spikes. Combined with the modified architecture for graph classification, we name our method TAS-GNN, and show that it outperforms existing works by a noticeable margin.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes] Justification: We reported error of confidence level in the main table.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).• The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)• The assumptions made should be given (e.g., Normally distributed errors).• It should be clear whether the error bar is the standard deviation or the standard error of the mean.• It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.• For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).• If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes] Justification: It refers to the appendix for experimental settings.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.• The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.• The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction). 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: SNN would be one of the breakthrough idea in respect of energy consumption.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.• Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.• The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.Justification: Considering our implemtation code is our asset, our work provides necessary license and documents for further usage.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.• The paper should discuss whether and how consent was obtained from people whose asset is used.• At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. Justification: Our work does not require IRB approvals and does not involve human subjects.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.• Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.• For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review. 
 
 Appendix: A.1 Limitation Currently, our work is experimenting with the small-scale dataset for the graph classification that is generally used. However, we will extend our work into the large-scale dataset that could apply to the real. In addition, we will continue our future work for theoretical proof for updating the initial threshold that is fused with adaptative changes in the timestep.The code which includes our implementation of this work is included in a zip archive of the supplementary material. The code is under Nvidia Source Code License-NC and GNU General Public License v3.0.Dataset Details Given the diverse properties of graph datasets, we selected five datasets from the well-known TUDatasets, commonly used for graph classification. We compiled statistics for these datasets to briefly represent their key properties. Network Architecture In this work, we consider the following three GNN architectures where the distinctions lie in their update rules:• Graph Convolution Network [27] (GCN): h(l+1) i = σ( j∈N (i) {i} W h (l) j √ |N (i)||N (j)|), where ϕ(•) is replaced by affine transformation W followed by nonlinearity σ.• Graph Attention Network [45] (GAT): h(l+1) i = α i,i W h (l) i + j∈N (i) α ij W h (l)j , where α ij is the normalized attention score between node i and j.• Graph Isomorphism Network [52] (GIN): h(l+1) i = M LP ((1 + ϵ)h (l) i + j∈N (i) h (l) j ),where ϵ is a learnable constant.For the GCN layers, 128 dimensions were used for hidden dimensions, and GAT layers were used for 4 multi-head attentions. GIN was used for 2-MLP layers for the above equation.We trained and evaluated our models using 10-fold cross-validation for all datasets. Note that the IMDB-BINARY dataset lacks inherent features, so we constructed features using the node degrees for the GNN layer. Additionally, we did not apply any multiplier to adjust the width of the sigmoid function. The details of our evaluation procedure are outlined below. Our experiment was evaluated on a single RTX-4090 GPU for the full batch GNN training.• Epochs: 1000• Surrogate function: σ(x) = 1 1+e -x• Learning rate(η): 0.01 (for main table)• Optimizer: Adamw for l = 1 to L do 7:for g in group G do 8:X g,(l) = GN N (l) (X g,(l) ) ▷ Operate by GCN, GAT, GIN architectures 9:for i = 1 to |Vg| do 10:X g i ,(l) = S g i ,(l) (t) = SN N (l) (X g i ,(l) ) ▷ X g i ,(l) represents i-th row of X g,(l) 11: S g,(l) (t) = 1 |Vg | i∈Vg S g i (t) 12:end for 13: V g,(l) th(t) = γV g,(l) th (t -1) + (1 -γ)S g,(l) (t) ▷ Update threshold through TAG Equation (10) 14:end for 15:end for 16:O t ← F C(P OOL(GN N (X (L) ))) + O t-1 17:end for 18:V th (0) = V th (0) -η∇ V th (0) L(O t=1 , Y ) 19: end forOur experiments were conducted on a number of degree groups. Please refer Table 5 for the sensitivity depending on the number of degree groups. Please note that the optimal values of the degree groups are different depending on the graph datasets. We reported to the max degree group setting that unseen nodes will use the initial values V init that represents the V th (0) that does not trained at all.Our experiments were conducted under various learning rate conditions η ∈ [0.001, 0.5] to assess their impact. As reported in Table 3 for the MUTAG dataset, we also present results for the PROTEINS, ENZYMES, NCI1, and IMDB-BINARY datasets across GCN, GAT, and GIN architectures. Our model's ability to learn V init demonstrates a sensitivity to learning rate similar to other ANN models.We found that the optimal performance was achieved at a learning rate of η = 0.01. The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and precede the (optional) supplemental material. The checklist does NOT count towards the page limit.Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:• You should answer [Yes] , [No] , or [NA] .• [NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.• Please provide a short (1-2 sentence) justification right after your answer (even for NA).The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include it (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.While "[Yes] " is generally preferable to "[No] ", it is perfectly acceptable to answer "[No] " provided a proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] " or "[NA] " is not grounds for rejection. While the questions are phrased in a binary way, we acknowledge that the true answer is often more nuanced, so please just use your best judgment and write a justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.• Delete this instruction block, but keep the section heading "NeurIPS paper checklist",• Keep the checklist subsection headings, questions/answers and guidelines below.• Do not modify the questions and only use the provided macros for your answers. • The answer NA means that the abstract and introduction do not include the claims made in the paper.• The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.• The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.• It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.• The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.• The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.• The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.• If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.• While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 


Decision: The paper proposes Topology-Aware Spiking Graph Neural Networks(TAS-GNN) to alleviate the neurons under starvation problem. The problem is very severe in the graph classification tasks. TAS-GNN leverages the topology of the graph to improve the performance of spiking neural networks in graph classification tasks. Experiments on diverse datasets show the empirical performance of the TAS-GNN.

During the review stage, several concerns are proposed by reviewers which are partially addressed by the authors with a lot of necessary changes for the current submission. After carefully considering all evaluations of other paper in the same batch, I recommend rejection for this submission. I hope the authors can improve the paper and submit it to a future venue.

Reviews: review: paper_summary: paper_summary: This paper proposes topology-aware spiking graph neural networks with adaptive thresholds based on a group of neurons for graph classification. The paper first diagnoses the poor performance as the existence of neurons under starvation caused by the graph structure. Then the paper proposes the adaptive threshold among neurons partitioned by degrees, as well as the learnable initial threshold and decay rate to reduce the sensitivity. Experiments on several datasets show superior performance of the proposed method.
strength_weakness: strength_weakness: 1. The proposed method seems to be a hybrid ANN-SNN model rather than a pure SNN design. The paper did not discuss how this will affect the deployment of the model on potential neuromorphic hardware, since SNNs mainly target those hardware to obtain energy efficiency.

2. The paper did not discuss the (theoretical) energy efficiency estimation, which is a major motivation for considering SNNs as stated in Introduction.

3. Or if the motivation is to get models with better performance than ANN, then Table 1 does not include state-of-the-art ANN results for comparisons.
questions: questions: Some recent works also study SNN for link prediction tasks in graphs [1] besides node-level classification, which may be discussed.

[1] Temporal Spiking Neural Networks with Synaptic Delay for Graph Reasoning. ICML 2024.
limitations: limitations: The authors discussed limitations in Appendix A.1.
paper_summary: paper_summary: The paper presents a novel approach called TAS-GNN (Topology-Aware Spiking Graph Neural Networks) to address the performance gap between spiking neural networks (SNNs) and artificial neural networks (ANNs) in graph classification tasks. The authors identify a "starvation" problem in spiking neurons within GNNs, where many neurons do not emit any spikes during inference, leading to severe information loss. This problem is more pronounced in graph classification tasks, where the test set graphs are independent from the training set, unlike in transductive or inductive learning settings.
strength_weakness: strength_weakness: 1.	The authors use the node degree instead of the concept of topology, there’s a large gap between the graph topology and node degree.
2.	The authors solve the graph classification task as a contribution, which is not a significant challenge for spiking graph neural networks.
3.	The advantage of Spiking Neural Networks (SNN) is their low energy consumption. However, the paper does not mention the feature, so it is unclear why graph neural networks should be combined with SNN. The motivation behind TAS-GNN is not clear.
questions: questions: The important points listed in weakness 1-3.
limitations: limitations: The authors adequately addressed the limitations.  The authors should discuss more details of the potential negative societal impact of the work.
paper_summary: paper_summary: This paper primarily discusses integrating Spiking Neural Networks (SNNs) into Graph Neural Networks (GNNs) to address several key challenges in graph classification tasks. Specifically, the paper proposes a new method called TAS-GNN (Topology-Aware Spiking Graph Neural Networks) which leverages the topology of graphs to improve the performance of spiking neural networks in graph classification tasks.
strength_weakness: strength_weakness: （1）The authors mention several application areas and challenges, but the references and comparisons to existing literature are not sufficiently comprehensive.
（2）Although the methodology section describes the main steps, it lacks detailed descriptions of some key aspects such as threshold initialization and the specific training process.
（3）Although there are some ablation studies, the analysis of the individual contributions of each component is insufficient, making it difficult to determine the specific impact of each component on the overall performance improvement.
questions: questions: （1）Could you provide more details on how the neuron starvation problem was diagnosed? Specifically, what metrics or observations were used to identify this issue in SNNs for graph classification?
（2）The paper mentions the use of learnable initial thresholds for neurons. Could you elaborate on how these initial values are set and what specific strategies or heuristics were used to determine them?
（3）Conduct a more thorough ablation study to analyze the independent contributions of each component (e.g., TAG, learnable initial thresholds) to the overall performance. This will help readers understand the significance of each part of the proposed method.
（4）The sensitivity analysis shows variations in performance with different initial thresholds and learning rates. Could you explain why certain thresholds or learning rates were more effective and how they were chosen?
（5）How does TAS-GNN scale with very large graphs in terms of computational efficiency and memory usage? Are there any specific optimizations or techniques used to handle large-scale datasets?
（6）While the paper compares TAS-GNN with several baseline methods, could you consider including comparisons with more recent or advanced GNN models that have shown strong performance in graph classification tasks?
（7）Have you tested TAS-GNN on any real-world applications or datasets beyond the ones mentioned? If so, could you share the results and insights gained from these experiments?
limitations: limitations: (1) While the paper discusses the neuron starvation problem and the sensitivity of initial thresholds, it does not explicitly outline the broader limitations of the proposed TAS-GNN method. It would be beneficial to include a dedicated section that explicitly lists and discusses the limitations of the current work.
(2) The paper does not thoroughly address how TAS-GNN scales with extremely large datasets or very high-dimensional graphs. Including an analysis of computational complexity and memory usage for larger graphs would provide a clearer understanding of the scalability limitations.
(3) While multiple datasets are used, the paper could further discuss the generalizability of TAS-GNN to other types of graph-based tasks beyond classification, such as regression, clustering, or even dynamic graphs.
paper_summary: paper_summary: There's a large performance gap for graph tasks, especially graph classification tasks, between the spiking neural networks and artificial neural networks. The authors proposes the problems as the neuron's under starvation and illustrated the reason of the problem. To solve the problem, TAS-GNN was proposed.

The main contributions of the paper are as follows:
1: Starvation problem of spiking neurone in GNNs in graph classification tasks are identified.

2: A strategy was proposed to address the spike frequency deviations on the basis of the correlation between graph topology and spike frequency patterns.

The authors conduct experiments on 5 popular datasets and use several different designs of GNN layer. The results show competitive potential of the TAS-GNN.
strength_weakness: strength_weakness: 1: The name of the paper is "Topology-Aware Spiking Graph Neural Networks". However, as I can tell the only graph topology used in the method is nodes degree, which is used to group the neurons. I wonder if it is appropriate to name it as "topology aware", or the author can explain it more.

2: The analysis regarding the performance of the method is lack of discussion. For instance, in some datasets, such as MUTAG and IMDB-Binary, the proposed method achieve quite competitive results while in PROTEINS it doesn't. It's betted to explain what cause the phenomenon, like the characteristics of the datasets? Also, in table 2, the results of GAT and GAT+TAG in IMDB-Binary are the same. It's better to make an explanation about them.

3: There're several typos and basic grammar mistakes in the paper that will affect the presentation of the paper. In line 120 " and apply is to"; The sentence in line 123 is hard to understand
questions: questions: 1: In section 3 the authors mentioned the hypothesis that the phenomenon mentioned above is caused by the topology of the real-world graphs. What motivates you to have the hypothesis?
limitations: limitations: na

Comments: title: TAS-GNN: Topology-Aware Spiking Graph Neural Networks for Graph Classification
Response to Reviewer 4pSD
Response to Author Rebuttal
Response to Author Rebuttal
comment: Thank you! We greatly appreciate the reviewer's thorough consideration of our response and the valuable insights shared with us.
I would like to thank the authors for their detailed responses, clarification, and additional results. Most of my questions are solved and I raise my score.
Thank you for taking the time to share your thoughts. We genuinely appreciate your comments and feedback, and we're confident your insights will help us further enhance our work.
Thank you for the response. I have read the response as well as the reviews and rebuttals of other reviewers. I will stand by my original recommendation.
Thank you! We sincerely appreciate the reviewer for the feedback. We will make sure to address the issue in the paper. If there are any additional points the reviewer would like to clarify, please let us know. We will be more than happy to address them.
Thank you for reading our rebuttal and sharing your thoughts. However, could we ask for a reconsideration? We believe the amount of revisions necessary from the current version would not be very extensive. 

Firstly, regarding W1, it would be sufficient to replace our term topology with a degree, as we used topology to refer to degree information in our explanation within the paper.

Secondly, regarding W2, we believe the significance of the graph classification task is already clearly demonstrated by the accuracy degradation of other baselines. Therefore, we believe there is less need for a major revision.

Finally, regarding W3, we think this concern can be addressed by simply adding this experiment table to the main manuscript. Adding these experiments to the paper will certainly be valuable. However, they would not significantly affect the overall flow of our paper, because reducing energy consumption is already a fundamental advantage of SNNs, and we focus on improving their accuracy for practicality.
The authors give a detailed reply to weaknesses 1-3. Since these three issues are crucial to the motivation and novelty of the paper, judging from the rebuttal statement, the author needs to make a large amount of modifications to address these issues. The current version is not suitable for public publication.
I would like to thank the authors for their detailed discussion and for addressing my concerns. As mentioned by one of the authors, the term 'graph topology' instead of 'degree' can be misleading. I hope the authors will further consider refining this expression. I've increased my score. Good luck!