NeuroGraph: Benchmarks for Graph Machine Learning in Brain Connectomics
Authors: Anwar Said
Roza G Bayrak
Tyler Derr
Mudassir Shabbir
Daniel Moyer
Catie Chang
Xenofon D. Koutsoukos
Venue: NeurIPS 2023 Datasets and Benchmarks Poster (2023)
Keywords: Graph Neural Networks
Benchmarking
Datasets
Neuroimaging
fMRI
TL;DR: This work presents a collection of graph-based Neuroimaging benchmark datasets along with a tool named NeuroGraph, for fMRI preprocessing

Abstract: Machine learning provides a valuable tool for analyzing high-dimensional functional neuroimaging data, and is proving effective in predicting various neurological conditions, psychiatric disorders, and cognitive patterns. In functional magnetic resonance imaging (MRI) research, interactions between brain regions are commonly modeled using graph-based representations. The potency of graph machine learning methods has been established across myriad domains, marking a transformative step in data interpretation and predictive modeling. Yet, despite their promise, the transposition of these techniques to the neuroimaging domain has been  challenging due to the expansive number of potential preprocessing pipelines and the large parameter search space for graph-based dataset construction. In this paper, we introduce NeuroGraph, a collection of graph-based neuroimaging datasets, and demonstrated its utility for predicting multiple categories of behavioral and cognitive traits. We delve deeply into the dataset generation search space by crafting 35 datasets that encompass static and dynamic brain connectivity, running in excess of 15 baseline methods for benchmarking. Additionally, we provide generic frameworks for learning on both static and dynamic graphs. Our extensive experiments lead to several key observations. Notably, using correlation vectors as node features, incorporating larger number of regions of interest, and employing sparser graphs lead to improved performance. To foster further advancements in graph-based data driven neuroimaging analysis, we offer a comprehensive open-source Python package that includes the benchmark datasets, baseline implementations, model training, and standard evaluation.

NeuroGraph: Benchmarks for Graph Machine Learning in Brain Connectomics
Anwar Said Vanderbilt University anwar wawar.siddeel www.daaddedadddbb......m........ edu
Roza G. Bayrak Vanderbilt University
roza. g. bayrak@vanderbiilt.ee edu
Tyler Derr Vanderbilt University LyLer.derr@vaadeebbbt..c.
Mudassir Shabbir
Vanderbilt University
Information Technology University mudassir.. shabbir@itu, edu, pkk
Daniel Moyer
Catie Chang
Vanderbilt University
Vanderbilt University
daniel m htw.ill.mooorrrraaarbbl.....
catie. changdvanddrrilt.edd
Xenofon Koutsoukos
Vanderbilt University
xenof on hot..oooooo.ioloeeeeeeb.........
Abstract
Machine learning provides a valuable tool for analyzing high-dimensional fune- tional neuroimaging data, and is proving effective in predicting various neurological conditions, psychiatric disorders, and cognitive patterns. functional magnetic resonance imaging (MRI) research, interactions between brain regions com- monly modeled using graph-based representations. The potency of graph machine learning methods has been established across myriad domains, marking trans- formative step in data interpretation and predictive modeling. Yet, despite their promise. the transposition of these techniques to the neuroimaging domain has been challenging due to the expansive number of potential preprocessing pipelines and the large parameter search space for graph-based dataset construction. In this paper, we introduce NeuroGraph! a collection of graph-based neuroimaging datasets, and demonstrated its utility for predicting multiple categories of behavioral and cogni- tive traits. We delve deeply into the dataset generation search space by crafting 35 datasets that encompass static and dynamic brain connectivity, running of 15 baseline methods for benchmarking. Additionally, we provide generic frame- works for learning on both static and dynamic graphs. Our extensive experiments lead to several key observations. Notably, using correlation vectors node features, incorporating larger number of regions of interest, and employing sparser graphs lead to improved performance. To foster further advancements in graph-based data driven neuroimaging analysis, we offer comprehensive open-source Python package that includes the benchmark datasets, baseline implementations, model training, and standard evaluation.
37th Conference on Neural Information Processing Systems (NeurIPS 2023) Track on Datasets and Benchmarks.

---

Table 1: Dataset statistics. |G| denotes the number of graphs, IN avg and Elagg denote the average number of nodes and edges, d indicates the degree, and K signifies the global clustering coefficient.
Dataset
[C
I avg
Staiistis
doaz
dawa
Kavg
Node Feut.
(dim)
#Chasses
Prodiction Task
HCP-Tisk
7443
400
7029.18
153
17.572
0.410
400
Graph Classification
Stafic
HCP H.Gendrrrr
1078
1000
45578.61
413
45.579
0,460
1000
Graph C Clsssiicaiion
45388.
45.585
0.406
Graph
1071
1000
45573.67
413
45.574
0.406
1000
Craph Kegression
H. POWM
1078
1000
40578.61
41.
45.579
0,400
1000
Crapph Negression
DynRC R- Task
7443
100
843.04
8.430
0.427
100
Graph Classifiation
Dynamkc
DyaHC Geedde
1080
100
874.88
8.749
0.439
100
Graph Classriictiion
DysHCP-Ase A-A
1087
875.
8.754
0.439
100
Graph Classifiation
DyaRt FEPT
1073
100
874.82
03
8.748
0.435
100
Graph Kegression
DyaHCP-WM WM
1080
100
874.88
53
8.749
0,439
100
Graph Kegression
Introduction
Graph Neural Networks (GNNs) have demonstrated remarkable efficacy in a variety of domains including recommendations, forecasting, and biomedical data analysis [1, 2, 3]. In human neuroimag- ing research, GNNs have proven valuable in capturing the complex connectivity patterns within brain functional networks but also enhance the modeling and analysis with other relevant informative features [4, 3J. For instance, examining synchronized fluctuations of functional magnetic resonance imaging (IMRI) signals provides a useful means of measuring functional network connectivity [5].
Neuroimaging and Graph Machine Learning (GML) are two rapidly evolving fields with immense potential for mutual collaboration. However, a significant challenge lies in bridging the gap between these domains and enabling seamless integration of neuroimaging data into state-of-ther fh-th-rrtt GML approaches [6]. This gap is primarily attributed to the expansive number of potential fMRI data preprocessing workflows, the absence of an intuitive tool to generate fMRI graph representations for graph- based learning approaches, and the knowledge gap between the fields of neuroimaging and advanced graph machine learning [2]. To address these challenges, the main objectives of this study are, first, a careful exploration of graph-based dataset generation with the goal of formulating a roadmap for graph-based representations of fMRI data. Second, we conduct a rigorous evaluation of graph machine learning methodologies, with a special emphasis on GNNs, examining their efficacy when applied to diverse fMRI data configurations.
The human brain, a complex network of interconnected regions, can be represented as graph, wherein nodes correspond to contiguous segments known as Regions of Interest (ROIs), and edges represent their relationships [7, 8]. Features of the functional connectome, such as correlations between the BOLD (Blood Oxygen Level Dependent) signals between different brain regions, typically employed for downstream machine learning tasks [6. 9], can be re-envisioned as node features within attributed graph representations. These representations pave the way for a rich assortment of graph-based data representations, wherein GNNs are exceptionally well-suited [10]. Yet, the vast potential offered by the intersection of fMRI datasets and GNNs remains untapped, due primarily to the expansive search space for data generation and the multifaceted nature of hyperparameters. In this study, we pioneer a rigorous exploration and benchmarking for GNNs, with the following primary contributions:
We introduce NeuroGraph, a collection of static and dynamic brain connectome datasets tailored for benchmarking GNNs in classification and regression tasks including gender, age, task classification, and prediction of fluid intelligence and working memory scores. This enables an extensive exploration of brain connectivity and its associations with various cognitive, behavioral, and demographic variables. Details of the proposed datasets are provided in Table 1.
We perform extensive exploratory study in search of optimal graph-based based data represen- tations for neuroimaging data, implementing 15 baseline models on 35 different datasets. Additionally, provide detailed benchmarking for the datasets we propose.
By offering NeuroGraph, we create an essential bridge between the neuroimaging and graph machine learning communities. Researchers in the neuroimaging field can more readily tap into the power of cutting-edge GNNs. Specifically, our datasets generation pipeline may guide researchers toward effectively transforming neuroimaging data into unified graph representation suitable for graph machine learning. This integration facilitates the adoption of state-of-the-are graph- techniques, unlocking new insights and accelerating discoveries in the neuroimaging field.

---

2
Related Work
While functional brain connectomes have long been recognized as a rich source of information in neuroscience and neuroinformatics [11, 12], their value has become increasingly evident in recent years [13]. Propelled by growth in data availability and methodological breakthroughs, ML has shown remarkable efficacy on tasks such decoding of cognitive processes [14, 15] and diagnosing mental health disorders [16, 17J. Simultaneously, there has been increased utilization of static and dynamic graph representation learning methods for brain analytics, which we briefly summarize in this section.
Static graph representations: GNNs have significantly evolved as a major field of exploration, offering an intuitive approach in learning from graph-structured data [18, 19. 20, 21, 22]. In a static setting, where individual data points are represented by single graphs, a variety of methods have been introduced [23, 21, 24, 22, 25, 26]. Recent studies have demonstrated the effectiveness of various approaches when applied to functional connectome data, which can be represented as different types of graphs, including weighted graphs [3, 27, 28], and attributed graphs [2, 29], among others. In benchmarking setup, BrainGB [27] stands out as a notable advancement, offering a unified framework for brain network analysis utilizing GNNs. Likewise, BrainGNN [3] introduces a specialized GNN architecture tailored for the discovery of neurological biomarkers from fMRI data. By leveraging the structured nature of the data and incorporating local information, GNNs not only enable learning from the functional connectivity patterns but also enhance modeling and analysis with other relevant informative features [9, 30, 22, 1, 31].
The field of learning dynamic graph representations in graph machine learning setting remains relatively unexplored, especially in the realm of brain imaging [32]. In neuroimaging, dynamic graphs are constructed to capture the time-varying interactions and connectivity patterns the brain [2, 33, 34]. While GMLL methods have not been commonly employed in this domain, recent years have seen the introduction of methods that yield impressive results on dynamic brain graphs[2. 35, 36, 37, 38). Specifically, Kim et al. [2] have made significant strides by introducing dynamic GNNs tailored specifically for brain graphs. These methods have showcased the potential of effectively capturing and analyzing the dynamic nature of brain connectivity, opening up new avenues for advancements in our understanding of brain function and neurological processes.
Dynamic graph representations:
3 NeuroGraph
The design space for graph construction from functional connectome is vast, since a variety of methods can be employed to generate various forms of graphs, such as simple undirected graphs [2], weighted graphs[3], attributed graphs [27], and minimum spanning trees [39, 40], among others. Thoroughly navigating this extensive design space and evaluating all potential parameter combinations challenging task. While recent efforts have been undertaken to leverage GNNs for predictive modeling on neuroimaging data, a consensus has yet to be reached regarding the preprocessing pipeline and hyper-parameter configurations best suited for generating expressive graph-based neuroimaging datasets [2, 3, 41, 28, 42). In addition, although there are multitude of GNNs models, no benchmark datasets have been created to evaluate GML approaches on brain connectome data. To fill this gap and provide a common ground between neuroimaging and GML communities, we use publicly available datasets and only minimally preprocess the data using standard IMRI preprocessing steps. We provide an illustration of the overall NeuroGraph preprocessing pipeline in Figure 1.
3.1 From IMRI to Graph Representations
fMRI data is typically represented in four dimensions, where the blood-oxygen level-dependent (BOLD) signal is captured over time in a series of 3-dimensional volumes. These volumes display the intensity of the BOLD signal for different spatial locations in the brain. However, since brain activity tends to exhibit strong spatial correlations, the BOLD signal is often summarized into a collection of special functional units, brain parcels. These units represent regions of interest (ROIs) whose constituent "voxels" (a smallest three dimensional resolution) exhibit temporally correlated activity.
The Human Connectome Project (HCP) [43] is publicly available rich neuroimaging dataset containing not only imaging data but also a battery behavioral and cognitive data. We select this dataset for benchmarking and utilize the established group level Schaefer [44] atlases to represent the
3

---

Static Graph
4D functional MRI data
Static Connectivity
Correlation Matrix
Rapr reprasentations
Time
Derse
Sparse
Temponal tercolution
Time
Woreeoo MAMMA WAYH
Window
Longth
Connectiv by
Strenggth
Dynamic Connectivnty
Sparial Resolution
R01 0007205
Window tength
Sliding
Dynamic Gragin Reprassntations
Group-Level iillss Summantzing Regions of interest (ROI)
Standard Minimal
LaNapo
Timm
Prep rocssiing
Conrect for: acarine dhifts
head mosion
Figure 1: An illustration of the preprocessing pipeline, demonstrating the transition from fMRI data to the construction of both static and dynamic graphs.
measured BOLD signal. These atlases provide parcellation of the cerebral cortex into hierarchically organized regions at multiple granularities (resolutions).
We use resting-state and seven task fMRI paradigms from the HCP 1200 dataset. All fMRI scans underwent the HCP minimal preprocessing pipeline [45]. We further regressed out six rigid-body head motion parameters and their derivatives, as well the low-order trends, from the minimally preprocessed data. The mean fMRI time series was extracted from all voxels within each ROI for different parcellation schemes. Individual (subject-wise) ROI time-series signals were temporally normalized to zero mean and unit variance. In summary, our proposed IMRI preprocessing pipeline encompasses five steps: a) brain parcellation, removal of b) scanner drifts and motion artifacts, c) subject-level signal normalization, d) calculation of correlation matrices and finally, e) construction of static and dynamic brain graphs. For further in-depth details, we refer the reader to Appendix B.3.
Our study of these datasets encompasses two distinct modes of analysis: static and dynamic graph construction. We apply different GNNs both types and perform benchmarking in five unique tasks. In the static graph construction, we investigate multiple parameters to build the graphs from the raw data, taking into consideration variations in node features, the number of nodes or regions of interest (ROIs), and the density of the graph. For node features, we take into account correlations. time-series signals, or a blend of both. For the number of nodes provided by [44] (i.c., ROIs), we examine three different resolutions: 100, 400, and 1000 nodes. As for graph density, we consider sparse, medium, and dense configurations. For the sparse setup, we choose the top 5% of values from the correlation matrix for edge selection, whereas for the medium and dense setups, we select the top 10% and 20% of values, respectively. We note that constructing brain graphs involves a number of parameters including the choice of parcellation methods, where various brain atlases can be employed to segment the brain into regions of interest. Equally crucial is defining the number exploring brain functions. The selection of ROIs
of ROIs, as it wields significant influence count permits the creation of brain graphs of varying sizes, allowing the opportunity to focus on both the global and granular levels of the brain. We have opted for those more likely to yield superior performance [3, 2]. Additional details about the complexity of the search space in benchmark dataset construction and the rationale behind these parameters are presented in Appendix B.4. We performed extensive experiments in the exploratory analysis to find the suitable combination of parameters and a total of 15 baseline methods for benchmarking. The baselines include 10 GNNs, 3 conventional machine learning methods and 2 new architectures. Using the optimal combination of parameters in the static setting, we generate benchmark datasets for corresponding tasks in the dynamic setting. In the subsequent sections, we first describe the generation of graph-based datasets, followed by the description of each task.
3.2 Graph Representation
The landscape of constructing brain graphs encompasses a variety of approaches. Previous research in this domain has explored various methodologies for constructing brain graphs and subsequent downstream tasks. For instance, in [2, 3, 46, 47], distinct measures such as mean activation, node index coordinates, spatial one-hot encoding, and correlations have been employed node features

---

with brain graphs. Additionally, a diverse range of measures including partial correlation, Pearson correlation, and geometric distances, among others, have found wide application in defining edges within brain graphs [48]. Our static graph representation encompasses the conventional methodology of generating a static functional connectome graph from an fMRI scan, see Appendix, B.3 for additional details. We define a connectome graph as G = (V, E, X), wherein the node set V {ui, U2,..., Un represents ROIs, while the edge set & [ V x V/ represents positive correlations between pairs of ROIs, determined via defined threshold. The feature matrix is denoted by X E [Rudd, where n signifies the total number of ROIs and refers to the feature vector's dimension. In our benchmarking setup, define correlation vectors as node features. Subsequently, we define a representation vector hg for the graph G. obtained via a GNN with an objective to perform the desired downstream machine learning task.
fMRI data comprise numerous timepoints within a scan, permitting the construction of dynamic graphs and thereby emphasizing the temporal information encapsulated within the data. This strategy has been evidenced be notably effective within the literature [2]. Within the dynamic context, we define a sequence of brain graphs over I' timepoints, denoted as G = {G,, G2.... GT}, wherein each graph G, captured at index t to t + from the fMRI scan. Here, I' signifies the window length, set to 50 with stride of 3 in our experiments. This setup allows us to capture functional connectivity within 36 seconds every 2.16 seconds, adhering to a common protocol for blddng-window analyses as outlined in [34]. Following the approach in [2]. opt randomly crop the ROI- c0l-timeseries data to a length of 150 timepoints. This procedure results in a total of 34 frames per subject, mitigating the computational and memory overhead in training complex models.
The procedure for constructing a graph at each time-point similar to the one applied to the static graph. The initial preprocessing, including parcellation, noise removal, and addressing head motions, remains consistent in order to construct the timeseries object. For each window, individual normaliza- tion has been performed, and then correlation matrices and corresponding graphs are constructed. Subsequently, G can be utilized to generate a dynamic graph representation haym to execute the desired downstream ML task. We refer the reader to the Appendix B.3 for further details.
3.3 Benchmark Datasets
The benchmark datasets are primarily divided into three main categories: those constructed for classification of demographics and task states, and those constructed for estimation of cognitive traits. Each category encapsulates distinct aspects of the collected data and serves unique analytical purposes. A brief description is provided below for each of these categories with some basic statistics presented in Table 1.
Predicting Demographics: The category of demographic estimation in our dataset is comprised of gender and age estimation [41]. The gender attribute facilitates a binary classification with the categories being male and female. Age is categorized into three distinct groups in [27]: 22-25, 26-30, and 31-35 years. A fourth category for ages 36 and above was climinated as it contained only 14 subjects (0.09%), to maintain a reasonably balanced dataset. We introduce four datasets named: HCP-Gender, HCP-Age, DynHCP-Gender and DynHCP-Age under this category. The first two are static graph datasets while the last two are the corresponding dynamic graph datasets.
The dask-dec involves seven tasks: Emotion Processing, Gambling, Language, Motor, Relational Processing, Social Cognition, and Working Memory. Each task is designed to help delineate a core set of functions relevant to different facets of the relation between human brain, cognition and behavior [49]. Under this category, we present two datasets: HCP-Task, a static representation, and DynHCP-Tass its dynamic counterpart.
Predicting Task States:
The cognitive traits category of our dataset comprises two signif- icant traits: working memory (evaluated with List Sorting) [50] and fluid intelligence (evaluated with PMAT24) [51]. Working memory refers to an individual's 's capacity to temporarily hold and manipulate information, a crucial aspect that influences higher cognitive functions such as reasoning, comprehension, and learning [52]. Fluid intelligence represents the ability to solve novel problems, independent of any knowledge from the past. It demonstrates the capacity to analyze complex relationships, identify patterns, and derive solutions in dynamic situations [20, 30). The prediction of both these traits, quantified as continuous variables in our dataset, are treated as regression problem. We aim to predict the performance or scores related to these cognitive traits based on the func-
Estimating Cognitive Traits:
5

---

Agprapale
Popn
prddicioo
Transformer
M,P
Baxch Nom
hy
Staph canviitinn
NU
+
Son Posing
Grgph convoltiin
Graghh Conciition
Gragh convoltioon
Fundioonl Connacto mo
Figure 2: (a). Illustration of the architecture for learning dynamic graph representations. (b). Visualization the GNN" architecture featuring residual connections and concatenated features.
tional connectome graphs. We generate four datasets under cognitive traits: HCP Fluid Intelligence (HCP-FI), HCP Working Memory (HCP-WM), DynHCP-FI and DynHCP-WM.
3.4 Learning models
The functional connectome, which effectively captures the network structure of brain activity, has proven to be valuable representation of fMRI data for machine learning, as demonstrated in numerous previous studies and process, we sought a suitable GNN framework that could effectively leverage the comprehensive functional connectome data through combination of message passing and neural network. After thorough exploration, we implemented GNN architecture, denoted as GNN" illustrated in Figure (b), that incorporates residual connections and concatenates hidden representations obtained from message passing at each layer. To further enhance the model's performance, we employ batch normalization and a multi-layer perceptron (MLP) effectively utilize the combined representations during training. While adaptive residual connections have been extensively explored in GNNs, we present this simple and unique architecture for brain graphs that effectively learns the representations for brain graphs [53].
experiments [4, 9]. Recognizing its significance in the learning
Recently, number of dynamic graph representation approaches in conjunction with recurrent neural networks (RNNs) such as GRU, LSTM, and transformers, have been introduced [33, 54). However, assessing the effectiveness of GNN models in unified dynamic setting using the existing approaches presents a significant challenge. Therefore, we implement a simple and generalized architecture tailored to process dynamic graphs for the graph classification problem, illustrated in Figure 2 (a). Our architecture comprises two distinct modules. The first is GNN-based learning module, responsible for deriving graph-level representations from each of the derived graph snapshot. Following this, a transformer module takes over, applying attention to the learned representations from the GNNs. Finally, the outputs are averaged into single dynamic graph representation vector, hdyn- This design offers a universally applicable method for evaluating multiple GNN methods within a dynamic graph setting for the downstream ML classification and regression problem.
Benchmarking Setup
In order to thoroughly evaluate the performance of brain graphs generated through different hyperpa- rameters, we propose a series of questions, defined as hyperparameter probe. These questions seek to identify the optimal hyperparameter setting for our graph-based neuroimaging analysis and ultimately enhance the performance of the predictive models derived from it.
Question 4.1 What are the optimal node feature configurations?
The first question aims to identify the best configurations for node features. This involves an exploration and comparison of various feature representations to discern their effectiveness on the performance of the derived predictive models. assessing node feature configurations, our analysis encompasses the correlation matrix, the time-series BOLD signals, well as their combination. The

---

correlation matrix is generated by calculating the correlation values amongst all ROIs. On the other hand, the BOLD signals are derived post the preprocessing of the input fMRI image, adhering the preprocessing pipeline outlined in Section 3.1.
Question 4.2 To what extent does the number of ROIs impact the performunce of predictive modeling on graphs?
The second question delves into the influence of varying the number of ROIs on the performance of predictive modeling. The objective is to assess how the granularity of ROIs affects the quality and the performance of the predictive models. We evaluate the use of 100, 400 and 1000 ROIs.
Question 4.3 To what degree does sparsifying brain functional connectome graphs impact the performance of predictive modeling? What threshold yields optimal performance?
Our third question investigates the impact of sparsifying brain functional connectome graphs on the performance of the predictive models. It aims to establish a threshold that leads to optimal model performance in graph machine learning setting. In our exploration, we consider the top 20%, 10%, and 5% percentile values from the correlation matrices to construct the graph edges.
Question 4.4 Which graph convolution approaches are preferable for the predictive modeling on brain graphs?
Our fourth and final question delves into the exploration of various graph convolution methods, assessing their suitability for predictive modeling on brain graphs. The aim here is not only to identify, but also to recommend the most effective techniques, considering the specific features and intricacies of neuroimaging data. In this endeavor, we have put to test over 12 GNNs, which include two of our own implemented frameworks, to gauge their comparative performance.
By addressing these questions, we aim to set a robust benchmarking framework for graph-based based ma- chine learning methods in neuroimaging, providing invaluable insights into their optimal application.
5 Benchmarking Results
In this section, we introduce the baseline models, describe our experimental setup, and present the results from our preliminary exploration study. Following this, lay out approach to benchmarking.
5.1 Baselines and Experimental Sctup
This section outlines the specifics of our unique, generalized experimental setup designed to evaluate a range of GNN models. We consider 10 well-established GNN models: G-GNN [22], GCN [18), GraphSAGE [19]. Unified Message Passing (UniMP)[25), Residual GCN (ResGCN) [21]. Graph Isomorphism Network (GIN), Chebyshev Convolution (Cheb) [24], Graph Attention Network (GAT) [23], Simplified GCN (SGC) [55], and General Convolution (General) [56]2. We also consider 3-layered Neural Network (NN), 2D Convolutional Neural Network (CNN) and Random Forest (RF) for the comparison.
In our experimental setup, we devise a graph classification architecture comprising three layers of GNNs, followed by a sort pooling aggregator [57]. Sort pooling sorts the node features based on the last channel, selecting only the first representations. Subsequently, sort pooling is advanced through two one-dimensional convolution layers, which are then succeeded by a two-layer Multi-Layer Perceptron (MLP). This architecture has been consistently utilized across all GNNs throughout the entire experimental setup. For the dynamic datasets, we utilize our baseline method with five different GNNs. For NN, we utilized 512, 256, and 128 hidden units in each layer, respectively. For the CNN, utilized a four-layer model with a stride of 2, 64 kernels of size 5, and padding set 2. This was complemented by three fully connected layers [58]. For the Random Forest (RF) [59], we opted for 100 estimators, leaving the remaining parameters at their Seikit-learn defaults. All of our experiments were carried out on a system equipped with an Intel(R) Xeon(R) Gold 6238R CPU operating at 2.20GHz with 112 cores, 512 GB of RAM. and an NVIDIA A40 GPU with 48GB of memory.
'e PyG implementations and default settings for running all these models
7

---

Table 2: Results of the gender classification using three distinct node feature configurations across three settings, evaluated on 10 GNNs. The configurations include CORRELATIONS (CORR), BOLD signals, and a combination of BOLD CORRELATIONS, evaluated across 10OROIs, 400ROIs, and 1000ROIs. Avg. column indicates the average results across the row and numbers under ROIS indicates average results across each ROI. The blue notation highlights the overall best results for
each GNN and red indicates average best performance across each ROI. Average best results are
obtained through 1000 ROIS with sparser graphs.
Dataset
I-GNN
GCN
SAGE
UniMP
ResGCN
GIN
Cheb
GAT
SGC
General
Avg.
CORR
65:33
08.98
68.70
08.33
56.06
08.21
03.91
69.49
68.45
64.50
67.30
100ROb
5037
31.07
51.30
$5.09
53.19
40395
51.90
51.11
CORRABOLD
52.78
51.02
50.28
50.99
50.00
54.91
49.44
20.37
51.57
51.30
51.36
CORR
7221
74:00
61:66
68.77
70009
7189
58.94
69.35
75.99
73099
69.56
400RO1.
ROTTS
51.16
3162
3339
31.90
52.11
3500
40007
50.46
33.34
300
5223
CORRABOLD
51.90
52.95
51.77
52.36
50.63
52.05
52.61
75.19
71.71
73.75
63.13
1000R3I
BOLD
48:15
46.99
49:33
5033
4799
36:48
47222
30393
49031
51.02
49:30
COREEBOLD
31.0
31.1
31.25
20.35
14.35
40766
51.22
31.34
31.37
31.31
Models training: We have carefully carried out the training and evaluation of each dataset in our study. Each dataset was partitioned randomly with 70% training. 20% testing, and 10% for validation. To ensure reproducibility and balance across the datasets, we employed fixed seed, 123, for the split in a stratified setting. This stratified approach facilitated an equitable distribution of classes in each partition. Each model underwent training for 100 epochs with learning rate of le-5 for classification, and for 50 epochs with a learning rate of le for regression problem. Across all experiments, we set dropout to 0.5, weight decay to 4, and designated 64 hidden dimensions for both the GNN convolution and MLP layers. Furthermore, for loss functions, we utilized cross entropy for classification and mean absolute error for regression problems. All benchmarks and their source codes can be accessed on GitHub². The static benchmark datasets are also available at PyG²
5.2 Exploratory Experiments and Results
Here address the questions outlined earlier by conducting series of experiments including the evaluation of different node feature configurations, the influence of varying numbers of ROIs, the implications of sparsity in brain graphs, and the effectiveness of diverse graph convolution approaches. Each experiment aligns with a question, thereby paving the way for comprehensive analysis and definitive conclusions.
Performance enhancement with correlations as node features: Our first step involves evaluating the interplay between the number of ROIs and the configuration of node features, with aim to streamline the overall search space. For this purpose, we engage in the gender classification problem using 10 different GNNs. The results of these experiments are presented in Table 2. It is clear that employing correlations as node features consistently enhances the performance across all evaluated numbers of ROIs. However, what caught our attention the significant variance in the results obtained through correlations and BOLD signals and the number of ROIs. The performance notably declines when correlations and BOLD signals combined, and the number of ROIs are reduced. This motivates further investigation on how to leverage BOLD signal or perhaps obtain features from the BOLD signals to be used for learning. Furthermore, the performance of different GNNs baselines does not consistently correlate with the number of ROIs or node features.
Our analysis extended to evaluating the efficacy of 10 GNNs on gender classification, using a varying number of ROIS and different graph densities. In addition to gender classification, we further incorporated task-state classification problem to strengthen our observations under different settings. For all the experiments, node features, decision driven by the consistent boost they offer in
Performance enhancement through large ROIs and sparse brain graphs:
we opted for correlations performance from the last experiment. The results are presented in Table 3. An important observation from our findings reveals that larger numbers of ROIs, (1000) demonstrate superior performance in gender classification. Similarly, significant number of GNNs exhibit improved results with the use of 1000 ROIs for the task-state classification problem. An analysis of the graph densities reveals an
htcrpe://gitabb..cm//aaaarr
/
8

---

Table Performance comparison terms of accuracy of 10 GNNs with different ROIs and varying graph densities on gender and task state classification problems. The blue highlights the overall best results per model each classification problem.
Dutaset
E-GNN
GCN
SAGE
UnIMP
ResGCN
GIN
Cheb
GAT
SGC
General
Spune
6333
72.96
69.35
69.72
68.66
69.72
63.70
70.28
70.37
67.22
Gender Classllicattio
100ROIS
63.5
6838
6833
68.33
Dense
61.30
6833
65.00
6506
63.00
66.39
64.26
69.22
68.45
61.76
Spund
0035
2234
69.86
67.56
7123
00.4
66445
72.22
28.25
26
400RO%%
Malhum
6361
6898
08.70
6833
0608
683
6304
20:49
68.43
6430
Spune
SE.3
75446
77.89
76.67
76:33
76.2
76.45
1000ROIs
Medium
73.80
15.19
71.23
75.14
78.25
00.22
71.43
72.34
23.25
03.13
Dense
61.77
7330
78.86
72.30
75,89
78.70
76.67
71.67
75.25
72.69
91.50
91.56
91.43
92.73
92.55
99.91
91.40
Task Classitcatio
Medium
91.83
92755
95.06
95.15
91.40
Danse
00.30
0111
03.15
05.28
0302
5712
0338
0308
00:40
8747
Spare
03.23
9421
94.78
94.72
94261
89.79
94.45
18.2
94.17
93.22
400ROTs
9502
03.39
Danse
05.76
04.78
Sparse
9330
9330
54.00
0339
3514
9332
34.66
032
9437
1000ROT:
Medium
V2.6
0087
04.30
03.79
9208
83.40
94.88
94.00
07.37
91.87
03.76
Table 4: Classification results in terms of accuracy on benchmark static datasets constructed with optimal setting. Blue indicates overall best results.
Unttawet
GCN
General
GNN
HCP Yask
97.79
55.85
18.9
93.23
94.31
98.78
9472
98M1
19.29
90.8
95.2
96.17
93.67
V8.30
Gender
80.67
76.99
699
83.19
75.46
77.9
7667
2839
75.96
5000
76.70
76.48
78.99
W9,07
HCPAN
46.3
4538
4034
4272
4376
4034
433
4000
44.99
4197
42.5
4544
41.03
50.35
intriguing trend. For instance, based on the results from Table 3, the ratio of sparse:mediummdenss on the gender classification dataset is 6 0 4, while the task dataset, it stands at 4 2 for 1000 number of ROIs Furthermore, the differences in results, especially exhibit lower performance, generally small. Recognizing the increased complexities stemming from memory usage, training demands, and the possibility of oversmoothing, we have chosen sparse graphs with the combination of large ROIs, and correlation features in benchmarking setup.
cases where sparse graphs
5.3 Benchmarking with Optimal Settings
Considering the optimal setting obtained through exploring search space presented in the previous section, here we present the experimental setup and benchmarking results on the proposed 10 datasets.
The classification accuracy of all baseline models detailed in Table 4. It is evident from the results that the GNN stands out as the leading performer. However, the Neural Network's performance also notably impressive. Similarly, the results pertaining the regression problems have been outlined in Table 5. The leading performer on the regression problems is again GNN These results distinctly demonstrate that residual connections coupled with message passing play a pivotal role in enhancing performance in brain networks. This synergy arises from the capacity of message passing glean meaningful representations from highly correlated features. Simultaneously, the inclusion of residual connections empowers the utilization of the input features with the learned representations obtained through message passing. This also underscores the effectiveness of correlation features in influencing the model's performance on brain graphs.
In Table 6, we lay out the classification and regression results obtained the dynamic datasets. Given the consideration of a basic dynamic baseline and the construction of dynamic datasets using limited dynamic lengths and number of ROIs, the performance does not quite match up to the static datasets. Nonetheless, it's worth noting that UniMP, despite the constraints, consistently demonstrates competitive performance.
6 Conclusion and Future Works
In this work, we introduce novel brain connectome benchmark datasets specifically tailored for graph machine learning, representing a promising avenue for addressing various challenges neuroimaging. The inherent symmetries and complex higher-level patterns found in brain graphs make them well-

---

Table Results for HCP-FI and HCP-WWM dataset using mean absolute error (MAE). Blue indicates
overall best results.
K-GNN
GCN
SAGE
UniMP
ReGGNN
GIN
Cheb
GAT
SGC
General
GNN"
HCP-FI
0,284
0.288
0,283
0.287
0.281
6,548
0.278
0.200
0.252
0,283
0,264
HCF-WM
0.818
0.825
0.810
0.812
0.530
1,032
0.789
0.804
0.828
0.819
0.751
Table 6: Models' performance in terms of accuracy and MAE across five dynamic datasets
Dataset
UniMP
E-GNN
Accuracy
GAT
SAGE
General
UmMPP
A-GNN
GAT
SAGE
General
DyaHC lask
89.00
73003
8967
90.33
68.84
DynHCP-91
3339
3.541
3,801
3,842
3,802
DvalICP
DyalICP-Ape
72.3
44.41
44.3
44.99
40.85
42.99
62.03
DysHCP-WM
10.589
10.596
10.592
10.507
10.571
suited for graph machine learning techniques. To advance this vision, we present NeuroGraph, comprehensive suite encompassing benchmark datasets and computational tools.
In our comprehensive exploratory study encompassing 35 datasets, we conducted a thorough analysis by running multiple machine learning models. Our key observations
follows: (1) increasing
the number of ROIs or employing large-scale brain graphs leads to improved performance compared datasets with fewer ROIs, (2) employing sparser graph setting enhances model performance and (3) while not intuitive, utilizing correlation as node features has significant potential to enhance model performance. Through a range of experiments across various learning objectives, we further highlight that GNNs exhibit superior performance compared to traditional NNs and 2D CNNs. These findings underscore the sigmificant potential of GNNs in achieving improved performance actoss diverse tasks and underscore their suitability for graph-baser Neuroimaging data analysis. Based on these insightful observations, we have developed NeuroGraph, a comprehensive benchmarks specifically designed for graph-based neuroimaging. Additionally, we provide computational tools to explore the design space of graph representation coming from Neuroimaging data, to facilitate the transformation of fMRI data into graph representations and showcase the potential of GNNs in this context. NeuroGraph serves as valuable resource, offering a road map for researchers interested in leveraging graph-based approaches for fMRI analysis and demonstrating the effective utilization of GNNs in this domain.
A Acknowledgments and Disclosure of Funding
This research incorporates contributions from the "Modeling and Model Integration" project of the Vanderbilt Institute for Software Integrated Systems, and by the National Science Foundation under Grant No. 2325417, 2321 2321684, 2239881 and NIH Grant No. RFI CFIMH125531 Work on "Modeling and Model Integration" is supported by Wellcome Leap part of the Multi-Channn Psych Program. Moreover, the data were provided [in part] by the Human Connectome Project, WU-Minn Consortium (Principal Investigators: David Van Essen and Kamil Ugurbil; 1U54MH0916577 funded by the 16 NIH Institutes and Centers that support the NIH Blueprint for Neuroscience Research; and by the McDonnell Center for Systems Neuroscience at Washington University.
References
[1] Jie Zhou, Ganqu Cui. Shengding Hu, Zhengyan Zhang, Cheng Yang, Zhiyuan Liu, Lifeng Wang, Changcheng Li, and Maosong Sun. Graph neural networks: A review of methods and applications. Al open, 1:57-81, 2020.
[2] Byung-Hoop Kim, Jong Chul Ye, and Jae-Jin Kim. Learning dynamic graph representation of brain connectome with spatio-temporal attention. Advances in Neural Information Processing Systems, 34:4314-4327 2021.
[3] Xiaoxiao Li, Yuan Zhou, Nicha Dvomek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence Staib, Pamela Ventola, and James $ Duncan. Braingnn: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis, 74:102233 2021.
[4] Andrew T Drysdale, Logan Grosenick, Jonathan Downar, Katharine Dunlop, Farrokh Man- souri, Yue Meng, Robert N Fetcho, Benjamin Zebley, Desmond Oathes, Amit Etkin, et al.
10

---

Resting-state connectivity biomarkers define neurophysiological subtypes of depression. Nature medicine, 23(1):28-33) 2017.
[5] Michael D Fox and Mareus E Raichle. Spontaneous fluctuations in brain activity observed with functional magnetic resonance imaging. Nature reviews neuroscience, 8(9):700-711, 2007. 2007.
[6] Lada Kohoutová, Juyeon Hco, Sungmin Cha, Sungwoo Lee, Taesup Moon, Tor D Wager, and Choong- Wan Woo. Toward a unified framework for interpreting machine-leaminng models in (999::1999-13933 2020.
neuroimaging. Nature protocols,
[7] Hejie Cui, Zijie Lu, Pan Li, and Carl Yang. On positional and structural node features for graph neural networks on non-attributed graphs. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, pages 3898-3902, 2022.
[8] Ed Bullmore and Olaf Sporns. Complex brain networks: graph theoretical analysis of structural 2009.
and functional systems. Nature reviews neuroscience,
[9] Anees Abrol, Zening Fu, Mustafa Salman, Rogers Silva, Yuhui Du, Sergey Plis, and Vince Cal- houn. Deep learning encodes robust discriminative neuroimaging representations to outperform standard machine learning. Nature communications, 12(1):353, 2021.
[10] Xuan Kan, Wei Dai, Hejie Cui, Zilong Zhang, Ying Guo, and Carl Yang. Brain network transformer. arXiv preprint arXiv: 10.06681, 2022.
[11] Olaf Sporns, Giulio Tononi, and Rolf Kötter. The human connectome: structural description of the human brain. PLoS computational biology, 144):e4,, 2005.
[12] Jonas Richiardi, Sophie Achard, Horst Bunke, and Dimitri Van De Ville. Machine learning with brain graphs: predictive modeling approaches for functional imaging in systems neuroscience. IEEE Signal processing magazine, 30(3):58-70, 2013.
[13] Danielle S Bassett and Olaf Sporns. Network neuroscience. Nature neuroscience, 20(3):353- 364, 2017.
[14] Hongming Li and Yong Fan. Interpretable, highly accurate brain decoding of subtly distinet brain states from functional mri using intrinsic functional networks and long short-term memory recurrent neural networks. Neurolmage, 202:1 16059, 2019.
[15] Armin W Thomas, Christopher Ré, and Russell A Poldrack. Interpreting mental state decoding with deep learning models. Trends in Cognitive Sciences, 26(1 1):97--986, 2022.
[16] Taeho Jo, Kwangsik Nho, and Andrew Saykin. Deep learning in alzheimer's disease: diag- nostic classification and prognostic prediction using neuroimaging data. Frontiers in aging neuroscience, 11:220, 2019.
[17] Taban Eslami, Vahid Mirjalili, Alvis Fong, Angela R Laird, and Fahad Saeed, Asd-diagnet: hybrid learning approach for detection of autism spectrum disorder using fmri data. Frontiers in neuroinformatics, 13:70, 2019.
[18) Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. arXiv preprint |6Xiv: (et9::60999999 2016.
[19] Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. Advances in neural information processing systems, 30, 2017.
[20] Mónica Emch, Claudia C Von Bastian, and Kathrin Koch. Neural correlates of verbal working
memory: An fmri meta-analysis. Frontiers in Human Neuroscience, 13:180, 2019.
[21] Xavier Bresson and Thomas Laurent. Residual gated graph convnets. arXiv preprint 171 2017.
[22] Christopher Morris, Martin Ritzert, Matthias Fey, William Hamilton, Jan Eric Lenssen, Gaurav Rattan, and Martin Grohe Weisfeiler and leman go neural: Higher-order graph neural networks. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 4602-4609, 2019.
11

---

[23] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio. Graph attention networks. arXiv preprint arXiv:1710.10903, 2017
[24] Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances in neural information processing systems, 29, 2016.
[25] Yunsheng Shi, Zhengjie Huang. Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for semi-supervised classification. arXiv preprint arXiv:2009.03500 2020.
[26] Anwar Said, Mudassir Shabbir, Saeed-UI Hassan, Zohair Raza Hassan, Ammar Ahmed, and Xenofon Koutsoukos. On augmenting topological graph representations for attributed graphs. Applied Soft Computing, 136:110104, 2023.
[27] Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, and Carl Yang. Braingb: benchmark for brain network analysis with graph neural networks. IEEE Transactions Medicul Imaging, 2022.
[28] Jeremy Kawahara, Colin J Brown, Steven P Miller, Brian G Booth, Vann Chau, Ruth E Grunau, Jill G Zwicker, and Ghassan Hamarneh. Brainnetenn: Convolutional neural networks for brain networks; towards predicting neurodevelopment. Neurolmage, 146: 1038-1049, 2017.
[29] David Ahmeet-Aristizabaat Mohammad Ali Armin, Simon Denman, Clinton Fookes, and Lars Petersson. Graph-based deep learning for medical diagnosis and analysis: past, present and future. Sensors, 21(14):445 (175) 2021.
[30] Simon Dahan, Logan ZJ Williams, Daniel Rueckert, and Emma C Robinson. Improving phenotype prediction using long-range spatio-temporal dynamics of functional connectivity. In Machine Learning in Clinical Neuroimaging: 4th International Workshop, MLCN 2021. Held in Conjunction with MICCAI 2021, Strasbourg. France, September 27, 2021, Proceedings 4, pages 145-154, Springer, 2021.
[31] Xuesong Wang, Lina Yao, Islem Rekik, and Yu Zhang. Contrastive functional connectivity graph learning for population-based fmri classification. In International Conference Medical Image Computing and Computer-Assisted Intervention, pages 221-230 Springer, 2022.
[32] Jiaxuan You, Tianyu Du, and Jure Leskovec. Roland: graph learning framework for dynamic graphs. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2358 2366, 2022.
[33] Aravind Sankar, Yanhong Wu, Liang Gou, Wei Zhang, and Hao Yang. Dynamic graph repre- sentation learning via self-attention networks. arXiv preprint arXiv: 18/2.09430, 2018.
[34] Maria Giulia Preti, Thomas AW Bolton, and Dimitri Van De Ville. The dynamic functional connectome: ofate-of-the-art and perspectives. Neuroimage, 160:41 2017.
[35] Jun Li, Junyu Chen, Yucheng Tang, Ce Wang, Bennett A Landman, and S Kevin Zhou. Trans- forming medical imaging with transformers? a comparative review of key properties, current progresses, and future perspectives. Medical image analysis, page 102762, 2023.
[36] Kanhao Zhao, Boris Duka, Hua Xie, Desmond J Oathes, Vince Calhoun, and Yu Zhang. A dynamic graph convolutional neural network framework reveals new insights into connectome dysfunctions in adhd. Neurolmage, 246: 1 18774, 2022.
[37] Alexander Campbell, Antonio Giuliano Zippo, Luca Passamonti, Nicola Toschi, and Pietro Lio. Dbgsl: Dynamic brain graph structure learning. arXiv preprint arXiv:2209. 13513, 2022.
[38] Kexin Ding, Mu Zhou, Zichen Wang, Qiao Liu, Corey W Arnold. Shaoting Zhang, and Dim- itri N Metaxas. Graph convolutional networks for multi-modality medical imaging: Methods, architectures, and clinical applications. arXiv preprint ar arxi:22002.080 2022.
[39] Giovanni Bonanno, Guido Caldarelli, Fabrizio Lillo, and Rosario N Mantegna. Topology of correlation-bass minimal spanning trees in real and model markets. Physical Review E, 6864):046130, 2003.
12

---

[40] Ke Zeng, Jiannan Kang, Gaoxiang Ouyang, Jingqing Li, Junxia Han, Yao Wang, Estate M Sokhadze, Manuel F Casanova, and Xiaoli Li. Disrupted brain network in children with autism spectrum disorder. Scientific reports, 7(1):16233, 2017.
[41] Soham Gadgil, Qingyu Zhao, Adolf Pfefferbaum, Edith V Sullivan, Ehsan Adeli, and Kilian M Pohl. Spatio-temport graph convolution for resting-state state fmri analysis. In Medical Image Computing and Computer Assisted Intervention-MICCA. 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part VII pages 528-535 Springer, 2020.
[42] Oscar Pina, Irene Gumplido-Mayorea Raffaele Cacciaglia, José María González-de Echávarri, Juan Domingo Gispert, and Verónica Vilaplana. Structural networks for brain age prediction. In International Conference on Medical Imaging with Deep Learning, pages 944-960. PMLR, 2022.
[43] David C Van Essen, Stephen M Smith, Deanna M Barch, Timothy EJ Behrens, Essa Yacoub, Kamil Ugurbil, Wu-Minn HCP Consortium, et al. The wu-minn human connectome project: an overview. Neuroimage, 80:62-79, 2013.
[44] Alexander Schaefer, Ru Kong, Evan M Gordon, Timothy Laumann, Xi-Nian Zuo, Avram Holmes, Simon B Eickhoff, and BT Thomas Yeo. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri. Cerebral cortex, 2899):3095-314 2018.
[45] Matthew F Glasser, Stamatios N Sotiropoulos, J Anthony Wilson, Timothy S Coalson, Bruce Fischl, Jesper L Andersson, Junqian Xu, Saad Jbabdi, Matthew Webster, Jonathan R Polimeni, al. The minimal preprocessing pipelines for the human connectome project. Neuroimage, 80:105-124, 2013.
[46] Xiaoxiao Li, Nicha C Dvornek, Yuan Zhou, Juntang Zhuang, Pamela Ventola, and James S Duncan. Graph neural network for interpreting task-fmri biomarkers. In Medical Image Computing and Computer Assisted inervention-MICCA. 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part V 22, pages 485-493. Springer, 2019.
[47] Byung-Hoon Hoon Kim and Jong Chul Ye. Understanding graph isomorphism network for rs- rs-fmri functional connectivity analysis. Frontiers in neuroscience, page 630, 2020.
[48] Xia Liang, Jinhui Wang, Chaogan Yan, Ni Shu, Ke Xu, Gaolang Gong, and Yong He. Effects of different correlation metries and preprocessing factors small-world brain functional networks: a resting-state functional mri study. PloS one, 7(3)x)2766, 2012.
[49] Deanna M Barch, Gregory C Burgess, Michael P Harms, Steven E Petersen, Bradley L Schlaggar, Maurizio Corbetta. Matthew Glasser, Sandra Curtiss, Sachin Dixit, Cindy Feldt, al. Function in the human connectome: fmri and individual differences in behavior. Neuroimage, 80: 10:1699 189, 2013.
[50] David Tulskky Noelle Carlozzi, Nancy Chiaravalloti, Jennifer I. Beaumont, Pamela A Kisala, Dan Mungas, Kevin Conway, and Richard Gershon. Nih toolbox cognition battery (nihtb-cb): List sorting test to measure working memory. Journal of the International Neuropsychological Society, 2066):599-61,, 2014.
[51] Tyler M Moore, Steven P Reise, Raquel E Gur, Hakon Hakonarson, and Ruben C Gur. Psycho- metric properties of the penn computerized neurocognitive battery. Neuropsychology, 29(2):235, 2015.
[52] Guixia Pan, Li Xiao, Yuntong Bai, Tony W Wilson, Julia M Stephen, Vince D Calhoun, and Yu-Ping Wang. Multiview diffusion map improves prediction of fluid intelligence with two paradigms of fmri analysis. IEEE Transactions on Biomedical Engineering, 688922552, 2020.
[53] Xiaorui Liu, Jiayuan Ding, Wei Jin, Han Xu, Yao Ma, Zitao Liu, and Jiliang Tang. Graph neural networks with adaptive residual. Advances in Neural Information Processing Systems, 34:9720-9733 9733, 2021.
13

---

[54] Seyed Mehran Kazemi, Rishab Goel, Kshitij Jain, Ivan Kobyzev, Akshay Sethi, Peter Forsyth, and Pascal Poupart. Representation learning for dynamic graphs: A survey. The Journal of Machine Learning Research, (1(1):2648-277 2720,, 2020.
[55] Felix Wu, Amauri Souza, Tianyi Zhang, Christopher Fifty, Tao Yu, and Kilian Weinberger. Simplifying graph convolutional networks. In International conference on machine learning. pages 6861-6871. PMLR, 2019.
[56] Jiaxuan You, Zhitao Ying, and Jure Leskovec. Design space for graph neural networks. Advances in Neural Information Processing Systems, 33: 7009--1001, 2020.
[57] Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI conference on artificial intelligence, volume 32, 2018.
(58) Alex Krizhevsky, llya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84 2017.
[59] Leo Breiman. Random forests. Machine learning, 45:5-32, 2001.
14

---

A Benchmarks Availability and Licensing
The fMRI data utilized in this research was sourced from the Human Connectome Project [1]. The proposed graph-based benchmark datasets can be accessed for download at github. io/ wawawwawsaad//aaaaaaaaaaaaaaaaaa wwwpssaiddneuorrrrpiii html. These datasets are provided PyG format, opti- mized for use with Graph Neural Networks (GNNs). However, they can also be conveniently incorporated into other platforms. Additionally, the associated code for downloading, preprocess- ing, and benchmarking open to the public at https: attpp:://ithhut com/ Anwar-Saile WaiddNeuroGraph. complete with comprehensive documentation.
B NeuroGraph and Neuroimaging Data
Neuroimaging, a powerful field of study, enables researchers delve into the complexities of the human brain by capturing detailed images and measurements. Recent advancements in technology have resulted in an abundance of neuroimaging data, particularly functional magnetic resonance imaging (IMRI), which offers invaluable insights into brain activity. However, understanding and analyzing fMRI data pose several challenges. Firstly, the high dimensionality of fMRI data presents a significant hurdle. Additionally, inherent noise and variability in IMRI signals can obscure underlying neural activity. Complex spatial and temporal dependencies further complicate fMRI data analysis, demanding advanced modeling techniques. Furthermore, the interpretation and analysis of fMRI data can be consuming and subjective. The graphical representation of fMRI data offers a plethora of opportunities to tackle these challenges. For instance, network science and graph theoretical approaches provide a diverse range tools to explore brain regions and their connectivity patterns [2]. Furthermore, the application of graph machine learning techniques, such GNNs particularly well-suited for analyzing neuroimaging data and have the potential to provide valuable insights. The provision of graph-based neuroimaging benchmarks and computational tools play crucial role to enhance the field, which is the main focus of this study.
B.1 IMRI Data Sources
Several initiatives have been undertaken in the past decade assemble comprehensive IMRI datasets. One notable source is the Human Connectome Project (HCP) dataset [1]. The HCP dataset offers an extensive collection of multimodal neuroimaging data, including resting- state fMRI, task-based fMRL, and structural MRI scans, from a large cohort of healthy individuals. In addition to large neuroimaging datasets curated by institutions or projects, some notable resources are OpenNeuro, OpenfMRI and fcon_10006 platforms, which host a diverse range of publicly available fMRI datasets contributed by researchers worldwide [3, 4]. These datasets cover various experimental paradigms (see Table 7), clinical populations, and research domains, providing researchers with a wealth of data for analysis and investigation.
We have chosen to utilize the HCP $1200 dataset from the Brain Connectome as a primary resource for our graph-based benchmarking [1]. This dataset well-suited for graph-based benchmarking due to its extensive coverage of brain regions and their interconnections. Additionally, the HCP S1200 dataset provides valuable demographic and behavioral information, enabling comprehensive analyses that consider various factors influencing brain connectivity. Its wide availability and standardized processing pipelines further contribute to its suitability for graph-based benchmarking, ensuring consistency and comparability across studies. Thus, the HCP S 200 dataset from the Brain Connectome represents robust choice for conducting graph-based benchmarking studies in the field of neuroimaging.
B.2 Reading HCP Dataset
Storing and reading fMRI datasets presents formidable challenge due to their substantial storage requirements, necessitating significant disk space allocation, e.g., each subject of HCP S1200 requires 1.1 GB of space on disk. Moreover, the preprocessing iMRI data calls for tools that are not only user-friendly but also highly efficient. Fortunately, the Human Connectome Database (HCP) offers
'nttpe://pyg.or/ htttp://fcoo_1000.ppp projectss.........ooom.oo mitre. hit/
15

---

Table Description of IMRI paradigms in HCP Young Adult dataset.
Dutaset
Volumes per run (TR)
Run duration (minl
Duration of tusk blocks (sec)
Descrittion
Rest
1200
14 : 24
slimali
Memary
4U5
5
Gambling
263
3 12
28
Winh, loss
284
body
Languaee
316
3
approx. 30
story, math
Relational Processing
232
2:
16
relational, control
Cognition
intersetion, interaction fnce, shage
Eaxction Processing
176
2: 16
18
Table 8: IMRI scans required disk storage. The storage information is obtained from Human Connectome Project website.
Task
Storage (GB)
1260.95
Working Memory
527.70
Gambling
387
Motor
115.81 .1
Language
426.72 72
Relational
343.4
Social
386.76
295.91
an AWS instance (s3 bucket) that allows for seamless data crawling. NeuroGraph, with its imple- mentation utilizing the boto3 Python package, provides an efficient solution for crawling the dataset. Boto3, a widely used Python package, enables scamless interaction with AWS services, facilitating efficient data retrieval and preprocessing the NeuroGraph framework. Our implementation offers users the flexibility to either store the datasets or preprocess them on the fly if storage space is limited (see Table 8 for disk storage). To access the HCP data, users are required obtain credentials from HCP? and provide them to NeuroGraph. Moreover, NeuroGraph also provides Python class for preprocessing data from the local storage.
B.3 Data Preprocessing
In close collaboration with domain experts from both the neuroimaging and graph machine learning fields, NeuroGraph's preprocessing pipeline is divided into five stages. These stages ensure the quality and reliability of the fMRI data. Initially, we utilize data that has already been processed using the HCP minimal processing pipeline [5].
Step 1 Brain Parcellation:
The first phase of our pipeline involves brain parcellation, process that divides the brain into smaller regions or parcels. This step allows for the analysis of functional connectivity within and between these parcels. In study, we employ the Schaefer atlases [6], widely used brain parcellation schemes that define neurobiologically meaningful features of brain organization. These atlases provide parcellation of the cerebral cortex into hierarchically organized regions at multiple resolutions. Using the population level atlases, extract the mean IMRI timeseries for each region of interest (ROI). This provides a representative measure of the average neural activity within each specific brain region, enabling subsequent connectivity analyses.
Step 2 Remove Scanner Drifts and Motion Artifacts: Next, remove linear and quadratic trends along with six rigid-body head motion parameters and their derivatives, from the fMRI data. Removal of the trends aims remove the scanner drifts in the fMRI signals that arise from instrumental factors. Removal of the motion parameters, that capture the movement and rotation of the subject's head during the scanning session, ensures that any potential confounding effects are minimized. By climinating these artifacts, we enhance the signal-to-noiss ratio and increase the sensitivity to neural activity.
Step 3 Subject- Signal Normalization: We perform subject-level normalization of the ROI timeseries signals. Specifically, we temporally normalize all subject signals zero mean and unit variance. This step allows for fair comparisons and facilitates the identification of meaningful variations in the functional connectivity patterns across subjects.
hhttps://b.. hwmancomnectome.com
16

---

(a) Reesingg-state graph
(b) Emotion graph
76
75
(c) Gambling graph
(d) Language graph
Figure 3: Visualization of the corresponding simple undirected graphs with 100 ROIs for a single subject during both the rest condition and while performing certain tasks. Note that the coloring of the graphs has been applied based the community structure, but solely for visualization purposes. Isolated nodes were removed.
We compute the correlation matrices from the ROI timeseries signals. Correlation matrices capture the strength of functional connectivity between different ROIs. By calculating pairwise correlations between the timeseries signals of each ROI, we obtain a matrix that represents the interregional functional connections within the brain. This step allows us to quantify and analyze the patterns of functional connectivity across the entire brain, and construct a graph. The correlation matrices serve as valuable tool for investigating the network-level organization of the brain and identifying regions that exhibit synchronous activity [7]. These matrices provide a representation of the functional architecture and can be further utilized for graph-based analyses, such as network characterization and identification of key brain hubs [7, 8]. In Figure 3, we provide the visualizations of the graphs correspond to one subject in certain conditions.
Step 4 - Calculate Correlation Matrix:
Step 5 Construct Static/Dynamii Attributed Graphs: Finally, we compute two types of graph-based datasets from the functional connectivity matrix: static and dynamic graphs. As discussed in Section 3 of the paper, the static graph is defined as G (V, E, X). Here, the node set V = {21, ......, , Un
represents ROIs, while the edge set E C V x V denotes positive correlations between pairs of ROIs, as determined by predefined threshold. The feature matrix is represented by X"xd where symbolizes the total number of ROIs, and d corresponds to the dimension of the feature vector. We explore the dataset generation search space by considering different numbers of ROIs, different thresholds, and node features to identify optimal parameters. The next section provides a comprehensive overview of the datasct construction search space.
17

---

Regarding the parameter setup for constructing our benchmark datasets. we opt for a sparse setup (top 5%) with 1000 ROIs for the HCP-Genner,, HCP-Age, HCP-WM, and HCP-FI datasets. However, for the HCP-Task dataset, we reduce the number of ROIs to 400 in order to manage memory overhead. In the dynamic setting, we employ sliding window approach with fixed window length (T) set to 50 and stride of 3. Considering memory constraints and computational overhead, we fix the dynamic length (!) to 150 and slide over the preprocessed timeseries matrix to construct dynamic graphs. For all dynamic graphs, we consider 100 ROIs and medium sparsity (top 10%). With this setting, the total number of dynamic graphs we obtain for each subject is ((1 (1--B) -T)/stride) +1
B.4 The Design Space Vast
The design space for constructing graphs from correlation matrices is substantial, given the multitude of available methods. We can construct diverse graph types employing various strategies. For instance, some of the potential graph types to consider include simple undirected graphs as demonstrated in [9], weighted graphs[10], attributed graphs [1 [11], and minimum spanning trees [12, 13], among others. Similarly, a range of parameters comes into play during this process, further expanding the design space for these constructions. These parameters include the number of ROIS, edge weights, density thresholding for edge selection, and node features, to name a few.
GNNs have shown considerable promise in handling attributed graphs, demonstrating their effec- tiveness in various domains [10, 14]. Attributed graphs, which include not only the graph topology but also node-level features, represent complex systems more accurately than simple graph. GNNs leverage these attributes to capture both local and global structural information, allowing for the development of more comprehensive graph representations. Considering the importance of attributed graphs, we opted to construct rich, brain attributed graphs.
Traditional methods for representing node features in graphs include using coordi- nates [15], one-hot encoding [8], and mean activation [16, 15]. Coordinates serve to provide spatial information about the nodes, while one-hot encoding are used for categorical features, effectively distinguishing different node types. Mean activation, on the other hand, can give insights about the average level of node's s activity or influence. While these methods provide a base level of information, they may not fully capture the rich complexity inherent in many data structures, such as brain graphs. To address this, we explore more powerful ways of representing node features, including using correlation vectors, BOLD signals and the combination of both. Correlation vectors can encapsulate the relationship between different nodes, providing insight into the connectivity and interaction within the graph. BOLD signals, give information about changes in blood flow in the brain, which can be an indicator of neural activity. By combining both of them, we may enrich models with a wealth of information, thereby capturing the intricate details and relationships present in brain graphs.
Node Features:
Number of ROIs: ROIs in brain graph construction may significantly impacts the granularity and overall scope of the resulting graph. Using a smaller number of ROIs, such as 100, can lead to a more generalized and coarser view of brain connectivity. This simplified perspective can be useful for broad overviews and initial exploration but might overlook intricate local interactions or specific clusters of activity. Conversely, using a larger number of ROIs, such as 400 or 1000, allows for more detailed and finer representation of the brain's 's connectivity. With more ROIs, the graph can capture more specific interconnections, potentially revealing sub-networks or localized activity patterns that a coarser graph might miss. However, larger graphs also present a challenge in terms of computational load and complexity, also prone to noise. Interestingly, different methods in the litcrature have adopted different numbers of ROIs for their analysis [9, 10, 11]. These varying approaches underscore the fact that the choice of ROIs number is not merely a matter of computational convenience, but can significantly influence the outcomes of the study.
In light of this, our research aims to explore these three ROIs sizes: 100, 400, and 1000. Our goal is to understand the impact of different graph granularity levels on the performance of GNNs. By doing so, we hope to provide deeper insights into how different levels of detail the graph structure affect the GNN's ability to capture and model brain connectivity. This investigation could potentially guide the selection of an optimal ROI size in future brain graph studies, striking a balance between capturing sufficient detail and maintaining computational feasibility.
18

---

Density Thresholding: Graph density is fundamental property that may impacts the performance of GNNs. Graph density refers the proportion of the possible connections in graph that are actual connections. It influences how information is propagated through the network, may potentially affect the accuracy and efficiency of the GNN. A sparse graph (low-density) might lead to information underflow, with some nodes being poorly connected, which might cause inadequate learning of node representations. On the other hand, a dense graph (high-density) could lead to an information overflow, with a significant amount of information being propagated, possibly causing noise and overfitting [17].
Thresholding, on the other hand, is crucial step in the construction of brain graphs. It's used to determine which correlations are strong enough to be included as edges in the graph. There are several approaches to thresholding. One is absolute thresholding, where fixed threshold value is selected, and all correlations in the matrix above this threshold are included as edges in the graph. However, the choice of an absolute threshold can be somewhat arbitrary, and may result in graphs of varying sizes and densities. This variability can complicate comparisons between graphs [18]. Proportional thresholding is another method, in which the strongest 2% of correlations are included as edges in the graph. This method ensures that all resultant graphs have the same density edges, which facilitates comparisons between them. However, it can also result in the inclusion of weak, potentially con-significant correlations in the graph. To avoid this issue, some studies consider only positive correlations, which allows the construction of graphs with various densities and avoids the complications of negative thresholding [19].
Indeed, there are numerous ways to conduct thresholding in brain graph construction, with several options available within each thresholding approach. Each method and option presents its unique set of advantages and potential limitations. In this context, we focus proportional thresholding with positive correlations, an approach that has shown encouraging results in previous research [10, 9J. Specifically, we explore three levels of density: those defined by the top 5%, 10%, and 20% percentile values from the correlation matrices. These densities represent different levels of graph sparsity, offering broad perspective on how the choice of threshold can impact the topology and interpretability of the resulting brain networks. We note that the terms "sparse" (5%) and "dense" " (20%) are relative and dependent the context of feasible ranges. Despite their different percentages of edges, both sparse and dense graphs exhibit a complexity of O(n?) edges. We observed that even in sparse datasets, the average degree is around 50 for 1000 ROIs, indicating substantial level of connectivity.
C NeuroGraph Benchmark Datasets
We propose a collection of ten datasets tailored to five distinct tasks, encompassing both static and dynamic contexts. These tasks are identified as HCP-Task, DynHCP-Task, HCP-Gender, DynHCP- Gender, HCP-Age, DynHCP-Age, HCP-WM, DynHCP-WM, HCP-FI, and DynHCP-FL. These datasets are derived from the HCP $1200 dataset, following sequence of preprocessing operations. For the creation of static datasets, we eliminated two subjects that contained fewer than 1200 scans and then applied the preprocessing as outlined in the previous sections. The resulting datasets are represented as sparse matrices with 1000 ROIs However, we 'v tailored the Activity dataset to include only 400 ROIS owing its larger size of over 7000 scans, as this adaptation was necessary to overcome memory constraints. As for the dynamic datasets. we 've standardized the dynamic length to 150, with window size of 50 and stride of 3. Moreover, to alleviate the substantial memory demands, we' ve limited the dynamic datasets to encompass only 100 ROIs. The distribution of classes for each dataset, as well as the values for regression tasks, have been visualized and are presented in Figure 4.
C.1 GNN" and Dynamic Graph Baselines
Our study also explores variation of residual GNNs, we named GNN*, the model that leverages both residual connections and a feature concatenation approach, enhancing the utilization of the functional connectome the training process. As delineated in Section 3.4 and visualized in Figure 2 of the main paper, GNN' employs a universal graph convolution layer, facilitating the use of any GNN convolution contingent on the project's 'ss requirements. Similarly, the dynamic graph baseline (depicted in Figure 2 of the main paper) also uses a general graph convolution, followed by Transformer
19

---

54.2%
500
400
34.6%
24.25
14.0%
240%L
144%
300
21.1%
100
Oratco Cand
(a) HCP-Gender
(b) HCP-Age
(c) HCP--Tsk
238
222
100
18.0
35.0
26.1
(d) HCP-WM
(c) HCP-FI
Figure 4: Illustration of class distribution for each dataset. For the regression task, histograms are presented depict the frequency distributions both Working Memory (WM) and Fluid Intelligence (FI) scores. In addition to these, Kernel Density Estimates are superimposed on the histograms, providing smoother representation of the distributions.
module. Throughout our experimentation, we employed UniMP with GNN' and tested five models using the dynamic baseline, the results of which are tabulated in Table of the main paper. All other parameters remain consistent with the detailed exposition experimental setup (Section 5. of the main paper.
D Memory and Running Time Analysis
Following comprehensive and rigorous exploration of the search space, have identified and established optimal datasets that strike a balance between minimizing memory requirements and maintaining an effective quantity of parameters. The trade-off achieved ensures that models are able to run smoothly on machines with reasonable computing power on our datasets, making them highly accessible wide range of users. This optimization also yields the additional benefit of reduced training times; our models are capable of training in mere minutes, significantly accelerating the model development cycle and promoting rapid iterative progress.
The specifics of this optimization are illustrated the context of Unified Message Passing (UniMP) model [20], which we use to showcase the efficient resource usage of datasets and approach. In Table 9, we offer detailed insights into the running times and memory requirements of UniMP model. We executed UniMP on each dataset for 100 epochs and recorded both GPU memory utilization and overall training time, which includes data loading. The number of hidden units for the GNN layer was 32 and 128 for the MLP layers. These data points provide a tangible representation of the efficiency gains achieved through our dataset size optimization process. Such optimizations are instrumental in ensuring datasets are not only computationally effective using any model but also highly accessible, enabling broader applicability for variety of hardware configurations. All experiments were executed on system equipped with an Intel(R) Xeon(R) Gold 6238R CPU operating at 2.20GHz with 112 cores, 512 GB of RAM, and an NVIDIA A40 GPU with 48GB of memory.
20

---

Table 9: Resource utilization analysis UniMP model on all benchmark datasets
Benchmark Dotaset
Disk storage (GB)
#Parameters
Minoory (MB)
Training time (sec)
TCCP-Task
4,0
265035
2463
854
HCP-iender
648870
T[CP-Age
3.6
648903
4293
355
HCP-WM
6551
ICCFFI
80336]
6762
890
Dy
7.3
309575
15881
11200
DynlTC
DynllCCP-A
1.0
309059
4113
1709
DyniCCP WM
J08N01
4359
1704
Dyn
30880
4335
1712
14
2
HCP-Task
(b) HCCP-ge
4227
1.004
0.44
0/22
1445
2.5
0.4
07'
800
(c) HCP-Gender
Figure 5: Models' performance: Accuracy and standard deviation on 10 runs with different seeds on HCP-Task, HCP-Age and HCP-Gender datasets.
E
Models Performance and Standard Error
We plot the accuracy along with the standard deviation of 10 runs, each with different seeds, for all the models three distinct datasets: HCP-Task, HCP-Age and HCP-Gender in Figure 5. We observed that the results reported a higher level of stability on both HCP-Task and HCP-Age datasets. This indicates that the models performed consistently and yielded more reliable results, suggesting greater degree of confidence in the accuracy measurements. On the HCP-Gender dataset, we observed slightly high standard errors across the models. Moreover, we provide the visualization of the hidden activations obtained from the last layer of G.N. N for the test and validation sets trained on HCP-Task and HCP-Gender datasets in Figure 6. We used TSNE for these visualizations.
References
[1] David C. Van Essen, Stephen M Smith, Deanna M Barch, Timothy EJ Behrens, Essa Yacoub, Kamil Ugurbil, Wu-Minn HCP Consortium, et al. The wu-minn human connectome project: overview. Neuroimage, 80:62-79, 2013.
[2] Cornelis J Stam. Modern network science of neurological disorders. Nature Reviews Neuro- science, 15(10):683-69) 2014.
21

---

Candin
-40
(a) HCP- Task test
(b) HCP-Task validation
1-5501 1P,
4.9
(d) HCP-Gender validation set Figure 6: Hidden layer activation on and validation sets of HCP-Task and HCP-Gender.
(c) HCP-Gender test set
[3] Russell A Poldrack, Deanna M Barch, Jason Mitchell, Tor D Wager, Anthony D Wagner, Joseph T Devlin, Chad Cumba, Oluwasanmi Koyejo, and Michael P Milham. Toward open sharing of task-based fmri data: the openfmri project. Frontiers neuroinformatiis, 7:12, 2013.
[4] Christopher J Markiewicz, Krzysztof J Gorgolewski, Franklin Feingold, Ross Blair, Yaroslav O Halchenko, Eric Miller, Nell Hardcastle, Joe Wexler, Oscar Esteban, Mathias Goncavles, et al. The openneuro resource for sharing of neuroscience data. Elife, 10:471774, 2021.
[5] Matthew Glasser, Stamatios N Sotiropoulos, J Anthony Wilson, Timothy Coalson, Bruce Fischl, Jesper Andersson, Junqian Xu. Saad Jbabdi, Matthew Webster, Jonathan Polimeni. et al. The minimal preprocessing pipelines for the human connectome project. Neuroimage, 80: 10:105-124, 2013.
[6] Alexander Schaefer, Ru Kong, Evan M Gordon, Timothy O Laumann, Xi-Nian Zuo, Avram J Holmes, Simon B Eickhoff, and BT Thomas Yeo. Local-global parcellation of the human cerebral cortex from intrinsic functional connectivity mri. Cerebral cortex, 2899):095-3114, 2018.
[7] Andrew T Drysdale, Logan Grosenick, Jonathan Downar, Katharine Dunlop, Farrokh Man- souri, Yue Meng, Robert N Fetcho, Benjamin Zebley, Desmond Oathes, Amit Etkin, et al. Resting-state connectivity biomarkers define neurophysiological subtypes depression. Nature medicine, 23(1):28-33) 38, 2017.
[8] Byung- Hoon Kim and Jong Chul Ye. Understanding graph isomorphism network for rs-fmri functional connectivity analysis. Frontiers in neuroscience, page 630, 2020.
22

---

[9] Hyung-Hoon Kim, Jong Chul Ye, and Jae-Jin Kim. Learning dynamic graph representation of brain connectome with namio-tt attention. Advances Neural Information Processing Systems, 34:4314 4327, 2021.
[10] Xiaoxiao Li, Yuan Zhou, Nicha Dvomek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence Staib, Pamela Ventola, and James S Duncan, Braingnn: Interpretable brain graph neural network for finri analysis. Medical Image Analysis. 74: 74:102233, 2021.
[11] Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, and Carl Yang. Braingb: benchmark for brain network analysis with graph neural networks. IEEE Transactions Medical Imaging, 2022.
[12] Giovanni Bonanno, Guido Caldarelli, Fabrizio Lillo, and Rosario Mantegna. Topology of pereelation-baee minimal spanning trees in real and model markets. Physical Review E, 68(4):046130, 2003.
[13] Ke Zeng, Jiannan Kang, Gaoxiang Ouyang, Jingging Li, Junxia Han, Yao Wang, Estate M Sokhadze, Manuel F Casanova, and Xiaoli Li. Disrupted brain network in children with autism spectrum disorder. Scientific reports, 7(1):16253, 2017.
[14] Anwar Said, Mudassir Shabbir, Saced-UI Hassan, Zohair Raza Hassan, Ammar Ahmed, and Xenofon Koutsoukos. On augmenting topological graph representations for attributed graphs. Applied Soft Computing, 136: 10104, 2023.
[15] Xiaoxiao Li, Nicha C Dvornek, Yuan Zhou, Juntang Zhuang, Pamela Ventola, and James Duncan. Graph neural network for interpreting task-fmri biomarkers. In Medical Image Computing and Computer Assisted Intervention-MICCN, 2019: 22nd International Conference, Shenzhen, China, October 13-17, 2019, Proceedings, Part V 22, pages 485-493. Springer, 2019.
[16] Soham Gadgil, Qingyu Zhao, Adolf Pfefferbaum, Edith V Sullivan, Ehsan Adeli, and Kilian M Pohl. Spatio-temporal graph convolution for resting-state fmri analysis. In Medical Image Computing and Computer Assisted Intervention-MICCA. 2020: 23rd International Conference, Lima, Peru, October 4-8, 2020, Proceedings, Part VII 23, pages 528-538. Springer, 2020.
[17] Qimai Li, Zhichao Han, and Xiao-Ming Wu. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI conference artificial intelligence, volume 32, 2018.
[18] Ed Bullmore and Olaf Sporns. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature reviews neuroscience, 10(3):186-19) 2009.
[19] Bernadette CM Van Wijk, Cornelis J Stam, and Andreas Daffertshofer. Comparing brain networks of different size and connectivity density using graph theory. PloS one, 13701, 2010.
[20] Yunsheng Shi, Zhengjie Huang, Shikun Feng, Hui Zhong, Wenjin Wang, and Yu Sun. Masked label prediction: Unified message passing model for supervised classification. arXiv preprint arXiv:2009.03509 2020.
23


Reviews:
[{"review_id": "aYliyeg24G", "reviewer": "Reviewer_vR2R", "summary": "", "questions": "", "limitations": "There are no obvious limitations.", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "- The proposed benchmark is useful for both the neuroscience community (provides novel insights for brain connectivity), as well as the graph learning community (a useful application of graph neural networks).\n- The benchmark is very extensive, the authors test various graph models across multiple tasks. \n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "lWWp0W1bcn", "reviewer": "Reviewer_Zh62", "summary": "", "questions": "", "limitations": "See *Opportunities For Improvement\n", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. They provide a pipeline extracting from neuroimage data to sparse graph data rather than just simple datasets. It can provide more datasets with the development of neuroimage datasets.\n\n2. The provided datasets collect both static and dynamic graphs, which enables novel tasks such as gender/age classification and mental state decoding.\n\n3. The authors provide an extensive exploratory study on the pipeline and provide insightful experimental analysis.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "LHsqvPViAW", "reviewer": "Reviewer_awZz", "summary": "", "questions": "", "limitations": "I do believe the authors could do a better job of explicitly calling out the limitations of their work. I am particularly concerned as to whether there are issues with encoding rich neuroimaging data in a graph (as expressive as they may be). Because the graph defines the structure, there is a real risk of information loss and introduction of bias that I do not see discussed in the current version of the manuscript. Some of this is covered in the supplemental but it would be good to have this in the main body of the text.", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "The authors provide a path for machine learning researchers to offer new tools to the neuroimaging community without delving into the detail of extracting data from fMRI scans. This work takes some of the guess work out of signal extraction, processing, and building the graphs that researchers will ultimately need. Running existing GNNs against this data further provides researchers interested in this field (or graph learning) in general a great starting point. The supplemental material does an excellent job introducing the underlying data.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "ZfrpLjL0iy", "reviewer": "Reviewer_g2aw", "summary": "", "questions": "", "limitations": "I don’t think this work poses any potential negative societal impact. Regarding the limitations listed before, I feel the authors could have obtained more complete datasets from the original HCP data, as discussed before.", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "From my point of view, the main strengths of the paper are:\n\n-\tConsidering the standard size of MRI datasets, the datasets compiled in the current paper have a large sample size, which enables the usage of novel GML methods, such as the ones outlined throughout paper.\n\n-\tThe benchmarking carried out by the authors is quite complete. They have tested many methods on different configurations of the data, in terms of feature and graph dimensionality.\n\n-\tThe authors achieve good results in many tasks, showing the potential of the data and the methods tested.\n\n-\tBesides, the authors present a competitive GNN architecture that proves also quite performing in many of the tasks.", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}, {"review_id": "VPVJIxFSiZ", "reviewer": "Reviewer_4Rsw", "summary": "", "questions": "", "limitations": "1.\t1.In Table -1 nomenclature of all symbols is not mentioned.  \n2.\tFigure-1 is not referred in the explanation.\n3.\tWhat represents edges in the generated graph.  \n4.\twhat are the challenges of the existing study?\n5.\tWhat are the six head motion parameter which are regressed out?\n6.\tThey haven’t mentioned the name of which GNN model performed well for all types of variations.\n7.\tCorrelation as a node features [a,b], and Sparser graph[c] are already used in these papers to get better  GNNs performance-\n a. Li, Xiaoxiao, et al. \"Braingnn: Interpretable brain graph neural network for fmri analysis.\" Medical Image Analysis 74 (2021): 102233.\nb. Ding, Kexin, et al. \"Graph convolutional networks for multi-modality medical imaging: Methods, architectures, and clinical applications.\" arXiv preprint arXiv:2202.08916 (2022).\nc. Pina, Oscar, et al. \"Structural networks for brain age prediction.\" International Conference on Medical Imaging with Deep Learning. PMLR, 2022.\n", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "1. Dealt with both static and dynamic graphs.\n2. Clear conceptual explanation. \n3. References are provided for all the technical explanations.\n", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": ""}]

Author rebuttal:

Meta-review: This submission generated much discussion. The reviewer appreciated the work that contributes an extensive benchmark of Graph Neural Networks for fMRI connectomics.

One suggestion that I (area chair) would have is to add non GNN baselines to the benchmark. Indeed, in a blinded challenge, Traut 2022 [1] found that GNNs underperformed simpler approaches.

[1] "Insights from an autism imaging biomarker challenge: Promises and threats to biomarker discovery", Traut et al 2022, NeuroImage
Decision: Accept (Poster)