Long-range Brain Graph Transformer
Authors: Shuo Yu
Shan Jin
Ming Li
Tabinda Sarwar
Feng Xia
Venue: NeurIPS 2024 poster (2024)
Keywords: long-range dependencies
brain graph transformer
brain disease prediction
graph learning
graph neural networks
Primary area: graph_neural_networks

Abstract: Understanding communication and information processing among brain regions of interest (ROIs) is highly dependent on long-range connectivity, which plays a crucial role in facilitating diverse functional neural integration across the entire brain. However, previous studies generally focused on the short-range dependencies within brain networks while neglecting the long-range dependencies, limiting an integrated understanding of brain-wide communication. To address this limitation, we propose Adaptive Long-range aware TransformER (ALTER), a brain graph transformer to capture long-range dependencies between brain ROIs utilizing biased random walk. Specifically, we present a novel long-range aware strategy to explicitly capture long-range dependencies between brain ROIs. By guiding the walker towards the next hop with higher correlation value, our strategy simulates the real-world brain-wide communication. Furthermore, by employing the transformer framework, ALERT adaptively integrates both short- and long-range dependencies between brain ROIs, enabling an integrated understanding of multi-level communication across the entire brain. Extensive experiments on ABIDE and ADNI datasets demonstrate that ALTER consistently outperforms generalized state-of-the-art graph learning methods (including SAN, Graphormer, GraphTrans, and LRGNN) and other graph learning based brain network analysis methods (including FBNETGEN, BrainNetGNN, BrainGNN, and BrainNETTF) in neurological disease diagnosis.

Long-range Brain Graph Transformer
Shuo Yu Yu!.?, Shan Jin Ming Li Tabinda Sarwar", Feng Xia
School of Computer Science and Technology, Dalian University of Technology, China
2 Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University Technology),
Ministry of Education, China
Sccooo Software, Dalian University of Technology, China
*Zhejiang Institute of Optoelectronics, China ZZhejiang Key Laboratory of Intelligent Education Technology and Application, Zhejiang Normal University. China
School of Computing Technologies, RMIT University, Australia
{shuo. yu, f xieeeieee, . jinshan0924@mail dlut edu. mingli@zjnu. edu. cn
tabinda www.inddassrrarmmmt.ccom www.:rrmittttuu..e.mmmmmommme
Abstract
Understanding communication and information processing among brain regions of interest (ROIs) is highly dependent on long-range connectivity, which plays a crucial role in facilitating diverse functional neural integration across the entire brain. However, previous studies generally focused on the short-range dependen- cies within brain networks while neglecting the long-range dependencies, limiting an integrated understanding of brain-wide communication. To address this limi- tation, we propose Adaptive Long-range aware TransformER (ALTER), brain graph transformer capture long-range dependencies between brain ROIs utilizing biased random walk. Specifically, we present novel long-range aware strategy explicitly capture
dependencies between brain ROIs. By guiding the walker towards the next hop with higher correlation value, our strategy simu- lates the real-world brain-wide communication. Furthermore, by employing the transformer framework, ALERT adaptively integrates both short- and long-range dependencies between brain ROIs, enabling integrated understanding of multi- level communication across the entire brain. Extensive experiments on ABIDE and ADNI datasets demonstrate that ALTER consistently outperforms generalized state-of-the-art graph learning methods (including SAN, Graphormer, GraphTrans, and LRGNN) and other graph learning based brain network analysis methods (including FBNETGEN, BrainNetGNN, BrainGNN, and BrainNETTF) in neuro- logical discase diagnosis. Cases of long-range dependencies also presented further illustrate the effectiveness of ALTER. The implementation is available at https://githun www.tttu....oouuueeeeeeeeeeeeeee wwt.///assoowiii/......... ALTER
Introduction
Brain networks represent blueprint of communication and information processing across different regions of interest (ROIs) 2 The interaction between anatomically connected ROIs within brain networks the foundation of brain network analysis tasks [3]. As shown in Figure D numerous studies have shown that brain networks exhibit not only short-range connectivity (i.e., shnrt- dependencies) but also extensive long-range connectivity (i.e., long-range dependencies) 4 6 7 Short-range dependencies rely the neighbourhood space, whereas long-range dependencies
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

reflect long distance communication among ROIs. Such long-range dependencies play vital role in theoretical analyses of brain function 8 dysfunction (9. organization [[0], dynamics [5]. and evolution [.]. Therefore, it is necessary to capture long-range dependencies within brain networks to better represent communication connectivity and facilitate brain network analysis tasks to
extract valuable insights. The communication connectivity is also known as functional connectivity,
representing the interaction between brain ROIs.
Several existing studies have been devoted to representing communication connectivity within brain networks via graph learning methods for network analysis tasks [12 1 T3]. However, as previously mentioned, they generally focus on the aggregation of neighborhood information, i.e., short-range dependencies, which still limits their effectiveness by neglecting the crucial long-range dependencies. Most of the studies build models to analyze node (ROIs) features and structures (inter- RÔI connectivity) using Graph Neural Networks (GNNs) with message-passingg mechanism. The limited expressiveness of GNNs fails to capture the long-range dependencies in brain networks. While the group- based graph pooling operations cluster ROIs, these are still limited to regional similarities and are not sufficient to represent long-range communication connectivities.
To address this limitation of solely consid- ering the short-range dependencies, we aim to develop a solution that leverages long- range dependencies to enhance brain net- work analysis tasks. Currently, the cap- turing the long-range dependencies has been addressed in different network analy- sis tasks outside the scope of brain studies, such as those related to social networks and molecular networks [16 17 18 19]. Among these, random walk methods are widely adopted, as they can explicitly cap- ture long-range dependencies by aggregat- ing structure information across the entire
Longraan Depadarcia
Human braia
Brain graph
Figure I: An illustration of long- Long-range dependencies and short-range dependencies within human brain.
random walk sequence 20 21 22 23]. In conventional random walk methods, the transition proba- bility from node to one of its neighbors is typically uniform. However, several studies have observed that different pairs of ROIs typically demonstrate varying communication strengths in brain activity, where stronger communication indicates greater dependencies among ROIS (24 [25] 26). Therefore, employing conventional random walk methods to sample the next hop with uniform probability renders it impossible capture the long-range dependencies within brain networks.
Based on the above observation, we are dedicated to capturing long-range dependencies in brain networks under the varying communication strengths among ROIs. In this paper, we propose Adaptive Long-range aware TransformER (ALTER). a brain graph transformer to capture long-range dependencies between brain ROIs extracted from biased random walk. Specifically, in order to capture long-range dependencies within brain networks, we firstly present an Adaptive Long-ranGe Aware (ALGA) strategy based random walk in Section 3.1 which explicitly samples random walk sequences based on varying communication strengths among ROIS. In this strategy, we initially calculate the inter-ROI correlations as adaptive factors to evaluate their communication strengths. Subsequently, the use of random walk is biased, subject to the next hop with a higher correlation value, thus explicitly encoding long-range dependencies long-range embeddings through random walk sampling. Furthermore, given the significance of both short-range and long-range dependencies in brain network analysis tasks, we introduce an effective brain graph transformer in Section [3.2 which can capture different levels of communication connectivities in human brains. Specifically, we inject the long-ran oong-range embeddings into transformer framework and integrate both short-range and long-range dependencies between ROIs using the self-attention mechanism.
The contributions of the paper summarized as follows: significance and challenges of capturing long-range dependencies in brain network analysis tasks, we propose a novel solution for capturing long-range dependencies within brain networks; 2) to address the limitations of previous studies that overlook long-range dependencies within brain networks, we introduce a novel brain graph transformer with adaptive long-range awareness, which leverages the communication strengths between ROIs to guide the capturing of long-range dependencies, enabling an integrated understanding of multi-level communication across the entire brain; 3) extensive
1) pioneering the explicit emphasis on the
2

---

experiments on ABIDE and ADNI datasets demonstrate that ALTER consistently outperforms generalized graph learning methods and other graph learning- -aased brain network analysis methods.
2 Related Work
2.1 Brain Network Analysis
Several studies have developed graph learning-based methods for brain network analysis tasks, such as neurological disease diagnosis and biological sex prediction [2
27 28]. The majority of studies have utilized GNNs to learn the information of ROIs and inter-ROI connectivity, For instance, Li al. [12] utilized GNNs with ROI-aware and ROI-selection to perform community detection while retaining critical nodes. Kan et al. (28) dynamically optimized a learnable brain network. Additionally, few models based on specific graph pooling operations were also proposed to retain the communication information of brain networks. Specifically, Yan et al. [14] designed group-based graph pooling operations to enable explainable brain network analysis. Kan et al. [15] considered the similarity property among brain ROIs and designed a graph pooling function based on clustering. However, these approaches are generally limited to the aggregation of neighborhood information, while neglecting the long-range connectivity that plays key role in brain network analysis tasks [5].
14
13
2.2 Graph Transformer
Several existing studies have focused on developing the transformer variants for graph representa- tion learning [29 30 31 32 B3]. Transformers have demonstrated competitive or even superior performance over GNNs. Dwivedi et al. 29 were the first to extend the transformer to graphs, defining the eigenvectors as positional embeddings. Kreuzer et al. (30 improved the positional embeddings and enhanced the transformer model by learning from the full Laplacian spectrum. Ying et al. 31 embedded the structural information of graph into a transformer, yielding effective results. Moreover, some studies have applied transformers to address unique issues in general graphs or domain-specific graphs. Wu et al. 32) utilized global attention to capture long-range dependencies within general graphs. Tao et al. 33 employed the transformer model to integrate both temporal and spatial information in social networks for disease detection.
3 Method
Within the brain graph, the collection of brain ROIs serves as the node set, and the features of these ROIs serve as the node features. The connectivity among brain ROIs is generally represented by the adjacency matrix. In this paper,
focus analyzing these brain graphs for neurological disease diagnosis. Formally, consider a set of subjects' brain network (Gi...
C 9 and their disease state labels {y1... ... } S YY where L the total number of individuals (size of the dataset). Each brain graph G contains N ROIs, defined as G (V, X, A), where V is node set, X E Nxx are node features with dimension d, and A E N X is an adjacency matrix. We aim to learn a representation vector he that will allow to predict the disease state of brain graph G, ie., yG = f (ha) where is prediction function. Notably, the proposed method can also deal with other brain network analysis tasks such biological sex prediction.
The overall framework of ALTER is illustrated in Figure 2 Briefly, we first extract the node features XG and adjacency matrix AG from the fMRI data. Subsequently, adaptive factors F& are calculated using the temporal features of the fMRI data. Next, using the adaptive factors FG and the adjacency matrix Ac. the long-range embedding EG representing the long-range dependencies is obtained through adaptive long-range encoding. The encoding is utilized by our Adaptive Long- ranGe A Aware (ALGA) strategy to explicitly encapsulate the long-range dependencies among ROIs as long-range embedding Eg. Finally, the long-range embedding EG is injected into the self-attention module, and graph-level representation of the brain network is generated using the readout function to the downstream tasks. The complete training process is supervised by the cross-entropy loss.
Detailed description of these steps discussed in the upcoming sections.
3

---

ROIHT M
Adaptive Long-range Aware Transformer
Calculatic Adaptive Factors
Downstreum tasks
RO112 M
ROINN M
Long-tange Dependency
Linear
CURT
0A4
Concatienale
X
0:5
hop
Scaled Dot-Product
Adaptive Long- range Encoding
Attention
A
K. hopp
Linear
Lancar
Linear
Eg
Injecting
Figure 2: The overall framework of the proposed ALTER.
3.1 Adaptive Long-range Aware (ALGA) Strategy
As previously mentioned, previous studies primarily focus on aggregating information from neigh- boring ROIs, generally neglecting their long-range connectivity. In light of this, goal is to design an efficient strategy to capture long-range dependencies among the ROIs. We now explain how the adaptive long-range
strategy achieves this through the computation of adaptive factors and the adaptive long-range encoding.
3.1.1 Adaptive Factors
The correlation between ROIs reflects their communication strength within specific time frames, which crucial for comprehending the functional organization, dysfunctions, and information propagation in the brain 2 34 Neuroscientific investigations have unveiled the existence of phase
synchrony neural oscillations among distinet ROIs, closely associated with the occurrence of perceptual motor behaviors and the integration of brain functional organization [24, 25 26). The degree phase synchrony indicates the strength of connectivity between ROIs, whereby higher phase synchrony signifies stronger communication and consequently higher correlation values among the respective ROIs. This manner simulates real-world brain-wide communication.
Considering the above observation, we first calculate the correlation between ROIs adaptive factors FG E RNN N evaluate the communication strengths between ROIs. Specifically, the adaptive factor fij. denoting the communication strength between node and node j within the brain graph G, defined as:
Cov(badd a(J)(())))))
if 0; and vj are connected,
füj
1,
fii=j. otherwise,
0,
where tu denotes the raw feature of the brain ROI U. (e.g., temporal feature of fMRI data). Cov (-) and (-) denote covariance and variance operations, respectively. The adaptive factors FG will influence the exploration mechanism of the random walk in the adaptive long-range encoding (Section [.1.2). Specifically, ROIs with stronger connectivities exhibit higher transfer probabilities of the next hop compared to ROIs with weaker connectivities in the random walk. Note that for simplicity, we choose Pearson correlation coefficient define the adaptive factors, without any modification.
3.1.2
Adaptive Long-range Encoding
The adaptive factors between ROIs, computed in the preceding step, are crucial for effectively capturing long-range dependencies between brain ROIs. Motivated by the random walk methods [16]
17 35), we approach the problem of capturing long-range dependencies as structural encoding task. By sampling and encoding node sequences into embeddings, effectively capture long-range

---

communication between brain ROIs. Hence. we compute long-range embedding using the walker sampling node sequences under the constraint of adaptive factors.
In a network, a random walk involves transitioning from one node to another. Specifically, when a walker moves from node i, the probability of the walker moving to node in the next step depends solely on the conditions of nodes i and j. This characteristic, where the probability of reaching node j is independent of the preceding step node defines Markov process. Thus, the random walk process inherently embodies Markovian nature.
Let Piş represent the probability that the walker walks from node i to node j in brain graph G, then Pij can be represented in the following matrix form:
P11
Pin
Pa
0 10SpO Pa 11, Ev= Pij
Poj=, 11
...
Pn1
Pun
where matrix Pe is the transfer matrix of brain graph G. Then the state vector is defined as:
T (k)) = (t, (R)) .2 (k) ,....,, tu (A)) Σ 1o (k)==)
(k: 1,
where & is denoted as the number of hops in random walks. ty (A) is the probability of the walker stops at node j after & times walk. tg (k) is called & steps state probability. According the total probability formula, we get:
(ks + Σ ty (k) 0
where K represents the total number of hops in random walk. So, we get the general recursive formula as T' (k) = T (0) Pa. Nevertheless, brain networks deviate from general networks as pairwise ROIs typically exhibit distinct communication strengths, indicative of the collaborative nature of ROIs in brain activity. Falsely treating pairs of ROIs with varying communication strengths equally may disrupt the collaborative dynamics within brain activity. Given this fact, the transfer probability of the walker in brain network can be fine-tuned using the adaptive factors, i.e., Pg = FG @ Pa. where denotes the dot product operation. Formally, considering the adjacency matrix AG and the corresponding diagonal degree matrix DG of a brain graph G, along with the obtained adaptive factors FG, we define the random walk kernel R for adaptive long-range encoding follows:
R (Fa @ Ag) D
(5)
In particular, The introduction of degree matrix help to obtain richer information about brain-wide communication and is very commonly used network analytics [36]. Since the degree matrix provides the number of degrees for each ROI, its ability to reflect the active state of the ROI in communication important in determining which ROIs play a key role in information propagation. Hence, degree matrix determines the transfer probability of node its neighboring nodes and highly influences the behavior walker (37]
In the K Kteppp random walk, the long-range embedding Eg initialized by the adaptive long-range encoding is defined as:
e = R, R2, RK E R
where denotes the identity matrix. e; denotes a long-range embedding associated with the i-th node, encapsulating the Long-range dependency asscoiated with i-th node. Through adaptive long-range range encoding, we can explicitly capture long-rang dependencies among ROIs in the brain graph G and encode them into the form of long-range embeddings Eg-
3.2 Long-range Brain Graph Transformer
In Section 3.1.2 we obtained the long-range embedding EG that explicitly encode the long-range connectivities within the brain network G. As short-range dependencies are also significant, we aim to present an effective brain graph transformer by first injecting long-ram oaggrange embeddings into brain network representation learning and then integrate both long-range and short-range dependencies for learning a more comprehensive representation. To achieve this objective, we begin by describing the process of injecting long-range embeddings EG into the brain graph transformer. Later, we explain how the self-attention mechanism can utilized to integrate long-range and short-range dependencies among brain ROIs
5

---

Injecting Long-range Embedding. The computed long-range embeddings Eg should be injected into the brain network transformer in a manner that enhances its utility. To acheive this, we introduce a fine-tuning procedure aimed at enhancing long-range embeddings FG and injecting them into the brain graph transformer. Specifically, we utilize linear layer as a remapping function for long-range embeddings Eg, facilitating the injection of long-range dependencies within the brain network. This process enables the acquisition of trainable long-range embeddings Eg with dimension k'. Formally, this procedure is defined as:
EG LL(E:; Wc) = WgEG + be ERNXN
where Wo E R xh and be E R denote learnable weight matrix and bias vector, respectively.
Self-attention Module. Transformer-baset models generally surpass conventional representation learning methods in their ability to capture pairwise token correlations and the influence of individual tokens. This stems from the self-attention mechanism's 's capability allow inter-token communication. Nonetheless, employing initial node features as input tokens insufficient for Transformer- based models effectively capture complex inter-dependencces within brain networks. Furthermore, pairwise ROIs often exhibit varying degrees of short- and long-range dependencies across various Hence, we need integrate both long-range and short-range communication among ROIs through self-attention module. To model this mechanism, we begin by constructing tokens through the combination of learnable long-range embeddings EG and initial node features Xg. Then, we utilize vanilla transformer encoder as the framework for the self-attention module.
brain network analysis tasks [38
39
40
Formally, we concatenate learnable long-range embeddings EG and initial node features Xa as tokens Xg. and then utilize transformer encoder with L-layer nonlinear mapping and M attention head to learn comprehensive node features Zg:
xa=
[Xal Ea] E Nx(++")
(8)
Qm Kmm.T' douu
Za Wo 11m=:2a RR Nxdoue 2ma softmax
Vmd R Nx
with Qm, = Wg W2Zeell " Kmm.. =( (Wizza and Vmd W.22 are the query matrix, the key matrix, and the value matrix, where Zy Na. || and [-|-] both indicate the concentrate operation, L and m denote the layer index and the head index, Wg. WR. W. E d xam.!-> and W. E Po xd"u. are learnable projection matrices. In the representation learning procedure, the employed Transformer framework enables the learned Zg to integrate both short-range and long- range dependencies between brain ROIs by introducing long-range embedding. This design allows our method to adaptively represent the communication connectivities human brains.
Readout Module. To accomplish brain network analysis tasks, we take the output Zo of the self-attention module as the criterion, and then utilize an efficient readout function to derive the entire brain graph representation to further enhance the performance. In addition, we train an additional classifier for downstream tasks. The final classification basis is obtained as follows:
Ye = Softmax (MLP (Readout (Zc)))
(10)
In Section 4.2 we evaluate the performance of various pooling methods. Ultimately, we employ clustering-basee pooling the readout function in the proposed method.
4 Experiments
In this section, we analyzed the following aspects to demonstrate the effectiveness of the proposed method and its capability to capture long-range dependencies within brain networks.
Q1. Does ALTER outperform other state-of-the-art models?

---

Q2. How does the proposed adaptive long-range aware strategy perform in different model architec- tures accompanied by various readout functions?
Q3. Does ALTER capture long-range dependencies within brain networks, and ALGA strategy considered a key component?
4.1 Experimental Settings
Datasets and Preprocessing. We evaluate the proposed method using two brain network analysis- related fMRI datasets. 1) Autism Brain Imaging Data Exchange (ABIDE I which contains 519 Autism spectrum disorder (ASD) samples and 493 normal controls. 2) Alzheimer's Disease Neu- roimaging Initiative (ADNI which contains 54 Alzheimer's disease (AD) samples and 76 normal controls. During the construction of the brain graph, we first preprocess the fMRI data using the Data Processing Assistant for Resting-State Function (DPARSF) MRI toolkit. Next, we define brain ROIs based on predefined atlases from preprocessed fMRI data and calculate the average time-series feature for individual brain ROI. Finally, we formalize the brain graph sample according to the average time-series features of brain ROIs. Specifically, node features X are functional connectivity matrix calculated by Pearson correlation, the adjacency matrix the thresholded functional connectivity matrix to generate binary matrix of Os or 1s, where the threshold is 0.3. The details of datasets and preprocessing can be found in Appendix A
XV, A) for each
Baselines. The selected baselines correspond two categories. The first category is general- ized graph learning methods (Generalized not specifies to brain networks), including SAN (30] Graphormer [31] GraphTrans [32], and LRGNN 42). The second category (Specialized) is the
brain graph-basee based methods including BrainNetGNNN [15]. FBNETGEN (28), BrainGNN [12]. BrainNETTF [TS]. A-GCL [43] and ContrastPool 44 Note that the original code shared by the
authors of these baselines is used for the comparative analysis. Please refer to the Appendix A for the
details.
Metrics. Given the medical application of neurological disease classification tasks, utilize both machine learning and medical diagnostic-specifie metrics to evaluate the performance of the proposed method. These include classification Accuracy (ACC), Area Under the Receiver Operating Characteristic Curve (AUC), FI-Score, Sensitivity (SEN), and Specificity (SPE). In the experimental results, report the mean and standard deviation across 10 random runs on the test dataset.
Implementation Details. In the proposed method, set the number of steps K for adaptive random walk to 16. The number of nonlinear mapping layers and attention heads M of the self-attention module are set to 2 and 4, respectively. For all datasets, we randomly divide the training set, evaluation set and test set by the ratio of 7 2. In the train processing, we adopt Adam as optimizer and CosLR scheduler by initial learning rate of 10-4 and weight decay of 10 10-4. The batch size is set to 16 and the epoch is set to 200. All experiments are implemented using the PyTorch framework, and computations are performed on one Tesla V100.
4.2 Performance Comparison
In this sections, we evaluate the performance of ALTER by comparison with existing baselines to address Q1.
Results. T reports the comparison results between the proposed and the baseline methods. ALTER is able to significantly outperform the two categories baseline methods for both datasets. In comparison generalized graph learning methods, the proposed ALTER exhibits a significant improvement in terms of the ACC metric (10.9% improvement on the ABIDE dataset and 6.8% improvement the ADNI dataset). For the case specialized graph learning methods, demonstrated superiority on both datasets in terms of the ACC metric (6.0% improvement on the ABIDE dataset and 1% improvement on the ADNI dataset). The reason for this performance improvement is that our method takes into account the communication strengths among brain ROIs
again
https://adni.ii wow..u.....d/

---

Table 1: Performance comparison with two categories of baselines on the two chosen datasets (%).
The best results are marked in bold and the standard deviations are in parentheses.
ABIDE
ADNI
Category
Method
ACC
AUC
ACC
SEN
SPE
SAN
(2.3 72.3
55.4
68.3(7.5)
68.1
62.6(5.2)
(2.4
(8.5)
Graphormer
(3.7
78.7 (2.77
(23.5
(5.6)
(3.7(3.1)
(0.1()13)
(13.5)
Generalized
GraphTrams
60.1 (6.7)
(7.8(47))
(10.3)
(11.5)
(3.7
(5.3
(7.2
(3.3
LRGNN
(4.3
66.1 (2.5)
(8.4
(6.8
(6.4)
(7.3
59.6(1.2)
(2.3)
FBNETGEN
75.6(1.2)
(8.0
(8.7)
(2.4
(3.5
65.0(2.6)
(2.3
(1.2
BrainNesGNNN
(1.3
(5.4
(3.9 (3775)
(011 (7.2
(6.2
(8.0)
BrainGNN
71.6(1.6)
(3.2)
(5.4
(3.4
(2.5)
61.5(3.2)
(3.4)
(4.5
Specialized
BraiNNETTF
80.2(1.0)
71.((.2)
(5.2
(9.3
(2.4)
69.0(2.7)
(7.7(
(8.0)
A-GCT.
(0.5
(0.6)
(5.0)
(6.5
(1.1
(2.2.0.8)
57.6(42.4)
(38.2)
ContrastPool
(0.3
(7.4
57.6(6.8)
57.0
(9.2 (3.5
(175()).2)
(2.42
Ours
ALTER
77.0 (1.1
(3.4
76.6 (4.6)
78.8 (2.1)
(6.5 (2.5)
(6.0
and utilizes this characteristic to guide the capture of long-range dependencies within the brain network.
Vairous Readout Function. In the experimental setups with and without the ALGA strategy, we compare the results of ALTER using various readout functions, including max pooling, sum pooling, average pooling, sort pooling [45,, and clustering- based pooling [13]. As illustrated in Table 2 and Figure Ba), method employing the ALGA strategy consistently achieved superior performance across all readout function settings. Particularly, the combination of clustering haaseerigg-baed pooling and the ALGA strategy yielded the best results. This phenomenon also addresses Q2
4.3 Ablation Study
In order to assess the performance of the model from the Q2 perspective, we conduct ablation studies on the ALGA strategy. This included performing evaluations with different architectures and readout functions.
Adaptive Long-range Aware with Varying Architectures. To verify the generalisability and effectiveness the ALGA strategy, we implement it within different architectures, including SAN and Graphormer on the two selected datasets. As shown in Table 2] we find that the AL.GA strategy can be adapted to different architectures, where this adaptation effectively improves the predictive power of models. This result demonstrates the effectiveness of this strategy in capturing long-range dependencies within brain networks. It should be noted that this analysis could only be performed on the generalized graph learning methods.
Table 2: Performance comparison with varying architectures on the two chosen datasets (%). The best results indicated by underlining and the standard deviations in parentheses.
ABIDE
ADNI
Method
AUC
ACC
SEN
SPF
AUC
ACC
SEN
SPF
Graphormer
(3.7)
(2.7)
(22.3)
36.7 (23.5)
(5.6)
(3.1
(11.3)
(7.7(13.5)
Graphomer++AAA. ALGG
67.2 (2.5)
64.1 (1.9)
(1.3)
45.9 (12.7)
(4.1)
(2.9)
(4.5
(2.4
SAN
71.3
(2.3)
(9.2)
68.3
(3.4)
(2.6
(2.())2)
(8.3
SAN +ALGG
(3.8) (1.5
(6.9
(4.1)
(2.3)
(3.7
(4.9
68.3 (6.2)
ALTER ALGA
(0.00
72.5( (1.0
(6.5)
(6.5
(2.7)
(7.1
75.0 (8.1)
ALTER
(L.8
(1.0)
(3.4
76.6 (4.6)
(2.1)
74.1 (2.5)
76.5 (6.1)
(0.5)
Adaptive Long-range Aware with Varying Readout Functions. To further demonstrate that the ALGA strategy plays a key role in the proposed method, we take the ABIDE dataset as a benchmark and attempt to vary the readout function under varying architectures (only the AUC metric is shown, the full result can be referred to the Appendix A). Specifically, we first take the proposed method, SAN, and Graphormer as the basic framework, then employ five approaches including max pooling, sum pooling, average pooling, sort pooling, and clustering- based pooling as the readout functions. This analysis will reveal the prediction ability of the frameworks with varying readout functions in the presence and absence of ALGA strategy. The results of different readout functions under various
8

---

architectures are shown in Figure 3 strategy enhances the performance of downstream tasks for various architectures compared those without ALGA.
We observe that, for any arbitrary readout function, the ALGA
ARALBA
MLAA
MEG
AUC/N)
AUC1%
"AVEMAGE
Graphormer
Figure 3: Performance comparison with varying readout functions (%)
4.4 In-depth Analysis of ALTER and ALGA Strategy
Here, we delve into the analysis of ALGA strategy and present cases to evaluate long-range depen- dencies to demonstrate the effectiveness of ALTER assess Q3.
First, we investigate the impact of key hyperparameter of ALGA strategy of ALTER, which the number of hops. In this experiment, we set the number of hops of ALGA to 2, 4, 8, 16, 32 for both selected datasets (only the AUC is shown here, the full result can be found in Appendix(A) Figure clearly shows that for both datasets, the predictive power of the proposed method generally increases as the number of hops increases. This phenomenon can be attributed to the presence of long- gang-range connectivity in communication and information processing within human brains, which our model effectively captures. Ignoring this characteristic will adversely affect the predictive power of graph learning methods in brain network analysis tasks.
Next, we investigate the impact of the adaptive factors in the ALGA strategy. Specifically, remove the adaptive factors and observe the change in the predictive ability of the proposed method. The experimental results demonstrate (Figure (a)) the performance of the proposed method is degraded when adaptive factors are not used to adjust the adaptive long-range encoding. The underlying reason for this result is that ROI correlations play crucial role in reflecting the communication strengths among brain ROIs Treating pairwise ROI connectivity equally could potentially have a detrimental effect on brain network analysis tasks that depend on inter-ROI communication.
Finally, we present the cases to demonstrate ALTER's s ability to capture long-range dependencies within brain networks. In this experiment, we randomly sample example brain graph from the ABIDE test set and used it to train our model to learn the corresponding node features without pooling operation, and thus compute the attention scores among the node features. Figure ((c) illustrates one example graph and the corresponding attention heatmap (Figure ())) More sample-level and group- level examples can be found Appendix A The attention heatmap demonstrates the communication patterns necessary for brain network analysis tasks. Specifically, certain ROIs receive higher attention scores from multiple other ROIs, irrespective of the distance between them. In particular, ROI and ROI 19 present higher attention score, despite the fact that these two ROIs are 5 hops apart (Figure ()).
5 Diseussions and Conclusion
Limitations. On one hand, while utilizing the brain graph transformer to integrate both short-range and long-range dependencies among brain ROIs, we still cannot ensure an optimal balance between them. In future research, we will explore how to achieve a better balance between short-range and long-range dependencies in brain network analysis, with the aim of achieving better research results. On the other hand, the experimental data we currently employ is limited to fMRI data. Although utilizing these data can demonstrate the crucial role of long-range dependencies in brain network analysis tasks, other forms of data, such as DTT data, are also worthy of exploration. In future work, we will delve into alternative forms of data and propose corresponding methods for capturing long-range dependencies within brain networks.

---

AUC(())
65
-0- MEEE
ADNUOR
Hops
(n) Influence of hops and adaptive factors
(b) Attention map
(c) Example graph
Figure 4: In-depth analysis ALTER and adaptive long-range aware strategy.
Conclusion. In summary, we present the ALTER model for brain network analysis, novel brain graph transformer that explicitly captures long-range dependencies in brain networks and adaptively integrates them with short-range dependencies. Extensive experiments on ABIDE and ADNI datasets demonstrate that ALTTE consistently outperforms generalized and specialized (specific brain network analysis method) graph learning methods. This study presents an initial attempt to capture long-distance dependencies within brain networks and provides a new insight into understanding brain-wide communication and information processing.
Acknowledgments and Disclosure of Funding
This work is supported by the National Natural Science Foundation of China (No. NN.U21A20473 No. 62172370) and the Jinhua Science and Technology Plan (No. 2023-3-003a). Thanks to Guangqing Bai and Yuelong Huang for their help with baselines during rebuttal.
References
[1] Farnaz Zamani Esfahlani, Joshua Faskowitz, Jonah Slack, Bratislav Misić, and Richard F. Betzel. Local structure-funcciion relationships in human brain networks across the lifespan. Nature Communications, 13(1):2053, 2022.
[2] Caio Seguin, Olaf Sporns, and Andrew Zalesky. Brain network communication: concepts, models and applications. Nature Reviews Neuroscience, 24(9):557-574, 2023.
[3] Danyal Akarca, Petra E. Vértes, Edward T. Bullmore, Kate Baker, and Susan E. Gathercole. generative network model of neurodevelopmental diversity in structural brain organization. Nature Communications, 12(1):4216, 2021.
[4] Dharmendra S. Modha and Raghavendra Singh. Network architecture of the long-distance path- ways in the macaque brain. Proceedings of the National Academy of Sciences, 107(30):13488 13490, 2010.
[5] G. Amulfo, S. H. Wang, V. Myrov, B. Toselli, J. Hirvonen, M. M. Fato, I.. Nobili, F. Cardinale, A. Rubino, A. Zhigalov, S. Palva, and J. M. Palva. Long-range phase synchronization of high-frequency oscillations human cortex. Nature Communications, 11(1):5363, 2020.
[6] Yuki Miura, Min-Yin Yin Li, Omer Revah, Se-Jin Yoon, Genta Narazaki, and Sergiu Paşea. Engineering brain assembloids to interrogate human neural circuits. Nature Protocols, 17(1):15- 35, 2022.
[7] Ming Jin, Changde Du, Huiguang He, Ting Cai, and Jinpeng Li. Pgen: Pyramidal graph convolutional network for eeg emotion recognition. IEEE Transactions on Multimedia, pages 1-13, 2024.
[8] Gustavo Deco, Yonathan Sanz Perl, Peter Vuust, Enzo Tagliazucchi, Henry Kennedy, and Morten Kringelbach. Rare long-range cortical connections enhance human information processing. Current 3120))4333200
2021.
10

---

[9] Anthony W. Harrington and David D. Ginty. Long long-iitance retrograde neurotrophic factor signalling in neurons. Nature Reviews Neuroscience, 14(3):177-177, 2013.
[10] Junhua Li, Nitish Thakor, and Anastasios Bezerianos. Brain functional connectivity in uncon- strained walking with and without an exoskeleton. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 28(3):730-739, 2020.
[11] Annalisa Paolino, Laura R. Fenlon, Peter Kozulin, Elizabeth Haines, Jonathan W. C. Lim, Linda J. Richards, and Rodrigo Suárez. Differential timing of a conserved transcriptional network underlies divergent cortical projection routes across mammalian brain evolution. Pro- ceedings of the National Academy of Sciences, 11sess,,1155 117(19)1055440055: 2020.
[12] Xiaoxiao Li, Yuan Zhou, Nicha Dvorek, Muhan Zhang. Siyuan Gao, Juntang Zhuang. Dustin Scheinost, Lawrence H. Staib, and James S. Duncan. BrainGNN; Interpretable brain graph neural network for IMRI analysis. Medical image analysis, 74: 10223,, 2021.
[13] Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, and Carl Yang. BrainGB: benchmark for brain network analysis with graph neural networks. IEEE Transactions Medical Imaging, 42(2):493-506 2023.
[14] Yujum Yan, Jiong Zhu, Marlena Duda, Eric Solarz, Chandra Sripada, and Danai Koutra. GroupINN: Grouping-basee based interpretable neural network for classification of limited, noisy brain data. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 772-782, 2019.
[15] Xuan Kan, Wei Dai, Hejie Cui, Zilong Zhang, Ying Guo, and Carl Yang. Brain network transformer. In Proceedings of the 36th Neural Information Processing Systems, pages 25586- 25599, 2022.
[16] Xiao Huang, Qingquan Song, Yuening Li, and Xia Hu. Graph recurrent networks with at- tributed random walks. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 732-740, 2019.
[17] Johannes Gasteiger, Chandan Yeshwanth, and Stephan Günnemann. Directional message passing on molecular graphs via synthetic coordinates. In Proceedings of the 35th Neural Information Processing Systems, volume 34, pages 15421-15433, 2021.
[18] Cai Zhou, Xiyuan Wang, and Muhan Zhang. Facilitating graph neural networks with random walk on simplicial complexes. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Proceedings of the 37th Neural Information Processing Systems, pages 16172-16206, 2023.
[19] Fei Ma, Ping Wang, and Xudong Luo. Random walks on stochastic generalized vicsek fractal networks: Analytic solution and simulations. IEEE Transactions on Network Science and Engineering, (135:1335-1344 2022.
[20] Giannis Nikolentzos and Michalis Vazirgiannis. Random walk graph neural networks. In Proceedings of the 34th Neural Information Processing Systems, pages 1621 1-622,, 2020.
[21] Feng Xia, Ke Sun, Shuo Yu, Abdul Aziz, Liangtian Wan, Shirui Pan, and Huan Liu. Graph learning: A survey. IEEE Transactions on Artificial Intelligence, 2(2):109-127, 2021.
[22] Pei-Kai Yeh, Hsi Chen, and Ming-Syan Chen. Random walk conformer: Learning graph representation from long and short range. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 10936-10944, 2023.
[23] Allan Jabri, Andrew Owens, and Alexei Efros. Space-time correspondence contrastive random walk. Proceedings of the 34th Neural Information Processing Systems, pages 19545-19560, 2020.
[24] Francisco Varela, Jean-Philippe Lachaux, Eugenio Rodriguez, and Jacques Martinerie. The brainweb: Phase synchronization and large-scale integration. Nature Reviews Neuroscience, 2(4):229-23) 2001.
11

---

[25] Peter J Uhlhaas and Wolf Singer. Neural synchrony in brain disorders: Relevance for cognitive dysfunctions and pathophysiology. Neuron, 52(1):155-168, 2006.
[26] Antoine Lutz, Jean-Philippe Lachaux, Jacques Martinerie, and Francisco J. Varela. Guiding the study of brain dynamics by using first-person data: Synchrony patterns correlate with ongoing conscious states during simple visual task. Proceedings of the National Academy of Sciences, 99(3):188-1551, 2002.
[27] Byung-Hoon Kim, Jong Chul Ye, and Jae-Jin Kim. Learning dynamic graph representation of brain connectome with spatio-temporal attention. In Proceedings of the 35th Neural Information Processing Systems, pages 4314-4327, 2021.
[28] Xuan Kan, Hejie Cui, Joshua Lukemire, Ying Guo, and Carl Yang. FBNETGEN: Task-aware gnn-based fMRI analysis via functional brain network generation. In Proceedings of the International Conference on Medical Imaging with Deep Learning, pages 618 (188667, 2022.
[29] Vijay Prakash Dwivedi and Xavier Bresson. A generalization of transformer networks to graphs. In Proceedings of the AAAI Workshop on Deep Learning on Graphs: Methods and Applications, pages 1-8, 2021.
[30] Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. In Proceedings of the 35th Neural Information Processing Systems, pages 21618-21622 2021.
[31] Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Tie-Yan Yan Liu. Do transformers really perform badly for graph representation? In Proceedings of the 35th Neural Information Processing Systems, pages 28877-28888, 2021.
[32] Zhanghao Wu, Paras Jain, Matthew Wright, Azalia Mirhoseini, Joseph E Gonzalez, and Ion Stoica. Representing long-range context for graph neural networks with global attention. In Proceedings of the 35th Neural Information Processing Systems, pages 13266-13277 2021.
[33] Yongfeng Tao, Minqiang Yang, Huiru Li, Yushan Wu, and Bin Hu. Depmstat: Multimodal spatio- temporal attentional transformer for depression detection. IEEE Transactions on Knowledge and Data Engineering, pages 1-12, 2024.
[34] Juergen Fell and Nikolai Axmacher. The role of phase synchronization in memory processes. Nature Reviews Neuroscience, 12(2):105- 118, 2011.
[35] Feng Xia, Jiaying Liu, Hansong Nie, Yonghao Fu, Liangtian Wan, and Xiangjie Kong. Random walks: A review of algorithms and applications. IEEE Transactions on Emerging Topics in Computational Intelligence, 4(2):95-101) 2020.
[36] Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph neural networks with learnable structural and positional representations. In Proceedings of the 10th International Conference on Learning Representations, 2022.
[37] Suman K. Bera and C. Seshadhri. How to count triangles, without seeing the whole graph. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, page 306-316, 2020.
[38] Júlia S. Pinho, Vincent Cunliffe, Kyriacos Kareklas, Giovanni Petri, and Rui F. Oliveira. Social and asocial learning in zebrafish are encoded by shared brain network that differentially modulated by local activation. Communications Biology, 6(1):633, 2023.
[39] Dheeraj S. Roy. Young-Gyun Park, Minyoung E. Kim, Ying Zhang, Sachie K. Ogawa, Nicholas DiNapoli, Xinyi Gu, Jae H. Cho, Heejin Choi, Lee Kamentsky. Jared Martin, Olivia Mosto, Tomomi Aida, Kwanghun Chung, and Susumu Tonegawa. Brain-wide mapping reveals that engrams for single memory are distributed across multiple brain regions. Nature Communica- tions, 13(1):1799) 199, 2022.
[40] Oana Bucsea, Mohammed Rupawala, Ilana Shiff, Xiaogang Wang, Judith Meek, Maria Fitzger- ald, Lorenzo Fabrizi, Rebecca Pillai Riddell, and Laura Jones. Clinical thresholds in pain-related facial activity linked differences in cortical network activation in neonates. PAIN, 164(5), 2023.
12

---

[41] Weiyi Chen, Oliver Mehikop. Alexandra Scharn, Hendrik Nolte, Paul Klemm, Sinika Henschke, Lukas Steuernagel, Tamara Botelo-Hitschiee Ecem Kaya, Claudia Maria Wunderlich, Thomas Langer, Natalia I.. Kononenko, Giavalisco Patrick, and Jens Claus Brüning. Nutrient-sensing agrp neurons relay control of liver autophagy during energy deprivation. Cell Metabolism, 35(5):786-80642 1077666-006113.... 2023.
[42] Lanning Wei, Zhiqiang He, Huan Zhao, and Quanming Yao. Search to capture long-range dependency with stacking GNNs for graph classification. In Proceedings of the ACM Web Conference 2023, page 588-598, 2023.
[43] Shengjie Zhang, Xiang Chen, Xin Shen, Bohan Ren, Ziqi Yu, Haibo Yang. Xi Jiang, Dinggang Shen, Yuan Zhou, and Xiao- Yong Zhang. A-GCL: Adversarial graph contrastive learning for fmri analysis to diagnose neurodevelopmental disorders. Medical Image Analysis, 90:102932, 2023.
[44] Jiaxing Xu, Qingtian Bian, Xinhang Li, Aihu Zhang, Yiping Ke, Miao Qiao, Wei Zhang, Wei Khang Jeremy Sim, and Balázs Gulyás. Contrastive graph pooling for explainable classification of brain networks. IEEE Transactions on Medical Imaging, pages 1-14, 2024.
[45] Muhan Zhang, Zhicheng Cui, Marion Neumann, and Yixin Chen. An end-to-end deep learning architecture for graph classification. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 4438 4445, 2018.
[46] T. Sarwar, Y. Tian, B.T.T. Yeo, K. Ramamohanarao, and A. Zalesky. diructureffunctioo coupling in the human connectome: A machine learning approach. Neurolmage, 226: 226:1176600 17609, 2021.
[47] Shangran Qiu, Matthew I. Miller, Prajakta S. Joshi, Joyce C. Lee, Chonghua Xue, Yunruo Ni, Yuwei Wang, Ileana De Anda-Duran, Phillip H. Hwang, Justin A. Cramer, Brigid C. Dwyer, Honglin Hao, Michelle C. Kaku, and et al. Multimodal deep learning for alzheimer's 's disease dementia assessment. Nature Communications, 13(1):3404, 2022.
13

---

A Additional Experiments
Due to page limitations, only the experimenta results for the AUC metric are included in the main text. For a more comprehensive study, the full results of the comparison experiments are provided in this appendix.
Datasets. We preprocess the IMRI using the Data Processing Assistant for Resting-Statt Function (DPARSF) MRI toolkit. Specifically, we removed the first 10 time points from the downloaded nii data according to the default mode and chose slice timing. where the middle layer was the reference slice. meanwhile, we set the head motion correction to 'Friston 24', and selected automask and Nuisance covariates regression. the others were basically set according to the default mode, Then, considering individual differences, we choose to perform 'Normalize by DARTEL', and for the definition of ROIs, we adopt the altas already available DPARSF. Finally, we construct brain networks for each fMRI. For parcellation, we utilize the Craddock 200 atlas, which defines 200 ROIs, for the ABIDE dataset. For the ADNI dataset, we apply the AAL atlas, comprising 90 cortical ROIs and 26 cerebellar ROIs
Baselines. The selected baselines correspond to two categories. The first category is generalized graph learning methods, including SAN [30], Graphormer [31], GraphTrans (32]. and LRGNN [42]. SAN and Graphormer are two more popular Transformer- based graph learning methods. GraphTrans capable of capturing long-range dependencies within general graphs. LRGNN combines neural architecture search to extract the long-range dependencies. The second category is the brain graph- based methods, including BrainNetGNN [TS], FBNETGEN (28) BrainGNN [2). BrainNETTF [55.
A-GCL 43). and ContrastPool [44) BrainNerGNN utilized ritention-bbsed GNN to learn the rep- resentation of brain networks. FBNETGEN employed task-aware GNN, and BrainGNN adopted GNNs with ROI-aware and ROI-selection. BrainNETTF modeled specific Transformer and readout function for brain network analysis. A-GCL utilizes adversarial graph contrastive learning to extract invariant features from brain networks. ContrastPool is capable of generating task-relevant, inter- pretable brain network representations. Although these methods show advantages in brain network analysis tasks, none of them analyzed and captured long-range dependencies within brain networks.
To ensure a fair comparison, we use the open-source codes of BrainGNN [12,, NETTE [33. FB- NETGEN (28 A-GCL [43] and ContrastPool 44. For SAN B30 Graphormer 31, LRGNN 42
and GraphTrans (32,, we adapt their open-source codes and modify them to suit the brain network datasets. For BrainNerGNN,, we implemente it ourselves following the settings described in the paper. During the parameter tuning. we follow the tuning BrainNETTF [T5 for SAN, BrainGNN, FBNETGEN, Graphormer, and BrainNETTE. For BrainNe:GNN, we search the number of GRU layers 1, 2, 3. For LRGNN, we vary the aggregation operations 8, 12 with the number of cell 1, 3. search the number of GNN layers 1, 2, 3, 4 with the hidden dimension of 100.
For Graph Trans, Regarding the construction of brain graphs for these baselines, we utilized functional connectivity matrix to compute brain graph for NETTT, which is computed by calculating the correlation between brain ROIs using the processed fMRI. The details of computing these correlation matrices is also incorporated to the revised paper. For BrainNetGNN and FBNETGEN, the models required the processed fMRI as input. ContrastPool, A-GCL, BrainGNN, SAN, Graphormer, LRGNN, and Graph Trans required the correlation matrix and adjacency matrix. As mentioned in the paper, the adjacency matrix is obtained by thresholding (2 0.3) the correlation matrix.
Adaptive Long-range Aware with Varying Readout Functions. The full results of the ablation study on the three frameworks, Vanilla Graphormer, and SAN, are presented in Table 3 4 and Based on the results from the three tables, it is evident that, with the aid of the ALGA strategy, each framework employing various readout functions demonstrates superior performance across AUC. ACC, and SPE metrics. Furthermore. we observe that each framework based on sesstring-based pooling exhibits significantly lower standard deviation in performance metrics when utilizing the ALGA strategy compared to those without it.
The Impact of Hops and Adaptive Factors. In Table 6 and 7 we present the results of four metrics under different hops and with/without adaptive factors. Based on the results from the both tables, we observe that the proposed method typically achieves better performance on AUC and ACC metrics when using adaptive factors. Besides, observe that, across both datascts, the proposed
14

---

Table 3: Performance comparison with varying readout functions on the VanillaTF framework (%).
The overall best results are highlighted bold, while better results with/without ALGA are indicated by underlining. Standard deviations are presented in parentheses.
w/o ALGA
AL.GA
Readout
AUC
ACC
SEN
SPE
AUC
ACC
SEN
SPE
MEAN
73.4(1.4)
67.4(1.2) 68.3(1.2) 66.7(1.2)
75.6(1.6)
71.6(1.3)
70.8(1.2)
69.4(1.2)
MAX
75.6(1.4)
68.4(1.3) 69.5(1.4) 67.7(1.2)
76.4(1.7)
72.6(1.4)
71.7(1.3)
70.7(1.1)
SUM
70.3(1.6)
62.4(1.3) 63.6(1.4) 67.6(1.2)
72.0(1.2)
68.3(1.3)
68.6(1.2)
67.5(1.3)
SORT
72.4(1.3)
65.2(1.2) 66.0(1.2 65.3(1.3)
74.4(1.4)
69.8(1.2)
69.9(1.3)
69.1(1.2)
CLUSTERING
80.2(1.0) 71.0(1)) 72.5(5.2) 69.3(6.5)
82.8(1.1)
77.0(1.0)
77.4(3.4)
76.6(4.6)
Table 4: Performance comparison with varying readout functions the Graphormer framework (%). The overall best results are highlighted bold, while better results with/without ALGA are indicated by underlining. Standard deviations presented in parentheses.
w/o ALGA
ALGA
Readout
AUC
ACC
SEN
SPE
AUC
ACC
SEN
SPE
MEAN
50.1(1.1)
48.6(2.2)
69.1(5.7)
39.6(6.2)
52.4(1.4)
51.8(1.)
72.7(5.2)
43.6(4.4)
MAX
54.5(3.6)
53.3(2.1)
74.7(5.2)
40.4(24.2)
56.1(1.4)
55.8(1.3)
73.9(6.2)
43.9(14.5)
SUM
54.1(1.3)
53.9(1.4
74.6(4.4)
39.7(12.2)
55.7(1.6)
57.8(2.1)
73.3(5.2)
50.7(10.2)
SORT
50.5(4.7)
49.6(5.2)
76.5(6.4)
40.6(19.3)
53.9(5.4) 52.9(5.3)
80.7(11.)
44.6(9.3)
CLUSTERING
64.9(2.7)
60.3(3.3
79.4(12.5) 41.7(20.1)
67.2(2.5)
64.1(1.9)
82.3(10.3) 45.9(12.7)
method generally performs exceptionally well with hop count of 16. Hence, we set the number of hop to 16 in our comparison experiments.
The Sensitivity of ALTER. We present experimental results on the ABIDE dataset for hop counts k ranging from 2 to 16 to analyze the sensitivity of ALTER. As shown in Table 8 we observe that as the number of hops increases, ALTER generally exhibits improved performance, achieving the best results at 16 hops. This indicates that method is influenced by the number of hops k. as ALTER relies on random walk sampling to capture long-range dependencies. Additionally, this phenomenon demonstrates the capability of the proposed method to capture long-range dependencies.
Attendin
Example graph
Attention map
Example graph
Cise
Attention
(c Example
Attention
Example
Figure 5: Example brain graphs from the ABIDE dataset and the corresponding attention heatmap.
15

---

Table 5: Performance comparison with varying readout functions on the SAN framework (%). The overall best results are highlighted bold, while better results with/without ALGA are indicated by underlining. Standard deviations are presented in parentheses.
w/o ALGA
ALGA
Readout
AUC
ACC
SEN
SPE
AUC
ACC
SEN
SPE
MEAN
63.7(2.4)
60.7(3.3)
56.7(1.3)
58.6(2.4)
64.4(1.6)
62.8(2.5)
57.6(1.5)
63.4(3.6)
MAX
61.9(2.5)
56.9(2.9)
54.2(3.1)
52.4(4.2)
63.1(1.7)
59,4(1.2)
(3.1(3.4)
59.9(3.1)
SUM
62.0(2.3)
57.8(2.6)
53.7(3.2)
60.7(4.7)
63.5(1.2)
60.7(1.4)
57.8(2.4)
61.6(5.2)
SORT
57.4(5.2)
55.6(5.2)
53.7(4.3)
56.7(3.2)
60.2(1.4)
58.9(4.7)
56.5(4.5)
59.8(3.6
CLUSTERING
70.6(2.4)
67.3(3.4)
56.7(7.5) 67.6(12.4)
72.5(19)
67.8(3.1)
58.9(6.5)
70.8(4.1)
Table 6: The Impact of Hops and Adaptive Factors on the ABIDE dataset (%). The overall best results are highlighted in bold, while better results with/without adaptive factors are indicated by underlining. Standard deviations are presented in parentheses.
w/o adaptive factors
adaptive factors
Hops
AUC
ACC
SEN
SPE
AUC
ACC
SEN
SPE
2
76.6(3.9)
69.0(..5)
70.1(3.5)
69.2(5.1)
78.7(3.9)
72.0(200)
71.9(3.1)
72.2(3.1)
4
77.4(3.))
71.0(3.0)
72.5(2.5) 71.1(4.5)
79.1(2.4)
73.0(2.5)
73.2(1.9
72.9(3.8
8
78.9(2.2)
73.0(2.0)
74.2(2.9) 75.4(5.9)
80.4(1.9)
72.0(4.0)
75.6(2.1)
74.8(5.6)
16
81.2(2.6)
75.0(1.5) 75.8(3.1)
75.1(4.))
82.8(1.1)
77.001.0)
77.4(3.4)
76.6(4.6)
32
78.8(1.9)
74.0(1.2) 76.4(2.2) 73.8(5.9)
79.6(2.1)
75.0(1.3)
76.3(2.7)
74.3(5.1)
The Cases of the ALTER. To further illustrate the ability the proposed method to capture long-range dependencies within brain networks, perform sample-level and group-level analyses, respectively. For the sample-leee analysis, we present four additional cases in the Figure From the Figure ( d). we observe that despite being separated by 6 hops, ALTER still able to capture the dependency between nodes 6 and 12. For the group-level analysis, we have computed the average across individuals to perform group-level analysis, this approach aligns with the methodologies commonly adopted similar studies [46]. The average graph and the corresponding attention heatmap are illustrated in Figure &())(()) of the global response. We can observe that ALTER captures group-level long-distance dependence, but it is not very significant relative to the individual- level. This may be due certain individual differences in patients, including age and gender, which can affect the brain-wide communication
We use the SHAP model for interpretability analysis on the ADNI dataset. We calculate the SHAP values of the attention matrix. From the Figure(((a), 4), can be observed that the hippocampal regions of AD cases have positive SHAP values and the Top- 10 ROIs with the highest SHAP values are almost always correlated with ADNI prediction, which is generally consistent with the results in [47.
The Interpretability of ALTER.
(a) Interpnctabilitty analysis based SHAP model
(b) Atiention
(6) Example graph
Figure 6: The interpretability analysis and group-level analysis.
16

---

Table 7: The Impact of Hops and Adaptive Factors on the ADNI dataset (%). The overall best results are highlighted bold, while better results with/without adaptive factors are indicated by underlining. Standard deviations are presented in parentheses.
w/o adaptive factors
adaptive factors
Hops
AUC
ACC
SEN
SPE
AUC
ACC
SEN
SPE
2
76.5(3.2)
71.5(2.5)
73.9(5.3)
69.1(5.8)
77.1(2.9)
71.8(1.)
71.2(6.2)
67.7(6.6)
4
76.9(3.4)
72.1(2.9)
74.0(5.2)
68.4(6.1)
77.3(3.2)
73.3(2.3)
74.6(6.9)
68.9(7.3)
8
77.6(2.5)
72.5(3.1)
75.6(6.5)
68.9(6.8)
78.2(2.8)
73.6(2.7)
75.4(7.2)
69.2(5.8)
16
78.1(2.9)
73.0(..1)
75.4(5.9)
71.2(6.2)
78.8 (2.1)
74.1 (2.5)
76.5 (6.1)
70.0 (6.5)
32
76.2(2.9)
73.0(1.2
74.0(4.5)
70.7(5.5)
77.9(2.6)
73.5(2.6)
74.9(5.2)
70.3(6.1)
Table 8: The sensitivity of ALTER on the ABIDE dataset. The best results are highlighted in bold, while standard deviations are presented in parentheses.
ABIDE
Hops
AUC
ACC
SEN
SPE
2
78.7(3.9)
72.0(2.0)
71.9(3.1)
72.2(3.1)
3
77.4(4.8)
70.4(22)
70.3(4.7)
71.3(5.1)
4
79.1(2.4)
73.0(2.5)
73.2(1.9)
72.9(388
5
78.6(4.9)
70.2(4.1)
72.3(3.2)
72.2(4.8)
6
76.6(4.3)
71.5(3.5)
72.0(2.2)
69.5(6.8)
7
78.0(3.2)
69.2(2.0)
72.9(2.1)
70.0(.66
8
80.4(1.9)
72.0(4.0)
75.6(2.1)
74.8(5.6)
9
79.8(2.9)
74.2(3.))
76.0(2.8)
71.7(34)
10
81.1(2.1)
73.(229)
71.7(6.2)
73.7(5.8)
11
77.4(.6)
71.0(3.0)
76.4(3.2)
71.4(6.3)
12
80.9(2.1)
74.0(3.5)
74.5(3.6)
72.2(4.2)
13
80.7(3.1)
73.2(3.2)
74.8(5.1)
73.4(5.2)
14
80.8(1.6)
75.0(2.0)
72.7(4.5)
70.5(6.5)
15
79.3(3.2
75.5(2.5)
72.2(4.6)
69.6(6.1)
16
82.8(1.1)
77.0(1.0)
77.4(344
76.6(4.6)
B Further Discussion
Possible Negatíve Societal Impacts. Given that the research in this paper involves neurological disease diagnosis, it is essential to declare the potential negative societal impacts of this study, even though it is currently in the research phase and has not yet been applied in practice. Specifically, in the process of Al Al-assisted disease diagnosis, erroneous results are inevitable. Such errors can have severe consequences for patients and society. Therefore, in real-world medical diagnostic scenarios, the final decision should always rest with the physician's 's diagnosis.
17

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Our abstract and introduction accurately reflect the paper's contributions and scope, namely our endeavor to address the limitation of overlooking long-range dependencies in brain network analysis tasks.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss the limitations of ALTER Appendix B
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (c.g.. independence assumptions, noiseless settings, model sell-speciiiatiinn asymptotic approximations only holding locally). ). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-tex, system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
18

---

Answer:
[NA]
Justification:
The paper does not include theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental ALTEERial, but they appear in the supplemental ALTERrial, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental ALTERrial.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: In the paper, we provide comprehensive details of all information necessary for reproducing the experimental results, including the experimental datasets, baselines, evaluation metrics, and relevant experimental settings, ensuring the reproducibility of the research.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
19

---

reproducible, they 5. Open access to data and code
Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental ALTERrial?
Question:
Answer: [Yes]
Justification: We offer links to the code and datasets to ensure the reproducibility of the experimental results.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://niis.cc.
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/nips. wwtpcc/pccppcpcceeeooooooop / seaibbmmssionPp. swiniisssoopiiiyyy Policyy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental ALTERrial (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We elucidate the training and testing details required for understanding the results in Section 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental ALTERrial.
7. Experiment Statistical Significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer:
[Yes]
Justification: All results are the average of random runs on test sets with the standard deviation. See metrics in Section[.1
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emoo bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
20

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments Compute Resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
The computer resources can be referred to in the experimental setup outlined in Section
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estiAL.TER the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary failed experiments that didn't make into the paper).
9. Code Of Ethics
Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics htttps://euurps..tt http///www......ooaoii///hhsaaaa
Question:
Answer: [Yes]
Justification: Our research is conducted in compliance with the NeurIPS Code of Ethics. Our experiments do not involve human subjects and potential harmful consequences for society.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- cration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Given that our study involves disease diagnosis, we have outlined its potential negative societal impacts in practical applications in Appendix B
21

---

Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point out. For example, it is legiti to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI...
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
The code uesd in the paper, we all explicitly cite original papers.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
22

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.coo//aaaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental ALTERrial is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
23

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
24


Reviews:
[{"review_id": "HQ0IjCU0pL", "reviewer": "Reviewer_dPgM", "summary": "This work proposes Adaptive Long-range aware TransformER (ALTER), a brain graph transformer to capture long-range dependencies between brain ROIs utilizing biased random walk.", "questions": "1. What is the final loss function for this task?\n2. How many times do you repeat the experiment? Have you conducted k-fold cross-validation to examine the result?\n3. How about the ALTER’s ability to capture long-range dependencies in the patient's group instead of choosing one example to show in Figure 4? Can it get the same conclusion in the group-level analysis?", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This work introduces a novel brain graph transformer with adaptive long-range awareness, which leverages the communication strengths between ROIs to guide the capturing of long-range dependencies.\n2. The result demonstrates that ALTER consistently outperforms generalized graph learning methods and other graph learning-based brain network analysis methods.", "weaknesses": "1. Even though the proposed ALTER is better than all selected baselines, some baselines like BrainGNN can provide both the prediction and interpretation. It is unclear whether the ALTER can also explain the important disease-specific pattern and find the biomarkers.\n2. It is unclear how to implement the comparable baselines and how to build the brain graph for them.\n3. It is unclear how to preprocess the fMRI in detail and how to conduct the quality control.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for the comments on our work. Below we address the questions raised. We hope we have well addressed your concerns. If there are any other issues remaining, we are pleased to address them further.\n\n**W1**\nAs shown in Figure 1(a) of the global response, we have analyzed the output using the SHAP model and confirmed that our results reflect AN-related ROIs. The proposed ALTER promotes an integrated understanding of brain-wide communication by capturing long-range dependencies and achieves superior performance in disease prediction tasks. Long-range dependencies, as complementary to short-range dependencies, can deliver unique insights into the organization and behavior of brain networks associated with neuropsychiatric disorders [1, 2, 3]. ALTER effectively captures long-range dependencies using the Markov process-based ALGA strategy, which has the ability to further explain the important disease-specific pattern and find the biomarkers. It is worth noting that the proposed model employs Transformers which are very well known for extracting the important features using its attention mechanism, which is effectively used for interpretability. \n\n**W2**\nWe apologize for not providing a clear description of how to implement the comparable baselines and build the brain graphs for them. We have added a section in the appendix to clarify the implementation details of the comparable baselines and the construction of the brain graphs.\n\nSpecifically, to ensure a fair comparison, we use the open-source codes of BrainGNN, BrainNETTF, and FBNETGEN. For SAN, Graphormer, LRGNN, and GraphTrans, we adapt their open-source codes and modify them to suit the brain network datasets. For BrainNetGNN, we implemente it ourselves following the settings described in the paper. During the parameter tuning, we follow the tuning of BrainNETTF [4] for SAN, BrainGNN, FBNETGEN, Graphormer, and BrainNETTF. For BrainNetGNN, we search the number of GRU layers {1, 2, 3}. For LRGNN, we vary the aggregation operations {8, 12} with the number of cell  {1, 3}. For GraphTrans, we search the number of GNN layers {1, 2, 3, 4} with the hidden dimension of 100.\n\nWe utilized functional connectivity matrix to compute a brain graph for BrainNETTF, which is computed by calculating the correlation between brain regions using the processed fMRI. The details of computing these correlation matrices is also incorporated to the revised paper. For BrainNetGNN and FBNETGEN, the models required the processed fMRI as input. BrainGNN, SAN, Graphormer, LRGNN, and GraphTrans required the correlation matrix and adjacency matrix. As mentioned in the paper, the adjacency matrix is obtained by thresholding (≥0.3) the correlation matrix.\n\n**W3**\nWe thank the reviewer for requesting a clear explanation of the preprocessing steps for fMRI. We preprocess the fMRI using the Data Processing Assistant for Resting-State Function (DPARSF) MRI toolkit. Specifically, we removed the first 10 time points from the downloaded nii data according to the default mode and chose slice timing, where the middle layer was the reference slice. meanwhile, we set the head motion correction to ‘Friston 24’, and selected automask and Nuisance covariates regression. the others were basically set according to the default mode. Then, considering individual differences, we choose to perform ‘Normalize by DARTEL’, and for the definition of ROIs, we adopt the altas already available in DPARSF. Then, we construct brain networks $G = \\left( {V,X,A} \\right)$ for each fMRI. \n\nDuring the experiment, for the ABIDE dataset, since we directly adopted the processed brain network from [4] and used it as the correlation matrix, its quality control refers to [4]. For the ADNI dataset, besides the above preprocessing, we performed head motion correction, slice timing correction, realigning and normalize. Please note that we have followed the standard protocol used by the other research studies to ensure that any bias and noise have been removed from the dataset.\n\n**Q1**\nThe final loss function for this task is the cross-entropy loss as the model address a classification task. This is already implemented in our open-source code as shown. We have now revised the paper to improve the clarity of the methodology.\n\n**Q2**\nFor all the experiments, we repeated them 10 times. Instead of k-fold cross-validation, we conducted repeated random split validation. To ensure that our results are trustworthy, we additionally performed 5-fold cross-validation (as shown in Table 4 in the global response).\n\n**Q3**\nWe have computed the average across individuals to perform group-level analysis, as this approach aligns with the methodologies commonly adopted in similar studies [5]. The average graph and the corresponding attention heatmap are illustrated in Figure 1(b)&(c) of the global response. We can observe that ALTER captures group-level long-distance dependence, but it is not very significant relative to the individual-level. This may be due to certain individual differences in patients, including age and gender, which can affect the brain-wide communication [6].\n\n**References**\n\n[1] Space-independent community and hub structure of functional brain networks. NeuroImage 2020.\n\n[2] Long-range connections are more severely damaged and relevant for cognition in multiple sclerosis. Brain 2019.\n\n[3] Engineering brain assembloids to interrogate human neural circuits. Nature Protocols 2022.\n\n[4] Brain network transformer. In NeurIPS 2022.\n\n[5] Structure-function coupling in the human connectome: A machine learning approach. NeuroImage 2021.\n\n[6] Local structure-function relationships in human brain networks across the lifespan. Nature 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1iKKkTXxtv", "reviewer": "Reviewer_u6uE", "summary": "The study employs the adaptive long-range aware graph transformer (ALTER) to tackle the challenge of weak comprehension in whole-brain communication, which arises from the failure to capture long-range dependencies in brain networks. Initially, the study encodes long-range dependencies into long-range embeddings through biased random walk sampling, thereby enriching the embeddings with information on long-range dependencies. Subsequently, the study takes into account the significance of both short- and long-range dependencies in brain network analysis tasks. It introduces the graph transformer, which uses a self-attention mechanism to integrate these dependencies between brain ROIs. The objective is to capture varying levels of communication connections within the human brain. Experimental results demonstrate that ALTER outperforms current SOTA graph learning baselines, achieving superior AUC and ACC scores on the ABIDE and ADNI datasets.", "questions": "In addition to what I wrote in the \"Weaknesses\" section, I have a couple of questions:\n\n1. How does the author define long-range dependence and short-range dependence?\n\n2. Did the authors attempt to choose other correlation measures than the Pearson correlation coefficient to define the adaptive factors?\n\n3. Could you elaborate on the need to introduce the graph transformer in the paper? Would replacing the graph transformer with a GNN affect the results?\n\n4. Could the authors elaborate on the correlation between the attention map and the example graph in Figure 5?", "limitations": "The paper emphasizes the possible negative social impact of work. However, in my opinion, the negative societal impact is not only medical errors due to prediction errors, but should also include the negative ethical impact of the model. Adding ethical concerns would help the reader better understand the potential impact of the proposed methodology.", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) This work is significant. How to facilitate the understanding of communication and information processing among brain ROIs is a key issue. The authors provide a clear overview of the necessity of long-range dependencies for understanding communication and information processing among brain ROIs, and propose an effective method to capture long-range dependencies in brain networks.\n\n(2) The paper is aesthetically pleasing in its writing form, especially the diagrams and charts that make it easy for the reader to understand the exact process of the work done. Meanwhile, the authors provide detailed preprocessing and implementation details. As a result, this paper is highly reproducible.\n\n(3) The paper provides complete experimental results. Besides comparisons with the baseline and ablation studies on modules, the appendix section validates the state-of-the-art of the adaptive long-range dependency aware strategy on multiple readout functions, further enhances reproducibility, and facilitates a deeper understanding of the method.\n\n(4) ALTER is an ingeniously crafted framework that is able to adaptively perceive both long- and short-range dependencies in brain networks. In addition, ALTER interestingly takes inter-ROI correlations into account in the capture of long-range dependencies in brain networks.", "weaknesses": "I have identified three main weaknesses that need to be clarified during the rebuttal process, which is the reason I gave a weak acceptance despite the strengths. If my points are properly addressed, I will be happy to review my scores based on the results of the rebuttal process. I numbered the comments to facilitate discussion.\n\n1. The technical contributions are neutral. The proposed ALTER seems to be a combination of the random walk and the graph transformer, but its novelty and difficulty are unclear.\n\n2. ALTER obtained SPE scores that were 5% lower than BrainNETTF on the ADNI dataset and SEN scores that were 1.3% lower than Graphormer on the ABIDE dataset. This performance lag is significant, but the authors do not seem to have clearly explained the reasons for the SPE and SEN lags.\n\n3. The authors state in line 202 that ALERT uses a linear layer to inject long-range dependencies into the brain graph transformer, but that direct collocation also accomplishes the above, and the authors do not state the necessity of introducing a linear layer.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your detailed assessment of our work and for highlighting the merits of our approach, as well as the importance of the problem. We address all concerns below, if there are any other issues remaining, we are pleased to address them further:\n\n**W1**\nBrain networks are inherently dense, which has led to the increasing use of transformers for their analysis. However, a significant limitation of transformers is their inability to effectively capture long-range dependencies, which limits the integrated understanding of brain-wide communication. The proposed ALTER is concerned with the capture of long-range dependencies between ROIs, not the combination of random walk and graph transformer. In particular, we design the ALGA strategy, which simulates real-world brain-wide communication by utilizing adaptive factors to evaluate the communication strength between ROIs and capturing long-range dependencies in brain networks through a Markov process. We have now re-emphasized the novelty and difficulty of the proposed ALTER in the revised manuscript.\n\n**W2**\nWe acknowledge that the proposed method exhibits a lower SPE on the ADNI dataset compared to BrainNETTF and a lower SEN on the ABIDE dataset compared to Graphormer. However, considering the standard deviation, our method demonstrates more stability, which can be attributed to its ability to capture long-range dependencies within the brain network. Moreover, our method significantly outperforms the baselines on other metrics across both datasets. In particular, on the ADNI dataset, our method shows a 10.3% improvement in the SEN metric compared to the sub-optimal method. We have provided a detailed explanation of this aspect in the revised manuscript.\n\n**W3**\nThank you for requesting clarity on the linear layer. The main objective of introducing a linear layer to map the initial long-range embedding to the transformer is to enable end-to-end learning of the long-range embedding to participate in updates and enhance its expressiveness. This method is able to enhance the expressiveness of the random walk embedding by capturing multiple types of graph propagation, which in turn facilitates the integration of global information in the brain graph. We have now added these details in the revised paper. If we do not use a linear layer and directly concatenate the initial long-distance embedding with the original features, this reduces the expressiveness of the long-distance dependencies in the brain graph learning. We additionally performed ablation experiments on ABIDE dataset to demonstrate the effectiveness of introducing a linear layer.\n\n|Method |AUC|ACC|SEN|SPE|\n|--- |-----------| ----------- | -----------| ----------- | \n| w/o Linear| 81.0(1.5) | 76.4(2.3)| 73.6(5.8) | 74.4(5.4) | \n| ALTER |82.8 (1.1) | 77.0 (1.0) | 77.4 (3.4) | 76.6 (4.6) | \n\n**Q1**\nwe utilize the proposed ALTER method to qualify these dependencies as they manifest in the data. In brain network analysis, long-range dependence and short-range dependence refer to interactions between neurons in the spatial dimension. Among them, short-range dependencies usually refer to interactions that occur in the same brain region or between anatomically neighboring brain regions. In contrast, long-distance dependence usually spans larger spatial brain regions and involves the information transmission between different functional regions. Such long-range dependencies assume the integration of global information, which plays a key role in complex cognitive functions. In the manuscript, we only deal with the capture of long-range dependence without defining it explicitly at the neuroscience level.\n\n**Q2**\nIndeed, we have not yet explored other correlation measures besides the Pearson correlation coefficient to define the adaptive factors in our study. The proposed method represents an initial attempt to capture long-range dependencies in brain networks. In future work, we plan to investigate some non-linear correlation measures, to further refine and potentially improve the adaptability of our method. It is worth noting that studies in neuroscience predominantly rely on Pearson correlation. \n\n**Q3**\nIn ALTER, the self-transformer mechanism is introduced to adaptively integrate the long-range and short-range dependencies between ROIs. Meanwhile, at the suggestion of reviewer s7Az, we try to replace the transformer with 1-layer GCN and 2-layer GCN. The experimental results are shown in the Table 5, which it is proved that transformer usually achieves better results due to its ability of adaptive learning.\n\n**Q4**\nIn Figure 5 we show additional examples of long-range capture at the individual level on the ABIDE dataset. In Figure 5(a)(b)(c) we can see that despite the fact that the two red labeled nodes are 5 hops apart, there is still a high attention value between the two ROIs. In Figure 5(d) we can see that there is a high attention value between 6 and 12 which are 6 hops apart.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NKjeA2Y4O2", "reviewer": "Reviewer_xvKE", "summary": "This paper highlights a significant gap in the existing literature on brain network representation learning, specifically the inadequacy of current methods to effectively capture long-range dependencies, leading to limited an integrated understanding of brain-wide communication. To bridge this gap, the paper introduces ALTER, an innovative model designed for adaptive awareness of long-range dependencies and brain network representation learning. The model consists of a long-range aware strategy for capturing long-range dependencies and a transformer framework for integrating multi-level communication understanding.", "questions": "1. Have the authors considered tasks other than binary categorization?\n2. Am I correctly understanding that the dimension of the original feature embedding and the long-range dependency embedding after splicing is equal to the original feature embedding dimension? In the representation in Figure 2, I get the impression that the dimension after splicing is unchanged, but this does not seem to be the case from the text. Could the authors clarify this point?\n3. The authors conducted experiments on fMRI data sets, and fMRI is only single-modal. Is the biased random walk strategy of ALTER still valid on multi-modal data?", "limitations": "See weaknesses and questions for details.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "(1) This paper is generally clear and well-written, providing a comprehensive analysis of the problem to be solved, supported by some convincing references, and emphasizing the urgent need to come up with solutions that can capture long-distance dependencies between brain ROIs. At the same time, the authors make the problem better understood with illustrations.\n(2) The logic of the paper is smooth. The authors clarify the issue, after which they elaborate on the theoretical causes of the issue, and finally design a specific biased random walk strategy for the theoretical causes and obtain encouraging experimental results.\n(3) I found the method presented here is technically sound with excellent results. From the experimental results, it is shown that the proposed adaptive long-range aware strategy is very effective in long-range dependencies capture and can greatly enhance the effectiveness of the disease diagnosis task. In addition, the ablation study conducted on various components reveals the superiority of the adaptive long-range aware strategy.\n(4) The authors clearly and frankly understand the drawbacks and strengths of the method, e.g. the method does not allow for a good trade-off between long- and short-range. This makes this method a clear one to build on top of, prompting further interesting work in highly important domain.", "weaknesses": "1. Although the results of the ablation experiments verify that biased random walk is efficient, it is difficult to convince me of the necessity of introducing this module based only on the theoretical description of biased random walk in the methods section. As far as I know, random walk is common in graph representation learning. Therefore, I would like to ask the authors to provide me with a more in-depth analysis of biased random walk, otherwise I do not understand why biased random walk is effective in capturing long-range dependencies in brain networks.\n2. Transformer-based graph representation learning is not original in brain network analysis. It is recommended that the authors provide a specific explanation for fusing long-range and short-range dependent embeddings using the graph Transformer.\n3. Lack of interpretable analysis. Interpretive analysis of brain network representation learning models geared toward brain disease diagnosis is important. In the context of brain network analysis, I suggest that papers should explain and identify the brain regions or networks that are most relevant to the task of brain disease classification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for acknowledging the novelty of the proposed method and for suggesting relevant analysis, which we have included in the global response. We hope to provide satisfying answers to the concerns raised. If there are any other issues remaining, we are pleased to address them further.\n\n**W1**\nOur model aims to capture long-range dependencies within brain networks leveraging the biased random walk, which has a Markovian nature that captures long-range communication among brain ROIs by sampling and encoding sequences into embeddings. We agree that random walk is common in graph representation learning. However, thedifferent pairs of ROIs in brain networks usually exhibit different communication strengths in brain activity. As a result, traditional random walk methods are usually not applicable to brain networks, and cannot capture long-range dependencies in brain networks.\n\n**W2**\nWe acknowledge that Transformer is commonly used in brain network analysis. The focus of the proposed method is on capturing long-range dependencies within brain networks. Transformer inherently has limitations in capturing long-range dependencies, which prevents a comprehensive understanding of brain-wide communication. Instead, the proposed ALTER method introduces an adaptive long-range aware strategy to explicitly capture long-range dependencies within brain networks. It then integrates long-range and short-range dependencies using the adaptive mechanism of Transformer, thereby achieving a multi-layered understanding of brain networks.\n\n**W3**\nAs shown in Figure 1(a) of the global response we supplemented the interpretability analysis. \n\n**Q1**\nIn the submitted manuscript, we evaluated performance solely on binary classification tasks using the ADNI and ABIDE datasets. To further demonstrate the effectiveness of the proposed method, we have conducted additional experiments on the PPMI dataset with four class, as suggested by reviewer s7Az. The experimental results are presented in the Table 1 of the global response. The results show that the proposed ALTER method achieved the best performance, as it effectively captures long-distance dependencies between brain ROIs. \n\n**Q2**\nThank you very much for your question. Since we concatenate the long-range dependency embeddings directly with the original feature embeddings without any other operation, the dimensionality of the concatenated features is not equal to that of the original feature embeddings. We have now revised the paper to improve the clarity of the methodology. \n\n**Q3**\nThank you very much for your question. The value of the biased random walk strategy in ALTER for multi-modal data is indeed a topic worth exploring. However, due to some fundamental differences between  functional and structural brain networks constructed based on fMRI and DTI, directly applying the biased random walk strategy to capture long-range dependencies in them may not be feasible and would require tailoring the ALTER model. Nonetheless, the biased random walk strategy can be adjusted by learning consistent and complementary communication patterns between functional and structural brain networks, which may enhance its adaptability to multimodal brain network data. We have now incorporated this as a future work of the study in the revised paper.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "lRD3W1MdNP", "reviewer": "Reviewer_s7Az", "summary": "This paper employs a random walk approach to capture long-range dependencies in the brain through a feature engineering scheme. The computed adaptive factors are incorporated into node features and used to train a Transformer model.", "questions": "1. Which atlas is used in the dataset?\n2. Which ROIs obtain high attention, and do these match domain observations for the diseases studied?", "limitations": "The limitations have been discussed in this paper.", "rating": 5, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The idea of capturing long-range dependencies in the brain is reasonable.\n2. The paper is well-written and easy to understand.\n3. The experimental results are strong on the two datasets tested.", "weaknesses": "1. Some design choices in the proposed method are not well-motivated:\n   - Using the designed adaptive factors (Eq. 1) instead of Pearson correlations.\n   - Introducing the degree matrix in the random walk kernel.\n2. The parameter \\( k \\) in Section 3.2 is not clearly defined; it is assumed that \\( k = K \\).\n3. The descriptions of the dataset construction and experimental settings are unclear. Specifically:\n   - Which atlas is used, and how many ROIs does it contain?\n   - What exact value is used to threshold the connectivity to obtain the adjacency matrix?\n   - Given that the number of hops is set up to 32, it is expected that the adjacency matrix is super sparse. However, such a high threshold is uncommon in brain network analysis.\n4. The paper conducts experiments on only two datasets, which is insufficient. The ADNI dataset used contains only two classes with 130 subjects. It would be better to extend the experiments to include more datasets as referenced in [1], which includes six datasets and over 1.3k subjects for ADNI.\n5. There is a lack of comparison with the latest related works. For example:\n   - [2] introduces a graph augmentation using ALFF features.\n   - [3] presents a global matching-based graph kernel that captures dynamic changes in evolving brain networks.\n   - [4] utilizes clustering-based graph pooling for readout.\n6. It would be valuable to incorporate the proposed ALGA method with GNN baselines such as GCN and GAT to evaluate whether it can improve their performance.\n7. The paper lacks deeper analysis from a neuroscience perspective. It is important to provide insights from the model results, such as identifying specific ROIs related to the target disease and whether these findings align with medical literature.\n\nReferences:\n[1] Data-driven network neuroscience: On data collection and benchmark. NIPS 2023  \n[2] A-GCL: Adversarial graph contrastive learning for fMRI analysis to diagnose neurodevelopmental disorders. MIA 2023  \n[3] Effective Graph Kernels for Evolving Functional Brain Networks. WSDM 2023  \n[4] Contrastive Graph Pooling for Explainable Classification of Brain Networks. IEEE TMI 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank you for your detailed feedback and your questions. We hope we have well addressed your concerns. If there are any other issues remaining, we are pleased to address them further.\n\n**W1.a**\nIn fact, we employ Pearson correlations as adaptive factors in random walk, without any modifcation. Since the correlation between ROIs reflects their communication strength, which is crucial for comprehending the functional organization, dysfunctions, and information propagation. We take the correlation between ROIs as adaptive factors to influence the exploration mechanism of random walk, which induces ROIs with stronger connectivities to exhibit higher transfer probabilities of the next hop. This manner simulates real-world brain-wide communication.\n\nWe recognize that this part of the presentation may not have been clearly articulated, leading to potential misunderstandings. Therefore, we have make the necessary revisions in the revised paper to enhance clarity. \n\n**W1.b**\nThe introduction of degree matrix can help to obtain richer information about brain-wide communication and is very commonly used in network analytics [1]. Since the degree matrix provides the number of degrees for each ROI, its ability to reflect the active state of the ROI in communication is important in determining which ROIs play a key role in information propagation. Hence, a degree matrix determines the transfer probability of a node to its neighboring nodes and highly influences the behavior of walker [2]. We have now incorporated these details in the revised paper.\n\n**W2**\nWe apologize for our mistake, we did mix up $k$ and $K$. As you say, what we use is $k = K$. $k$ is generally denoted as the number of hops in random walks. We apologize for not defining it explicitly, which has now been addressed in the revised paper.\n\n**W3&Q1**\nWhile nearly all of the experimental settings are presented in the source code, we acknowledge that our descriptions of these settings were indeed insufficient. We have now incorporated the details in Appendix A.\n\nSpecific details requested by the reviewer includes: \n- For the ABIDE dataset, we use the Craddock 200 atlas with 200 ROIs. For the ADNI dataset, we use the AAL atlas with 90 cortical ROIs and 26 cerebellar ROIs. \n- The threshold is 0.3. \n- This is a mistake in the number of hops reported in the paper. The experiments were performed with 16 hops. We tried the hops commonly used for long-range dependency capture in network analysis by random walks through a grid search, and chose 16 hops. As shown in Figure 4(a) in the paper, larger hops do not lead to better results since brain graphs are relatively dense.\n\n**W4&W5**\nUpon receiving comments, we promptly review the referenced papers and their corresponding open-source codes. As shown in Tables 1&2&3 of the global response, we have added additional datasets (PPMI, Matai, TaoWu, and Neurocon) and baselines (A-GCL [3] and ContrastPool [4]).\n\nRegarding the complete ADNI dataset [5], we discovered from the GitHub repository provided by the authors that due to data protocol issues, it has not yet been made publicly available. Regarding [6], we find that the original papers did not provide corresponding code links, and also not released in github. We also contacted the authors, but as of the end of the rebuttal, we have not received responses. However, we need to state that [6] more focuses on learning time series of fMRI using a global and local matching-based graph kernels to construct dynamic brain networks, whereas ALTER is more concerned with capturing long-distance dependencies from static brain networks using random walk kernel, which are fundamentally different from each other.\n\nIn the field of neuroscience, it is often challenging to obtain large amounts of data due to various constraints. And often use a smaller dataset (≤130 individuals), which are sufficient to support conclusions [7, 8, 9, 10]. We initially focused only on the ADNI and ABIDE because they are widely used for disease prediction. However, due to the significant challenges in preprocessing, we were unable to process the complete fMRI data from the ADNI dataset. While only two classes and 130 subjects were included, our method still shows excellent performance. This demonstrates the effectiveness of ALTER with limited data, suggesting that it may better adapt to other datasets with similarly limited quantities. This adaptability can potentially lead to more successful outcomes in other brain science-related medical tasks. \n\n**W6**\nAs shown in the Table 5 of the global response we did additional experiments and achieved better performance. This is because ALGA can capture the long-range dependencies in the brain network.\n\n**W7&Q2**\nBased on the SHAP model, we have supplemented the experiment in neuroscience perspective, and concluded that ALTER is can recognize disease-related regions and give high attention. The results are illustrated in Figure 1(a) of the global response. \n\n**References**\n\n[1] Graph neural networks with learnable structural and positional representations. In ICLR 2022.\n\n[2] How to Count Triangles, without Seeing the Whole Graph. In KDD 2020.\n\n[3] A-GCL: Adversarial graph contrastive learning for fMRI analysis to diagnose neurodevelopmental disorders. MIA 2023.\n\n[4] Contrastive graph pooling for explainable classification of brain networks. TMI 2024.\n\n[5] Data-driven network neuroscience: On data collection and benchmark. In NeurIPS 2023.\n\n[6] Effective graph kernels for evolving functional brain networks. In WSDM 2023.\n\n[7] Spatio-Temporal Graph Hubness Propagation Model for Dynamic Brain Network Classification TMI 2024.\n\n[8] RH-BrainFS: Regional Heterogeneous Multimodal Brain Networks Fusion Strategy. In NeurIPS 2023.\n\n[9] Functional brain network reconfiguration during learning in a dynamic environment. Nature Communications 2020.\n\n[10] Increased global integration in the brain after psilocybin therapy for depression. Nature Medicine 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for their insightful assessment of our work, as well as the useful feedback and actionable suggestions they provided. We are pleased that they found our work to be meaningful (reviewer u6uE) and reasonable (reviewer s7Az), that the experimental results are strong (reviewers s7Az, dPgM and xvKE), and that the manuscript is clear (reviewer xvKE). We will incorporate their suggestions to improve the presentation in future revisions of the paper. 

Below we provide additional empirical analysis based on the recommendations raised by the reviewers. Each reviewer's individual questions will be answered in separate responses. The new results include:

**Interpretability Analysis.** We used the SHAP model for interpretability analysis on the ADNI dataset. We calculated the SHAP values of the attention matrix. From the results, it can be observed that the hippocampal regions of AD cases have positive SHAP values and the Top-10 ROIs with the highest SHAP values are almost always correlated with ADNI prediction, which is generally consistent with the results in [1].

**More Sufficient Experiments.** we evaluate the proposed ALTER on additional datasets (PPMI, Matai, TaoWu, and Neurocon). Meanwhile, we add more models including A-GCL [2] and ContrastPool [3].

**More Ablation Studies.** (1) We used a 5-fold cross validation replacing the repeated random split validation for evaluating the proposed ALTER. (2) We evaluate the performance of the ALGA strategy in combination with different GNN baselines, including 1-layer GCN, 2-layer GCN, 1-layer GAT, and 2-layer GAT.

**Group-level Analysis.** We have computed the average across individuals to perform group-level analysis, and then presented the average graph and the corresponding attention heatmap.

**References**

[1] Multimodal deep learning for Alzheimer’s disease dementia assessment. Nature communications 2022.

[2] A-GCL: Adversarial graph contrastive learning for fMRI analysis to diagnose neurodevelopmental disorders. MIA 2023.

[3] Contrastive graph pooling for explainable classification of brain networks. TMI 2024.

Meta-review: The paper introduces a novel brain graph transformer, ALTER, with adaptive long-range awareness, which effectively captures long-range dependencies among brain regions of interest (ROIs) by leveraging their communication strengths. The approach is well-motivated, addressing the critical issue of understanding communication and information processing among brain ROIs, and shows promising results, consistently outperforming other graph learning methods in brain network analysis. The method is technically sound, and the experimental results demonstrate its effectiveness across multiple datasets, with detailed ablation studies further validating the adaptive long-range dependency-aware strategy. The paper is well-written, with clear diagrams and charts that enhance understanding and reproducibility. 

However, the paper has some limitations. ALTER's interpretability in identifying disease-specific patterns remains unclear compared to methods like BrainGNN. The technical novelty, involving random walks and graph transformers, could be better explained, with some design choices appearing under-motivated. Details on baseline implementation, fMRI preprocessing, and brain graph construction are lacking, which could affect reproducibility. Additionally, while ALTER shows good performance, its scalability and comparative advantage over all baselines require further evaluation.

Despite these weaknesses, the strengths of the paper, particularly in addressing a key problem with a well-supported and innovative approach, justify its acceptance.
Decision: Accept (poster)