Biologically Plausible Brain Graph Transformer
Authors: Ciyuan Peng
Yuelong Huang
Qichao Dong
Shuo Yu
Feng Xia
Chengqi Zhang
Yaochu Jin
Venue: ICLR 2025 Poster (2025)
Keywords: brain
graph learning
transformer
graph representation
brain networks
Primary area: learning on graphs and other geometries & topologies

Abstract: State-of-the-art brain graph analysis methods fail to fully encode the small-world architecture of brain graphs (accompanied by the presence of hubs and functional modules), and therefore lack biological plausibility to some extent. This limitation hinders their ability to accurately represent the brain's structural and functional properties, thereby restricting the effectiveness of machine learning models in tasks such as brain disorder detection. In this work, we propose a novel Biologically Plausible Brain Graph Transformer (BioBGT) that encodes the small-world architecture inherent in brain graphs. Specifically, we present a network entanglement-based node importance encoding technique that captures the structural importance of nodes in global information propagation during brain graph communication, highlighting the biological properties of the brain structure. Furthermore, we introduce a functional module-aware self-attention to preserve the functional segregation and integration characteristics of brain graphs in the learned representations. Experimental results on three benchmark datasets demonstrate that BioBGT outperforms state-of-the-art models, enhancing biologically plausible brain graph representations for various brain graph analytical tasks

Published as a conference paper at ICLR 2025
BIOLOGICALLY PLAUSIBLE BRAIN GRAPH TRANS- FORMER
Ciyuan Peng'. Yuelong Huang?, Qichao Dong', Dong",Shue Shuo Yu², Feng Xia Chengqi Zhang,, Yaochu Jin Federation University Australia, Dalian University of Technology, 3Zhejiang Gongshang University, RMIT University, HHnng Kong Polytechnic University, Westlake University
ABSTRACT
State-of-the-art brain graph analysis methods fail fully encode the small-world architecture of brain graphs (accompanied by the presence of hubs and func- tional modules), and therefore lack biological plausibility some extent. This limitation hinders their ability to accurately represent the brain's 's structural and functional properties, thereby restricting the effectiveness of machine learning models in tasks such as brain disorder detection. In this work, we propose a novel Biologically Plausible Brain Graph Transformer (BioBGT) that encodes the small-world architecture inherent in brain graphs. Specifically, we present a network entanglement based node importance encoding technique that captures the structural importance of nodes in global information propagation during brain graph communication, highlighting the biological properties of the brain structure. Furthermore, we introduce functional module-aware self-attention to preserve the functional segregation and integration characteristics of brain graphs in the Iearned representations. Experimental results three benchmark datasets demonstrate that BioBGT outperforms state-of-the-are models, enhancing biologically plausible brain graph representations for various brain graph analytical tasks
INTRODUCTION
Brain graphs, also known as brain networks, are a primary form to present the complex in- teractions among regional activities, functional correlations, and structural connections within the brain (Seguin al. 2023 Wu et al. 2024bb Zhu et al. (2024). Brain graphs are constructed based on information extracted from brain data, such as functional magnetic resonance imaging (fMRI), with regions of interest (ROIs) nodes and the correlations among ROIs as edges. One of the most important characteristics of brain graphs is their small-world architecture, with scientific evidence supporting the presence of hubs and functional modules in brain graphs (Liao al.) [2017] Swanson al. [2024]. First, it is demonstrated that nodes in brain graphs exhibit high degree of difference in their importance, with certain nodes having more central roles in information propagation (Lynn & Bassett 2019 Betzel al. 2024). These nodes perceived as hubs, shown in Figure (a) (the visualization based findings by (Seguin et al. ), which are usually highly connected so as to support efficient communi- cation within the brain. Second, human brain consists of various functional modules (c.g., visual cortex), where ROIS within the same module exhibit high functional coherence, termed functional integration, while ROIs from different modules show lower functional coherence, termed functional segregation (Rubinov & Sporns, 2010 Seguin et al. 2022). Therefore, brain graphs are characterized by community structure, reflecting functional modules. Figure (b) visualizes the functional con- dataset. The functional module labels are empirically
Fundiosal Connectidy
Nokkn
Centrol
Centrl
Vicua
Conte
Hubta intormation procegation
odlir indicuto ahotge comriillarr
Figure 1: Smoll-world architecture of brain graphs.
nectivity of sample brain from ADHD
"Corresponding authors: f xi f.w.ieeeeee.or.ooo htt.::/www.......ooooooosssssaa. edu. cn
"Our code is available at wwt.:a/lw...taeeettt.a.aoaa//o//

---

Published as a conference paper at ICLR 2025
provided based on Dosenbach et al. (2010). ROIs in the same module have strong connections (high temporal correlations), while those from different modules show weaker connections.
With the significant ability of graph transformers in capturing interactions between nodes (Ma et al. 2023a Shchzad et al. 2024; Yi et al. 2024). Transformer based brain graph learning methods have gained prominence (Kan et al. [222 Bannadabhavi et al.) 2023). Despite these advancements, there is still a lack of tailored design for brain graphs that consider the small-world architecture as an essential feature. Consequently, the learned representations of current methods may lack sufficient biological plausibility for brain graphs. This limitation can be understood from two perspectives. First, most current studies consider connections within the brain as pairwise correlations between nodes, and typically treat all nodes equally. For example, Brain Network Transformer (Kan et al. 2022) assumes that all nodes in a brain graph have the same degree and each node is connected with all the other nodes. However, nodes can have significantly different roles in terms of propagating information within the brain graph (Lynn & Basseti 2019). Second, current methods often encode the correlations between nodes simply based on node-level similarities, ignoring the existence of functional modules within a brain graph. Unfortunately, the existing labeling of functional modules is largely empirical and lacks precision (Kan et al. 2022). Therefore, this limitation is particularly evident in brain datasets where functional module labels are unavailable or inaccurate. This impedes preserving the functional segregation and integration characteristics of the brain.
To this end, this paper proposes a brain graph representation learning technique that departs from existing methods. We aim to improve the alignment of the learned representations with biological properties, particularly by encoding small-world features commonly observed in brain graphs. We propose Biologically Plausible Brain Graph Transformer (BioBGT), which aligns brain graph representations with biological properties through two main components: node importance encoding and functional module encoding. (1) For brain graphs as communication networks, node importance is reflected by how crucial a node is in propagating information across the network (Seguin et al.) 2023). Thus, we propose a node importance encoding technique based on network entanglement Given the topology of a brain graph, the global information diffusion process is modeled through quantum entanglement, wherein the importance of a node is measured by the changes in the density matrix-based spectral entropy before and after perturbing the local connections surrounding the node. The encoding of node importance is thereafter embedded into node representations, reflecting the small-world architecture in terms of the presence of hubs. (2) We then present functional module-aware self-attention to preserve the functional segregation and integration characteristics of brain graphs in the learned representations. Particularly, we design community contrastive strategy-based functional module extractor to refine nodes' similarities at the functional modular level, instead of merely calculating node correlations the node level. Therefore, we can obtain functional module-awar node representations for the self-attention mechanism.
Contributions. This paper highlights that brain graph representations obtained from learning models should align closely with the biological properties of the brain. Under this perspective, i) we propose a new Biologically Plausible Brain Graph Transformer entitled BioBGT that encodes the small- world architecture of brain graphs to enhance the biological plausibility of the learned representations: fii we present a network entanglement- based node importance encoding technique, capturing node importance in the information propagation across brain graphs; iii) we introduce functional module- aware self-attention, yielding functional module-aware node representations with the functional segregation and integration characteristics of brain graphs preserved: iv) experimental results show the effectiveness of our model design and the superiority of our model performance, especially in brain disease detection tasks.
2 PRELIMINARIES
2.1 PROBLEM DEFINITION
brain graph presents the connectivity between ROIS, characterized by the small-world architecture. A brain graph with n nodes (ROIs) is denoted as G = (V, E, X), where V stands for the node set, E is the edge set, and X E R xd represents the feature matrix with the i-th row vector x; E Rd indicating the feature of node i. Here, d is the hidden feature dimension. Hubs and functional modules are two crucial indicators of the small-world brain graph (Rubinov & Sporns| 2010). This paper suggests that the biological plausibility of brain graph representations be enhanced by
2

---

Published as a conference paper at ICLR 2025
capturing these features to extent. For given brain graph, model aims to learn representations that incorporate biologically relevant features and achieve accurate brain graph analysis.
2.2 GRAPH TRANSFORMERS
Transformer architecture is composed of multiple Transformer layers, each of which contains self-attention module followed by feed-forward network (FFN) (Vaswani et al. (017). In the self- attention module, the imput feature matrix E Ix* is first projected query matrix Q. key matrix K, and value matrix V by the corresponding projection matrices E Raxdx, W E Rdxdy and Wv E dddde
Q == XWQ:
K= XWK, V
Then, the self-attention is calculated as:
A QK dd
All((X) = softmar A)V.
(2)
Here, A indicates the attention matrix representing the similarity between queries and keys, dx is the dimension of Q, K, and V. Extending Equation (2) to the multi-head attention is common and straightforward. Afterwards, the output of the eeff-attetiion module fed FFN module:
x At),
X WyReLU(W ())).
Here, ReLU(.) stands for the activation function. W₂ and W₁ are the projection matrices.
Graph transformers are proposed for applying Transformers graph data, which introduces the structural information of graphs as structural encoding (SE) or positional encoding (PE). such as
Laplacian PE, spatial encoding, and edge encoding (Dwivedi al. 2022, Geisler al. [023 Deng
et al. (2024] Xing et al. 2024). However, these methods exhibit limitations when applied to brain
graphs because they do not adapt the specific small-world characteristic, including the presence of hubs in information propagation and functional modules.
3 BIOLOGICALLY PLAUSIBLE BRAIN GRAPH TRANSFORMER
In this section, we present BioBGT in detail. We describe how to enhance the biological plausibility of brain graph representations by focusing on two key aspects: node importance encoding and functional module encoding. First, we design network entanglement-based node importance encoding method in the input layer, denoted as ((.). Then, to encode functional modules, we present functional module-aware self-attention, denoted as FM- FM-Attn(.). Therefore, for each node, rewrite the left part of Equation
3) as:
(xx)) FM-Attn(i).
Figure 2 shows the overall framework of our model. We will introduce the functions of ((-) and FM-Artn(-) in Section 3.1 and Section 3.2 respectively.
3.1 NETWORK ENTANGLEMENT-BAR
NODE IMPORTANCE ENCODING
We measure node importance in information propagation based network entanglement, importing quantum entanglement into brain graphs. Quantum entanglement a phenomenon quantum mechanics, describing the correlations between particles YYu et al.) 2023). Mathematically, quantum entanglement is often represented by a density matrix of quantum entangled states, which captures the entangled relationships between particles in the entire entangled system (Weedbrook et al. 2012). When combined with network information theory, concepts from quantum entanglement can provide powerful lens for analyzing the global topology and information diffusion of graphs (Huang et al.) 2024). Inspired by this, we treat the brain graph as an entangled system, where nodes and their connections reflect interdependent states. The density matrix is used quantify structural information. This upproach enables us to capture the intricate entangled relations between nodes, offering insight into both the global topological features and the information diffusion process within brain graphs.
Proposition (Density matrix structural information). The structural information of a brain graph G. including the connection strength between nodes and the degree distribution of nodes, is encoded by its density matrix, which stands normalized information diffusion propagator and formulated
3

---

Published as a conference paper at ICLR 2025
Community Contrastive Strategy-based Functional module Extractor
GNN
ancher
Xl.
Negatidd
Postive samples
Layers
FFN
samples
Concat
GNN
Functional modules
FM- Attr
Functional Module-Aware Sell-Attention
Node importance encodlngg
X Ru
4p (X)
Brain graph
Linear
Linear
Linear
Node Entanglement
S(G)
S(G
node perturbation
NEG (S S(G)
Transformer Layer
Network Entanglement-based Node Importance Encoding
Figure 2: Overall framework of BioBGT.
as PG == Here, e is the information diffusion propagator, "I denotes the positive parameter, the Laplacian matrix of G, and Tr(e->hy represents the partition fanction of G.
Appendix A. I gives the complete proof. In quantum information theory, von Neumann entropy is
used to measure the uncertainty or randomness of quantum systems (Huang et al. 2024). quantifies the degree of entanglement present in quantum systems. When it comes to the complex graph scenario, density matrix-basee spectral entropy is considered as the counterpart of von Neumann entropy, capturing the global topology and information diffusion process of graphs (De Domenico & Biamonte 2016). It is formulated as:
S(G) -e-rr log, PG),
where S()) is the density mntrix-based spectral entropy G, and Tr(-) indicates the trace operation computing the trace of the product of the density matrix pc and its natural logarithm. The perturbation of a single node on the whole graph can quantified by the change of density matrix-barn based spectral entropy, defined as node entanglement (NE) (Huang al. 2024). We define node importance degree based on the NE value.
Definition (Node importance degree). The node importance degree of node is defined as its NE value, formulated as NE [|S(G.) - SS where G; denotes the i-control graph obtained after the perturbation of node i. A node with higher NE value considered more important and possesses hub attributes, exhibiting greater disparity between the density matrix-based -bssed spectral entropy of the original graph and that of the perturbed graph.
Notably, NE measures the node importance in terms of the influence of one node the global topology and information diffusion throughout the graph. Compared to other methods, such as degree centrality (DC), betweenness centrality (BC), closeness centrality (CC), and eigenvector centrality (EC), which emphasize the local structure or local message passing, NE is more reliable for node importance measuring (especially in communication networks like brain graphs). Appendix gives detailed discussion. The next theorem shows the quantification analysis of entanglement.
Theorem (Quantification analysis of entanglement). Assume that the number of connected compo- nents in the i-control gruph is the same the original graph, denoted as Or a. The NE value of node i approximated as
2mym2 2
AZ
Z, + loga!
NE
(6)
2n a) ZZ,
where, n and m are the numbers of nodes and edges, respectively. Z, stands as the functtoon for Gü and AZ Z Z.
4

---

Published as a conference paper at ICLR 2025
The complete proof is given in Appendix A.2 by assigning each node the learnable embedding vector of its node importance degree in the input layer. For node i, its node representation in input layer is updated to x'x:
Then, we design node importance encoding ((-)
x'= (x,) X; + XN8(8)
X( is the learnable embedding vector specified by (1
3.2 FUNCTIONAL MODULE-AWARS SELF-ATTENTION
In this section, we first propose a community contrastive strategy-based functional module extractor, which can capture the functional segregation and integration characteristics of the brain. Then, the obtained functional module-aware node representations from the extractor are learned by an updated self-attention mechanism, which can calculate node similarity at the functional module level.
3.2.1 COMMUNITY CONTRASTIVE STRATEGY- FUNCTIONAL MODULE EXTRACTOR
Given a brain graph G, the representation of node i after node importance encoding x',, we then obtain its updated representation after our functional module extractor v5, indicated as h : :i, (i), where M, stands for the functional module node i belongs to.
In dh. first utilize unsupervised community detection method, Louvain algorithm (Blondel et al. 2008), to highlight the functional modules. This approach particularly addresses the challenge posed by the absence of functional module labels, which is limitation encountered in many empirically labeled datasets. Then, we apply graph augmentation to generate two graph views of G by modifying its structural information and enhancing functional modules. Particularly, we apply an edge dropping strategy (Rong et al.) 2020; Chen et al.) (023) to achieve graph augmentation. The main idea of the edge dropping strategy dropping less important edges while preserving the functional module structure. Details of edge dropping strategy are given in Appendix C After graph augmentation, we can obtain two augmented graph views G and G??
Then, G and G are fed into graph neural network- view encoder GNN(.) to obtain the representations of two graph views, denoted as H' GNN(G")) and H GNN( To enhance inter-module differences and intra-module similarities, we design contrastive objective strategy by setting nodes from the same function module as positive samples, while those from different function modules as negative samples. We adopt the InfoNCE (Oord et al. 2018) the contrastive loss function:
C= C--- n
erp(Sim(h)) h)")))
log Ne j==1= cxp(Sim(( (h). hj)) + Σj= exp(Sim (h) h2))
Here, Sim(.) () the score function measuring the similarity between two nodes. For an anchor node i in G1, its representation is h,, we consider the nodes within functional module M, from both graphs G and G2 as the positive samples, denoted as h otherwise they are considered as negative samples. Neg indicates the number of negative samples in graph view. Consequently, the updated functional module-aware representation of node can be obtained, denoted as h;-
3.2.2 UPDATED SELF-ATTENTION MECHANISM
After obtaining the functional module aware node representations, we design an updated self-attention mechanism. Inspired by Mialon et al. (202I). we design the self-attention mechanism as kernel smoother to capture the similarity between each pair of nodes. Particularly, we define trainable exponential kemelss on functional module-aware node representations. The updated self-attention is formulated as:
exp exp( cap((Wobi (Woh.. /
FM-Attn(i) = Σ
f(b;).
(9)
JEV Suevepp (Wah, Wghh)/
Here, exp (Woho, Wy /ddx is non-negative kernel, where (..)) indicates the dot product.
f(+) is linear value function.
5

---

Published as a conference paper at ICLR 2025
This updated self-attention mechanism can capture node similarity from the functional module level, without destroying the coherence of functional module-aware node representations. Representations of nodes in the same functional module closer, while those from different modules keep farther. Therefore, the obtained node representations are more biologically plausible, preserving functional segregation and integration characteristics. The next theorem guarantees that our self-attention function can controllably preserve functional modules.
Theorem 2 (Controllability analysis of functional module-aware self-attention). Assume the func- tional module extractor is bounded by constant Co. For any two nodes a and the distance between their representations after the functional module-awar self-attention is bounded by:
|FM-Attn(a) FM-Attn(b)]l S CMllba- - -hol-
(10)
h == Ma) and ha Mb) are representations of nodes and b after the functional module extractor, respectively. CM is constant.
This theorem demonstrates that node representations will maintain their relative distances after undergoing the functional module-aware self-attention mechanism. For example, after the self- attention, two nodes within the same functional module will remain close to each other, while two nodes from different functional modules will remain distant from each other. This crucial for ensuring that the self-attention mechanism preserves functional modules while capturing the similarity between nodes. The proof of this theorem is provided in Appendix A..3
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Datasets. We conduct experiments on IMRI data collected from three benchmark datasets. (1) Autism Brain Imaging Data Exchange (ABIDE) dataset. This dataset contains resting-state fMRI data of 109 anonymous subjects (age range: 6-64 years old) including 516 Autism spectrum disorder patients and 493 normal controls. The ROIs of brain graphs in ABIDE are defined by Craddock 200 atlas (Craddock et al. 2012). (2) Alzheimer's 's Disease Neuroimaging Initiative (ADND) dataset. The collected dataset comprises total of 407 subjects, including 190 normal controls, 170 mild cognitive impairment (MCI) patients, and 47 Alzheimer's disease patients, carefully matched for both age and sex ratio. The ROI definition in ADNI dataset is based on AAL atlas 2002). (3) Attention Deficit Hyperactivity Disorder (ADHI-200) dataset. This dataset contains 459 subjects from 7 to 21 years old. 230 subjects are typically developing individuals and 229 subjects ADHD patients. The ROI definition in ADHD dataset is also based Craddock 200 atlas. The number of ROIS in ABIDE, ADNI, and ADHD-200 datasets are 200, 90, and 190, respectively. Notably, brain graphs are constructed by computing the Pearson correlation coefficient (PCC) (Cohen ett al. (009) between ROIs based on the collected fMRI data. In particular, thresholds are set to keep edges with higher PCC values (weights) and drop those with lower PCC values (see Table 4 in Appdendix D.I.
et al.)
Evaluation Metrics. We evaluate our model on the graph classification task. For ABIDE and ADHD-200 datasets, our model aims to detect whether the subject is patient or normal control. Therefore, the classification tasks in these two datasets are binary classification problems. For ADNI dataset, there are three groups including normal controls, mild cognitive impairment patients, and Alzheimer's s disease patients. Thus, disease detection in ADNI dataset is a multiple classification problem. We use five metrics to evaluate the model performance: (1) Test accuracy (ACC) indicates the ratio of brain graphs that are correctly classified out of all samples; (2) Fl score is the harmonic mean of precision and recall; (3) Area under the receiver operating characteristic curve (AUC) shows the trade-off between true positive rate and false positive rate; (4) Sensitivity (Sen.) refers to true positive rate; (5) Specificity (Spe.) gives the true negative rate. For the multiclass classification task on the ADNI dataset, we use macro averaging for the Fl score, sensitivity, and specificity. All results are the average values of 10 random runs on test sets with the standard deviation.

---

Published as a conference paper at ICLR 2025
Baseline Methods. We compare our model with state-of-the-ar methods: (1) typical machine learning
(ML) methods, including SVM (with a linear kernel) and Random Forest; (2) graph transformer
models, including SAN (Kreuzer al., 2021).
Graph Transformer (Graph Trans.) (Dwivedi &
Bresson 2020), Graphormer (Ying et al. 2021 BRAINNETTE (Kan et al.) ) SAT (Chen
al. 2022), Polynormer (Deng al.) 2024); Gradformer (Liu al. (024a), and GTSP (Liu
ct al.
2024b); ; (3) graph neural networks for brain graph analysis, including GAT (Velickovic et al.
2018
BrainGNN (Li al., 202T), BrainGB (Cui al. 2022). MCST-GCN (Zhu et al. (024), and
GroupBNA (Peng et al. 2024). For the SAT model, we consider its two variants as baselines: SAT
without positional encoding (SAT-PE) and SAT with positional encoding (SAT+PE).
Implementation Details. Our model is implemented using Py Torch Geometric v2.0.4 and Py PyTochh 11.9.1. Model training is performed on an NVIDIA A6000 GPU with 48GB of memory. Our model is trained using the AdamW optimizer (Loshchilov & Hutter] (019). and the cross-entropy loss is
used for classification tasks. Each dataset randomly split, with 80% used for training. 10% for validation, and 10% for testing. Full implementation is given in Appendix D.I
4.2 RESULTS
The experimental results (ACC and AUC) on the three datasets are summarized in Table [ The results for F1, Sen., and Spe. are provided in Appendix As experimental results show, the overall performance of BioBGT is superior to that of other baselines on all three datasets. For example, in the ADHD-200 dataset, we can see that the performance of BioBGT is distinguished, with the best accuracy and Fl score. Notably, our Fl score is around 4.21% % higher than the second-best baseline. In the ABIDE dataset, BioBGT achieves a 5.76% improvement accuracy over the second-best baseline. The experimental results demonstrate that model excels in various brain disorder detection tasks.
Table 1: Results (mean margin of error) on three datasets (%)
ADHD-200
ABIDE
ADNI
Method
ACC
AUC
ACC
AUC
ACC
AUC
SVM
53.56++2.73
54.66++..40
19.01
49.05
2..29
49.88
ML Methods
Random Forest
58.96:42.77
59.49+2.38
51.1443.008
51.41=3.23
49.03=11227
58.1842.31
SAN
$1.09++2.00
51.22+2.21
19.80=
50.20=2.34
34.44+4.61
49.23+22.67
Graph Trans.
50.76:12.07
51.4911.15
50.20:10.50
48.20=0.16
40.28
52.3112..44
Graphormer
61.60::0.00
11.50
58.40:10.68
57.61=0.72
35.64:2..17
48.19:12..9
Graph
SAT-PE
60.00:1..73
59.68:12.60
60.60:1.111
59.14:4.56
39.96:11.51
18.17:1..57
Transformer
SAT+PE 64.44113.45
64.2113.40
58.76:14.88
69.29-15488
41.5111.011
12.13.15.74
Models
BRAINNETTF
70.80+-2.70
79.36+3.44
68.2442.24
78.38-343
47.39+33.11
55.72+7.13
Polynormer
64.78+22.34
63.61+2.43
57.03++0.96
16.42-1.56
41.85+2.12
54.34+4.37
Gradformer
68.9443.118
67.8344.66
61.56:2.113
61.75=4.29
46.5442.72
53.88+22377
GTSP
61.70++3.81
61.41+2.90
61.37+3.59
60.43=3.47
47.27+3.81
53.59+3.226
GAT
55.38-+3.18
54.97+3.28
53.51++2.54
53.41=2488
34.99-4..43
51.73++6.66
BrainGNN
15.76:11.20
58.00+0.49
51.3411.17
54.27=0.66
43.33114.08
50.21
Graph Neural
BrainGB
68.20+7781
74.64-10.10
65.1243.90
70.32-3.66
44.3443.90
62.24414.68
Networks
MCST-GCN
59.06:12.69
39.05
54.22:12.40
55.18:2.35
48.4411.112
62.25
GroupBNA
69.87:13.02
71.1614.53
63.14:12.65
7..30=3.81
46.72-11.33
50.8511.10
Our Model
BioBGT
71.06:10.08
71.6411.14
74.00-12.01
73.33=2.37
52.08-12.08
62.33-15.98
4.3 ABLATION STUDIES
We conduct a series of ablation studies on three datasets to validate the effectiveness of each component in BioBGT. To verify how the network entanglement-basee node importance encoding benefits the model performance, we conduct an ablation experiment by removing the node importance encoding, denoted as "-NE". In addition, to show the effectiveness of our functional module-aware self-attention, we remove the community contrastive strategy- bbased functional module extractor and replace our FM- FM-Attn(-) with normal self-ateention Attn(-) (see Equation (2)), denoted as "-FM-Attn", Figure commpares the performance of BioBGT with the altered models on three datasets. BioBGT achieves superior performance compared the model without node importance encoding
7

---

Published as a conference paper at ICLR 2025
(-NE), indicating that node importance encoding method is crucial for the model performance. Furthermore, BioBGT shows better performance than the altered model -FM-Attn. This indicates that it is essential to encode node similarities from the functional module level and preserve functional segregation and integration characteristics of brain graphs.
70-
40
20-
5ps.
34m
AUC
5p4
ADHD-200
(b) ABIDE
(c) ADNI
Figure 3: Model performance of BioBGT and its altered models.
4.4 COMPARATIVE ANALYSIS OF NODE IMPORTANCE MEASUREMENT
To validate the effectiveness of NE in measuring node importance, we perform a comparative analysis between BioBGT and its variants, which replace NE-based node importance encoding with (1) Laplacian matrix- -based positional encoding, denoted as "+PE"; (2)degree centrality encoding. denoted as "+DC"; (3) Laplacian matrix and degree centrality encoding, denoted "+PE+DC"; (4) betweenness centrality encoding, denoted as "+BC"; (5) closeness centrality encoding, denoted "+CC", (6) eigenvector centrality, denoted "+EC". The results summarized in Table and Table 8 (see Appendix D.2). The overall performance of BioBGT is significantly better than other variants. This indicates that our node importance encoding method is crucial for the model performance, suggesting NE is more reliable for node importance measuring.
Table 2: The results (F1, ACC, AUC) for BioBGT and its variants on three datasets (%).
ABIDE ACC
ADNI ACC
ADHD-200
F1
AUC
F1
AUC
F1
ACC
AUC
+PE
54.00-1-2.97
60,60:2..25
60.1 2:05
30.09:1..36
49.4342442
$2.14=2.41
69.21:17.14
67,56.33.24
67.391:2.80
+DC
59.73.14.23
61.20
61.28
27.27
50.57
55.45
74.22.1..35
69.78.2.33
70.18:1.288
PEADC
56.64412.40
63.0011..6
63.3221.55
27.77
1..94::..25
55.08-3.94
73.60-14228
70.6714.00
70.79:4.10
+BC
$2.77=:1.30 70.00:26.12
63.6247.29
26.70:4226
48.11#:3.00
$1.38=7.88
72.20+3.61
71.12=0888
70.09=1.05
62.43:t1.53
48.1113361
50.68111.24
73.78:15.11
72.69-1.888
69.38
+BC
53.13:11.62
71.25:18.00
66.67.19888
27.75.19.12
47.64
84.39
73.99
71.1111111
68.99
BioBCT
68.41:12.19 74.00:12.01 7.333
32.29-12.11
52.08:12.088
61.33-3.98
74.63-11118
71.06:10.08
71.6421.14
4.5 BIOLOGICAL PLAUSIBILITY ANALYSIS
We assess the biological plausibility of our node impor- lance encodings and functional module-aware node rep- resentations by proving their consistency with existing neuroscience knowledge and providing reasonable explain- ability.
Biological Plausibility in Node Importance Encod- ing. Node efficiency (NEff) one of the key metries for complex graphs, which is used to measure the effi- ciency of information propagation between a given node and all other nodes in graph (Latora & Marchiori] 2001). can effectively quantify the communication strength of nodes, capturing their ability facilitate information prop- agation across the graph. A high NEff value suggests that the node plays crucial role hub for information
Figure The NE and NEff values of 50 randomly selected nodes from sample the ABIDE dataset.
propagation, enabling close and effective
8

---

Published as a conference paper at ICLR 2025
communication with the rest of the graph. Notably, NEff has been widely utilized in the field of neuroscience to identify critical regions in functional brain graphs (Bullmore & Sporns 2009).
Therefore, compare the NE values of each node with their corresponding NEff values to evaluate the validity of NE as a measure of node importance. Figure 4 presents the NE and NEff values of 50 randomly selected nodes from sample randomly chosen in the ABIDE dataset. In addition, visualizations of NE and NEff values for all nodes in a randomly selected graph across three datasets are provided in Appendix D.3 As shown, the changing trend of the NE curve aligns closely with that of the NEff curve, demonstrating that nodes with high NE values also exhibit high NEff values, and vice versa. This consistency suggests that NE aligns with biological plausibility as a measure of node importance in information propagation. To further validate the effectiveness of NE in measuring node importance, we also compare each node's NE value to its average functional connectivity (FC) strength with all of the other nodes (see Appendix D.4h.
Biological Plausibility in Functional Module-Aware Self-Attention. Figure | displays the heatmaps of the average self-attention scores from the ADHD test set, output by Graphormer (a), SAT (b), and our functional module-aware self-attention mechanism (c). Based on the empirical labels of brain regions and functional modules (Dosenbach et al. 2007 2010), the ROIs are classified
into functional modules, including visual cortex (Vis), motion control (MC), cognition control (CC), auditory cortex (Aud), language processing (LP), and exccutive control (EC). The detailed functional module division is provided in Appendix D.5 As illustrated in Figure 3 compared to the heatmaps produced by Graphormer and SAT, the heatmap of model clearly shows that the learned attention scores of our model align better with the division of functional modules. For example, nodes within the visual cortex exhibit higher attention similarity. This observation verifies that our functional module-aware self-attention preserves the characteristics of functional segregation and integration within brain graph representations better. To further analyze the relationships between the attention patterns learned by our model and the biological patterns of target diseases, we visualize the self-attention scores of samples from each class in the three datasets (see Appendix [D.]
V4
A.d
Ast
NC
NC
3
5
(a) Graphormer
(b) SAT
(c) BioBGT
Figure 5: The heatmaps of the average self-attention scores. Compared to other methods, heatmap (c) shows that learned attention scores of BioBGT align better with the division of functional modules.
5 RELATED WORK
5.1 BRAIN GRAPH ANALYSIS
Brain graphs, reflecting the connections in human neural system, are constructed from various brain health data, such as functional magnetic resonance imaging (fMRI, positron emission tomography (PET), and electroencephalographh (EEG) (Bullmore & Sporns 2009; Bessadok et al. 2022 Yu
al.) 20224). Recently, graph learning-basee based brain graph analysis has attracted increased attention. dominating range of tasks (e.g., brain disease detection and treatment recommendation) (Sun et al.)
2024. Kan et al. 2022 Liu et al. 2023 Ding et al.) (023). NeuroGraph (Said et al., 2024) collects various brain connectome datasets for benchmarking graph learning models in brain graph analytical tasks (e.g., gender identification). BrainPrint (Wang et al. 2020b) develops network estimation module and graph analysis module to embed EEG features. BrainGNN (Li et al. 2021 contains special ROI-aware graph convolutional layers to capture the functional information of brain networks for fMRI analysis. BrainGB (Cui et al. 2022) summarizes the pipelines of brain graph construction. BRAINNETTF (Kan et al. (022) utilizes a Transformer-bar
model to analyze brain graphs, while ignoring the structural encoding of ROIs and failing preserve the small-world architecture of brain

---

Published as a conference paper at ICLR 2025
graphs. MSE-GCN (Lei et al. 2023) applies multiple parallel graph convolutional network layers to encode brain structural and functional connectivities to detect early Alzheimer's ' disease (AD). GroupBNA (Peng et al. 2024) constructs group-specific brain networks via group-adaptive brain
network augmentation strategy.
5.2
GRAPH TRANSFORMERS
Graph transformers attempting to generalize Transformer models to graph data have shown significant performance in graph representation tasks (Kong et al. 2023 [uo et al. 2024b a). SAN (Kreuzer al. 202T) leverages the full Laplacian spectrum as the learned of input nodes, emphasizing the global structural information of the graph. Graphomer (Ying et al. 202 introduces three SE methods to the Transformer architecture, including centrality encoding, spatial encoding and edge encoding, for graph representation learning. SAT (Chen et al., 2022) proposes structure-awar self-attention mechanism to extract subgraph representations of nodes. SGFormer (Wu et al., (2024a) presents single-layer attention model utilizing linear complexity to capture global dependencies among nodes. Geoformer (Wang et al., 2024) considers atomic environments as the PE of nodes in molecular graphs. EXPHORMER (Shirzad et al. 2023) proposes sparse attention mechanism based on virtual global nodes and expander graphs for large graph representation learning. GRIT (Ma et al. 2023b) proposes learned PE based on relative random walk probabilities and flexible self-attention mechanism aiming to update both node and node-pair representations.
6 CONCLUSION
This paper presents the Biologically Plausible Brain Graph Transformer (BioBGT) model with network entanglement-ben based node importance encoding technique and an updated functional module- aware self-attention mechanism. Extensive experiments on three benchmark datasets demonstrate our BioBGT outperforms state-of-the-art baselines, as well as enhances the biological plausibility of brain graph representations. Significantly, BioBGT offers valuable insights into enhancing the efficacy of brain graph analytical tasks, notably in the realm of improving disease detection. Our work could potentially advance digital health. Importantly, this work contributes to the intersection of neuroscience and artificial intelligence by proposing a brain graph representation learning technique that enhances biological plausibility.
While these findings are encouraging, some limitations remain. Firstly, according to current neuro- science knowledge, the biological properties of the brain highly complex and remain uncertain, with many underlying mechanisms still requiring further research. Therefore, it is unlikely that a fully biologically plausible brain graph can be constructed. Instead, we can strive to build brain graphs that enhance biological plausibility by incorporating insights from existing knowledge, such as the brain's 's small-wond world architecture as one of its key features. Then, the computation complexity of network entanglement and the quadratic complexity of our functional module-awan restrict the applicability of BioBGT. Although we have managed to keep the number of parameters comparable to other models, computational efficiency still needs future improvement. Therefore, it is worth exploring how to trade off the biological plausibility brain graph representations and model computation complexity.
self-attention module
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Anushree Bannadabhavi, Soojin Lee, Wenlong Deng, Rex Ying, and Xiaoxiao Li Community-awar transformer for autism prediction in fMRI connectome. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 287-297. Springer, 2023.
Alaa Bessadok, Mohamed Ali Mahjoub, and Islem Rekik. Graph neural networks in network neuroscience. IEEE Transactions Pattern Analysis and Machine Intelligence, 4555):5833, 2022.
Richard Betzel, Maria Grazia Puxeddu, and Caio Seguin. Hierarchical communities in the larval drosophila connectome: Links to cellular annotations and network topology. Proceedings of the National Academy of Sciences, 22220771112 2024.
Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks. Journal of Statistical Mechanics: Theory and Experiment, 2008 (10):P10008, 2008.
Ed Bullmore and Olaf Sporns. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience, 10(3):186-199 2009.
Dexiong Chen, Leslie O'Bray, and Karsten Borgwardt. Structure-awars transformer for graph representation learning. In International Conference on Machine Learning, pp. 3469-34899 PMLR, 2022.
Han Chen, Ziwen Zhao, Yuhua Li, Yixiong Zou, Ruixuan Li, and Rui Zhang. CSGCL: community- strength-enhanced graph contrastive learning. In Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, IJCAI, pp. 2059-2067 2023.
Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. Noise Reduction in Speech Processing, pp. 1-4, 2009.
R Cameron Craddock, G Andrew James, Paul E Holtzheimer III, Xiaoping P Hu, and Helen S Mayberg. A whole brain fMRI atlas generated via spatially constrained spectral clustering, Human Brain Mapping, (38))19144-1928 2012.
Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang He, Ying Guo, and Carl Yang. BrainGB: A benchmark for brain network analysis with graph neural networks. IEEE Transactions on Medical Imaging, 42(2):493-506, 2022.
Zhengjia Dai and Yong He. Disrupted structural and functional brain connectomes in mild cognitive impairment and Alzheimer's disease. Neuroscience Bulletin, 30:217-232, 2014.
Kousik Das, Sovan Samanta, and Madhumangal Pal. Study on centrality measures in social networks: survey, Social Network Analysis and Mining, 8:1-11, 2018.
Manlio De Domenico and Jacob Biamonte. Spectral entropies as information-theoreticc theoretic tools for complex network comparison. Physical Review X, 6(4):041062, 2016.
Chenhui Deng, Zichao Yue, and Zhiru Zhang. Polynormer: Polynomial-exprestie graph transformer in linear time. In Proceedings of the International Conference on Learning Representations, ICLR, 2024.
Yi Ding, Neethu Robinson, Chengxuan Tong, Qiuhao Zeng, and Cuntai Guan. Lggnet: Learning from local-global-grapp representations for brain-computer interface. IEEE Transactions on Neural Networks and Learning Systems, 2023.
Nico UF Dosenbach, Damien A Fair, Francis M Miezin, Alexander L Cohen, Kristin K Wenger, Ronny AT Dosenbach, Michael D Fox, Abraham Z Snyder, Justin L Vincent, Mareus E Raichle, et al. Distinct brain networks for adaptive and stable task control in humans. Proceedings of the National Academy of Sciences, 104(26): 1073-11078, 2007.
11

---

Published as a conference paper at ICLR 2025
Nico UF Dosenbach, Binyam Nardos, Alexander L Cohen, Damien A Fair, Jonathan D Power, Jessica A Church, Steven M Nelson, Gagan Wig, Alecia C Vogel, Christina N Lessov-Schlaggee et al. Prediction of individual brain maturity using fMRL. Science, 12965997) 2010.
Vijay Prakash Dwivedi and Xavier Bresson. generalization of transformer networks to graphs arXiv preprint arXiv:2012.09699 2020.
Vijay Prakash Dwivedi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. Graph neural networks with learnable structural and positional representations. The Tenth International Conference on Learning Representations, ICLR. OpenReview.net, 2022.
Bolin Gao and Lacra Pavel. On the properties of the sofumax function with application in game theory and reinforcement learning, arXiv preprint arXiv:1704.00805, 2017.
Johannes Gasteiger, Stefan Weißenberger, and Stephan Günnemann. Diffusion improves graph learning. Advances Neural Information Processing Systems, 32, 2019.
Simon Geisler, Yujia Li, Daniel J Mankowitz, Ali Taylan Cemgil, Stephan Günnemann, and Cosmin Paduraru. Transformers meet directed graphs. In International Conference on Machine Learning, pp. 11144-11173 172. PMLR, 2023.
C Lee Giles, Kurt D Bollacker, and Steve Lawrence. CiteSeer: An automatic citation indexing system. In Proceedings of the Third ACM Conference on Digital Libraries, pp. 89-98, 1998.
Yiming Huang, Hao Wang, Xiao-Long Ren, and Linyuan Lü. Identifying key players in complex networks via network entanglement. Communications Physics, 7(1):19, 2024.
Takashi Itahashi, Takashi Yamada, et al. Altered network topologies and hub organization in adults with autism: a resting-state fMRI study. PloS one, 9()): 15, 2014.
Edward G Jones. The thalamus. Springer Science & Business Media, 2012.
Xuan Kan, Wei Dai, Hejie Cui, Zilong Zhang, Ying Guo, and Carl Yang. Brain network transformer. Advances in Neural Information Processing Systems, 35:25586-25599, 2022.
Kezhi Kong. Jiuhai Chen, John Kirchenbauer, Renkun Ni, C Bayan Bruss, and Tom Goldstein. GOAT: A global transformer large-scale graphs. In International Conference on Machine Learning, pp. 17375-17390. PMLR, 2023.
Devin Kreuzer, Dominique Beaini, Will Hamilton, Vincent Létourneau, and Prudencio Tossou. Rethinking graph transformers with spectral attention. Advances in Neural Information Processing Systems, 34:21618 126621629, 2021.
Vito Latora and Massimo Marchiori. Efficient behavior of soold-world networks. Physical Review Letters, 19190 2001.
Baiying Lei, Yun Zhu, Shuangzhi Yu, Huoyou Hu, Yanwu Xu, Guanghui Yue, Tianfu Wang, Cheng Zhao, Shaobin Chen, Peng Yang, al. Multi-scale enhanced graph convolutional network for mild cognitive impairment detection. Pattern Recognition, 134: 109100, 2023.
Xiaoxiao Li, Yuan Zhou, Nicha Dvomek, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H Staib, Pamela Ventola, and James S Duncan. BrainGNN: Interpretable brain graph neural network for fmri analysis. Medical Image Analysis, 74: 74:102233, 2021.
Yibo Li, Xiao Wang, Hongrui Liu, and Chuan Shi. A generalized neural diffusion framework on graphs. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 8707-8715, 2024.
Xia Liang, Jinhui Wang, Chaogan Yan, Ni Shu, Ke Xu, Gaolang Gong, and Yong He. Effects of different correlation metrics and preprocessing factors woald-world brain functional networks: resting-state functional MRI study. PloS One, 7(3)xe32766, 2012.
Xuhong Liao, Athanasios Vasilakos, and Yong He. world-world human brain networks: perspec- tives and challenges. Neuroscience & Biobehavioral Reviews, 77:286-300, 2017.
12

---

Published as a conference paper at ICLR 2025
Chuang Liu, Zelin Yao, Yibing Zhan, Xueqi Ma, Shirui Pan, and Wenbin Hu. Gradformer: Graph transformer with exponential decay. Proceedings of the Thirty- International Joint Confer- on Artificial Intelligence, Jeju, South Korea, August 3-9, pp- 2171-2179, 2024a.
Chuang Liu, Yibing Zhan, Xueqi Ma, Liang Ding, Dapeng Tao, Jia Wu, Wenbin Hu, and Bo Du. Exploring sparsity in graph transformers. Neural Networks, 174: 106265, 2024b.
Lingwen Liu, Guangqi Wen, Peng Cao, Tianshun Hong, Jinzhu Yang, Xizhe Zhang, and Osmar R Zaiane. BrainTGL:: A dynamic graph representation learning model for brain network analysis. Computers in Biology and Medicine, 153:106521, 2023.
Iyya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR. OpenReview.net, 2019.
Renqiang Luo, Huafei Huang, Shuo Yu, Xiuzhen Zhang, and Feng Xia. FairGT: fairness-aware graph transformer. In Proceedings of the 33rd International Joint Conference on Artificial Intelli- gence, 2024a.
Yuankai Luo, Veronika Thost, and Lei Shi. Transformers over directed acyclic graphs. Advances in Neural Information Processing Systems, 36, 2024b.
Christopher W Lynn and Danielle S Bassett. The physics of brain network structure, function and control. Nature Reviews Physics, 1(5):318-32,, 2019.
Anjun Ma, Xiaoying Wang, Jingxian Li, Cankun Wang. Tong Xiao, Yuntao Liu, Hao Cheng, Juexin Wang, Yang Li, Yuzhou Chang, et al. Single-cell biological network inference using heterogeneous graph transformer. Nature Communications, 14(1):964, 2023a.
Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soree Soriio, Puneet K Dokania, Mark Coates, Philip Torr, and Ser-Nam Lim. Graph inductive biases in transformers without message passing. In International Conference on Machine Learning, pp.
PMLR, 2023b.
Andrew Kachites McCallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3:127-163, 2000.
Grégoire Mialon, Dexiong Chen, Margot Selosse, and Julien Mairal. Graphit: Encoding graph
structure in transformers. arXiv preprint arXivv2216..00 05667, 2021.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint 1rXiv:1807 2018.
Ciyuan Peng, Mujie Liu, Chenxuan Meng, Shuo Yu, and Feng Xia. Adaptive brain network aug- mentation based on group-aware graph learning. In The Second Tiny Papers Track at ICLR 2024, 2024.
Josef P Rauschecker and Sophie K Scott. Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing. Nature Neuroscience, 12(6):718 7241 2009.
Yu Rong, Wenbing Huang, Tingyang Xu, and Junzhou Huang. DropEdge: Towards deep graph convolutional networks on node classification. 8th International Conference on Learning Representations, ICLR, 2020.
Mikail Rubinov and Olaf Sporns. Complex network measures of brain connectivity: uses and interpretations. Neurolmage, 52(3):1059-1069 2010.
Anwar Said, Roza Bayrak, Tyler Derr, Mudassir Shabbir, Daniel Moyer, Catie Chang, and Xeno- fon Koutsoukos. Neurograph: Benchmarks for graph machine learning in brain connectomics. Advances in Neural Information Processing Systems, 36, 2024.
Emesto Sanz-Arigita, Menno M Schoonheim, et al. Loss of wsolll-world' networks in Alzheimer's disease: graph analysis of fMRI resting-state functional connectivity. PloS one, 5(11)813378 2010.
13

---

Published as a conference paper at ICLR 2025
Caio Seguin, Olaf Sporns, Andrew Zalesky, Fernando Calamante, et al. Network communication models narrow the gap between the modular organization of structural and functional brain networks. Neurolmage, 257:119323, 2022.
Caio Seguin, Olaf Sporns, and Andrew Zalesky. Brain network communication: concepts, models and applications. Nature Reviews Neuroscience, 24(9):557-574, 2023.
Ahsan Shehzad, Feng Xia, Shagufta Abid, Ciyuan Peng. Shuo Yu, Dongyu Zhang, and Karin Verspoor. Graph transformers: A survey. arXiv preprint arXiv:2407.09777, 2024.
Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica Sutherland, and Ali Kemal Sinop. Exphormer: Sparse transformers for graphs. In International Conference Machine Learning, pp- 31613-31632. PMLR, 2023.
Chandra S Sripada, Daniel Kessler, and Mike Angstadt. Lag in maturation of the brain's intrinsic functional architecture in aitentioo-deffccihyyeeaciiiii disorder. Proceedings of the National Academy of Sciences, 11133))14259-14466 2014.
Ke Sun, Ciyuan Peng, Shuo Yu, Zhuoyang Han, and Feng Xia. From EEG data to brain networks: Graph learning based brain discase diagnosis. IEEE Intelligent Systems, pp. 1-9, 2024.
Larry W Swanson, Joel D Hahn, and Olaf Sporns. Neural network architecture of a mammalian brain. Proceedings of the National Academy of Sciences, 121(39)):2413422111,, 2024.
Nathalie Tzourio Mazooe, Brigitte Landeau, Dimitri Papathanassiou, Fabrice Crivello, Octave Etard, Nicolas Deleroix, Bernard Mazoyer, and Mare Joliot. Automated anatomical labeling of activations in SPM using a macroscopic anatomical parcellation of the MNI MRI single-subject brain. Neurolmage, (5())2273-289 2002.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. Graph attention networks. In 6th International Conference on Learning Representations, ICLR, 2018
Mengjing Wang, Zhishan Hu, Lu Liu, Haimei Li, Qiujin Qian, and Haijing Niu. Disrupted functional brain connectivity networks in children with cenationndeficcdyyyeeaettttt disorder: evidence from resting-state functional near-infrared spectroscopy. Neurophotonics, 7():015012-015012, 2020a.
Min Wang, Jiankun Hu, and Hussein A Abbass. BrainPrint: EEG biometric identification based on analyzing brain connectivity graphs. Pattern Recognition, 107310, 2020b.
Yusong Wang, Shaoning Li, Tong Wang, Bin Shao, Nanning Zheng, and Tie- Yan Liu. Geometric transformer with interatomic positional encoding. Advances in Neural Information Processing Systems, 36, 2024.
Christian Weedbrook, Stefano Pirandola, Raúl García-Patrón, Nicolas J Cerf, Timothy C Ralph, Jeffrey Shapiro, and Seth Lloyd. Gaussian quantum information. Reviews of Modern Physics, 84(2):621, 2012.
Qitian Wu, Wentao Zhao, Chenxiao Yang, Hengrui Zhang, Fan Nie, Haitian Jiang, Yatao Bian, and Junchi Yan. Simplifying and empowering transformers for large-graph representations, Advances in Neural Information Processing Systems, 36, 2024a.
Yuon-hao Wu, Ella Podvalny, Max Levinson, and Biyu J He. Network mechanisms of ongoing brain activity's 's influence on conscious visual perception. Nature Communications, 15(1):5720, 2024b.
Yujie Xing, Xiao Wang, Yibo Li, Hai Huang, and Chuan Shi. Less is More: on the over wer-globaliziig problem in graph transformers. In Proceedings of the International Conference on Machine Learning, ICML. 2024.
14

---

Published as a conference paper at ICLR 2025
Yun Yi, Haokui Zhang, Rong Xiao, Nannan Wang, and Xiaoyu Wang. NAR-Former V2: Rethinking transformer for universal neural network representation learning. Advances Neural Information Processing Systems, 36, 2024.
Chengxuan Ying, Tianle Cai, Shengjie Luo, Shuxin Zheng, Guolin Ke, Di He, Yanming Shen, and Yan Liu. Do transformers really perform badly for graph representation? Advances in Neural Information Processing Systems, 34:28877-28888. 2021.
Shuo Yu, Ciyuan Peng, Yingbo Wang, Ahsan Shehzad, Feng Xia, and Edwin R Hancock. Quantum graph learning: Frontiers and outlook. arXiv preprint arXiv:2302.0088 2023.
Shuo Yu, Shan Jin, Ming Li, Tabinda Sarwar, and Feng Xia. Long-range brain graph transformer. In The eighth- Annual Conference on Neural Information Processing Systems, 2024.
Xiaohu Zhao, Yong Liu, et al. Disrupted small-world brain networks in moderate Alzheimer's disease: resting-state fMRI study, PloS one, 7(3)e33340, 2012.
Qi Zhu, Shengrong Li, Xiangshui Meng, Qiang Xu, Zhiqiang Zhang, Wei Shao, and Daoqiang Zhang. Spatio-temporal graph hubness propagation model for dynamic brain network classification. IEEE Transactions on Medical Imaging. 2024.
15

---

Published as a conference paper at ICLR 2025
A PROOFS
A.1 PROOF OF PROPOSITION
Proposition T (Density matrix as structural information.) The structural information a brain graph G. including the connection strength between nodes and the degree distribution of nodes, is encoded by its density matrix, which stands as a normalized information diffusion propagator and formulated as PG z Here, e e->h is the information diffusion propagator, denotes the positive parameter, L is the Laplacian matrix of G, and Z Tr(e=>h) represents the partition function of G.
Proof. According to the theory of diffusion processes, information diffusion process in graph can represented in exponential form as e>> (Gasteiger et al., 2019 et al. 2024). Therefore, the density matrix can be regarded as diffusion matrix along with structural information. In particular, the Laplacian matrix, L = - .A, indicates the difference between the degree matrix D and the adjacency matrix A. The degree matrix D is diagonal matrix with its diagonal element dy representing the degree of nodes i. The elements ai of the adjacency matrix A represent the connection strengths between nodes i and Therefore, the density matrix PG captures the connection strengths and degree distribution information among nodes in the graph G, encoding the global structural information of the graph G.
A.2 PROOF OF THROREM
Theorem [ (Quantification analysis of entanglement.) Assume that the number of connected components in the i-control graph is the same the original graph, denoted as a. The NE value node approximated as
2mr7n² 2
AZ
Zi 7
NE x in a²2 ZZ;
+ logz
where, and m are the numbers of nodes and edges, respectively. Z, stands as the partition function for G,, and AZ = Z; Z.
Proof. According to Equation spectral entropy of graph G is
(5) and spectral decomposition theory, the density matrix-
S(G) S(G))
Tr(pp loga PG)
(pG) logz (pG))
where {y(p)) -0y(0))
Therefore, we have
S(G) In 2
(L)A + +log22 Z
(A,(L)))(P))) +log22 7
1n2
(L)()) + +oog Z.
1n
The approximation comes from mean- mean-field approximation. Assume the number of X satisfying A (L) is the same the number of connected components Refer Huang al. (0024)
16

---

Published as a conference paper at ICLR 2025
get
(A(())) - 1
Σ AL
n j=1
Σ
(L)
n a
j=
2m
11 --
Likewise,
1 n (T)
(A/(P)))
n1 j=l
Z
(Ty()
Σ
Z
j=a-1 1
(1
Z
n1
Therefore, S(G) can be approximated as
2mn
S(G)
(1 - + loog Z
In 2
2myn² 2
+ logz Z.
In 2 )²
Then, we can quantify the perturbation of node i on graph G by capturing the changes of density matrix-based spectral entropy from S(G) to S(G.), obtaining N/((i)
NE(I) ||S(G.) - S(G)]l
2mym²
(
2mmyn²
a Z)1 log, 2 2)
~ In 2(n a;)3 (1
+ +log2 Zi)
(1
Zi
In 2( a)²
Suppose a; = a, we can get
2m 2
2myyn²
NE
(1
+ logg Zi
(1
+ log2 Z
2 - a²²
Z,
In 2 a)²
z
2myn2 2
1
2,
+ loga
In 2 a:)2
Z;
2myn2 2
AZ
Z
+ logz
In 2(n )²² ZZ,
Z
Here, AZ 2, Z.
A.3 PROOF OF THEOREM 2
Theorem 2 (Controllability analysis of functional module-aware self-attention.) Assume the functional module extractor is is bounded by constant Cg.- For any two nodes and b, the distance between their representations after the functional module-aware self-attention is bounded by:
[FM-Attn(a) FM-Attn(b)] s CM lb hall-
h2 == (aa and h2 := (b, M₁,) are representations of nodes a and after functional module extractor, respectively. CM is constant.
Proof. The similarity representations of nodes a and b can be denoted as
2, ((Woha: W Wxhi))ev E RR,
Z₁ ((Wobs, W Wgh E R..
17

---

Published as a conference paper at ICLR 2025
Then, the softmar(z) with the k-th coefficient is
exp(2k/ /dx/
so softmar(z);
ΣØ= /xk))
Afterwards, we can get
(MMAttt(aa) FM-Attn(D]]- =
Σ
so amtaa (2.)s
Σ softmaa((+)) (2b)
NEV
KEV
(sof (sofbmar(Za); s0flmaz((z))) s(b:)
so sofftma(zz)) softmaz(zo))
Σ (f,)) iEV
$
1m - 26ll
The first inequality is based on simple matrix norm inequality. and the second inequality is based on the fact that so fomar function is -Lipschitz (Gao & Pavel 2017). C' is the Lipschitz constant. Then, we can infer that
|| 2. - 20012=
Σ
((W@oh ( WWhh, Wxh.) - (Woh,, Wxchi)
1EV
(We(he - hR), Wxhh)) 2
EV
[Wo h.) W h. 11²)
iEV
$ I| 1 lbb
h C W 1
= c W
Wol & bb. - h.
The first inequality comes from the Cauchy-Schward inequality. The second inequality uses the definition of spectral norm, and constant Co the bound of the functional module extractor sh. Therefore, we can infer that
FMM-Att(a) FM-Aun(b)]
S
-1 22oll7nCC.
Vdk
n dx
CC Woll [h. hbsl
V
A constant C'M can be defined as
n
Cm
CC W
Vdx dx
Consequently, we can obtain the inequality in the Theorem
2
FM-Attn(a)) FM-Attn(b)] S C. -hll-
B RELIABILITY OF NE FOR NODE IMPORTANCE MEASURING
Given node i, its importance degree in the information propagation across the graph is defined its NE value, which is obtained by measuring the disparity between the density matrix- matrix-based spectral entropy of the original graph and that of the i-control graph. The density matrix-based spectral entropy
18

---

Published as a conference paper at ICLR 2025
captures both the global topology and information diffusion process of the graph (Huang al. 2024). Therefore, NE can fully reflect the significance of node on the information propagation across the graph. For example, if the density matrix- based spectral entropy of the i-control graph changes greatly compared to the original graph, it indicates that induced perturbation of node i can lead to significant changes in the global topology and information diffusion pattern of the graph. This means node plays an important role in the graph in terms of information propagation. Particularly, brain graphs are communication networks, in which information propagation a crucial aspect (Seguin et al., 2023). Hence, it is essential to measure node importance based on NE reflecting on the changes the global topology and information diffusion of brain graphs.
(a) Node (DC=4) plays an important role in the information diffusion within the graph.
(b) Node b (DC=6) is less important for information diffusion within the graph.
Figure 6: An example of the fragility of DC in measuring node importance in brain graphs High- lighted edges represent paths of information diffusion.
60
Figure 7: Visualization of the DC (a), BC (b), CC (c), EC (d), and NE (e) values of nodes in brain graph from the ABIDE dataset. The colors of nodes are proportional to the normalized values of the DC and NE.
Due to the capability of NE to reflect the global topology and information diffusion, it is more reliable for node importance measuring brain graphs compared other methods, such as centralities (Das) al. 20.8). Four representative centralities, including degree centrality (DC), closeness centrality (CC), betweenness centrality (BC), and eigenvector centrality (EC), emphasize the local structure or local message passing, having limitations to fully capture both global topology and information diffusion process. Therefore, they are not reliable for measuring node importance, especially for brain graphs. For example, Figure Tigure/Jshows shows the fragility of DC in measuring node importance in brain graphs. Node a plays a more important role in information propagation than node b, while its DC value (=4) is lower than that of node b (=6). We also visualize the DC. BC, CC, EC, and NE values of nodes in real brain graph from the ABIDE dataset (see Figure 7). As shown in Figure NE can better distinguish node importance in the entire brain graph. contrast, other methods may mistakenly identify marginal nodes with many direct connections but less significance in information propagation as important nodes. For example, especially in Figure and (d), many nodes exhibit disproportionately high CC and EC values, highlighting their inability to effectively differentiate between truly important and less significant nodes in information propagation.
19

---

ADHD-200 Published as a conference paper at ICLR 2025
C EDGE DROPPING STRATEGY
Following Chen ct al. (2023), we apply the edge dropping strategy to achieve graph augmentation. The main idea of edge dropping strategy is dropping less important edges while preserving the functional module structure. Particularly, our edge dropping strategy is based on inter-modular edges first rationale. That is inter- mnter-modular edges less important than intra-module edges. We define a scoring function (.) to calculate the importance of each edge, it must meet the following condition:
IM > IM (Cinter).
(11)
Here, Cintra and Einter are intra-module and inter-modular edges, respectively. If nodes and j from the same functional module, the scoring function of their edge eij is:
(eij) Wey + max (w),
(12)
where wey is the weight of edge and mar indicates the highest edge weight in the graph. On the other hand, if nodes and j are from different functional modules, the scoring function of their edge eig is:
IM,e)) Wes, (w)
(13)
Consequently, the scoring function of inter-module edges will keep lower than that of intra-module edges. Our edge dropping strategy will first consider dropping inter-module edges with lower importance scores. Therefore, the functional module structure can be preserved in two augmented graphs.
D EXPERIMENTAL DETAILS AND ADDITIONAL RESULTS
D.1 IMPLEMENTATION DETAILS
The detailed hyperparameter settings for training BioBGT on three datasets are summarized in Table
Table 3: Hyperparameters for training BioBGT three different datasets.
Hyperparameter
ABIDE
ADNI
ADHID-200
#1 Layers
3
#Attention heads
8
Threshold of edge weight
0.3
0
0
Hidden dimensions
128
128
128
FFN hidden dimensions
256
256
256
Dropout
0.5
0.1
0.1
Readout method
mean
mean
mean
Learning rate
3e-4
3e-4
3e-4
Batch size
128
128
128
#Epoccs
200
200
200
Weight decay
1e-4
1e-4
1e-4
Warm-up Steps
10
10
10
Table 4: The number of parameters for different models on three datasets.
Method
ABIDE
ADNI
Polynormer
9.78M
9.80M
9.83M
Random Forest
21K
18K
9K
Gradformer
455K
446K
468K
GTSP
427K
424K
430K
BioBGT 465K
455K
454K
Number of parameters and computation time. The number parameters for BioBGT are 465K, 455K, and 454K on ABIDE, ADNI, and ADHD-200 respectively. The running time for training BioBGT on ABIDE, ADNI, and ADHD-200 is 650 s/epoch, 265 s/epoch, and 251 s/epoch, respectively. Table presents the number of parameters for different models on three datasets.
20

---

Published as a conference paper at ICLR 2025
D.2 ADDITIONAL EXPERIMENTAL RESULTS
Table Table 6 and Table give the additional results (F1, Sen., Spe.) for BioBGT compared to state-of-the we-the-ar methods on ADHD-200, ABIDE, and ADNI datasets, respectively. The overall experimental result reveals that BioBGT outperforms other methods, suggesting its superiority in various brain graph analysis tasks. Please note that due to the randomness of experimental results, the reproduced results of some baselines may be little different from those in the original papers. Table | summarizes the results of sensitivity and specificity for BioBGT and its variants on three datasets.
Table 5: Additional results on ADHD-200 (%)
Method
Sen.
Spe.
SVM
58.9115.47 57.07=4.13
68.75=16.11
40.581:18558
MI. Methods
Random Forest
55.01+9.30
63.95-8.04
SAN
48.19499.76
50.43-19.32
51.744:20.16
Graph Trans.
55.58.1.118
62.39.1943
39.13.1
Graphormer
69.2511..05
83.3412.90
33.96_6.10
Graph
SAT-PE
65.37
73.9113.73
13.45
Transformer
SAT+PE
68.30=3.83
75.65=7.44
52.73+10.67
Models
BRAINNETTF
70.42=3.06
66.69:2.93
75.75=4.63
Polynormer
18.62=#1293
51.67=17.68
19.63=11.59
Gradformer
69.01
71.66:49.77
13.99
GTSP
64.59
10.40
97.22411.844
GAT
57.85
64.54=13.97
15.10
BrainGNN
55.89=1.21
56.09:2.113
55,43=3.11
Graph Neural Networks
BrainGB
63.73+11.93
58.26-15.82
78.20-8.19
MCST-GCN
54.76+1.220
30,42+2.17
68.05-1.799
GroupBNA
70.61 +2.35
74.05++4.60
72.93-6.49
Our Model
BioBGT
7..63
89.3915.666
14.07-2.1
Table 6: Additional results on ABIDE (%).
Method
F1
Sen.
Spe.
SVM
52.02-+2.16 51.46++3.87
54.41=5.44
43.69+1.90
ML Methods
Random Forest
51.23 82
51.58+5.75
SAN
47.52-13.74
44.95.16.53
54.85.17.48
Graph Trans.
57.98:10.52
67.38
32.32-10.45
Graphormer
64.17-12.89
8..64
46.49-11.51
Graph
SAT-PE
67.21+3.84
64.76:t8.62
53.52+15.34
Transformer
SAT+PE
68.33:12.88
69.2945.48
43.24+18.03
Models
BRAINNETTF
68.20::2.31
69.3911.22
66.9515.28
Polynormer
44.20+111.13 38.16+17.04
15.46
Gradformer
62.12+33.77
19.1674.77
10.99
GTSP
65.36+66.22
69.86+11.57
10.99+12.59
GAT
56.12-1..25
12.93
48. 12.44
BrainGNN
50.09+
54.85+4.12
47.96+22.2
Graph Neural
BrainGB
66.95++5.08
17.01+10.00
60.07+8.53
Networks
MCST-GCN
52.63-t6.33
48 6.81
45.85+10.19
GroupBNA
61.64-11.3 1.20
64.72-15.43
65.28.16.27
Our Model
BioBGT
68.41 18.41-1..119
70.00-16.25
76.67
D.3 ADDITIONAL NE AND NEFF CURVES FOR THREE DATASETS
We provide visualizations of NE and NEff values for all nodes in brain graphs across three datasets. The graphs from the ABIDE, ADHD-200, and ADNI datasets contain 200, 190, and 90 nodes,
21

---

Published as a conference paper at ICLR 2025
Table Additional results on ADNI (%).
Method
F1
Sen.
Spe.
SVM
26.69+44.56
27.33+2..12
74.91=0.63
ML Methods
Random Forest
29.18.12.13
32.33.1..11
79.03.1..34
SAN
14.73.13.01
23.24:12.65
74.30.11.41
Graph Trans.
20.96+:1511
26.65:2.224
76.14=1.22
Graphormer
21.63+5..85
25.72::4.97
75.78=3.47
Graph
SAT-PE
24.08+4.118 30.67=4.19
76.16=0.29
Transformer
SAT+PE
19.66:1:.113 26.72:13.46
76.1642.115
Models
BRAINNETTF
35.64:4:.997
35.25:4..29
80.03+4.43
Polynormer
16.75++3.05
25.72++092
75.47=10.60
Gradformer
23.80::2.61
29.90=1.92
18.1811.224
GTSP
25.4811..14
30.39=1.91
78.26=1100
GAT
25.70++3.54
30.92+4.01
17.72+1.40
BrainGNN
23.23++4.67 29.52++3.11
77.47+1.80 1.80
Graph Neural Networks
BrainGB
32.22-+7.96 34.04+6648
18.7
MCST-GCN
37.4411.112 38.96.1..99
70.52.12.71
GroupBNA
35.85
34.98
78.25.16.78
Our Model
BioBGT
32.2942.31
3..65+1.43
80.39:10822
Table 8: The results (Sen. and Spe.) for BioBGT and its variants on three datasets (%).
ABIDE
ADNI
ADHD-20
Sen.
Spe.
Sen.
Spe
Sen.
Spe
+PE
45.49-t4.36
76.33++..05
34.50-22.36
1..5
74.78+
10.00++2.80
+DC
57.26+t9.11
65.31+9..32
32.68=11.05
79.69+00773
88.42++9.08
51.94+12.25
PE+DC
47.45.12.84
79.18.12.28
32.99-1..88
79.82-10.73
82.53111..77
19.05
+BC
43.95
87.50.14.17
31.53
78.71
77.12-11.778
74.7511..46
+CC
16.25
85.4113.61
31.50-13.20
78.84-1180
15.17
71.59119.4
+EC
43.75=t8.30
89.58+3.61
33.99:17.13
1..9
18.52
70.46+8.37
BioBGT
70.00:16.25
76.67+9.77
35.65++1443
80.39:0082
89.39-5.66
84.07-22.19
respectively. Figure provides the curves for all 200 nodes from randomly selected sample in the
ABIDE dataset. Figure shows the curves for all 190 nodes from randomly selected sample in the ADHD-200 dataset. Figure I0 shows the curves for all 90 nodes from randomly selected sample in the ADNI dataset.
10.00
0.1
120
-0%
120
125
330
275
Figure 8: The NE and NEff values of 200 nodes from a randomly selected sample in the ABIDE dataset.
22

---

Published as a conference paper at ICLR 2025
Mmm
Nc VOLU
60
-10
130
150
0.0
Mode
Figure 9: The NE and NEff values of 190 nodes from a randomly selected sample in the ADHD-200 dataset.
Volua
02
Node
Figure 10: The NE and NEff values of 90 nodes from randomly selected sample in the ADNI dataset.
23

---

Published as a conference paper at ICLR 2025
y Mayp
Figure 11: The NE value and FC strength of 200 nodes from randomly selected sample in the ABIDE dataset.
04
0.0
Figure 12: The NE value and FC strength of 190 nodes from randomly selected sample in the ADHD-200 dataset.
D.4 NE AND FC STRENGTH CURVES FOR THREE DATASETS
To further validate the effectiveness of NE in measuring node importance, we compare each node's NE value its average functional connectivity (FC) strength with all of the other nodes. FC strength is widely regarded as suitable metric for capturing the global topological properties of functional brain graphs (Liang al. 2012). Notably, FC strength commonly quantified using the Pearson correlation coefficient (PCC) between nodes, a standard measure in brain graph analysis (Li et al. 2021}. A node's average FC strength (average PCC value) reflects communication strength with other nodes, providing additional perspective on its importance in information propagation. Figures [I 12 and [3 illustrate the NE and FC strength curves for all nodes in randomly selected samples from the ABIDE, ADHD-200, and ADNI datasets, respectively. The results reveal an obvious alignment between the trends of the NE and FC strength curves, indicating that nodes with high NE values also exhibit high FC strength, and vice versa. This consistency further supports the validity of NE as a measure for quantifying node importance in information propagation.
D.5 FUNCTIONAL. MODULE DIVISION
Table Doole/givve gives the empirical labels of ROIs and functional modules. The functional modules used in this study are based on empirical labels. These labels represent the best effort to categorize regions based on known functional associations, but they are inherently limited due to the complex biological properties of the brain graph. Empirical functional modules often encompass ROIs from diverse brain regions, resulting in heterogeneity within each module. For example, in the auditory cortex, both temporal lobe regions (e.g., 'temporal 103') and thalamic regions (e.g.. 'thalamus 57") included due to their involvement in auditory processing (Rauschecker & Scott) 2009; Dones 2012). This diversity may reduce the uniformity of high self-artention scores within the module. In addition,
24

---

Published as a conference paper at ICLR 2025
MW hhw MM
Figure 13: The NE value and FC strength of 90 nodes from randomly selected sample in the ADNI dataset.
Table 9: The labels of ROIs and funetional modules.
Functional Module
ROI
'occipital 146', 'occipital 141', 'occipital 136',
'occipital 137',
'occipital 92', 'occipital 149', 'occipital 148',
'occipital 145',
'occipital 147', 'occipital 142', 'occipital 139', "occipital 135',
Vis
'occipital 133', 'occipital 129", 'occipital 126',
'occipital 118",
'occipital 119,, 'occipital 106', 'post occipital 160', "post occipital 158',
'post occipital 159', 'post occipital 157', 'post occipital 156',
'post occipital 153', 'post occipital 154', 'post occipital 152'
inf cerebellum 155', 'inf cerebellum 150', 'inf cerebellum 151',
inf cerebellum 140', 'inf cerebellum 131', 'inf cerebellum 122',
'inf cerebellum 121', 'inf cerebellum 110', 'lat cerebellum 128',
MC
'lat cerebellum 113', 'lat cerebellum 109', 'lat cerebellum 98'
'med cerebellum 143', 'med cerebellum 144", 'med cerebellum 138',
'med cerebellum 130', 'med cerebellum 127', 'med cerebellum 120',
'post parietal 99', 'SMA 43', 'basal ganglia 71', "basal ganglia 38",
'basal ganglia 39', 'basal ganglia 30'
'post cingulate 115', 'post cingulate 111', 'post cingulate 108',
'post cingulate 93', 'post cingulate 90', 'post cingulate 73',
'vIPFC 15', 'vmPFC 13', 'vmPFC 11',
CC
'vmPFC 7', 'vmPFC 6', 'vmPFC
'IPL. 107', 'ILL 104', 'IPL 101",
IPL 96', 'IPL. 88', 'IPS 116', 'IPS 114'
'sup temporal 100', 'temporal 103', 'temporal 95',
'temporal 78', 'thalamus 57', 'thalamus 58"
Aud
'thalamus 47', 'inf temporal 91', 'inf temporal 72',
'inf temporal 63', 'temporal 123'
'angular gyrus 102', 'mid insula 61', 'mid insula 59',
LP
'mid insula 44', 'aPFC 5', 'angular gyrus 124',
'angular gyrus 117', 'aPFC 2', 'aPFC 3'
'dACC 27', 'dFC 36', 'dFC 34', 'dFC 29', ddPPFC 24', 'dIPFC 22', VPFCC23' 'vent aPFC 10', 'vent aPFC 9¹, 'v,'VIFFC 12', 'dIPFC 16', 16','dFC
EC
because of limited label availability, there are no available labels for the atlas of the ADNI and ABIDE datasets. Therefore, can only provide labels for the ADHD--20 dataset.
25

---

Published as a conference paper at ICLR 2025
D.6 ANALYSIS OF ATTENTION PATTERNS OUTPUT BY BIOBGT
Comparsion between NCs and ADHD patients. Studies in neuroscience have shown that functional connectivity (FC) in ADHD patients be either enhanced weakened varying degrees, disrupt- ing functional modularity (Wang et al. 2020a). For instance, Sripada el al. demonstrated that the FC strength within the cognition control (CC) module reduced in individuals with ADHD (Sripada the heatmaps of the average atte-nttention scores for normal controls (NCs) and ADHD patients from the ADHD-200 test set. It is evident that the functional modules in the heatmap of ADHD patients are less distinct, while the NC group's heatmap shows clearer functional modules, such as Vis, CC, and MC modules. Specifically, the CC module in the NC group exhibits higher internal strength and stronger modularity compared to the CC module in the ADHI) group, which aligns with Sripada et al.'s findings in neuroscience.
et al. 2014). Figure
A20
EC
NC
8
2
8
(a) NC
(b) ADHD
Figure 14:
The heatmaps of the average self-attention scores of normal controls (a) and ADHD patients (b) in the ADHD-200 test set.
Comparsion between NCs and ASD patients. Studies have shown that the small-world properties of brain graphs in autism spectrum disorder (ASD) patients are generally lower than those of typically developing individuals (Itahashi et al., 2014). Particularly, the hub properties of nodes are weakened in brain graphs of ASD patients. Figure 15 shows the heatmaps of the average self-attention scores for NCs and ASD patients from the ABIDE test set. It can be observed that the attention heatmap of NCs contains nodes with strong hub properties, indicating high correlations with most other nodes. In contrast, the attention heatmap of ASD patients shows weaker hub properties, which aligns with the finding that the small-world properties of functional connectivity ASD patients are diminished.
(a) NC
(b) ASD
Figure 15: The heatmaps of the average self-attention scores of normal controls (a) and ASD patients (b) in the ABIDE test set.
Comparsion between NCs, MCI patients, and AD patients. Different stages of Alzheimer's disease exhibit distinct functional connectivity patterns "Sanz-Arigit et al.) 2010). Specifically, as the disease progresses, the modular characteristics of functional brain graphs decline (Dai & He 2014 Zhao et al. 2012). The brain graphs of NCs display more distinct modular structures, while the functional connectivity patterns of mild cognitive impairment (MCI) patients begin to show signs of disorganization. For patients with Alzheimer's disease (AD), the partitioning of functional modules almost disappears, and modularity is significantly diminished. Figure presenns the heatmaps of the average self-attention scores for NCs (a), MCI patients (b), and AD patients (c) in the ADNI test set. is evident that the brain graphs of NCs exhibit more distinct functional modules compared to those of MCI and AD patients.
26

---

Published as a conference paper at ICLR 2025
(a) NC
(b) MCI
(c) AD
Figure 16: The heatmaps of the average self-attention scores normal controls (a), MCI patients (b), and AD patients (c) in the ADNI test set. Table 10: Model performance on Citeseer and Cora (%)
Citeseer ACC
Cora
FI
AUC
FI
ACC
AUC
GAT 45.45:15.17
70.01-0.68
81.06:1..11
81.02:10.46 81.7410.53 0.03
SAT+PE 6..04+1.22 68.80 08.5
87.69+1.99 1.99
81.62:0..56 82.84=0.87 94.63=4..44
BioBGT 64.80:10.34
69.02=0.70
88.04=1.11
82.03+1.33
83.1420.80
94.58:10.24
D.7 MODEL GENERALIZABILITY ANALYSIS
We suggest that BioBGT not only performs well on brain graphs but may also generalize effectively to other networks with similar structural characteristics, such as the presence of hubs and modules. To prove this, we apply BioBGT to other types of networks, such as citation networks. We train our model the Citeseer (Giles et al., 1998) and Cora (McCallum et al. (2000) datasets for node classification tasks. Table hiighhhhss the superiority of BioBGT on both Citeseer and Cora datasets. Therefore, our model shows generalizability in extending to other networks that contain hubs and modules.
27


Reviews:
[{"review_id": "5amJZcsXz4", "reviewer": "Reviewer_vQGg", "summary": "The paper introduces a brain graph representation learning framework aimed at enhancing biological plausibility, with a focus on capturing the small-world architecture of brain networks. Specifically, the authors propose a novel framework called BioBGT, which emphasizes node importance encoding and functional module encoding. The framework consists of two key components; 1) Network entanglement-based node importance encoding, which captures the significance of nodes in the process of information propagation, and 2) Functional module-aware self-attention, which generates node representations that reflect both the functional segregation and integration properties of brain networks. The experimental results demonstrate the effectiveness of this design and its superior performance in brain disease detection tasks.", "questions": "- Contributions and novelty need to be clarified as well as the questions raised in the Weakness.\n\n- As seen in [2] and Eq. (9), the self-attention formulations are nearly identical. However, the proposed method claims to enhance functional module encoding by making the exponential kernel trainable. The explanation regarding the benefits of learning the kernel seems insufficient. Could you elaborate on the advantages of this approach? Additionally, the experiments only compare the proposed method with the self-attention mechanism from Eq. (2). A more thorough discussion of the differences between the proposed approach and the method presented in [2] would strengthen the analysis. \n\n- Why did the authors use 50 randomly selected nodes to validate the biological plausibility in node importance encoding? The number of nodes of the brain networks doesn’t seem to be too large. Also, are the results consistent across all datasets?\n\n- In Figure 5, the authors present the average self-attention scores as a heatmap. However, in the results of the proposed BioBGT, the block-like color patterns corresponding to functional modules do not consistently appear across all ROIs. Could you explain the reason for this discrepancy? (For example, between ROIs 40 and 60, the block-like regions with high values are not observed.) Also, I think the caption should be more informative for the figure. \n\n- Moreover,  the biological plausibility in functional module-aware self-attention seems to be ``plausible'' as in its block-like patterns, but do they comply with the neuroscientific prior knowledge? Adding analysis on which functional module match each block may help the biological plausibility of your method.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- As emphasized by the authors, their method highlights the biological properties of the brain structure via biological plausibility.\n- Overall, the presentations are clear and easy to understand their framework and results.\n- The experiments are well-structured with multiple brain datasets, and the proposed model demonstrates strong performance in most cases.", "weaknesses": "- The authors propose three main methods: network entanglement-based node importance encoding to reflect the different importance of each node like the hub node, community contrastive strategy-based functional module extractor and an updated self-attention mechanism for functional module-aware self-attention mechanism. However, I couldn’t find much difference from the original works of each methods the authors mentioned they were inspired by. On top of that, the equation 6 and its proof in the appendix needs to be double checked (shouldn’t log2(Z_i Z) be log2(Z_i/Z)?).\n\n- As shown in [1] and the proof in the appendix, the two equations appear identical. However, due to an error in the calculations within the proof, the resulting expressions differ. Therefore, in Eq. (6) of Theorem 1, the second term should be log_2(Z_i/Z) instead of log)_2(Z_iZ). Additionally, while defining the hub characteristics in the brain through node entanglement (NE) seems reasonable, it is difficult to identify significant novelty compared to previous studies.\n\n- Even though the authors provided citations on the quantum entanglement methods, it would be very helpful for the readers to have more details on the definition and why they need to be utilized to quantify the structural information.\n\n- The term “with their biological plausibility preserved” seems to be misleading in Line 345, since the classification performance alone does not directly demonstrate the effect of the biological plausibility, which is never dealt until much later in section 4.4.\n\n- Tables 1 and 2 suggest that the baselines were not compared fairly. Their high standard deviations undermine the credibility of the experimental results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P7X3CXZFjm", "reviewer": "Reviewer_75RC", "summary": "This paper proposes BioBGT, a framework that incorporates node importance encoding and community-aware Graph Transformer for learning the network representation of the brain. \nStrengths of this work include that 1) the proposed method is motivated by incorporating prior knowledge of the brain, and 2) it shows clear improvement over the presented baselines.\nSome concerns and suggestions are outlined in the `Questions` section.", "questions": "### Major concerns\n- The biological properties of the brain graph are very complex and not much known yet. Strong assumptions and statements about \"biologically plausible\" properties can be generally toned down throughout the manuscript.\n- The proposed methodology does not seem to significantly incorporate the biological property of the human brain as the authors claim, given that the theoretical and empirical findings are not very supportive. Specifically:\n  - 1) In Section 4.4, using PCC values (which seem to reflect the average functional connectivity strength across the nodes) as a representative measure of biological plausibility is logically tenuous. Even if we assume that the PCC value is representative of biological plausibility, the quantitative similarity between NE values and PCC values has not been studied. \n  - 2) The BioBGT output demonstrated in Figure 5 (c) does not seem to represent a biologically plausible network, which deviates from the modular patterns of brain functional connectivity.\n- Section 4.5 focuses more on generalizability across datasets than on comparing the scale of the model parameter size, data, and computation. Please consider revising the possibly misleading term 'scalability' to 'generalizability'.\n- The generalizability of BioBGT to non-biological networks shown in Section 4.5 seems to contradict the authors' emphasis on the strength of the BioBGT, that it introduces 'biological' property of the human brain to the framework. Please revise the logical consistency of Section 4.5 since it reads as if any network that contain hubs and modules are considered 'biological'.\n- Merging Tables 1, 2, and 6 and reducing the number of presented metrics (e.g., AUC and ACC) would improve the readability of the results.\n- Ablation studies seem to show redundant node importance ablations with various graph metrics. Please consider reducing the number of node importance ablations and merging the result with FM-Attn ablation results. \n- In Eq. (7), please elaborate on the rationale for incorporating node importance through addition rather than multiplication or concatenation. \n\n\n\n### Minor concerns\n- Clarification needed on “high correlation” (line 054): please clarify if this refers to the correlation of temporal changes.\n- The category \"Brain Graph Learning Models\" can be revised to \"Message-Passing Neural Networks (MPNNs)\" or \"Graph Neural Networks (GNNs)\". GAT is classified as a Graph Transformer, but it is more often considered as a GNN. Please consider re-categorizing the model.\n- Please consider adding more recent Graph Transformer-based studies on functional connectivity as baseline methods.\n\n\n### Recommendation\nGiven the above concerns, especially the weakness of the experimental results in supporting the authors' claim, I initially recommend reject of the paper.\n\n\n### Recommendation after rebuttal\nGiven the authors' response to the concerns, I recommend weak reject of the paper.", "limitations": "", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Motivation to incorporate prior knowledge of the brain into the framework\n- The proposed method shows clear improvement over the presented baselines", "weaknesses": "- The authors’ emphasis on biological plausibility is weakly supported both theoretically and experimentally.\n- The baseline model performance in the experiments is generally lower than expected, so verification may be necessary.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WTXL3msXOV", "reviewer": "Reviewer_drzD", "summary": "The Biologically Plausible Brain Graph Transformer (BioBGT) is introduced here as a graph transformer specifically aimed at creating brain graph representations rooted in biological principles. Unlike prior models that allow representations to form without these constraints, BioBGT uses the concept of network entanglement - borrowed from quantum computing - to more accurately assess node importance than traditional centrality metrics. Furthermore, the authors implement a community contrastive training method, enabling node representations to account for network communities. These two contributions are aligned with core characteristics of fMRI-based graphs, underscoring their relevance for neuroimaging-focused models.\n\n\n_____________________\nEDIT (after rebuttal period): I appreciate the answers and discussions that the authors had with me and the other reviewers. Based on all this I'm increasing the score from 6 to 8 (accept), as well as increasing the contribution score from 3 to 4. There are still some small weaknesses that I've mentioned in this discussion and that is why I do not increase the score of this paper to the maximum value. I thank the authors for the insightful discussions and the time taken to address my concerns.", "questions": "I will be happy to revise my final score if the authors tackle the weaknesses I have identified. Furthermore, I have the following questions, more or less in the order they appear in the paper:\n1. Equation 7 mentions \"learnable embedding vectors\", but I do not see those in the equation. Can the authors specify this better?\n2. Did the authors try other contrastive loss functions beyond the one in equation 8?\n3. How did the authors calculate the F1/sensitivity/specificity scores for the ADNI dataset, where we have 3 classification labels?\n4. In Conclusion, the authors argue that they \"managed to keep the number of parameters comparable to other models\". Where is this shown in the paper?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "I found this paper compelling and I believe it has strong potential for ICLR. The writing is clear and well-organized, making it highly readable. The paper’s originality lies in its combination of concepts from another field, applied thoughtfully to the fMRI domain, with evidence that this approach yields positive outcomes. In this sense, the results in tables 1 and 2 are quite impressive, while those in Table 6, though somewhat less robust, still demonstrate solid performance.\n\nThe authors leverage three widely recognized neuroimaging datasets and employ two different parcellation methods. I feel the choice of baseline models is appropriate for the study’s goals, with an exception that I will provide in the next section. The quick experiments in Section 4.5 are noteworthy, showing that the model can achieve comparable performance on non-neuroimaging datasets with similar underlying motivations.", "weaknesses": "1. No experiment is conducted with a smaller parcellation. The results using the ADNI dataset - where a smaller parcellation was used - are not as strong, which in my opinion are an indication that a smaller number of ROIs could impact the performance of this method. Given the focus on community structure and node importance of this work, I think it would have been important to understand whether the size of the parcellations are an important factor in this models performance.\n2. Even though the ablation study makes a good job in comparing different alternatives to the node importance encoding by comparing other centrality measures, there is no ablation study for the community contrastive strategy. \n3. The work lacks comparison with more \"traditional\" ML models like random forests or SVM. In a few fMRI works/papers that I have seen in the past, and also based on my experience with these datasets, I typically see is that more \"traditional\" models achieve similar or even better performance than deep learning when trained on the upper-triangle of the correlation matrices (which has been historically widely explored in the connectomics field). In this sense, for a conference like ICLR, I find the lack of comparisons with traditional ML models in this specific context a weakness of this paper.\n4. This work has an entire section (4.4) focused on the biological plausibility of its model, but these results do not discuss whether these modules correspond to any known brain structure or medical knowledge. Furthermore, it is not clear how these different modules contribute to the final classification, and thus it is not clear how plausible and good these representations are.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)