Pre-train Graph Neural Networks for Brain Network Analysis
Authors: Yi Yang
Hejie Cui
Carl Yang
Venue: Submitted to ICLR 2023 (2023)

Abstract: Human brains, controlling behaviors and cognition, are at the center of complex neurobiological systems. Recent studies in neuroscience and neuroimaging analysis have reached a consensus that interactions among brain regions of interest (ROIs) are driving factors for neural development and disorders. Graph neural networks as a powerful tool for analyzing graph-structured data are naturally applied to the analysis of brain networks. However, training of deep learning models including GNNs often requires a significant amount of labeled data. Due to the complicated data acquisition process and restrictions on data sharing, brain network datasets are still small compared to other domains (e.g., molecules, proteins). Moreover, real clinical tasks (e.g., mental disorder analysis) are often conducted on local datasets with even smaller scales and larger noises. To this end, we propose to leverage pre-training to capture the intrinsic brain network structures regardless of specific clinical outcomes. Specifically, we characterize the contributions in this work from two perspectives: (1) We design brain-network-oriented unsupervised pre-training techniques to utilize large-scale brain imaging studies without highly relevant task labels. (2) To facilitate effective knowledge transfer across studies with different ROI systems, we propose to develop a data-driven parcellation atlas mapping pipeline. The proposed pre-training techniques are validated with various GNN models. Extensive experiments demonstrate consistent improvement in performance as well as robustness.

Under review as a conference paper at ICLR 2023
PRE-TRAIN GRAPH NEURAL NETWORKS FOR BRAIN NETWORK ANALYSIS
Anonymous authors Paper under double-blind review
ABSTRACT
Human brains, controlling behaviors and cognition, are at the center of complex neurobiological systems. Recent studies in neuroscience and neuroimaging anal- ysis have reached a consensus that interactions among brain regions of interest (ROIs) are driving factors for neural development and disorders. Graph neural networks (GNNs) as powerful tool for analyzing graph-structured data are nat- urally applied to the analysis of brain networks. However, training of deep learn- ing models including GNNs often requires a significant amount of labeled data. Due to the complicated data acquisition process and restrictions data sharing, brain network datasets are still small compared to other types of graphs (e.g.. so- cial networks, molecules, proteins). Moreover, real clinical tasks (e.g.. mental disorder analysis) are often conducted on local datasets with even smaller scales and larger noises. To this end, we propose leverage pre-training to capture the intrinsic brain network structures regardless of specific clinical outcomes, for obtaining GNN models that are easily adaptable downstream tasks. Specifi- cally, (1) we design brain-network -et--ottt--ororoteeeeddd oriented unsupervised pre-training techniques to utilize large-scale brain imaging studies without highly relevant task labels; (2) we develop a data-driven parcellation atlas mapping pipeline to facilitate effec- tive knowledge transfer across studies with different ROI systems. The proposed framework is validated with various GNN models, with extensive empirical results demonstrating consistent improvement in performance as well as robustness.
INTRODUCTION
In recent years, the analysis of brain networks has attracted considerable interest in neuroscience studies. Brain networks are essentially graphs, where anatomical regions of interest (ROIs) given parcellation atlas are formed into nodes, and the connectivities among ROIs are formed into edges. Based on brain networks constructed from different modalities such as Diffusion Tensor Imaging (DTI) and functional Magnetic Resonance Imaging (fMRI), effective graph analysis plays piv- otal role in understanding the biological structures and functions of complex neural systems, which can be helpful in the early diagnosis of neurological disorders and facilitate neuroscience research Martensson al. (2188); Yahata et al. 2016); Lindquist 20088 Smith 2012).
Deep learning has replenished the fields of computer science and beyond. Among various modern deep learning models, the emerging graph neural networks (GNNs) have demonstrated superior per- formance and even plausible interpretability on variety of network datasets, including social net- works, recommender systems, knowledge graphs. protein and gene networks, molecules and so forth
Kipf & Welling (201)) Hamilton et al.] (2017); Schlichttrull et al. (2018) Vashishth et al. (2020); Xu al. (2019); Ying et al. (2018); He et al. (2020b); Zhang et 2220 ; Liu et (2022) Xiong et al. (2020). due to its powerful representations and efficient computations of complex graph struc- tures towards specific downstream tasks. Such achievements on other types of networked data pro- pel studies on GNNs for brain networks, especially models for graph-level classification/eegrsssoo Ying et al. 2 8 al. (2019); Luo et al. (2020); : Vu & Thai (2020), towards tasks such as connectome-basee disease prediction and multi-leval neural pattern discovery. However, the training of powerful deep learning models including GNNs often requires significant amounts of labeled data Hu et al. (2020a); You jet al.) (2020); Zhu et al. (2021a) For brain network analysis, there are limited big imaging datasets from few large-scale national neuroimaging studies such the ABCD Casey al. (2018). ADNI
Xu et al. 2019 Errica et 2020) and important vertex/edge identification Ying

---

Under review as a conference paper at ICLR 2023
Hinrichs et al. (2009), and PPMI Aleksovski et al. (2018), but such datasets are still rather small compared with graph datasets in other domains (e.g.. datasets with 41K to 452K graphs on OGB Hu et al. 2020b) and datasets with thousands to millions of graphs on NetRepo Rossi & Ahmeed
(2016))
One solution toward data scarcity is transfer learning which transfers models trained the large- scale brain network datasets onto small-scale local studies while retaining favorable performance. However, one limitation of transfer learning is its reliance on the availability of similar tasks as supervision during training in the source dataset. In reality, similar tasks in the smaller local studies may not always be available in the large-scen in the fields of computer vision He et al. (2020a); Chen et al.] (2020b), natural language processing
public studies. Pre-training training has shown its effectiveness
Devlin et al. ) Radford et al. (2018), as well as graph mining Sun et al. (2022). We explore GNN pre-training on brain networks without supervision and study its effectiveness in the prediction
of specific clinical outcomes. However, unique challenges impede the direct application of existing
GNN pre-training paradigms to brain networks. For example, brain networks within one study usually share the same node system, which is not properly exploited, whereas different studies often different node systems, which hinders the transferability of pre-trained models.
To fully unleash the power of GNNs for brain network data by collaborating across datasets, we propose to develop unsupervised multi-dataset GNN pre-training framework for brain networks. Specifically, we adapt the popular data-efficient framework of MAML with carefully designed two- level contrastive learning that works in concert with the functional neural system modules in brain networks. In addition, to address the problem of dataset misalignment, we propose a novel data- driven atlas mapping technique based on an auto-encoder that transforms the original features in multiple datasets into low-dimensional representations in a uniform embedding space. The trans- formed features are aligned via variance-based projection with locality prriett--reereinn.. module-aware and pparsity-orientee regularizations.
In summary, our contributions are three-folded:
We identify the intrinsic data-scareity issue in brain network learning and formulate our problem into an unsupervised pre-training objective.
We propose novel brain-network -rreert--rreeeeed two-level contrastive sampling strategy for multi- dataset GNN pre-training (Section (.2). In addition, we implement a data-driven brain atlas map- ping strategy with customized regul legularizations and variance-based sorting to facilitate cross-dataset model sharing (Section B.3).
conducted compare our proposed framework with various baselines (Section 4.1 and we further the analysis through in-depth studies on the contributions of each constituent component of our framework (Sections 4.2
Extensive experiments
2 RELATED WORK
Unsupervised Graph Representation Learning. Most graph representation learning framework often leverages specific label knowledge as ground truth references in the objective function. Unfor- tunately, brain connectome data often lacks sufficient expert annotation making most supervised op- timizations underperform. Therefore, we adopt unsupervised learning techniques for GNN training on brain networks. Motivated by context-based token embedding techniques such Skip-Grau and Continuous Bag-of-Wordd (CBOW) Mikolov et al. (2013), Node2vec Grover & Leskovec (2016) and DeepWalk Walk Perozzi al. (2014) propose to learn node representation by leveraging similarity w.r.t its random-walk induced neighbors. However, this framework is transductive and thus cannot be properly pre-trained and adapted
new dataset. VGAE Kipf Welling (2016) proposes to learn representations capable of reconstructing the graph connectivity. However, this design may still adapt poorly to tasks other than link prediction. Recently, the advancement in visual represen- tation learning Chen et al. (2020a): He et al. (2020a) has inspired contrastive learning on graphs. For instance, GBT Bielak et al. (022) designs a Barlow Twins Zbontar et al. (202I) loss function based empirical cross-correlation of node embeddings learned from pair of augmented graph views Zhao et al.] (022). Similarly, GraphCL You et al.] (2020) contrasts among graph represen- tations learned from different graphs. Generally, graph contrastive learning captures robust latent representations on well-attributed graphs with a sufficient number of graph samples. However, its accountability may be undermined in sample-scarce and atrributelacking brain networks.
2

---

Under review as a conference paper at ICLR 2023
Graph Neural Networks Pre-training, Pre- Pre-training of deep neural networks has proven immense success in computer vision Cao et al. (2020); Grill et al. (2020), natural language processing Devlin et al. (2019a); Radford et al. (2018), and mull-modality (e.g. text-image pair) learning Radford et al. 2021 Li et al. (2022): Yao et al. )022;; Wang et al. (2022). Specifically, the pre-training pipeline is formulated by defining pretext tasks such as masked token prediction in NL.P image
recovery from Gaussian blurring in CV. Following similar intuition, GPT-GNN Hu et al. (2020c) proposes graph-oriented pretext tasks, for instance, masked attribute and edge prediction. L2P- GNN Lu et al. (2021) introduces dual-adaptation by simultaneously optimizing the encoder on a node-level link prediction objective and a graph--eveesllf--uueresssss sulf-supervision task similar DGI Velickovic et al. (2019). Interestingly, PHD Li et al. (2021a) leverages half-graph representations and optimizes on connected component prediction objective. Others, such as GMPT Hou et al. (002) devises inter-graph message passing to obtain contextual node embedding and optimizes concurrently under supervision and sulf-superv.sion. Overall, most GNN pre-training strategics emphasize the effective formulation of pretext tasks and data augmentation. However, existing techniques cannot be directly localized to brain networks because they do not consider the special properties of brain connectome data.
UNSUPERVISED BRAIN NETWORK PRE-TRAINING
Problem Definition. Formally, given collection of brain network datasets S (D1, (D.,D,....D.)) each D; = -9t,, 1 is composed of N, subjects of weighted graphs Gi.. Specifically, each graph Gig described by a node set V. { M an edge set Eigj V.j Vijj and a weighted adjacency Aij E TRM.,XM... We define 0 parameterized GNN f()) that learns mapping relation fe(gr)) = 2.. where 2i, denotes the learned representation(s). The goal of our framework focuses on pre-training f() on a subset Ssource of S via self-supervision, to obtain an initialization Og that can efficiently converge toward a target optimal 0 on a dataset Durget E S.
3.1 MULTI-DATASET GNN PRE-TRAINING
Model pre-training aims at finding proper initialization that can be efficiently adapted toward down- stream evaluations. Specifically, in our framework, the model is first jointly pre-trained on multiple brain network datasets under self-supervision, and later applied to target dataset with a supervised objective. Importantly, our pipeline differs from traditional transfer learning since it requires mea- surable similarity between the source and target data as well as their learning objectives, whereas transferable task formulations are often missing for brain network analysis.
To enable faster convergence, our multi multi-dataset pre-training adapts the popular data-efficient train- ing techniques presented in MAML Finn et al. (2017). Specifically, at each training iteration, we partition each input dataset into an inner-loop support set and an outer-loop query set. The model is first trained on the support set without explicitly updating the parameters. Instead, we temporarily store the updates as fast weights Ba et al. (2016), which we later evaluate on the query set compute the actual gradients. In other words, the training makes use of approximating higher-order deriva- tives Tan & Lim (2019) at each step achieve quicker descent along the optimization trajectory. In our scenario, the joint optimization sums over the loss on each brain network dataset, that is, for number of datasets along with their respective temporary fast weights 0' and query partition, the ii Equery, foz (-).
step-wise update of the model parameter at time t states as 0++1 0 V Σi
3.2 BRAIN NETWORK ORIENTED TWO-LEVEL CONTRASTIVE LEARNING
Considering the high cost of acquiring informative task labels for brain networks, our pre-training pipeline resorts to a label-free free learning strategy. in particular, contrastive learning (CL). Generically, given an anchor point of investigation mutual information (MI) T(::-) against its positive samples X and minimize MI against its negative samples X- Formally, we express the cost function as below:
from a data distribution H. CL seeks to maximize the
Joon = E
1
Σ
(1)
H
X.X
W
3

---

Under review as a conference paper at ICLR 2023
Atla Projection Variance busee Sorting
PC
Dataset
Moilt- Dulsst GNN
Adlptation
P4
Target Task
Parumeter Initialization
Source Dalasel
Initial
Transformed
Sorted
Features
Features
Contrativve Objoctiive
Figure 1: Overview of proposed framework. We first project the source data features into fixed dimensional representation through atlas transformation followed by crriance-based feature alignment. We then pre-train GNN encoder multiple datasets under self-supervision via novel two-level contrastive learning. Finally, learned parameters are fine-tuned on target task.
Applying this concept toward graph-level CL, given anchor node representation Eo and an arbi- trary set S containing a collection of node representations sampled from the training batch. Based on the Jensen-Shannon divergence, the MI between Fa and the representations in S is elaborated as:
I (Fa: S) Σ JSD(=ai
1 2
log 1 exp
T
Fa
The end goal of our framework focuses localizing effective GNN CL learning Zhu et al. (2020) 202 for brain networks. To this end, suppose a dataset DD and an anchor node i from graph Gp E D with the learned representation Zi,p, we propose to categorize the possible sample selections into three fundamental types (a gentle visualization is shown in Figure
S₁: {zj : j E (i,,))); which refers the set of node representations within the anchor's
k-hop neighborhood in graph 9p-
S2: {2t,p j & Ni, p)}, which refers to the set of remaining node representations graph Gp that are not in the anchor's k-hop neighborhood.
S3: {zig Gg E D, j E Gg & p}, which refers to the set node representations all other graphs in the dataset D.
Notice that our framework leverages the k-hop substructure of the anchor node to further differ- entiate S1 and S2 for contrastive optimization. This design motivated by two perspectives: (1) Regarding GNN learning. Since the node representation is learned from information aggregation of its h-pop neighborhood, maximizing the MI of an anchor courages lossless message passing of GNN convolutions. (2) Regarding the uniqueness of brain networks. Brain networks can be anatomically segmented into smaller-scale neural system mod- ules Cui et al.] (2022b), thus capturing subgraph-level knowledge can provide useful signals for brain-related analysis.
k-hopp neighbors naturally en-
On top of these three fundamental types of samples, we further introduce additional type by leveraging the property of brain networks that ROI identities and orders are fixed across samples. can further promote GNN to extract shared substructure knowledge by evaluating the MI an anchor against its presence in other graphs. Given an anchor representation i of node i from graph Gp E D, the novel inter-graph sample type is defined as:
Taking this advantage,
S4: [z E No (i, 4) n (i, 9, E D, * p}. which refers to the set of node represen- tations within the k-hop neighborhood of node all other graphs in D. Conceptually, S, is special subset of S3.
Notice that for an anchor node i, its k-hop neighborhood structures might not be identical among different graphs. Thus, we only consider shared neighborhoods for multi-graph MI evaluation.
To encourage the learning of unique neighborhood knowledge single brain network instance, and shared substructure knowledge across the entire dataset, configure S₁ and S4 positive samples while S2 and the set Sz Sn as negative samples, as illustrated in Figure

---

Under review as a conference paper at ICLR 2023
S2
$
1
3
$1
Positive
Negative
iiOi!! S4
gp
9q
S
Figure Visual example of the sample types where Xtp is the anchor and Sy/S4 are sampled as 1-hop neighbors.
Figure Sampling configu- rations of our framework.
In addition, our sampling categorization can also help un- derstand the objective formulations in various state-of-the art graph CL frameworks Velickovic et al. (2019); Qiu et al.
Table 1: Sampling configurations of existing graph CL methods.
[2200;] Xia et al. (2022); (unn al. (2019); Zhu et al. 20) We hereby summarize our findings in Table | Specifically, "+" denotes positive sampling: "_" denotes negative nagpling
S1
S2
Sa
S4
DGI
+
+
/
/
and "f" means that the sample type is not considered. Observe
InfoG
+
+
/
that since DGI and InfoGraph leverage graph representation pooled from node representations essentially equivalent to jointly considering S, and S2 with- explicit differentiation. On the other hand, GCC and EGI, which are more intuitively connected to our framework, lever- age neighborhood MI maximization on single graph, but fail extend this consideration multi-graph setting like ours.
GCC
+
/
special sample, it is
EGI
+
/
Ours
3.3 DATA-DRIVEN BRAIN ATLAS MAPPING
Motivation. When a pre-trained model fine-tuned on a new data domain, misalignment between source and target signals may negatively affect downstream adaptation. This problem is especially concerning in brain networks where datasets are built from various brain atlas templates each map- ping unique system of ROIs. For example, the HIV dataset is parcellated from the AAL99 template, resulting in 90 defined ROIs; whereas the PPMI dataset adopts the
template, resulting in 84 defined ROIs. Thus, brain networks in the two datasets will have differ- ent numbers and semantics numbers and orders of nodes, when construct the most informative ROI features the brain connection profiles Cui et al.] (2022a), the feature dimensions and their meanings will be different. Although manual conversion
nodes. Although GNNs can naturally handle graphs without fixed
be done translate among various templates, an accurate one- to-one mapping is impossible, and even obtaining a coarse mapping still needs tremendous expert effort. To this end, to facilitate an effective target adaptation, we propose a data-driven atlas mapping solution that transforms the original node features into oover-dimensional embedding spaces that not only preserves original connectivity information but also achieves cross-dataset feature alignment.
3.3. AUTOENCODER WITH BRAIN NETWORK ORIENTED REGULARIZERS
To address information preservation, structure. Specifically, AE features a linear projection encoder W and a transposed decoder W The general objective is to learn low-dimensional projection that easily reconstruct to its original presentation. Formally, the loss function is defined as minimizing the reconstruction Cis. (1/n) X XWW''I for an input X R and projection W E Rdxm Hinton & Zemel (199). To achieve a more informative feature compression besides dimension reduction and to guide the overall AE optimization, we propose to implement several regularizers that take special consideration of brain network characteristics:
adopt one-layer linear autoencoder (AE) base
Regularizer: Following section 3.2 a significant portion brain network analysis focuses learning neighborhood structure. Naturally, the compressed feature should pre- serve the neighborhood information from its original profile. Considering that each ROI atlas con- tains unique set of 3D coordinates specifying the scaled actual locations of ROIs in the human
5

---

Under review as a conference paper at ICLR 2023
brain, one important type of neighborhood is its physical adjacencies based on the 3D projection. Hence, we regularize our AE objective as neighborhood preserving He et al. (2005), which can be written as Lioe (1/n)][Y TYII?, where Y E R=xm denotes the AE projected features and T E R is the transition matrix built from the k-NN graph of the 3D) coordinates.
Module-Aware Regularizer. Brain networks can be segmented into various neural system mod- ules each characterizing a subset of ROIs. In graph terminology. they are community structures. Given this, the projected feature could also incorporate information regarding neural system belong- ings. Unfortunately, knowing such ground-truth segmentations requires significant prior knowledge. Hence, we resort to community detection methods on graphs, and in particular, our design follows the modularity maximization paradigm in which the regularizer (alha- et al. (2022) can be formally written as minimizing
1 2m
a
kij
Loom
Ai
exp( 11y: - 1,
(3)
2m
where A E RRxx denotes graph adjacency, h; denotes degree of node i, and y; denotes the projected features. Essentially, the optimization minimizes the Le distance of representations among nodes within the same communities measured by the modularity and maximizes the distance otherwise.
Oparsity-Orientee Regularizer. Sparse networks have demonstrated robustness in learning repre- sentations from Joong et al. (2017); : (hh et al. (2019); Makhzani & Frey (2014) and it has been shown that, in brain connectome analysis, sparsity leads to a better interpretation of task-specific ROI connections in generation and classification tasks Kan et al. (2022). To this end, we adopt the popular KL--dieereence smoothing to enforce parameter sparsity, which is given as:
d
P Pij
(1-2))
CKL
plog
+ (1 (1-
p) log (1- Pij
(4)
i=1
where p is the target sparsity value usually set to a small positive float, and púj is the element-wise activation of the n x d encoder projection matrix.
3.3.2 VARIANCE- BASED DIMENSION SORTING
In addition to dataset-specific feature transformation, cross-dataset alignment of feature signals also greatly facilitates model adaptation. Taking a closer look the one-layer AE transformed feature vectors, essentially, each dimension in the projected space is formed by a weighted (nonlinear) com- bination of multiple dimensions in the original space. Localizing this concept to brain networks, the process is ultimately grouping the ROIs and their signals to derive new feature dimensions, which can be regarded as virtual ROIS. Although different datasets devise unique atlas templates, the base brain anatomy construction process follows fundamental theorems and practices in the neuroscience literature [Brodmann] (1909); Zhou et al. 2020). The final variations in ROI parcellations are results of post-processing due to different analytical and clinical conventions. Therefore, it is reasonable to assume a shared virtual ROI system underlying different parcellation systems. Incorporating the objectives of community learning and neighborhood preserving regularizers introduced in section 3.3 we are essentially training the AE to capture such shared virtual ROIs from a data-driven per- spective. Our final step is to align the discovered virtual ROIs across datasets such that, regardless of origins, each virtual ROI characterizes the same functional unit in the human brain.
Observe that one-layer linear AE shares similar objective with PCA (more details in Appendix butt with greater customizability (i.e., adding regularizers). In particular, PCA automatically orders the dimensions based on decreasing levels of sample variance [Htelling (933)). Taking this inspiration, we can leverage the learned parameters of the AE projection to approximate a variance estimate for each virtual ROI (i.e., projected feature dimension). Intuitively, each virtual ROI is computed with the sample variance indicating its representativenesss of the original data variations. Based on the previous discussion on shared patterns across different parcellation systems, we assume similar virtual ROIs in different datasets with different atlas templates would give similar variance scores, at least regarding the order of the scores. Thus, by separately sorting the same number of virtual ROIs according to the sample variance in each dataset, expect to achieve cross- dats-ett virtual ROI alignment, regarding the actual functional unit in the human brain. We summarize this procedure in detail in Appendix A.2 Algorithm]] 1

---

Under review as a conference paper at ICLR 2023
4 EXPERIMENTS
We analyze our framework by conducting extensive experiments on real brain network datasets, with a focus on these research questions: RQI: How does our method perform compared to other popular unsupervised GNN pre-training frameworks adapted to our setting? RQ2: How does each major component in our framework contribute to the overall performance? RQ3: How do the difference in sampling consideration influence model convergence and adaptation performance? RQ4: How does our variance- based sorting align virtual ROIs among different parcellation systems?
Datasets, configurations, and metrics. Our experiments use brain network datasets of diverse modalities from three IRB approved sources, including Parkinson's Progression Markers Initiative (PPMI), Bipolar Disorders (BP), and Human Immunodeficiency Virus Infection (HIV). Specifically, the PPMI is a public dataset that includes brain networks constructed from three tractography algo- rithms namely the Probabilistic Index of Connectivity (PICo), Hough voting (Hough), and FSL for 718 subjects, where 569 subjects are Parkinson's 's Disease (PD) patients and the rest 149 are Healthy Control (HC) ones; while BP a private dataset composed of the resting-state fMRI and DTI image data of 52 Bipolar subjects who are in euthymia and 45 HCs, and HIV involves 70 subjects, with 35 early HIV patients and 35 HCs. More details about the dataset origin and preprocessing are provided in Appendix B We pre-train the model on the PPMI data where brain networks constructed from each tractography algorithm are treated as separate pre-training dataset and then evaluate the target performance on BP and HIV. For backbone selection, we employ GCN as our base GNN encoder. Detailed
settings regarding the entire pipeline and GCN setup are described in Ap- pendix C In addition, we also benchmark our framework GAT and GIN, and the performance is reported in Appendix [D.T] Our target evaluation of disease prediction is formulated as binary graph classification problem. To this end. adopt the two widely used metrics in the medical discipline Li et al.] (2021b); Cui et al. (2022a), namely, accuracy score (ACC) and the area under the receiver operating characteristic curve (AUC) to assess the downstream performance.
4.1 OVERALL PERFORMANCE COMPARED WITH BASELINES (RQ1)
We present a comprehensive comparison of the target performance between our framework and existing unsupervised baselines in Table 2 The baselines are grouped based on their optimiza- tion strategies: Group 0: No pre-training (NPT), where model parameters are randomly initial- ized before fine-tuning on the target. Group 1: Non-CL-based frameworks, including Node2 WeepWalk, and VGAE, whose cost functions focus on co-occurrence agreement or link reconstruc- tion. Group 2: Single-scale CL, including GBT, ProGCL, and GraphCL, whose CL optimizations leverage either node- or graph-level representations. Group 3: Multi-scale CL, including DGI and InfoGraph (InfoG), whose CL. optimizations depend on examining both nodes- and graph-level rep- resentations. Group 4: Ego-graph sampling, including GCC and EGI, whose contrastive samplings consider k-hop ego-networks as discriminative instances, which are the most similar to our method.
Table 2: Disease prediction performance comparison. All results are averaged from 5-fold cross- validation along with standard deviations and have passed the significance test with p 0.05.
BP-MMRI
BP-DTI
HIV-MMI
HIV-DTI
Method
ACC
AUC
ACC
AUC
ACC
AUC
ACC
AUC
NPT
50.07a1330
50.1laisss
49.51214
51.831138
56.27a1584
57.16
51.30ato42
53.82a1404
Node2Vec
48.5 18.312
49.68
50.83
46. 702103
52.61a1038
50.75
49.6521030
51.22
DeepWalk
50.28-mm
51.59
52.
48.36
54.81
55.55-141
52.67-1142
50.88
VGÂE
$6.71s9ss
$5.24.11.a
54.63=1209
$4.21a1144
62.76
61.25=1111
56.90
55.35
GBT
57.21a
57.3241048
56,2923.s
55.27am04
65.7321000
66.082mas
57.37
GruphCL
59.79ass
59.10ame
57.57
$7.35mmsi
67. 67.08
69.17amms
60.43
60.03am04s
ProGCL
61. 61.36a530
61.61
60.26
61. 61.67a8.6
70.52e9.1
76.16
61 61.48ai0.s
60. 60.94
DGI
12
60.75
15-mm
58.95
40.22
70. 12-224
60.83
62.0641015
InfoG
62.87awa
62.37
60.88
60.44auai
72.46axm1
72.94axs
61.75amm
61.37asxs
GCC
63.45
62.39
60.4423.4
60.292m035
70.9741031
72.48
21.27
61. 11.3821002
EGI
63.38-431
63.58
61. 11.82
61.57
73.46sa.0 on
73.28ss.5
60,89
62. +2.5
Ours
68.84.xm
68.45
66.57.ta7
68.31 v0.39
77.80
77.62-k74
67.51.x4m
67.74.x.9

---

Under review as a conference paper at ICLR 2023
Overall, highlight the following observations:
Our framework demonstrates consistent improvements over the baselines. In particular, our method achieves 1.34%-13.30%% relative gain over the best performing baselines and 31.80% 38.26% relative gain over the NPT setting across all metrics.
Compared with the transductive methods of Vec and DeepWalk, the GNN pre-trained by representations and achieves the most promising results in group 1, but the performance still visibly inferior to Ours.
VGAE learns cruueture-preservvie
Maximizing MI between augmentations from a singular instance may prevent GNN from learning shared knowledge about the entire dataset. For baselines in groups 2-4, con-augmentationnbaat
CL pre-training generally reports 4.36% relative gain across both metrics and 7.63% relative decrease performance variance compared to augmentation-basse counterparts. For this reason,
Ours does not deploy data augmentation.
Multi-scale MI encourages capturing of effective local (i.e., node-level) representations capable of summarizing global (i.e., graph-level) information of the entire network. Hence, on average, group outperforms group by a relative gain 2.68% in ACC and 3.27% in AUC.
Group 4 presents the strongest baseline which benefits from its considerations of k--hpp neigh- borhood, indicating the importance of local neighborhoods in brain network analysis. Ours the only method that captures nodes' local neighborhoods through both nodes- and graph-level CL.
4.2
ABLATION STUDIES (RQ2)
BP-IMRI
HIV-IMRI
BP-IMRI
HIV IMRI
68
Metric
Metric
Metric
Metric
ACC
76
ACC
77
ACC
66
AUC
AUC
ALC
AUC
76
64
72
75
Var Var.2 Var. 3 Var.d
Poll Vac.2
Var.
Yar.
1
Phll
+1.8
-CIR
Pull
-50
+1.8
CC
Poll
Figure Ablation comparisons on contrastive sampling considerations (left two) and atlas mapping regularizers (right two). The y-axis refers to the numeric values of evaluated metrics (in %).
Figure shows the contribution of each constituent part of our framework to the overall performance in the 1MRI view
Table 3: Sampling variants overview
(the DTI version is in Appendix D.2). Specifically, the two sub-figures on the left investigate the different sampling con- siderations of $ S4- In particular, we list four possible variants detailed Table 3 To this end, we deliver three in- sights regarding the comparison: (1) leveraging k-hop neigh- borhood (i.e., positive S2) MI maximization brings visible performance gain, confirming its benefit in brain structure learning; (2) extending to multi-graph CL (i.e., consideration of Sz) encourages extraction of unique ROI knowledge, thus leading improved results in Var. 3/4; (3) Var. 4 performs better than Var. likely due to its summarization of global (iee,, graph-level) information in local node representations.
S
S2
Sz
S4
Var.
/
/
Var. 2
+
1
1
Var. 3
+
Var. 4
+
Meanwhile, the two sub-figures on the right focus on atlas mapping regularizers by comparing the downstream results without the sparsity regularizer (-SR), the locality regularizer (-LR), and the community regularizer (-CR) respectively against our full framework. We hereby remark two key observations: (1) removing SR leads to the greatest performance drop, indicating its importance in leamnngg noise robust projections; (2) LR and CR are both crucial atlas mapping optimization, validating our intuition in learning neighborhood-awee -wwre and blockwise feature information.
4.3 IN-DEPTH ANALYSIS OF TWO-LEVEL CONTRASTIVE SAMPLING (RQ3)
Figure Sshow
shows the further investigation of the four sampling variants and the full framework in terms of pre-training convergence, target adaptation progression, and pre-training runtime consumption. Overall, we highlight three important observations: (1) From Figure 5(a) we can observe that the pre-training convergence is generally efficient across all variants with the MAML-inspired multi- dataset joint optimization technique. addition, the full model presents the most optimal conver- gence confirming the benefit of learning shared neighhorhood information in brain network data
8

---

Under review as a conference paper at ICLR 2023
300
0.10
Full
Full
Var.
Var. 4
69
Var.
Var.
Full
Var.
Var.
-0.05
Var.
20
Var.
0
50
100 150
200
50
100
150
200
50 100
150
(a) Pre-train loss on PPMI
(b) Test ACC on HIV-IMRI
(c) Pre-train time (secs) on PPMI
Figure 5: In-depth comparison among the four variants and our full model. The z-axis is epochs.
via two-level node contrastive sampling. (2) From Figure 5(b) the effectiveness learning shared substructure knowledge within fixed ROI system directly reflected in superior downstream adaptation compared to other variants. (3) From Figure 5(c) the more sophisticated the sampling considerations, the greater the computational complexity for MI evaluation, resulting in larger time for each pre-training epoch. However, the total time consumptions are all on the same scale.
4.4 IN-DEPTH ANALYSIS OF VIRTUAL ROI ALIGNMENT (RQ4)
RL
(a) PPMI mapping
(b) BP mapping
HIV mapping
Figure 6: Virtual ROI Mapping across the three investigated datasets.
To further validate our designs of variance-basse virtual ROI sorting, for each atlas template (i.e., dataset), we select the top virtual ROIs that have the highest sample variances and backtrack to locate their projected ROIs. We then illustrate in Figure O visualization highlighting the original ROIS on a 3D brain surface. Specifically, we remark on two key takeaways: share similar ROI projection regions in the medial (i.e., bottom left and right) views of the brain surface, reflecting the working effectiveness our ROI alignment design. (2) There exist certain variations among ROI projections from the lateral (i.e., top left and right) view, which shows limitation of unsupervised atlas mapping but is still better than no alignment at all.
(1) The three atlases
5 CONCLUSION
Brain network analysis for task-specific disease prediction is challenging for conventional GNN frameworks. Firstly, effective knowledge extraction for deep neural models requires a significant amount of training data and task labels which are often lacking in neuroimaging studies, especially for local downstream tasks. Moreover, the absence of a unifying brain atlas definition prevents efficient knowledge transfer across different domains. Through extensive benchmarking on real- world brain connectome datasets, our framework reports superior and robust performance in disease prediction and clearly outperforms various -ftt-off-the-art baselines. Specifically, our framework implements unsupervised multi-datase GNN pre-training via two-level node contrastive sampling to address the data-scarcity concern. Concurrently, our framework also features atlas mapping via brain-network-oriennee regularizers and variance-based sorting which presents data-driven solu- tion to the incompatible ROI parcellation systems for cross-dataset model adaptation. Due to the difficulties in obtaining many brain network datasets, experiments focused three widely used brain imaging studies, and is natural to further validate the generalizability of our framework with more brain network datasets when they become available.

---

Under review as a conference paper at ICLR 2023
REFERENCES
Darko Aleksovski, Dragana Miljkovic, Daniele Bravi, and Angelo Antonini. Disease progression in parkinson subtypes: the ppmi dataset. Neurol. Sci., 19:1971-1976, 2018.
Jimmy Ba, Geoffrey E Hinton, Volodymyr Mnih, Joel Z Leibo, and Catalin lonescu. Using fast weights to attend the recent past. NeurIPS, 2016.
Piotr Bielak, Tomasz Kajdanowicz, and Nitesh V Chawla. Graph barlow twins: A self-supervised representation learning framework for graphs. Knowl Based Syst, 2022.
Korbinian Brodmann. Vergleichende Lokalisationslehre der Grosshirnrinde in ihren Prinzipien dargestellt auf Grund des Zellenbaues. 1909.
Bokai Cao, Liang Zhan, Xiangnan Kong, Philip Yu, Nathalie Vizueta, Lori I. Altshuler, and Alex I)
Leow. Identification of discriminative subgraph patterns in fmri brain networks in bipolar affective disorder. In International Conference on Brain Informatics and Health, 2015.
Yue Cao, Zhenda Xie, Bin Liu, Yutong Lin, Zheng Zhang, and Han Hu. Parametric instance classi- fication for unsupervised visual feature learning. NewrlPS, 2020.
BJ Casey, Tariq Cannonier, May I Conley, Alexandra O Cohen, Deanna M Barch, Mary M Heitzeg, Mary E Soules, Theresa Teslovich, Danielle V Dellarco, Hugh Garavan, et al. The adolescent brain cognitive development (abed) study: imaging acquisition across 21 sites. Dev. Cogn. Neu- rosci., 32:43-54, 2018.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020a.
Ting Chen, Simon Korbblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. ICML, 2020b.
Hejie Cui, Wei Dai, Yanqiao Zhu, Xuan Kan, Antonio Aodong Chen Gu, Joshua Lukemire, Liang Zhan, Lifang Hc, Ying Guo, and Carl Yang. Braingb: A benchmark for brain network analysis with graph neural networks. arXiv preprint arXi::2204.0707
2022a.
Hejie Cui, Wei Dai, Yanqiao Zhu, Xiaoxiao Li, Lifang He, and Carl Yang. Interpretable graph neural networks for connectome-basee brain disorder analysis. 2022b.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019a.
Jacob Devlin, Ming-Wes Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In NAACI. 2019b.
Federico Errica, Marco Podda, Davide Bacciu, and Alessio Micheli. fair comparison of graph neural networks for graph classification. In ICLR, 2020.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-leariin for fast adaptation of deep networks. In ICML, 2017.
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, et al. Bootstrap your own latent-a new approach to self-supervised learning. NeurIPS, 2020.
Aditya Grover and Jure Leskovec. node2vec: Scalable feature learning for networks. SIGKDD, 2016.
Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In NeurlPS, 2017.
Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, 2020a.
10

---

Under review as a conference paper at ICLR 2023
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgen: Simplifying and powering graph convolution network for recommendation In SIGIR, 2020b.
Xiaofei He, Deng Cai, Shuicheng Yan, and Hong-Jiang Zhang. Neighborhood preserving embed- ding. In ICCV, 2005.
Chris Hinrichs, Vikas Singh, Lopamudra Mukherjee, Guofan Xu, Moo K Chung, Sterling C John- son, Alzheimer's Disease Neuroimaging Initiative, et al. Spatially augmented Ipboosting for ad classification with evaluations on the adni dataset. Neurolmage, 48: 138-149, 2009.
Geoffrey E Hinton and Richard Zemel. Autoencoders, minimum description length and helmholtz free energy. NeurIPS, 1993.
Harold Hotelling. Analysis of a complex of statistical variables into principal components. J Educ Psychol, 1933.
Yupeng Hou, Binbin Hu, Wayne Xin Zhao, Zhiqiang Zhang, Jun Zhou, and Ji-Rong Wen. Neural graph matching for pre-training graph neural networks. In SDM, 2022.
W Hu, B Liu, Gomes, M Zitnik, P Liang, V Pande, and Leskovec. Strategies for pre-training graph neural networks. In ICLR, 2020a.
Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: Datasets for machine learning on graphs. 2020b.
Ziniu Hu, Yuxiao Dong, Kuansan Wang, Kai-Wei Chang, and Yizhou Sun. Gpt-gnn: Generative pre-training of graph neural networks. In SIGKDD, 2020c.
Seongah Jeong, Xiang Li, Jiarui Yang, Quanzheng Li, and Vahid Tarokh. Dictionary learning and sparse coding- based denoising for high-resolution task functional connectivity mri analysis. In International Workshop on Machine Learning in Medical Imaging, 2017.
Xuan Kan, Hejie Cui, Joshua Lukemire, Ying Guo, and Carl Yang. Fbnetgen: Task-aware gnn-based fmri analysis via functional brain network generation. MIDL, 2022.
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. NIPS Workshop on Bayesian Deep Learning, 2016.
Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional net- works. In ICLR, 2017.
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: training for unified laniun-l understanding and generation. In ICML, 2022.
Bootstrapping language-image pre-
Pengyong Li, Jun Wang, Ziliang Li, Yixuan Qiao, Xianggen Liu, Fei Ma, Peng Gao, Sen Song, and Guotong Xie. Pairwise half-graph discrimination: A simple graph-level self-supervised strategy for pre- pre-traiming graph neural networks. In IJCAI, 2021a.
Xiaoxiao Li, Yuan Zhou, Nicha Dvorekk, Muhan Zhang, Siyuan Gao, Juntang Zhuang, Dustin Scheinost, Lawrence H Staib, Pamela Ventola, and James S Duncan. Braingnn: Interpretable brain graph neural network for fmri analysis. Med Image Anal, 2021b.
Martin A Lindquist. The statistical analysis of fmri data. Stat Sci, 23:43994666 2008
Shengchao Liu, Hanchen Wang, Weiyang Liu, Joan Lasenby, Hongyu Guo, and Jian Tang. Pre- training molecular graph representation with 3d geometry. ICLR, 2022.
Ilya Loshchilov and Frank Hutter. SGDR: stochastic gradient descent with warm restarts. In ICLR, 2017.
Yuanfu Lu, Xunqiang Jiang, Yuan Fang, and Chuan Shi. Learning to pre-train graph neural networks. In AAAI, 2021.
11

---

Under review as a conference paper at ICLR 2023
Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong. Haifeng Chen, and Xiang Zhang. Parameterized explainer for graph neural network. In NeurIPS, 2020.
Guixiang Ma, Chun-Ta Ta Lu, Lifang He, S Yu Philip. and Ann B Ragin. Multi-view graph embedding with hub detection for brain network analysis. In ICDM, 2017.
Alireza Makhzani and Brendan Frey. K-sparse autoencoders. ICLR, 2014.
Gustav Martensson, Joana Pereira, Patrizia Mecocci, Bruno Vellas, Magda Tsolaki, Iwona Kloszewska, Hilkka Soininen, Simon Lovestone, Andrew Simmons, Giovanni Volpe, et al. Sta- bility of graph theoretical measures in structural brain networks in alzheimer's disease. Scientific reports, 8:1-15 2018.
Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word represen- tations in vector space. In ICLR, 2013.
Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representa- tions. In SIGKDD, 2014.
Jiezhong Qiu, Qibin Chen, Yuxiao Dong, Jing Zhang, Hongxia Yang, Ming Ding, Kuansan Wang, and Jie Tang. Gee: Graph contrastive coding for graph neural network pre-training. SIGKDD, 2020.
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language under- standing by generative pre-training, 2018.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In ICML, 2021.
Ryan A Rossi and Nesreen Ahmed. An interactive data repository with visual analytics. SIGKDD, 2016.
Guillaume Salha-Galvan, Johannes Lutzeyer, George Dasoulas, Romain Hennequin, and Michalis Vazirgiannis. Modularity- aware graph autoencoders for joint community detection and link pre- diction. Neural Netw, 2022.
Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. Modeling relational data with graph convolutional networks. In ESWC, 2018.
Rui Shi, Jian Ji, Chunhui Zhang, and Qiguang Miao. Boosting sparsity-induced autoencoder: A novel sparse feature ensemble learning for image classification. Int. Adv. Robot. Syst., 2019,
Stephen M Smith. The future of fmri connectivity. Neurolmage, 62:1257-1266, 2012.
Fan-Yo Yun Sun, Jordan Hoffman, Vikas Verma, and Jian Tang. Infograph: Unsupervised and semi- supervised graph-level representation learning via mutual information maximization. In ICLR, 2019.
Mingchen Sun, Kaixiong Zhou, Xin He, Ying Wang, and Xin Wang. Gppt: Graph pre-training and prompt tunigg to generalize graph neural networks. In SIGKDD, 2022.
Hong Hui Tan and King Hann Lim. Review of second-order optimization techniques in artificial neural networks backpropagation. IOP Conference Series: Materials Science and Engineering, 2019.
Nathalie Tzourio-Mazoyerr Brigitte Landeau, Dimitri Papathanassiou, Fabrice Crivello, Olivier Etard, Nicolas Delcroix, Bernard Mazoyer, and Mare Joliot. Automated anatomical labeling of activations in spm using macroscopic anatomical parcellation of the mni mri single-subject brain. Neurolmage, 2002.
Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha Talukdar. Composition multi- relational graph convolutional networks. In ICLR, 2020.
12

---

Under review as a conference paper at ICLR 2023
Petar Velickovic, William Fedus, William L Hamilton, Pietro Liö, Yoshua Bengio, and R Devon Hjelm. Deep graph infomax. ICLR, 2019.
M Vu and MT Thai. Pgm-explainer: Probabilistic graphical model explanations for graph neural networks. In NeurIPS, 2020.
Zirui Wang, Jiahui Yu, Adams Wei Yu, Zihang Dai, Yulia Tsvetkov, and Yuan Cao. Simvim: Simple visual language model pretraining with weak supervision. ICLR, 2022.
Jun Xia, Lirong Wu, Ge Wang, Jintao Chen, and Stan Z Li. Progel: Rethinking hard negative mining in graph contrastive learning. In ICML, 2022.
Wenhan Xiong, Jingfei Du, William Yang Wang, and Veselin Stoyanov. Pretrained encyclopedia: Weakly supervised knowledge-pretraing language model. ICLR, 2020.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? In ICLR, 2019.
Noriaki Yahata, Jun Morimoto, Ryuichiro Hashimoto, Giuseppe Lisi, Kazuhisa Shibata, Yuki Kawakubo, Hitoshi Kuwabara, Miho Kuroda, Takashi Yamada, Fukuda Megumi, et al. A small number of abnormal brain connections predicts adult autism spectrum disorder. Nat. Commun., 7:1-12, 2016.
Lewei Yao, Runhui Huang, Lu Hou, Guansong Lu, Minzhe Niu, Hang Xu, Xiaodan Liang, Zhenguo Li, Xin Jiang, and Chunjing Xu. Filip: Fine-grained interactive language-image pre-training. ICLR, 2022.
Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. Gnnexplainer: Gen- erating explanations for graph neural networks. In NeurIPS, 2019.
Zhitao Ying, Jiaxuan You, Christopher Morris, Xiang Ren, Will Hamilton, and Jure Leskovec. Hi- erarchical graph representation learning with differentiable pooling. In NeurlPS, 2018.
Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. Graph contrastive learning with augmentations. In NeurlPS, 2020.
Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. Barlow twins: Self-supervised learning via redundancy reduction. In ICML, 2021.
Shijie Zhang, Hongzhi Yin, Tong Chen, Quoe Viet Nguyen Hung. Zi Huang, and Lizhen Cui. Gen- based user representation learning for unifying robust recommendation and fraudster detection. In SIGIR, 2020.
Tong Zhao, Yozen Liu, Leonardo Neves, Oliver Woodford, Meng Jiang, and Neil Shah. Data aug- mentation for graph neural networks. In AAAI, 2021.
Zhen Zhou, Xiaobo Chen, Yu Zhang, Dan Hu, Lishan Qiao, Renping Yu, Pew-Thian Yap, Gang Pan, Han Zhang, and Dinggang Shen. A toolbox for brain network construction and classification (brainnetclass). Hum Brain Mapp, 2020.
Qi Zhu, Carl Yang, Yidan Xu, Haonan Wang, Chao Zhang, and Jiawei Han. Transfer learning of graph neural networks with ego-graph information maximization. In NeurlPS, 2021a.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Deep Graph Contrastive Representation Learning. In ICML Workshop on Graph Representation Learning and Beyond, 2020.
Yanqiao Zhu, Yichen Xu, Feng Yu, Qiang Liu, Shu Wu, and Liang Wang. Graph Contrastive Learn- ing with Adaptive Augmentation. In WWW, 2021b.
13

---

Under review as a conference paper at ICLR 2023
A ANALYSIS OF THE AUTOENCODER STRUCTURE
A.1 BRIDGING RECONSTRUCTION MINIMIZATION AND VARIANCE MAXIMIZATION
In this subsection, we briefly discuss how the reconstruction minimizing objective in one-layer AE be cast variance maximizing objective in PCA. Assume given data matrix X E R its covariance matrix Σ = =X"X R"x", and single-laye AE projection matrix W E R with parameters randomly initialized from the continuous uniform distribution (0, 1). the reconstruction objective is:
X - XX-WWWTIP - tr ((X -
- -XXXX"Y) ((XWWTX))
- (Ir(X"") (
+ + ((XXWWWW'X'') WW
(cl xXXWWXX) + +IXXTT
-c w(wpw''))) +o)
= C3 C4 u(W''''XWW
= tr(wT
Notice that ; C2, C3, C4 non-negative scalar constants that do influence the overall optimiza- tion schedule. Hence, alternatively, the optimal AE projection also maximizes the sample variance tr( EW), achieving an identical end goal of PCA transform. Specifically, according to PCA, variance maximization is realized by constructing the projection W to contain the set of orthonormal eigenvectors of Σ that gives the largest cigenvalues HHoteling (1933). That is, there is an orthogo- nality constraint on W. Minimizing the MSE reconstruction also results in an orthogonal W:
x - XWWW 0 XWW = = WW =I
Therefore, the optimal AE projection W is also capturing a set of variance- variance-maxxiiizing maximizing orthogonal vectors. Note that the AE optimized W is theoretically equivalent the eigendecomposition of E and only the reconstruction loss is 0. Therefore, practice, the AE is, at best, an approximate solution to variance maximization.
A.2 VARIANCE-BASED SORTING PROCEDURE
Following the discussion in A.T assuming a perfect optimization, the linear one-layer AE behaves similarly to PCA. and there Notice that in PCA. the cigenvalue of the covariance matrix Σ signifies the intensity of data variation along the direction of its corresponding eigenvector, which is essentially column entry of the transformation matrix. Then intuitively, given an optimized AE projection W, we can examine, for each column of W, its representativeness (i.e., data variance) of the data covariance with scalar estimate (i.e., an eigenvalue-like scoring). Inspired by the properties of eigendecomposition, can approximate these estimates by measuring the distance of W w.r.t the product of linearly transforming W through Σ by scaling factor of X. More specifically, we want to solve for À such that Ew Xw for every column vector w W. Under the PCA perspective, contains the variance estimate for each column- wise individual projection of W. To this end, we detail the sorting procedure in Algorithm
equivalence relation between their respective objective functions.
B DATASET DETAILS
All datasets used, which are introduced below, are processed anonymously with no personally iden- tifiable information. In addition, all related studies conducted according the Good Clinical
14

---

Under review as a conference paper at ICLR 2023
Algorithm 1 Overview procedure for variance-based sorting
Input: Original feature matrix X E [ AE optimized projection matrix W E Rdxm Initialize: Scalar vector À E mR" Small positive float
Output: Sorted AE projection matrix W
1: Normalize the feature matrix: Xn f X||
2: Compute data covariance matrix: Σ 4 Xn
T Xn
3: Solve for À such that EW A W| <
4: Sort column vectors w W according to sorted indexation of À obtain W
Practice guidelines and U.S. 21 CFR Part (Protection of Human Subjects) and under the approval of Institutional Review Boards.
Parkinson's Progression Markers Initiative (PPMI): We pre-train the model on large-scale real-life Parkinsons Progression Markers Initiative (PPMI) data which is publicly available at consists of 718 subjects, where 569 subjects are Parkinson's 's Disease (PD) patients and the rest 149 are Healthy Control (HC) ones. Eddy-curree and head motion correction are performed using FSIRannd and the brain networks are extracted using the same tool. The EPI induced susceptibility artifacts correction is handled using Advanced Normalization Tools (ANT In the meantime, 84 ROIs are parcellated from T1-weighted structural MRI using Freesurfe The brain networks are constructed using three whole brain tractography algorithms namely the Probabilistic Index of Connectivity (PICo), Hough voting (Hough), and FSL. Each resulted network for each subject is 84 x 84. Each brain network normalized by the maximum value to avoid computation bias for the later feature extraction and evaluation, since matrices derived from different tractography algorithms differ in scales and ranges.
Bipolar Disorders (BP): This local and private dataset is composed of the resting-state fMRI and DTI image data 52 Bipolar subjects who are in euthymia and 45 Healthy Controls (HCs) with matched age and gender Cao et al. (2015); Ma et al. (017)) The fMRI data was acquired 3T Siemens Trio scanner using a T2" echo planar imaging (EPI) gradient-echo pulse sequence with integrated parallel acquisition technique (IPAT) and DTI data were acquired
Siemens 3T Trio scanner. The brain networks are constructed using the CONN toolbox We performed the normalization and smoothing after first realigning and co-registering the raw EPI pictures. After that, the signal was regressed to remove the confounding effects of the motion artifact, white matter, and CSF. The 82 cortical and subcortical gray matter regions produced by Freesurfer were identified, and pairwise signal correlations were used to build the brain networks.
5
Human Immunodeficiency Virus Infection (HIV): Collected from the Early HIV Infection Study at Northwestern University, this private dataset involves fMRI and DTT brain networks for 70 subjects, with 35 of them early HIV patients and the other 35 Healthy Controls (HCs). These two groups of subjects do not differ in demographic distributions such age and biological sex. The preprocessings for IMRI including brain extraction, slice timing correction and realignment are managed with the DPARSH toolbox. while the preprocessings for DTI such distortion correction are finished with the help ESL toolbox. Finally, brain networks with 90 regions of interest are constructed based on the automated anatomical labeling (AAL) Tzoyrio-Mazoyer et al. 2002).
C HYPERPARAMETER SETTINGS
GNN Setup. The GCN encoder is composed of graph convolution layers with hidden dimensions of 32, 16, 16, and Similarly, the GAT encoder is built from 4 graph attention layers with hidden dimensions of 32, 16, 16, and 8. Regarding GIN, which is slightly different, the encoder consists of MLP layers with each MLP containing linear layers with a unifying hidden dimension of 8.
http://ww..iirr..orr/roooocooi
htt:://ffnr..org/ppa...
15

---

Under review as a conference paper at ICLR 2023
Pre-training Pipeline Setup. For two-level node contrastive sampling, we set kt 2 the radius regarding k-hop neighborhood sampling for S1 and S1- To enable efficient computation on multi- and we set default batch size of 32. In addition,
graph MI evaluation, resort to we leverage the popular Adam Kingma & Bal (2015) optimizer with the learning rate set to 0.002 as well the cosine annealing scheduler Loshhhiilo & Hutter (2017) to facilitate GNN training. In general, a complete pre-training cycle takes 400 epochs with an active deployment of early stopping.
Atlas Mapping Regularizers Setup. Following the discussion in section 3.3 of the AE projection is given as:
the total running loss
(5) in particular, we set ox B = 0.8 and 7 0.01. The one-layer AE encoder transforms the feature signals from all given datasets into universally projected dimension of 32. Regarding the details of locality-press preserving regularizer (i.e., Lioc), the transition matrix T is built from the 5-nearest- neighbor graph from the 3D coordinates each atlas templates. For the sparsity-oriented regularizer (i.e., ). the target sparsity value P is set to 1e- The overall optimization process, which is similar to model pre- trainingn, takes a total of 100 epochs with a learning rate of 0.02.
= + afLire + &oom + CKI,,
Downstream Evaluation Setup. For each target evaluation, the fine-tuning process features a 5- fold cross-validation, which approximately splits the dataset into 70% training, 10% validation, and 20% testing. To prevent model over-fitting, we implement L2 penalty with coefficient of -4. Overall, the model fine-tuning process, which is nearly identical to the other two training procedures, takes total of 200 epochs with a learning rate of 0.001 and a cosine annealing scheduler.
D ADDITIONAL EXPERIMENT
D.1 DOWNSTREAM PERFORMANCE ON GAT AND GIN
Table 4: Disease prediction performance of our framework using GAT and GIN.
BP-MMRI
BP-DTI
HIV-IMRI
HIV-DTI
Method
ACC
AUC
ACC
AUC
ACC
AUC
ACC
AUC
Ours w/ GCN
68.84.a
68.45.4
66.57
68.31
77.80
77.22-a7
67.51xx
67.74-mm
Ours w/ GAT
66.96.a7
69,68.98
64.23
63.76410.0
74.93x1035
75.78=11
65, 85.84
66.51-1207
Ours w/ GIN
66.30asm
68.92.937
64, 28
66.44
75.96
77.63
67.36
65.95=112
Treporsss the downstream performance of our full framework using GAT and GIN as backbone encoders. In general, the two encoders deliver inferior performance compared to GCN, which sug- gests that complex GNN convolutions (e.g., GAT and GIN) might not be effective as they seem when learning on bran network datasets.
D.2 ADDITIONAL ABLATION STUDIES ON DTI VIEW
BP-DII
HIV-DTI
BP-DIT
HIV-DIT
68
Metric
Metric
Metric
Metric
ACC
66 AUC
AUC
AUC
AUC
65
65
Yac. Var. Var. 3 Var.4
Pull
Var. Var.2 2 Var. Var.
Pell
-012
+1.0
-031
P'll
-00
+1.0
433
Tull
Figure 7: Additional ablation comparisons on DTI views. The left two subfigures refer to con-
trastive sampling considerations and the right two subfigures refer atlas mapping regularizers. The y-axis refers to the numeric values of evaluated metrics (in %).
presents our ablation studies on the DTI view following the same setup as discussed in Section We draw similar conclusions from the DYTI-based analysis where each constituent component of our two-level sampling consideration as well the atlas mapping mechanism has proven positive contribution and significance towards the overall performance and robustness.
Figure 7
16


Reviews:
[{"review_id": "fmLVrKRYUym", "reviewer": "Reviewer_Y7QB", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a method for graph-based learning from brain imaging datasets, where they 1) pretrain GNN using 2-level contrastive learning, which leverages some brain network knowledge, under a MAML optimization approach , and 2) perform a data-driven atlas transformation with various regularizations and variance sorting to provide some alignment of ROIs across different datasets. The method is tested using 3 datasets, where the large PPMI dataset is used for the pretraining and the transfer to task learning is tested on the other 2 datasets. Experiments are performed comparing multiple baselines with and without pretraining and studying effect of model components through ablation studies.", "summary_of_the_review": "While the work largely builds off of existing prior work, the new approach to defining positive samples for contrastive learning of GNNs for brain datasets is interesting and makes sense, and a number of experiments have been performed to demonstrate the potential advantage of the proposed methods. However, some clarifications are needed.", "strength_and_weaknesses": "Strengths:\n1. The method considers leveraging additional brain network information not previously considered in GNN pretraining - using samples from corresponding node positions in other graphs as positive examples for the anchor node. This seems like a reasonable way to define more positive samples utilizing other graphs for contrastive learning.\n\n2. The approach considers the problem of combining brain imaging datasets that used different parcellations and thus have different graph node configurations.\n\n3. The experiments have meaningful comparisons to baselines separated into different groups (e.g., no pretraining, baselines using different levels of node/graph representation in contrastive learning) to help understand the improvements in model decision.\n\n4. There are several ablation experiments included to see the effect of different parts of the design approach.\n\n\nWeaknesses: \n1. The motivation for the data driven atlas transformation is to get a common graph configuration for all the datasets, since different datasets may use different parcellations. However, what if one fixed atlas were used and applied to the existing imaging data - this is not compared? Is the idea that it is not always available? Furthermore, my understanding is each dataset transformation is learned separately - then how can the transformed space be aligned? I understand that the \"virtual ROIs\" are set according to some variance ranking, but this does not mean the ROIs would be aligned across datasets - as seen in Fig. 6 which is supposed to demonstrate overlap, I would argue that the ROIs do NOT look aligned. Then, if these ROIs are not aligned, does the pretraining of a model to use on a different dataset make sense?\n\n2. It is unclear to me if the atlas transformation is learned separately from the pretraining step? And whether each transformation mapping is learned separately for each dataset? Also, is it based on all the data, or trained using just training data? It would be unfair using all data as the representation is then fit to the test data as well.\n\n3. In comparing the proposed method to other methods with pretraining, I presume this is with the proposed atlas transformation, while the comparison method does not have any such mapping estimated - is this correct? Or are other methods also learned using the atlas transformation? If not, then it is unclear the contribution of the proposed contrastive learning formulation vs. the atlas transformation. If could perform the atlas transformation for comparison methods, then can more fairly compare just the contrastive learning for pretraining. Or, do not include any atlas transformation, and then compare the contrastive learning approaches.\n\n4. Some of the experimental details are confusing to me. For example, the plots in Fig. 5 show pretraining and learning on the task dataset for 150 epochs, while the appendix says pretraining goes for 400 epochs and the task learning for 200 epochs. In Fig. 5b, it seems that the curves for all the approaches may not have converged yet, so the comparison is not quite fair, even though at that epoch (which is not 200) the proposed method is most accurate. \n\n5. At a higher level, I wonder whether expecting the same parcellation of brain regions for fMRI vs DTI data makes sense - it may be that the natural groupings based on fMRI data may not align with natural groupings based on structural DTI data. Yet both these types of datasets are simply combined for analysis. Could the authors comment on this?\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "_judopGVt6", "reviewer": "Reviewer_khKm", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The manuscript proposed data-driven brain atlas mapping and contrastive pretraining for brain network analysis. The authors show improved results over previous baselines, ablate the proposed method based on different strategies, and perform a visual examination of ROI alignment. Contrastive learning is based on Jensen-Shannon divergence with positive and negative pairs defined based on the graph. The data-driven brain atlas mapping incorporates an autoencoder with brain-oriented regularizers based on locality, brain modules, and sparsity. In addition, variance-based dimension sorting is proposed to align ROIs of the different dataset and their atlases.", "summary_of_the_review": "Overall, I like the proposed approach and comparison with multiple baselines. The proposed framework seems to improve over previous baselines. However, I have some minor issues that I want to clarify. \n\nFurther, I think the authors should propose some methodology for the data-driven atlas mapping, as the visual evaluation does not seem enough.", "strength_and_weaknesses": "#### Strength\n- The experiments were performed on multiple datasets.\n- There are multiple graph baselines.\n- There are ablation studies for choosing contrastive sampling strategies and atlas mapping regularizes.\n\n#### Weaknesses\n- It is not clear what statistical significance test has been used. The t-test should not be used as you can use it only if you pass the normality assumption. Further, you need to report statistics in a valid format. In most scenarios in Table 2 and Table 4, the variance is so high that most performance metrics should overlap, so I don't think the metrics should pass the significance testing. In addition, have you explored the reason behind such a huge variance?\n- There is no final holdout-test set, only 5-fold cross-validation. The validation set has to be used only to select checkpoints or hyperparameters.\n- Alignment of the virtual ROIs is only done visually. Consider segmentation measures (e.g., DICE (F1)) or other measurable methodology to evaluate the performance of data-driven brain atlas mapping concerning the original atlas. \n- There are no non-graph-based baselines or discussions. \n- There is no discussion on related work for Neuroimaging domain:\n  - That describes the use of GNNs or pretraining:\n    - Mahmood, Usman, et al. \"Multi network InfoMax: A pre-training method involving graph convolutional networks.\" arXiv preprint arXiv:2111.01276 (2021)\n    - Mahmood, Usman, et al. \"Deep Dynamic Effective Connectivity Estimation from Multivariate Time Series.\" arXiv preprint arXiv:2202.02393 (2022).\n  - That describes data-driven brain atlas mapping (e.g., via spectral clustering):\n    - Geenjaar, Eloy, et al. \"Spatio-temporally separable non-linear latent factor learning: an application to somatomotor cortex fMRI data.\" arXiv preprint arXiv:2205.13640 (2022)\n- Could you clarify how the matching of the nodes for a positive pair between the source datasets is enforced? It seems that nodes should be paired based on spatial alignment. How do you guarantee that nodes are spatially aligned to the same brain region after the autoencoder?\n- I am unsure if you used the same MAML framework for all the baselines. Could it be the main reason for the improvement? Could you compare your model by pretraining on two datasets and perform transfer learning by fine-tuning on the third?\n\n\n\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "NbvPRsHZXwH", "reviewer": "Reviewer_TtZF", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents an unsupervised GNN pre-training technique for brain networks. The authors first designed a two-level contrastive learning method and then propose a data-driven atlas mapping technique for mapping different ROI systems. The method was evaluated in three brain network datasets. The results showed the proposed method outperformed other pretraining frameworks in disease classification problems.", "summary_of_the_review": "The second motivation of this paper is problematic, so it is not suitable for published in ICLR.", "strength_and_weaknesses": "The second work of this paper about brain atlas mapping is problematic. The author states that they need to convert networks between different ROI systems because the brain networks in the two datasets will have different numbers and semantics of nodes. However, to train the model, you do not need to convert the network between different brain templates, you only need to generate data with the same brain template. Because raw MRI image data is voxel-based, an easy way is to generate a brain network based on the same brain template. So, I do not see a reasonable motivation for using deep learning methods to solve these simple problems. The authors may argue that the available data are based on different brain templates and need to convert. However, keep in mind that brain networks are just an analytical representation of neuroimaging data, and different networks can be generated based on different templates and parameters. Therefore, it is simpler and more accurate to generate a unified brain network directly from raw image data using the same brain template. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "1: The main claims of the paper are incorrect or not at all supported by theory or empirical results.", "technical_novelty_and_significance": "1: The contributions are neither significant nor novel.", "empirical_novelty_and_significance": "Not applicable"}, "extra_text": {}}, {"review_id": "IZjgHASCgS9", "reviewer": "Reviewer_XzZd", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The authors propose to leverage an unsupervised pre-training strategy to capture the intrinsic brain network structures regardless of specific clinical outcomes for obtaining GNN models that are easily adaptable to downstream tasks. The method utilizes large-scale brain imaging dataset that may not be related to a specific target task to improve the target task, and a data-driven parcellation atlas mapping pipeline is proposed to interpret knowledge transfer across studies with different ROI systems. ", "summary_of_the_review": "The proposed model has some merits as shown in the performance improvement but there are many places where the description of the framework are not very clear. The authors should better demonstrate their ideas such that this paper can be read and understood by a broader audience. ", "strength_and_weaknesses": "Strength:\n- The proposed method can propose a globally sound initialization for a graph neural network (GNN) to be trained. \n- The proposed method let one combine multiple datasets to come up with the initial representation.\n- Experiments on various datasets demonstrate that the method improves training of other GNN module. \n\nWeakness:\n- The authors mentioned that obtaining a large-scale brain imaging dataset is difficult but the model is proposed assuming that a large-scale dataset exists for training the proposed method. \n- Problem definition is not formally written. According to the introduction, as $D_i$ are all different, I assume that dataset sizes are different across $D_i$ but all of them are presumably given as $N$. It is not clear if there exists one $A_j$ per dataset, and the objective of GNN $f()$ (i.e., input and output) is also not clear. Moreover the description of $\\theta^{t+1}$ seems not right with $L_{query_i} f(\\theta\\prime_i^t)$ as the $f()$ should have a graph $G$ as an input and multiplying L and f doesn't make sense; please correct me if I am wrong. Is there any description of measurement on the nodes? \n- Lack of problem description lead to confusion in the method section. Contrastive learning is performed with $X$ but it is not clear what the $X$ is. Is this measurements on the graph nodes? \n- At the beginning of 3.2, X was described as a sample. Then, $\\bf S$ is named as an arbitrary sample but eq (2) sums over $\\bf S$ where $X_S$ are its elements and not samples which is causing confusion. \n- S1 and S4 are considered as positive samples but S4 is a subset of S3 which is a set of negative samples. Is there a reason why the data were divided into subsets such that they contradict to each other? Also, there is no equation or loss function explaining how these subsets are contrastively trained. \n- It is not clearly explained how the authors deal with dataset heterogeneity. \n- A summary table of individual dataset can be helpful.  \n- Ablation study is performed on the variants of subsets for contrastive learning. A reader perhaps would be more interested how much each regularization term designed by the author is contributing to the performance. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject