SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiaxing Xu $^{1*}$ , Yunhan Yang $^{2*}$ , David Tse Jung Huang $^{2*}$ , Sophi Shilpa Gururajapathy $^{1*}$ , Yiping Ke $^{1}$ , Miao Qiao $^{2}$ , Alan Wang $^{3,4,5}$ , Haribalan Kumar $^{6,7}$ , Josh McGeown $^{7\dagger}$ , Eryn Kwon $^{3,7\dagger}$  partially for the Alzheimer's Disease Neuroimaging Initiative $\ddagger$
$^{1}$ School of Computer Science and Engineering, Nanyang Technological University, Singapore
$^{2}$ School of Computer Science, The University of Auckland, New Zealand
$^{3}$ Auckland Bioengineering Institute, The University of Auckland, New Zealand
$^{4}$ Faculty of Medical and Health Sciences, The University of Auckland, New Zealand
$^{5}$ Centre for Brain Research, The University of Auckland, New Zealand
$^{6}$ General Electric Healthcare Magnetic Resonance, Australia & New Zealand
$^{7}$ Mātai Medical Research Institute, Tairāwhiti-Gisborne, New Zealand
{jiaxing003, sophi.sg, ypke}@ntu.edu.sg; {h.kumar, j.mcgeown}@matai.org.nz
{yunhan.yang,dtj.huang, miao.qiao, alan.wang, e.kwon}@auckland.ac.nz
This paper presents a comprehensive and quality collection of functional human brain network data for potential research in the intersection of neuroscience, machine learning, and graph analytics. Anatomical and functional MRI images have been used to understand the functional connectivity of the human brain and are particularly important in identifying underlying neurodegenerative conditions such as Alzheimer's, Parkinson's, and Autism. Recently, the study of the brain in the form of brain networks using machine learning and graph analytics has become increasingly popular, especially to predict the early onset of these conditions. A brain network, represented as a graph, retains rich structural and positional information that traditional examination methods are unable to capture. However, the lack of publicly accessible brain network data prevents researchers from data-driven explorations. One of the main difficulties lies in the complicated domain-specific preprocessing steps and the exhaustive computation required to convert the data from MRI images into brain networks. We bridge this gap by collecting a large amount of MRI images from public databases and a private source, working with domain experts to make sensible design choices, and preprocessing the MRI images to produce a collection of brain network datasets. The datasets originate from 6 different sources, cover 4 brain conditions, and consist of a total of 2,702 subjects. We test our graph datasets on 12 machine learning models to provide baselines and validate the data quality on a recent graph analysis model. To lower the barrier to entry and promote the research in this interdisciplinary field, we release our brain network data and complete preprocessing details including codes at https://doi.org/10.17608/k6.auckland.21397377 and https://github.com/brainnetuoa/data_driven_network_neuroscience.
${}^{ \dagger  }$  Contribution was made exclusively to the data collection and community engagement of the Mātai dataset.
$^{\ddagger}$ Part of the data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database (adni.loni.usc.edu). As such, the investigators within the ADNI contributed to the design and implementation of ADNI and/or provided data but did not participate in the analysis or writing of this report. A complete listing of ADNI investigators can be found at: http://adni.loni.usc.edu/wp-content/uploads/how_to_apply/ADNI_Acknowledgement_List.pdf
Neuroscience unveils the principles and mechanisms underlying complex brain functions on normal subjects and subjects with neurological, psychiatric, and/or neurodevelopmental conditions. Recent years have witnessed an expansion in the size, scope and complexity of human neural data acquisition. Such an expansion joined with a rapid progress in machine learning and graph analytics has drawn a growing attention in network neuroscience – the neuroscience that comprehends the structure/function of the human brain using graphs (called brain networks) [14]. Each neuron/brain region can be modelled as a node and the interconnectivity between two neurons/brain regions as an edge. Brain networks capture rich structural and functional information that traditional manual examinations fail to capture: communications associated with spontaneous or task-evoked brain activities, dynamic patterns of neural signalling, interactions/disconnections associated with certain diseases, etc. [7].
Network neuroscience leverages on graph-based machine learning for clinically important applications. For example, a community in brain networks could represent areas of co-activation that could be weakened on brains with neurodegenerative conditions [34]. Graph classification [19] can help differentiate subjects with neurological diseases from healthy ones. Graph ordinal regression [38] can identify subjects with different stages of neurological diseases based on the severity. Nonetheless, the potential of graph-based machine learning in clinical applications is hindered by the scarcity of available brain network datasets in this important interdisciplinary field.
In this study, we focus on functional magnetic resonance imaging (fMRI), which monitors changes in the blood flow, i.e., blood-oxygen-level-dependent (BOLD) signals to capture functional activities [24]. The conversion of fMRI scans to brain networks has two stages, preprocessing and parcellation, both requiring intensive domain inputs. Specifically, to ensure the image quality for subsequent tasks, the preprocessing of raw MRI images needs proper quality control over a number of steps, including motion correction, realigning, field unwarping, normalization, bias field correction, and brain extraction. Different preprocessing choices lead to a large variation in the output images. Parcellation translates the preprocessed MRI images to regions-of-interest (ROIs) as nodes and the co-activation between ROIs as weighted edges. Each generated brain network contains i) a weighted adjacency matrix that characterizes the connectivity between ROIs and ii) a feature matrix that captures the attributes of ROIs in terms of the aggregated BOLD signals. Choosing a different brain atlas/scheme for parcellation leads to a different brain network. Existing study typically selects a single parcellation scheme to generate a fixed set of nodes for all subjects in the study, while the effect of different choices of group-wise data-driven parcellation schemes remains largely unexplored.
The above conversion poses a high barrier to entry the research on brain networks, limiting the development of data-driven network neuroscience. Specifically, domain knowledge in neuroimage preprocessing is required to select and guide the proper pipeline and tools used, image processing and graph extraction lead to high computational costs, and large-scale imaging studies require a complex setup with multi-site, multi-scanner, and multiple acquisition protocols. This paper aims to bridge the gap by making more brain network data available to the public. We believe that releasing this brain network collection will promote research in the interdisciplinary field of network neuroscience, machine learning, and graph analytics, and advance graph-based and clinical studies such as the detection of neurodegenerative conditions. Our main contributions are highlighted below.
1. We release a large resting-state functional brain network collection to the public. The collection was originated from 6 raw rs-fMRI image sources with 5 well recognized ones in neuroscience and one new data source, covering 3 neurodegenerative and one brain injury conditions, i.e., Autism, Alzheimer, Parkinson, and mTBI. The collection consists of ABIDE (N=1025), ADNI (N=1327), PPMI (N=209), Mātai (N=60) and two other sources totalling to 2,702 subjects.
2. We test the datasets on one recent graph analysis model [34], 6 conventional machine learning (ML) models, as well as 6 representative graph ML models. The experimental results demonstrate that the quality of our datasets is not compromised by the conversion process and can serve as a domain benchmark for subsequent research.
Data Collection. Preprocessed Connectomes Project (PCP) [17] is an initiative to preprocess part of the raw MRI images in the International Neuroimaging Data-sharing Initiative (INDI) database
Table 1: Statistics of our datasets and the generated resting-state functional brain networks. Each subject has a graph (brain connectivity network) generated under each Parcellation Method (PM) of AAL, HarvardOxford (HO), Schaefer, k-means and Ward Clustering (see Table 2 for details). The number of nodes in a graph generated under a PM is the number of ROIs of the PM. We call an edge non-zero if its weight has absolute value  $>10^{-2}$ . The number of non-zero edges varies under different parcellations. The number of node features is the length of the BOLD signals.
Dataset	Condition	# of Graphs (# of Subjects)	# of Classes	Avg # AAL (116)	Non-Zero HO (48)	Edges under PM (# of Nodes)	Avg # of Node Features
Schaefer (100)	k-means (100)	Ward (100)
ABIDE	Autism	1025	2	6402	1112	4811	4698	4729	201
ADNI	Alzheimer	1327	6	6447	1112	4824	4734	4715	344
PPMI	Parkinson	209	4	6512	1122	4866	4795	4684	198
Mātai	mTBI	60	2	6433	1112	4832	4750	4731	198
TaoWu	Parkinson	40	2	6481	1116	4846	4724	4766	239
Neurocon	Parkinson	41	2	6455	1114	4830	4677	4779	137
and make the preprocessed neuroimages publicly available. Within PCP, one relevant dataset that has gone through functional preprocessing pipelines is the ABIDE dataset on Autism. Note that the pipeline used for preprocessing ABIDE was proposed in 2012 [9] while the state-of-the-art functional preprocessing pipeline is fMRIPrep [20] which introduces less uncontrolled spatial smoothness [20] compared to other preprocessing tools. Some work converts preprocessed neuroimages to brain network datasets which, however, are predominately binarized, i.e., the edge weight can take only two values 0 or 1. For example, in [34], the ABIDE dataset was converted to binarized brain networks. In [42, 1], around 80 samples for Attention Deficit Hyperactivity Disorder (ADHD) were released in three datasets KKI, OHSU, and Peking_1 in the form of binarized brain networks. There still lacks a large collection of quality brain network datasets available to the public. Our collection uses fMRIPrep on data from 6 sources (see Section 3 for details) on 4 clinical conditions of interest under different parcellation schemes and wraps the whole conversion process from raw MRI images to brain networks in a holistic manner. We release the codes of the entire processing pipeline and keep future efforts in refining the pipeline and/or enriching the collection open.
Benchmark. Network neuroscience that uses machine learning and graph analytics has attracted an increasing attention [23]. Along this line of research, most recent studies [10, 46, 37, 30, 34] apply machine learning models to perform connectivity analysis on the ABIDE dataset with the generated brain networks. In our benchmark, we tested our datasets on one recent graph analysis model [34], 6 conventional ML models and 6 representative graph ML models: the quality of our datasets is not compromised by the conversion process and can serve as baselines for this line of research.
3 Dataset Sources: Raw Neuroimages
This section describes our selected sources of raw neuroimages and their selection and acquisition settings. Table 1 summarizes our datasets and the generated brain networks. This paper focuses on resting-state functional connectivity of the brain using rs-fMRI, a potent method for detecting neurodegenerative conditions [24], leaving alternative modalities for future exploration. To achieve quality image preprocessing, each rs-fMRI image needs a structural T1-weighted (T1w) image that was acquired from the same subject in the same scan session. T1w image provides structural details which allow brain mask extraction, image alignment and BOLD time series normalization. Other pipelines such as DPARSF [54] have the same requirement.
Autism Brain Imaging Data Exchange (ABIDE) The ABIDE initiative aggregated functional brain imaging data collected from laboratories around the world to support the research on Autism Spectrum Disorder (ASD). ASD has stereotyped behaviors such as irritability, hyperactivity, depression, and anxiety. Subjects are classified into typical controls and those suffering from ASD.
Alzheimer's Disease Neuroimaging Initiative (ADNI) ADNI [4, 3, 52] is a longitudinal multisite study for the early detection and tracking of Alzheimer's Disease (AD). AD is a progressive neurologic disorder that causes the brain to shrink and brain cells to die and is the most common cause of dementia that affects a person's ability to function independently. ADNI data used in this study were obtained from the ADNI database (adni.loni.usc.edu) which was launched in 2003 as a public-private partnership, led by Principal Investigator Michael W. Weiner, MD. The primary goal of ADNI has
been to test whether serial MRI, positron emission tomography (PET), other biological markers, and clinical and neuropsychological assessment can be combined to measure the progression of mild cognitive impairment (MCI) and early Alzheimer's disease (AD). Subjects are from 6 different stages of AD: cognitive normal (CN), significant memory concern (SMC), mild cognitive impairment (MCI), early MCI (EMCI), late MCI (LMCI), and Alzheimer's disease (AD) [8].
Parkinson's Progression Markers Initiative (PPMI) PPMI aims to identify biological markers of Parkinson's risk, onset and progression. Parkinson's disease is a progressive nervous system disorder that mainly affects movement [41]. The study is ongoing and contains multimodal, multi-site MRI images similar to ADNI. The PPMI dataset contains subjects from 4 classes: normal control, scans without evidence of dopaminergic deficit (SWEDD), prodromal, and Parkinson's disease (PD).
Mātai Mātai is a longitudinal single site, single scanner study designed for detecting subtle changes in the brain due to a season of playing contact sports. This new dataset consists of the brain networks preprocessed from the data collected from Gisborne-Tairāwhiti area, New Zealand, with 35 contact sport players imaged at pre-season (N=35) and post-season (N=25) with subtle brain changes confirmed using diffusion imaging study due to playing contact sports. Note that this dataset does not release raw data nor metadata and has been preprocessed so that no ancestral history is able to be extracted.
TaoWu and Neurocon TaoWu and Neurocon datasets are released by ICI [6] and are two of the earliest image datasets released for Parkinson's. The datasets consist of age-matched subjects captured using a single machine and on a single site. We include these two datasets in our collection as they could be used in studies that aim to minimize or contrast the variability introduced from different image acquisition settings. It includes normal controls and patients labelled with PD. Neurocon and Taowu label patients with a diagnosis of PD who have been under treatment (most under levodopa) as PD. PPMI's PD definition involves patients with a diagnosis of PD for two years or less and who are not taking PD medications. Under these definitions, Neurocon and Taowu are more similar when compared to PPMI. It is worth noting that while these two have similar scanning protocols, they used different scanners (with Taowu being higher in resolution). In [6], the authors compared these scans and argued that they can be treated similarly. We believe there could be more such explorations with the data available, which is one of the main reasons why we want to release this collection.
4 From MRI Images to Brain Networks: Design Choices
We adopt the common functional processing pipeline in network neuroscience [21] to convert raw MRI images (rs-fMRI and T1w) into brain networks, as depicted in Figure 1. Under this general pipeline, a number of choices need to be made in data selection, data formats, neuro-image preprocessing tools, parcellation schemes, network edge formation, etc. We worked closely with our domain experts to ensure that our design choices are sensible and state-of-the-art. Specifically, Step A collects raw MRI images based on the selection criteria (Section 4.1). Step B converts the images into BIDS format (Section 4.2). Step C preprocesses the images using fMRIPrep (Section 4.3). Step D parcellates the preprocessed data into different ROIs (Section 4.4). Steps E-F extract the connectivity matrix and the feature matrix and form the brain network (Section 4.5). Domain experts have guided our preprocessing by providing advice/feedback on i) the selection of images from the data sources, ii) the choices of using the state-of-the-art fMRIPrep pipeline, iii) parcellation strategies, iv) quality check of fMRIPrep outputs, and v) the selection of confounds.
4.1 Data Collection and Selection Criteria
Our data sources involve multi-site, multi-scanner images. Thus, the inclusion criteria of our data collection is based on the availability, validity, and quality of the MRI images of our required types.
For ABIDE, TaoWu, and Neurocon, the original source images were already collated with 1 raw rs-fMRI image paired with 1 T1w image for each subject. We included all subjects provided by these data sources except for those that had quality issues. A subject has quality issues if it has an incomplete image (i.e., not containing the full brain) and/or damaged data (i.e., with error reported by any subsequent preprocessing steps). Available metadata of each subject from the source is included in our collection, such as age and gender. The gender distributions of our datasets closely match
A. Data Collection and Selection
B. BIDS Format Conversion
C. fMRIPrep Preprocessing
D. Parcellation and ROI Definition
E. Connectivity Matrix Extraction
F. Graphical Brain Network
Figure 1: Brain Network Construction Pipeline
those of the original data sources. The only exception is ADNI: our released data contains  $0.6\%$  more females than the original ADNI data.
For longitudinal study data sources ADNI, PPMI, and Mātai, it is possible that some subjects have had multiple scans over the course of several years. Depending on the scanning protocol for the study, different types of images were taken at various times (e.g., baseline, 1 year follow-up, 2 year follow-up, etc.). The baseline study (the first scan) is usually the most comprehensive one that would cover a wide range of modalities. Thus, as suggested by our domain experts, we consider the baseline study which is likely to be the set of scans with both an rs-fMRI image and a T1w image taken on the same date. In the case that multiple rs-fMRI and T1w scans exist, we selected the first available one.
ABIDE, ADNI, and PPMI are multi-site neuroimaging sources. We chose not to apply data harmonization techniques to these datasets in our preprocessing due to the following reasons. (1) For fMRI connectivity, there is no well-grounded harmonization method for benchmark. (2) Even though no fMRI connectivity harmonization was applied, [43] has shown that fMRI connectivity still has some good repeatability. (3) The site and scanner information are available and researchers have the flexibility of performing harmonization based on our data collection for further analysis.
For validity, we examined each image manually to make sure the images conformed to their labels in the database and did not have obvious data format issues for preprocessing. We had to check this manually because the image databases often had inconsistent labeling.
4.2 BIDS Conversion
Raw MRI images are typically in the Digital Imaging and Communications in Medicine (DICOM) format. DICOM images are then converted to the Neuroimaging Informatics Technology Initiative (NIfTI) format using dcm2niix [36], which includes a JSON file that details various imaging parameters such as scanner model, magnetic field strength, flip angle, slice timing, echo time and repetition time. The NIfTI and JSON files are then organized into a folder hierarchy with a precise naming convention known as Brain Imaging Data Structure (BIDS) [27]. After conversion, BIDS formatted data can be applied to preprocessing pipelines in a highly reproducible and transparent manner.
4.3 fMRIPrep Preprocessing
BIDS compliant datasets are then preprocessed using fMRIPrep [20], a state-of-the-art tool for preprocessing fMRI. fMRIPrep performs basic processing steps (coregistration, normalization, noise component extraction, skull stripping, etc.) and uses a combination of tools from well-known software packages, including FMRIB Software Library (FSL) [31], Analysis of Functional NeuroImages (AFNI) [16], Advanced Normalization Tools (ANT) [5] and FreeSurfer [22]. This pipeline was designed to use the best software implementation for each step of preprocessing. fMRIPrep can be
Table 2: Parcellation Methods
Name	# ROIs	Generation Method
AAL [49]	116	Delineated with respect to anatomical landmarks by following the sulci course in the brain.
HarvardOxford (HO) [40]	48	Created by subdividing neocortex by topographic criteria into 48 parcellation units corresponding to the principal cerebral gyri.
Schaefer [47]	100	Using gradient-weighted Markov Random Fields (gwMRF) to automatically group similar fMRI regions.
k-means Clustering [39, 2]	100	Top-down clustering algorithm that partitions voxels into non-overlapping predefined number of regions.
Ward Clustering [51, 2]	100	Bottom-up hierarchical clustering algorithm that agglomerates together voxels progressively into regions.
Figure 2: Extracted ROIs and Glass Brain Connectome (right bottom corner)
installed via container technologies such as docker or singularity. fMRIPrep automates the pipeline by using the scanner outputs from the images (e.g., slice timing correction parameters are taken automatically from the scanner outputs of the original image data). Therefore, the user does not need to specify these parameters manually. fMRIPrep can be configured to include or exclude specific workflow steps, such as ignoring slice timing. We followed the default automatic fMRIPrep workflow with surface reconstruction enabled. No further modifications to the settings were made as the default workflow met our expectations for a functional preprocessing pipeline. As part of our validation process of fMRIPrep, we have passed the preprocessed images over to our MRI imaging experts: they have carefully examined the outputs to ensure the data quality. fMRIPrep outputs conform to the BIDS Derivatives specification for compatibility and include the preprocessed BOLD images and the confounds file, which records fluctuations during MRI data acquisition, also known as nuisance regressors. fMRIPrep is computationally expensive. In our case, one run on one subject of fMRIPrep using 8 threads on an Intel(R) Core(TM) i9-10940X CPU@3.30GHz took on average 4-5 hours.
4.4 Parcellation Strategies
Parcellation defines regions in the brain known as region-of-interest (ROI). The parcellation step takes in the two outputs from fMRIprep for each subject to generate the parcellations: the preprocessed BOLD image and the confounds file. We adopted the standard 9 parameters (9P) confounds setting widely used in functional connectivity studies [25, 26] for denoising: white matter, cerebrospinal fluid, global signal, and the 6 rigid-body motion parameters for rotation and translations at  $x$ ,  $y$  and  $z$  axes. To parcellate the brain, we selected a list (Table 2) of Parcellation Methods (PMs) that cover both atlas-based and clustering-based PMs frequently used in the domain. Brain regions partitioned through predefined regions are atlas-based PMs. We selected the most commonly used atlas-based PMs, AAL, HarvardOxford (HO), and Schaefer, that are generated using different strategies and