dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgements The authors express our deepest gratitude to the participating patients and their families. We also acknowledge our user/next-of-kin participants for their help and constructive criticism along the way. This project was part of the strategic program "Personalized medicine-biomarkers and clinical studies", which was supported by the  Western Norway Regional Health Authority. The project was also supported by grants from the  Folke Hermansen Foundation  and the  Norwegian Cancer Society (National Group of Expertise on Pancreatic Cancer).
Prediction of chronological age from neuroimaging in the healthy population is an important issue because the deviations from normal brain age may highlight abnormal trajectories towards brain disorders. As a first step, ML models have emerged to predict chronological age from brain MRI, as a proxy measure of biological age. However, there is currently no consensus w. r. t which Machine Learning (ML) model is best suited for this task, largely because of a lack of public benchmark. Furthermore, new large emerging population neuroimaging datasets are often biased by the acquisition center images are coming from. This bias heavily deteriorates models generalization capacities, especially for Deep Learning (DL) algorithms that are known to overfit rapidly on the simplest features (known as simplicity bias).
Here we propose a new public benchmarking resource, namely Open Big Healthy Brains (OpenBHB), along with a challenge for both brain age prediction and site-effect removal through a representation learning framework. OpenBHB is large-scale, gathering > 5K 3D T1 brain MRI from Healthy Controls (HC) and highly multi-sites, aggregating > 60 centers worldwide and 10 studies. OpenBHB is expected to grow both in terms of available modalities and number of subjects. All OpenBHB datasets are uniformly preprocessed, including quality check, with container technologies that consist in: 3D Voxel-Based Morphometry maps (VBM from CAT12), quasi-raw (simple linear alignment of images), and Surface-Based Morphometry indices (SBM, from FreeSurfer). The OpenBHB challenge is permanent and we provide all tools, materials and tutorials for participants to easily submit and benchmark their model against each other on a public leaderboard.
In this context, harmonization methods  Fortin et al. (2018)  ;  Garcia-Dias et al. (2020)  ;  Kia et al. (2020)  have emerged to remove this undesired variability from the data. However, such harmonization models estimate their parameters on the entire dataset, or at least, on a great portion of it containing all sites. It is also a limitation in the context of personalized medicine, where MRI data coming from new hospitals would mean to re-train the whole model before making a new prediction. Besides, these methods are also sensitive to the number of samples per site as some statistics (mean or variance) are estimated for each site separately. Other recent approaches  Bashyam et al. (2020a)  ;  Dewey et al. (2020)  ;  Liu et al. (2021)  ;  Liu and Yap (2021)  ;  Robinson et al. (2020)  are integrating DL to perform imageto-image translation (e.
All these approaches use different validation procedure and they are hardly comparable to one another (as they generally do not even use the same datasets and modalities). As a result, we propose the OpenBHB Challenge on brain age prediction with site-effect removal. This challenge is based on the largescale (  > 5000 ) multi-site brain MRI dataset OpenBHB that contains both minimally preprocessed data along with VBM and SBM measures derived from raw T1w MRI. All images in OpenBHB have passed a semiautomatic visual quality check, and the data are publicly available on the online IEEE Dataport platform. The challenge consists in learning a representation of the data such that i) brain age variability is preserved and ii) site-related information is removed. The submitted models should output a vector representing input data such that brain age can be easily predicted (i.
We plan to extend OpenBHB with additional subjects, longitudinal data and other modalities (e. g. , resting-state functional MRI and diffusion MRI) that bring complementary structural and functional information to the current T1w images. In summary, in this paper our main contributions are: • OpenBHB, a new large-scale (  > 5000 ) brain MRI dataset publicly available that includes: • preprocessed quasi-raw, VBM and SBM T1w data; • a visual quality check; • a training, validation and test splits used for the OpenBHB challenge; • a new challenge for brain age prediction with site-effect removal. • a leader-board for the comparison of submitted models with a new metric. • an online platform to submit the trained models.
Public datasets aggregated in OpenBHB OpenBHB aggregates 10 publicly available datasets, namely IXI 1  , ABIDE 1 2  , ABIDE 2 3  , CoRR  Zuo et al. (2014)  ,  GSP Buckner et al. (2014)  ,  LOCALIZER Orfanos et al. (2017)  , MPI-Leipzig  Babayan et al. (2019)  ,  NAR Nastase et al. (2021)  ,  NPC Sunavsky and Poppenk (2020) , and RBP Follmer et al. (2018)  ;  Li and Clariana (2019). Currently, OpenBHB is focused only on Healthy Controls (HC) since the main challenge consists in modeling the (normal) brain development by building a robust brain age predictor. As a result, we only included HC from ABIDE 1 and 2, and we left out the subjects with Autism Spectrum Disorders in the current release. OpenBHB contains  = 5330 3D T1 brain MRI scans from HC acquired on 71 different acquisition sites with eventually multiple acquisition protocols per site (see Table  1  ).
The overall age and sex distributions for OpenBHB are plotted Fig. 2. It should be noticed that sex distribution is globally well balanced for all age bins. Age distribution contains 2 main modes centered around 10 years old (during synaptic pruning) and 25 years old with a long tail above 40 until 88 years (and fewer samples in this range). Additionally, we performed appearance analysis from VBM data, as it preserves cortical and sub-cortical information from raw images (as opposed to SBM) and ROI measures can be derived easily to reduce data dimensionnality. Specifically, we represented the t-SNE visualization of ROI extracted from VBM data in Fig. 3  per study. This plot clearly suggests that age and site effect are driving the representation, thus justifying the objective of the OpenBHB challenge.
Datasets with both a large number of sites and a large age range cover wider regions in t-SNE space than others (e. g CoRR covers almost all regions while GSP and IXI cover only middle and upper regions; even if they are lifespan, they only include 6 sites together vs 18 sites for CoRR). This is even more obvious with MPI-Leipzig that covers mostly small left and upper regions while it is also lifespan.
Preprocessing and derived anatomical features All data are preprocessed uniformly with container technologies comprising quasi-raw, CAT12 VBM, and FreeSurfer (see Fig. 2. 2 ), which allows us to control the different software versions over time. The project hosting the codes is freely accessible at https://brainprep. readthedocs. io. We conducted a semi-automatic quality control (QC) guided with quality metrics leading to a selection of images that meet the quality criteria for all three pre-processing pipelines (see Fig. 5  and the detailed QC per pre-processing below).
(1999)  (measuring the depth/height of a vertex above the average surface) are also accessible on the high-resolution seven order icosahedron. To allow inter-hemispheric cortical surface-based analysis, we further transform the right hemisphere features into the left one, using the symmetric "  _  " Freesurfer template and the "xhemi " routines  Greve et al. (2013). The final vertex-wise cortical features comprise 163,842 nodes per hemisphere. ). Private data (used to score the models submitted to the OpenBHB challenge) are composed of 2 subsets: an internal test (stratified on age, sex and site from the OpenBHB dataset) and an external test (independent from OpenBHB and with acquisition centers distinct from the public data). Importantly, the validation set is built in a similar fashion from public OpenBHB data to allow participants to derive all the challenge's metrics.
7  and Table  2  ). External test. In the context of the OpenBHB Challenge, we also built an external test, fully independent from OpenBHB that also preserves age and sex statistics (see Fig. 7  and Table  2  ) but that contains images acquired on independent sites (i. e. , with no site overlap with OpenBHB). These sites are kept private to avoid any bias or data leakage during the challenge. The external test set is used to evaluate the generalization capacity of ML models on never-seen sites for brain age prediction (see Section 3 for more details). As for the OpenBHB challenge, MRI scans from this external test have been acquired on both 1. 5T and 3T in geographically distinct locations, across 8 acquisition centers. Region coverage analysis between training, validation and tests Previously we showed that training, validation and the 2 test sets share the same age and sex distributions (along with the same site distribution between training and internal test).
Data organization and accessibility Data sharing. A data sharing platform distributes the prepared OpenBHB dataset. All up-to-date information are centralized at this location https://baobablab. github. io/bhb. Data organization. Currently, we are only sharing the training and validation splits for the OpenBHB challenge, in order to avoid data leakage. All modalities (quasi-raw, VBM, SBM) are stored in NumPy (. npy) format to allow easy cross-platform implementation. A resource folder contains information about the actual geometry of all manipulated data (i. e VBM and quasi-raw MNI templates as well as ROI labels for the Desikan  Desikan et al. (2006)  and  Destrieux Fischl et al. (2004)  atlases used by FreeSurfer and for the Neuromorphometric atlas used by CAT12). The data are stored in N-dimensional arrays, where the first two dimensions are sample size and number of modalities (or channels, equal to 1 here), followed by the data dimension.
As a result, we discretized all these pairs to create the confounding variable "siteXacq " that is defined in this file. All participants in the OpenBHB Challenge should use this variable to remove "site effect ". Thus, when referring to "site removal ", we implicitly englobe both the scanner and acquisition protocol used to generate the final MRI image. We propose a challenge to compare the models capacity to encode a relevant representation of the data that preserves the biological variability associated with age while removing the site-specific information. All required information, data loading, models submission, etc. is described on the web page: https://baobablab. github. io/ bhb/challenges/age_prediction_with_site_removal.
Background Predicting phenotype (such as age and sex) from brain imaging data is a key challenge to answer several exciting questions (e. g. , biomarker discoveries for psychiatric disorder or neurocognitive impairment with brain age, personalized medicine with normative modeling, etc. ). Recent efforts in ML/DL for neuroimaging along with the availability of largescale datasets have led to the development of models increasingly accurate, capable of predicting chronological age within 2-3 years, and sex with 99% accuracy from brain MRI. However, these models are severely biased by non-biological variability, particularly associated with acquisition sites (different acquisition protocols, manufacturers, magnetic fields, etc. ). These sources of variability inevitably limit the performance of such models and can even bias the neuroimaging community towards over-optimistic results, mainly when using small samples size.
Challenge description Consequently, we propose a representation learning challenge using the OpenBHB dataset. The aim is to learn a representation of the 3D T1 anatomical MRI pre-processed with the three pipelines described previously that full-fills 2 key properties: 1) the representation should be predictive of biological age; 2) site information should be removed from the representation. Thus, we aim to compare the capacity of the proposed models to encode a relevant representation of the data (feature extraction and dimensionality reduction) that preserves the biological variability associated with age while removing the site-specific information. The algorithms submitted must output a vector of features with  < 10 4 dimensions for each input data. Both quasi-raw, VBM, and SBM features (including ROI-based and voxel-wise features for VBM and SBM data, see Section 2.
Name-That-Site performance In order to assess the current bias in OpenBHB, we played at the game "Name-That-Site ", inspired by  Torralba and Efros (2011)  on natural images and  Wachinger et al. (2021)  on brain MRI (originally created as "Name-That-Dataset "). We train a classifier on different input data  to classify between the 64 pairs (site, acquisition setting) in the OpenBHB Challenge and we test it by following the challenge train-ing+validation/test splits defined Section 2. 3. We used a CNN for VBM and Quasi-Raw inputs and linear logistic regressions for ROI surfacebased measures and age.
Training details We considered 3D adaptation of three representative CNN architectures: AlexNet inspired from  Abrol et al. (2021)  (5 convolutional layers and 2. 5M parameters), ResNet18  He et al. (2016)  (18 convolutional layers and 33. 2M parameters) and DenseNet121  Huang et al. (2017)  (121 convolutional layers and 11. 3M parameters) for whole-brain imaging data (VBM and quasi-raw). Their implementation is available on challenge web page : https://baobablab. github. io/bhb/challenges/ age_prediction_with_site_removal. We trained them for 300 epochs with an initial learning rate  = 10 - 4  reduced by factor 0. 9 every 10 epochs. We used the standard cross-entropy loss optimized with Adam  Kingma and Ba (2015)  optimizer (  1 ,  2 ) = (0. 9 , 0. 999). As for SBM data (both ROI-based and mesh-based features), we simply optimized a logistic regression using scikit-learn  Pedregosa et al.
Results Results in Table  3  first indicate that all modalities preserve site information, especially input data in their rawest form (Quasi-Raw) with 83% BAcc. Second, age give small hints about site ( 2. 86% BAcc) indicating that participants inclusion is biased for some datasets (e. g ABIDE) but it remains negligible compared to other modalities (always > 42% BAcc). These results are in line with recent literature  Wachinger et al. (2021)  and it highlights the high non-biological heterogeneity that remains in MRI images even after non-linear registration and normalization (e. g VBM). Finally, these results suggest that Bacc is an appropriate metric to quantify the bias in models representation. As a final remark, we notice that even if BAcc using age is low ( < 3% ), it is still not equal to the theoretical random level ( 1   = 1∕64 ).
Towards transfer learning for computer-aided diagnosis This challenge is a first step towards building new algorithms for phenotype prediction robust across sites. However, as recently proven  Dufumier et al. (2021b)  , it is also possible to leverage such multicenter large-scale dataset to significantly improve classification performance on other hard computer-aided diagnosis (CAD) tasks such as Alzheimer's detection or schizophrenia diagnosis, with Transfer Learning (TL). First, pre-training algorithms to remove site effect is critical for computer-aided diagnosis since most clinical datasets with moderate size (  > 100 ) are multi-site (e. g  ADNI Jack Jr et al. (2008)   2019 ) since they are able to leverage large un-annotated dataset to improve performance on several downstream tasks with TL.
Towards multi-modal integration for new bio-markers discovery In the context of brain age prediction as a tool for CAD, sMRI data provide meaningful information about subtle anatomical modifications in cortical and sub-cortical structures (e. g atrophies or hypertrophies). On the other hand, resting-state fMRI and Diffusion Weighted Images (DWI) give hints about functional and structural brain connectivity, that can be altered for patients with psychiatric disorders (e. g autism spectrum disorder, schizophrenia or bipolar disorder Baker et al. (  2014 ) ). However, little is known about the predictive power of a multi-modal approach combining both sMRI, DWI and resting-state fMRI for brain age modelling or CAD. A future release of the OpenBHB dataset might bring some answers. Almost all datasets included in the current OpenBHB release -IXI and NAR excepted -have rfMRI data available for all participants and only a few of them (IXI, NPC, RBP) also have DWI (see Table  1  ).
Acknowledgments We are exceedingly grateful to  Alexandre Gramfort ,  Francois Caud ,  Guillaume Lemaître ,  Thomas Moreau  ( Université Paris-Saclay, Inria, CEA ) and  Roman Yurchak  ( Symerio SME, Paris area ) to provide access and active support to the challenge platform: https://ramp. studio. This work is supported by  Big2Small , a teaching and research chair in AI of  French National Research Agency  and  R-LiNK  and  Horizon 2020   EU grant  ( H2020-SC1-2017 ,  754907 ).
We address the problem of scalable content-based search in large collections of music documents. Music content is highly complex and versatile and presents multiple facets that can be considered independently or in combination. Moreover, music documents can be digitally encoded in many ways. We propose a general framework for building a scalable search engine, based on (i) a music description language that represents music content independently from a specific encoding, (ii) an extendible list of feature-extraction functions, and (iii) indexing, searching, and ranking procedures designed to be integrated into the standard architecture of a text-oriented search engine. As a proof of concept, we also detail an actual implementation of the framework for searching in large collections of XML-encoded music scores, based on the popular ElasticSearch system.
The MIDI format encodes information related to the production of sound by a MIDI device  [5]. Music notation is the most elaborate way of describing music at this symbolic level  [6]. It has been traditionally used for engraving musical scores  [7] , but, since the advent of digital encodings such as the **kern format  [8]  or XML-based variants (MusicXML  [9] , its next-generation MNX  [10] , or MEI  [11, 12] ), music notation can also be seen as a support for music information processing. **kern is for instance explicitly designed as a digital encoding of scores that feeds the music analysis modules of the Humdrum toolkit  [13]. Large collections of digitally codified music scores are now available, either as results of long-running academic efforts  [14, 15] , or as a side-effect of the generalized production of music scores with editing software that encode their documents in one of the above-mentioned formats (e.
g. , MuseScore  [16] ). Such collections are examples of datasets where the music content is described in a structured and well-organized way, apt at supporting sophisticated computer-based operations. To the best of our knowledge, however, most existing search tools for large music collections highly rely on metadata. This is the case for search engines incorporated in music streaming services like Deezer or Spotify  [17] , and for renowned digital music databases like Discogs  [18]  and AllMusic  [19]. Musixmatch  [20]  allows lyrics search with access to libraries of major music streaming platforms. Shazam  [21]  allows searching audio recordings by indexing the fingerprints of files, and its result are therefore highly dependent on the specificities of audio music encoding. SoundHound  [22]  offers a Query by Humming  [23]  functionality that relies on the measurement of melodic similarity, thus it cannot search other aspects of music.
The few approaches that address search operations applied to symbolic representation propose an exhaustive scan of the digital encoding, such as, for instance, the Humdrum tools based on Unix file inspections  [13]  or the search methods incorporated in the Music21 toolkit  [24]. They do not scale to very large music datasets. We expose in the present paper the design of a general framework for scalable contentbased search in large digital collections of music documents. Here, scalable means a sublinear search complexity, delivering very fast response time even in the presence of very large collections; search operations are content-based because they rely on a structured representation of music inspired by music notation principles, and can thus refer to specific aspects of a music document (e.
Related Work Our approach relies on an abstract music content model. It consists of a tree-based decomposition of a music score that reflects its temporal organization. This draws heavily from  [25] [26] [27] , which introduced into the music information retrieval literature some ideas and tools from the fields of databases systems and computer linguistics (e. g. , hierarchical decomposition of musical content and context-free grammars). Recently, the authors of  [28]  used a similar graph-based representation to study music similarity. The process that extracts instances of our model from digital music documents depends on their specific representation. Automatic Music Transcription (AMT) applies to audio files (e. g. , either pulse-code modulation representation, or (un)quantized-MIDI) and produces symbolic data (generally quantized MIDI).
A Short Discussion So far, we presented a set of features that all relate to the monodic aspect of music content. Some are mathematically founded (chromatic, rhythm), others are applicationdependent (diatonic). They illustrate the design of our search framework as a producer of features derived from a normalized, high-level music content description. They all result in a linear representation, akin to being assimilated to textual data in a standard search engine. Another design choice is to simplify the feature representation so that it favors recall over precision. Since a feature captures only one aspect of the content (either rhythmic, or melodic-based), two descriptions that are close with respect to this aspect, but highly different with respect to another, might match in spite of important differences.
Implementation In this section, we detail an implementation of our proposed framework for symbolic music collections, i. e. , music in a notation-based format such as MIDI, XML, and MEI. Since the core musical elements such as structure, melody, and rhythm are represented in symbolic music, it is straightforward to develop an extraction of music content descriptors. We offer a publicly available Docker image at https://hub. docker. com/repository/docker/ traversn/scoresim (accessed on 17 January 2022), for the community to experience the proposed search engine. The code is in open access on Github (components implementation: https: //github. com/cedric-cnam/scoresim (accessed on 17 January 2022)) under the GNU General Public License v3. 0. In the remainder of the section, we first present the global architecture, before delving into some specific components: descriptor extraction and feature production, integration into a standard text-based information retrieval system (centered around ElasticSearch) with several search modes available, customized highlighting, and ranking procedures.