dataesr/scientific-paragraphs-categorization huggingface dataset:
MRi-Share The MRi-Share database is a subcomponent of a larger, prospective cohort study on French university students' health, called iShare (internet-based Student Health Research enterprise, www. i-share. fr). The detailed study and MRI acquisition protocol have been described in  Tsuchida et al. , (2021). For the training and evaluation of the SHIVA-WMH detector, we used the MRI data and manually-traced WMH from the same sub-sample of 50 subjects described in  Boutinaud et al. , (2021) , drawn from the total sample of 1,867 MRi-Share subjects (mean age 22. 1 years, range 18 to 35, 72% female). We also used the T1w and FLAIR data from additional 360 subjects, selected from the 1,817 subjects without any manual tracing of WMH, to enhance the training dataset, as described in the section SHIVA-WMH detector: training and enhancement.
MICCAI 2017 WMH challenge dataset (MWC) We supplemented the training and evaluation datasets with the MRI and manual tracing of WMH from 60 subjects provided as training datasets in the MICCAI 2017 WMH segmentation challenge (http://wmh. isi. uu. nl/;  Kuijf et al. , 2019). The set of 3D T1w and 2D or 3D multi-slice FLAIR images were acquired at three different institutes: the University Medical Center (UMC) Utrecht, Vrije Universiteit University Medical Centre (VU) Amsterdam, and the National University Health System (NUHS) in Singapore (20 subjects per site; Table  1 ). The 3D FLAIR images had been resampled into the transversal direction with a slice thickness of 3 mm by the MWC organizers to make them similar to other 2D FLAIR images in the dataset and to save time for manual annotation.
MRi-Share Subjects for manual tracing of WMH were selected based on the visual inspection of a neuroradiologist (BM) who reviewed the raw T1w and FLAIR images of the entire dataset, to cover varying degrees of both WMH and visible PVS, from no detectable WMH or PVS to many visible WMH (>10) and/or PVS. A trained investigator (AT) then performed voxelwise manual segmentation of each WMH on the raw FLAIR images using Medical Image Processing, Analysis and Visualization (MIPAV) software (v 7. 4. 0). Specifically, WMH was segmented on each axial slice of the FLAIR image, viewed along with coronal and sagittal views using the 3D view setting of the MIPAV to check the 3D shape and extent of hyperintense signals. Any punctate region of increased intensity within the white matter, as well as hyperintense rims around the ventricles that were thicker than 2 mm, were segmented.
This included punctate hyperintense regions sometimes found around the PVS visible on T1w images. For each hyperintense region found, the 'paint grow' tool of the MIPAV was applied to automatically paint every neighboring voxels that have a higher intensity than and within a 3 mm distance from the selected voxel. Following the initial segmentation of the first ten subjects, they were reviewed and modified by a second expert (LL). Any discordance between the two raters was then reviewed together to reach a consensus. Subsequently, the remaining 40 MRI datasets were manually segmented by the first expert only.
MWC We used the publicly available manual tracing of WMH for the 60 MWC training dataset. The details of the procedure are described in  Kuijf et al. , (2019). Briefly, manual tracing of WMH, as well as any other pathologies, was performed by two expert raters by consensus, in which the tracing performed by the first rater was reviewed by the second rater and corrected by the first rater.
UKB A small sample of 11 subjects was selected to cover a range of estimated WMH load (0. 7 to 16mL, representing values in the first and last decile of the entire dataset) from a pool of 13554 subjects (available at the time of the selection) with 'usable' quality T1w and FLAIR images as well as the WMH load estimated by the Brain Intensity Abnormality Classification Algorithm tool (BIANCA:  Griffanti et al. , 2016). The same rater who performed the manual tracing of WMH for the MRi-Share database (AT) manually traced WMH on the raw FLAIR images using the 3D Slicer tool  (version 4. 11. 20210226 : https://www. slicer. org), using the same criteria applied during the segmentation of WMH for MRi-Share. With the 3D Slicer tool, the 'Threshold' effect in the Segment Editor module was used to specify an intensity range that preliminarily isolated visible hyperintensities from the surrounding white matter in any given location.
, 2015). The resulting scalar images of DTI and NODDI metrics were coregistered to the native T1w image space using antsRegistrationSyNQuick script of Advanced Normalization Tools (ANTs, version 2. 1: http://stnava. github. io/ANTs/) package. For the present work, we focused on the neurite density index (NDI) from NODDI, fractional anisotropy (FA), and mean diffusivity (MD) from DTI metrics, which all have been shown to be altered in WMH in MS  (Alotaibi et al. , 2021)  or in older subjects  (Muñoz Maniega et al. , 2015; Riphagen et al. , 2018). To compare NDI, FA, and MD values inside WMH and NAWM in the 50 MRi-Share subjects with the manual tracing of WMH, manually traced WMH masks in the native FLAIR space were coregistered linearly to the T1w image by applying the transformation matrix generated by the coregistration of FLAIR to T1w with flirt tool from the FSL  (Jenkinson et al.
Preprocessing In order to prepare the input image arrays for the SHIVA-WMH detector, the following preprocessing steps were performed on the T1w, FLAIR, and manually-traced WMH masks: 1. Reorientation to match either LAS or RAS (left or right/anterior/superior) orientation using fslreorient2std tool from the FSL. 2. For the MWC dataset, images were resampled to 1mm isotropic using flirt from the FSL  (Jenkinson et al. , 2002) , with -applyisoxfm and -noresampleblur options. 3. Linear coregistration of FLAIR to T1w image was performed with flirt  (Jenkinson et al. , 2002)  for MRi-Share and UKB datasets, and the generated transformation matrices were applied to the WMH masks to bring them to the reference T1w images. This step was skipped for the MWC dataset, which had already been aligned by the organizers.
8. • The dropout rate applied at each stage (after the max pooling for encoding or after the last convolution for decoding blocks) was increased from 0. 1 to 0. 5, except in the first encoding block, which had a reduced dropout rate of 0. 05 to increase the rate of retention, reflecting the general recommendation for optimal dropout rate across a wide range of networks and tasks  (Srivastava et al. , 2014). • Convolution blocks now use a Swish activation function (equation (  1 )) rather than the rectified linear unit (ReLU;  Glorot and Bengio, 2010)  used previously, since it has shown an advantage over ReLU on deeper models across a number of challenging datasets  (Ramachandran et al. , 2017). equation (  1 ) 𝑓(𝑥) = 𝑥 • 𝑠𝑖𝑔𝑚𝑜𝑖𝑑(𝑥) Figure  1  provides a schematic overview of the resulting architecture of the SHIVA-WMH model.
The modified model has slightly less trainable parameters (40 million rather than 44 million in the  Boutinaud et al. study)  and also extracts a higher number of features at higher resolution upper stages and less at deeper levels, which we found to be advantageous for the detection of small lesions like PVS and small WMH. In addition to the network architecture modifications, we performed the following data augmentations to the training dataset to increase the model robustness: (i) flipping on the midsagittal plane, (ii) voxel translations (up to plus or minus five voxels in X, Y, and X axes), (iii) non-linear voxel intensity value transformation using a Bézier curve, in the similar fashion as in  (Zhou et al. , 2021) , with two endpoints set to [0, 0] and [1, 1] and two control points within this range generated randomly.
The probability of each type of augmentation for an image at a training epoch was set to 0. 5, 0. 9, and 0. 9, respectively. In particular, the non-linear voxel intensity value transformation was found to significantly improve the generalizability of the detector when predicting lesions in unseen datasets (unpublished observation from the PVS detector). We implemented the network in Python 3. 7, using Tensorflow 2. 7 with Keras backend, scikit-learn (1. 0. 1), and scikit-image (0. 18. 3). The network was trained on a computer (Ubuntu 22. 04) with a Xeon ES2640, 40 cores, 256 Gb RAM, and a Tesla V100 GPU with 32 Gb RAM; inferences can be done on any GPU compatible with the Tensorflow version with at least 8Gb RAM. We used a 5-fold cross-validation scheme to train the network, stratified on WMH voxel load and cohort.
To overcome the problem of imbalances, we took the advantage of the large pool of still unannotated MRi-Share dataset to perform semi-supervised learning approach using the predictions of "MRi-Share-specific" model that could segment small WMH with relatively high accuracy (see 4. 2. 2 for a review of such approach:  Tajbakhsh et al. , 2020). We first obtained the predictions of WMH in 100 randomly selected MRi-Share subjects out of 1,817 subjects without the manual tracing of WMH. We filtered out those with very low or high predicted load of WMH by removing subjects in the top and bottom 5 percentile of the total estimated WMH volume, and used the WMH labels in the 90 subjects to supplement the original training data from 90 subjects ("enh90" model, for enhancement with 90 predicted data).
FLAIR-only version We initially focused on developing a model that uses both T1w and FLAIR as inputs, following an earlier observation of the superior performance of multi-modal over FLAIR-only models. However, to formally compare the impact of having only FLAIR as an input and to potentially allow quantification of WMH in datasets with only FLAIR images, we created the modification of the SHIVA-WMH detector with FLAIR only input. The training process of the FLAIR-only version and its performance comparison with the multi-modal version is described in the Supplemental Material.
Performance evaluation We evaluated the performance of the SHIVA-WMH detector and existing methods in the held-out test dataset from the three cohorts, including ten unseen data each from MRi-Share and MWC and 11 data from UKB. The UKB test data represent a dataset coming outside of cohorts used in training (Table  1 ), with the levels of WMH severity being the intermediate between those of MRi-Share and MWC.
PGS (Park et al. , 2021) The PGS tool is a 2D Unet-style model with a multi-scale highlight foreground method to augment the influence of small lesions or voxels lying in the lesion boundaries, and is currently the best-ranking method that has been submitted to the MICCAI 2017 WMH challenge. The pre-trained model submitted to the challenge is available as the docker-contained code from the challenge website (https://wmh. isi. uu. nl/results/pgs/). It has been trained on the 2D axial slices of T1w and FLAIR images from the same MWC dataset used in the present study. Because our MWC test dataset is part of the training dataset in the challenge (and therefore in the PGS model), evaluation of this tool in the MWC test dataset was not performed. Since this tool had been trained on the bias-field corrected data prepared by the challenge organizers (using SPM12), we used the bias-field corrected T1w and FLAIR images of the MRi-Share (based on SPM12, as described in  Tsuchida et al.
HyperMapper (HPM: Mojiri Forooshani et al. , 2022) The HPM tool is a 3D Unet model with Monte Carlo dropout layers incorporated in the encoding layers, and reported high accuracy on a large and diverse test dataset representing 160 subjects from seven different studies. Of particular interest from our perspective, it also reported very high accuracy on the subsample of 50 test subjects with mild WMH cases with the average of 2 mL of lesions. Unlike the PGS tool and many other recently published methods (including all other tools submitted to the MICCAI 2017 WMH challenge), the HPM did not use the MWC dataset to train the models. As such, its performance on the MWC test dataset could be evaluated against the SHIVA-WMH. The HPM tool is publicly available from https://github. com/AICONSlab/HyperMapp3r.
They indicate the clear advantage of the SHIVA-WMH detector in the MRi-Share test dataset, both at the voxel-and lesion cluster-level: Our tool shows both higher TPR and PPV than any of the three reference methods, resulting in the significantly higher VL-and CL-Dice scores (all paired t-tests p < 0. 001 for VL-Dice and p < 0. 0001 for CL-Dice; Figure  4  and Table  3 ). FLAIR-only version shows a similar pattern, albeit with slightly lower VL-and CL-Dice scores compared to the multi-modal version and weaker significance when compared against other reference methods (Supplemental Figure  2 ). In the MWC test dataset, both multi-modal and FLAIR-only versions of SHIVA-WMH have significantly higher CL-Dice (all p < 0. 001) and numerically better VL-Dice scores than the LST-LPA or HPM, suggesting that our tool is detecting small lesion clusters better than these tools.
Discussion Long dismissed as the "normal" radiological finding in the aging brain, WMH is now firmly established as the most common marker of covert cSVD  (Debette and Markus, 2010; Wardlaw et al. , 2015). In the present work, we leveraged the high-quality research scans from the MRi-Share study  (Tsuchida et al. , 2021)  to demonstrate that young adults in their twenties already exhibit signs of compromised white matter integrity as small punctate WMH. Comparisons of DWI-based metrics in NAWM and manually-traced WMH revealed decreases in both FA and NDI and an elevated MD in these small punctate WMH relative to NAWM. We further described the SHIVA-WMH detector, trained with both the MRi-Share and publicly available MWC dataset from older subjects  (Kuijf et al. , 2019) , with a specific aim to optimize WMH detection across a wider range of WMH burdens than existing methods.
We demonstrated the superior performance of our tool relative to three reference methods in accurately segmenting WMH in the test dataset composed of three databases representing young, middle-aged, and older individuals with varying degrees of WMH burden. When performance was compared using Dice score, both at the voxel-by-voxel-level (VL-Dice) and at the level of each lesion cluster (CL-Dice), to evaluate the similarity between the manually-traced and predicted WMH segmentations, our tool had the highest average scores across the test dataset, with the overall VL-Dice and CL-Dice scores of 0. 66 and 0. 71, respectively. Detailed evaluation in each cohort separately indicated that the SHIVA-WMH detector achieved significantly higher VL-and CL-Dice scores than all three reference methods in the MRi-Share test set, and significantly or nominally higher Dice scores in both UKB and WMC test datasets.
, 2018). Together, it underscores the importance of accurately charting the early emergence and progression of WMH for a better understanding of its pathophysiology, its genetic and environmental determinants, and ultimately for early intervention. To this end, we combined the WMH labels of the MRi-Share with the publicly available MWC dataset to develop the SHIVA-WMH detector, based on our prior work that used the 3D Unet-based architecture to detect PVS, another marker of covert cSVD  (Boutinaud et al. , 2021). Even though there has been an increasing number of studies applying Unet-based models for WMH detection, it has not been used to push the limit of early detection in young, neurologically asymptomatic cohorts. Further, with few exceptions  (Tran et al. , 2022; Umapathy et al. , 2021) , most work on automatic segmentation methods for age-related WMH so far has focused on developing and evaluating their tools on more conventional 2D FLAIR images with thick slices (typically 3 to 5 mm).
However, there is an inherent limitation in accurately quantifying the amount of small WMH with 2D FLAIR acquisitions, since small lesions out of the plane of acquisition cannot be detected. More modern 3D FLAIR acquisitions with isotropic resolution are known to achieve a better signal/contrast-to-noise ratio and allow greater sensitivity to WMH lesions than 2D acquisitions  (Bink et al. , 2006; Chagla et al. , 2008). They are also increasingly available in large neuroimaging databases in population-based studies (e. g. UKB;  Alfaro-Almagro et al. , 2018; ADNI-3; Gunter et al. , 2017; Rhineland Study; Lohner et al. , 2022 ). Yet, several recent works on age-related WMH detection methods that explicitly evaluated their methods in participants with mild WMH burden (< 5 mL) had not included 3D FLAIR datasets for training or evaluation  (Khademi et al.
, 2021; Mojiri Forooshani et al. , 2022; Ong et al. , 2022; Rachmadi et al. , 2018). Our work is unique in taking advantage of the 1mm isotropic 3D FLAIR scans from young adults and the detailed delineation of every visible WMH in every slice to train the 3D Unet-based model, with the objective to apply our tool in other cohorts with similar acquisitions to accurately describe WMH burden across the adult lifespan. We evaluated the performance of the SHIVA-WMH detector in the 10 subjects each from the MRi-Share and MWC datasets that had been set aside (i. e. not seen during training), as well as in the 11 additional subjects from the UKB dataset, representing adults from the general population in the age range and with the level of WMH burden in between MRi-Share and MWC. As the reference tools for comparison, we selected one conventional signal intensity-based method (LST-LPA) and two existing Unet-based methods (PGS and HPM) that could be used out-of-the-box (i.
e. with pretrained models available for Unet-based methods). The LST-LPA is based on the logistic regression model that uses intensity and location information of FLAIR images to classify WMH. It was originally developed to segment WMH in MS patients  (Schmidt, 2017a) , but has been applied widely to quantify age-related WMH as well  (Ribaldi et al. , 2021)  and has also been demonstrated to show robust performance across diverse datasets with age-related WMH  (Heinen et al. , 2019; Vanderbecq et al. , 2020)  These scores are the highest among several recent works that explicitly evaluated their methods in participants with mild WMH burden of less than 5 mL  (Khademi et al. , 2021; Mojiri Forooshani et al. , 2022; Ong et al. , 2022; Rachmadi et al. , 2018). For the purpose of comparison with our tool, it also had the advantage of being trained with multisite imaging datasets that did not include the MWC dataset, allowing for a fair comparison of performance with our tool in this cohort, unlike the PGS, whose training data included MWC testing data set aside in the present study.
Of the three reference methods, LST-LPA had the lowest overall Dice scores, and performed progressively worse in the cohorts with lower overall WMH burden. It had comparable Dice scores as SHIVA-WMH in the MWC test set, but only at the voxel-level. Lesion-wise CL-Dice was significantly worse than SHIVA-WMH even in this test cohort with the largest overall WMH burden. It suggests that while it is able to detect relatively large WMH in subjects with moderateto high-lesion burden, small lesions found in these subjects are missed by LST-LPA. This observation is consistent with other studies demonstrating the superior performance of Unet-based methods over LST-LPA primarily in subjects with lower overall lesion burden  (Khademi et al. , 2021; Li et al. , 2022). Among the three reference methods, PGS had the highest sensitivity in the MRi-Share and UKB test datasets, with comparable VL-and CL-TPR in UKB and lesion-wise CL-TPR in MRi-Share as SHIVA-WMH, attesting to the stated advantage of their approach.
However, the relatively high sensitivity came at the cost of low precision in both test datasets, resulting in the significantly lower VL-and CL-Dice scores in MRi-Share and CL-Dice score in UKB compared to SHIVA-WMH. The lower precision likely results from the 2D input they use for their Unet model, since islands of cortical ribbons on some axial slices are difficult to distinguish from WMH without the 3D context. The underperformance of HPM was somewhat surprising, given their high reported performance in subjects with mild WMH burden and the fact that it has been trained with a large and diverse dataset representing 432 individuals from 4 multicentre studies, using the 3D input for their Unet model as in SHIVA-WMH. Although speculative, we suspect that the reason may be the nature of the training dataset they used: All their training and testing data were from 2D FLAIR with 3 mm slice thickness, which may have limited the advantage of full 3D model.
Further, in order to prepare the large number of reference WMH labels to train their model, they used a semi-automated pipeline that generated intensity-based segmentations. Although these labels were then reviewed and manually edited by trained human annotators, it is generally more difficult and time consuming to add lesions missed by the automated method than rejecting false positives during such manual editing, which can result in more conservative labels missing small lesions not detected by the semi-automated method. In contrast, we used the high-quality, high-resolution manual labels for the MRi-Share training dataset consisting of 40 subjects to train a model specific to this cohort, then used the predictions generated by the model in unannotated MRi-Share subjects to iteratively train our model until optimal performance was reached in the manually-traced MRi-Share and MWC validation sets.
Such an approach has been recently suggested as one of the effective solutions for enhancing limited high-quality annotations for training DL-based models  (Tajbakhsh et al. , 2020) , here applied specifically to enhance performance for more difficult cases of subjects with very low WMH burden. The superior performance of the SHIVA-WMH detector relative to other reference methods in the UKB test set is worth being emphasized, as this test set represents data coming from an unseen cohort, and thus constitutes an important test case for the transferability of our detector to cohorts not seen during the training. Further, the UKB dataset has been acquired with a modern scanner with high-quality 3D FLAIR acquisitions similar to MRi-Share, and represents one of the intended target populations to apply our tool in future studies to characterize the full extent of WMH in neurologically asymptomatic adults.
Relative to SHIVA-WMH, only PGS showed comparable sensitivity to WMH found in UKB, but at the cost of lower precision, resulting in the lower and more variable Dice scores in this cohort than our tool. It suggests that our tool can predict variable burden of WMH in this cohort more accurately and consistently than the reference tools tested here. It should be noted that we compared the performance of SHIVA-WMH against PGS and HPM without re-training the latter two methods with the same training data used by SHIVA-WMH, since our aim was to evaluate the direct applicability of these publicly available tools, rather than to test the ability of different Unet-based models to learn new dataset. Although beyond the scope of this study, it is possible that innovations in the model architectures, such as the multi-scale highlighting forgrounds approach in PGS, can further improve small lesion detection when used in combination with the high resolution training data we used.