dataesr/scientific-paragraphs-categorization huggingface dataset:
Our results for intensity distributions, brain volumes, image quality metrics and radiomic features indicated that the MRI characteristics at the various sites had been effectively homogenized. Next, brain age prediction experiments and the observed correlation between the gray-matter volume and age showed that thanks to an appropriate training strategy and despite biological differences between the dataset populations, the model reinforced biological patterns. Furthermore, radiologic analyses of the harmonized images attested to the conservation of the radiologic information in the original images. The robustness of the harmonization model (as judged with various datasets and metrics) demonstrates its potential for application in retrospective multicenter studies.
However, this type of approach relies on prior feature extraction and is not suited to the harmonization of whole brain MR images. Image-level harmonization is mainly based on deep learning approaches and has shown encouraging results for structural MRI harmonization. Supervised learning using cohorts with "traveling subjects" (i. e. participants having been assessed at more than one center) have enabled to set up frameworks that do not require large datasets  [11, 12]  but are not applicable in many situations when such subjects are not available. Other types of harmonization methods have been designed for specific tasks  [13] [14] [15]  but require different training phases for each application and are limited to supervised predictions. In the present study, we focus on the use of unsupervised deep learning frameworks for the harmonization of T1-weighted (T1w) brain MR images.
Palladino et al. [19]  and Chen et al. [17]  developed 3D solutions, but they were limited to low-volume patches. The resulting loss of spatial and contextual information was not justified or was dictated by material limitations  [18, 19, 22]. Along with these methodological challenges, the evaluation of the quality of the harmonized results is also a key aspect. The main approach has been similarity metrics based on small traveling-subject datasets for which ground truth harmonization can be assumed  [23, 25, 24, 18, 22]. Some studies also showed better prediction of biological information such as sex  [18, 26] , age  [27, 26] , disease  [23, 25, 18]  and brain tissue segmentation  [17, 21, 19]. However, in most works the validations are not extensive enough to assess the robustness of the models to various applications.
Furthermore, site effects have been reported for features extracted from T1w brain images: tissue volumes  [28] , radiomic features  [29]  and image quality metrics (IQMs)  [30]. In this work, we propose a 3D CycleGAN model for the inter-site harmonization of T1w brain images that enables the processing of 3D images while preserving stability and optimizing computation resources. Using six datasets, we measured inter-site differences in intensity distributions, brain volumes, radiomic features and IQMs to evaluate our approach. We used age to quantify biological patterns through brain age prediction and the correlation between age and gray-matter (GM) volume, and we used specific scales to rate the conservation of radiologic patterns. We tested the model on cohorts with different age distributions and managed to harmonize them efficiently while avoiding overcorrection thanks to an appropriate training strategy.
Independent datasets We obtained 3D T1w brain images from public data-sharing sources: IXI, 1 OASIS-3  [31] , NKI-RS  [32]  and NMorphCH  [33]  and then created six datasets of images acquired with six different machines (Table  1 ). All the participants were only present in a single dataset. As specified in the individual study protocols, all participants were healthy controls. Each participant had given his/her informed consent at the local study site, and each contribution was ethically approved. The selection of the MRI datasets is described in detail in the Suplementary Materials (section 1).
MRI preprocessing We first skull-stripped the T1w brain images (using volBrain software  [34] ) and corrected them for magnetic field inhomogeneity effects (using N4ITK algorithm  [35] ). Next, we linearly registered the MR images to 1 × 1 × 1 mm MNI space with FSL-FLIRT tool  [36, 37]  and scaled the intensities by setting the median within each brain to 500. We found median normalization to be less sensitive to outliers than maximum normalization, which is widely used when applying deep learning to MR images  [23, [38] [39] [40] [41] [42]. To facilitate processing with CNNs, we padded and cropped the images to have dimensions 192 × 192 × 192 without removing any brain voxel.
Harmonization procedure 2. 3. 1. Framework and architectures The model developed is based on a CycleGAN configuration  [16]. Given two image domains, it learns to translate an image from one domain to the other. The training needs a set of images from each domain but does not require ground truth (i. e. the two sets are not paired). With a multisite MRI dataset, one site is set as the reference and the images from the other sites are transferred towards the reference domain (MR images from the reference site are not modified). We adapted the U-net generators and patchGAN discriminators described by Isola et al. [43]  with 3D convolutions for processing whole 3D MR images (Fig. 1 ). The patchGAN discriminator had a receptive field of 38 × 38 × 38, which we found to be a good compromise between accessing contextual information and focusing on local imaging details.
Model training We implemented several strategies for increasing the training phase's stability and the model's robustness. Firstly, we pretrained a generator to replicate the input with all the datasets and an L1 loss; the generators of our CycleGAN were then initialized with the learned weights. Secondly, in order to make the early training stages less sensitive to variance, we also applied a linear decay from 200 to 100 for the hyperparameter weighting the cycle consistency constraint  [45]. Thirdly, while we set the batch size to 1 for the generators, we stabilized the discriminators by training them with a history of 50 generated images  [46]  and at every step, each one was updated with 4 real images, 2 newly-generated images and 2 formerly-generated images. In order to save GPU memory and speed-up the computations, we applied a mixed precision policy  [47].
Configuration In a first set of experiments, we harmonized three pairs of datasets: Site1 vs. Site2, Site3 vs. Site4, and Site5 vs. Site6. In each pair, the two sites had similar age distributions (statistical assessment in section 3 of the Supplementary Materials); this prevented confusion between site effects and biological variabilities during the training and evaluation steps. The three pairs were chosen to address various potential site effects: a difference in scanner field strength between Site1 and Site2, different MR scanners at Site3 vs. Site4, and a difference in acquisition parameters between Site5 and Site6 (Table  1 ). We also performed an overall harmonization in which five sites were harmonized against a single reference site. Although Site1's images were acquired with a 1.
5T scanner, we chose it as the reference site because it contributed a relatively large number of MR images and had a broad age range (from 20 to 86). Hence, we trained five models: Site1 vs. Site2, Site1 vs. Site3,. , Site1 vs. Site6. To avoid correcting age effects because of differences in the datasets' age distributions (section 4 of the Supplementary Materials), we divided the MR images into several age ranges for each training session and assigned a probability of sampling for each age range; hence, at each step, the two training sets had the same probability of age distribution. To avoid undersampling the smallest populations, we developed an algorithm that, given the two training sets, gave us the probability of sampling in the age ranges. The algorithm is described in the Supplementary Materials (section 5), together with the sampling probabilities used for the various harmonizations vs.
Brain age prediction Brain age prediction consists in training a machine learning model to predict an individual's age from brain MRI data and has been widely investigated in recent years For each of the three paired-site experiments, we selected the site with the largest number of MR images as the harmonization reference (i. e. Site1, Site3 and Site5), trained an age prediction model and evaluated the latter against the images from the other sites (i. e. Site2, Site4 and Site6) before and after harmonization. In order to measure baseline performances, we randomly split the three reference sites' MR images into training and test sets with age stratification; this resulted in 50, 117 and 30 test MR images for Site1, Site3 and Site5, respectively. V. Roca, G. Kuchcinski, J. -P. Pruvo et al. The ratio between the WM's median intensity and the 95th percentile of the full-intensity distribution in the brain.
rpve_gm An estimation of the residual partial volume effect for GM, WM and CSF. rpve_wm rpve_csf fwhm Full-width at half-maximum, an estimation of the blurriness of the image using AFNI 3dFWHMx. GM: gray-matter; WM: white-matter; CSF: cerebrospinal fluid. For the multisite experiment, we evaluated the model trained on MR images from Site1 (the reference set) against the other five datasets before and after harmonization. We also set up a brain age prediction model to assess the value of harmonization for a large multicenter training set. We randomly split our data into a training set of 1863 images and a nonoverlapping test set of 448 images, while conserving the proportion of images within each site. Next, we trained and evaluated two brain age prediction models: one without harmonization and the other with images harmonized against Site1.
Statistical inferences We compared the sites with regard to the tissue volumes and IQMs by using two-tailed t-tests for the paired-site and a one-way analysis of variance (ANOVA) for the multisite experiments. For the age prediction results, we used twotailed Wilcoxon signed-rank tests to compare the prediction errors before and after harmonization. We compared linear correlation coefficients using a two-tailed Steiger's test  [67]. Using the Benjamini-Hochberg procedure, we corrected p-values for multiple comparisons (i) for each comparison of tissue volumes and IQMs, (ii) for the three brain age prediction comparisons in the pairedsite experiments, and (iii) for the five brain age prediction comparisons in the multisite experiment with the model trained on Site1's MR images. To ensure independence between samples, we averaged the data for each participant in each test.
Harmonization on traveling subjects We used our traveling subject dataset (section 2. 1. 2) to evaluate the ability of our model to transform images from one site to their equivalent in the other. We reused the harmonization models previously trained for the paired-site experiments (Site3 vs. Site4, section 2. 4. 1) to harmonize the 76 traveling subjects in both directions. We computed the structural similarity index measurement (SSIM)  [68]  for each image pair, with an intensity range fixed to 1000. Before computing SSIM, we removed the background slices. We compared our 3D approach with a 2D CycleGAN adapted from Zhu et al. [16]. To exploit the three orientations, we trained three 2D models and generated the final output volumes with a 2. 5D inference  [23, 11]. For training, we used slices that contained above 1% of non-zero pixels after processing, following the approach of Bashyam et al.
Brain age prediction Fig. 6  illustrates the results related to the three age prediction experiments. In all three experiments, the MAE of the test dataset was lower than that of the generalization dataset, and harmonization resulted in a significant decrease of the prediction errorseven if the size of the decrease varied (Fig. 6a ). For the three generalization sets, the MPAD was closer to 0 with harmonization (particularly for the Site5/Site6 pair), meaning that under/overestimation patterns caused by site effects were partly corrected on the harmonized MR images (Fig. 6b ).
Site-related variabilities of MRI features The measure for each tissue volume in each age range varied significantly from one dataset to another (Fig. 9a, 9c, 9e ). The variability was lower after harmonization (Fig. 9b , 9d, 9f), except for the CSF volumes in the 60-70 age range (Fig. 9f ). Large inter-site differences in almost all the IQMs were apparent on the original MR images (Fig. 10a , 10c, 10e). Overall, the differences were reduced by harmonization (Fig. 10b, 10d, 10f ) except for fwhm and snr_csf. Based on the PCA components extracted from the radiomic features, site clusters were easily distinguished for the original data (Fig. 11a, 11c, 11e ). An exception was seen for the 20-30 age range at Site1, the components of which were superposed on those of Site5 (Fig. 11a ). The clusters were not more easily distinguished after harmonization (Fig.
Brain age prediction 3. 2. 3. 1. Single-site training set Fig. 12  illustrates the results produced by the brain age prediction model trained on Site1 and applied to all our MRI datasets. Fig. 12a  shows that, as in the paired-site experiments, the test set's MAE was lower than those of the generalization sets. After harmonization, the prediction errors fell significantly for Site2, Site3, Site4 and Site6. In contrast, they were significantly higher for Site5. It can be seen in Fig. 12b  that, after harmonization, the MPAD was closer to zero for Site2, Site3 and Site4 but further from zero for Site5 and Site6; this finding suggests that site-related under/overestimation patterns were attenuated in the first case and accentuated in the second. For each set of predictions (other than for Site1), the MPAD had the same sign as the TMD, which reflects RTTM.
Harmonization on traveling subjects Table  3  reports the SSIMs obtained with the traveling-subject dataset. Our 3D model resulted in a significant increase in SSIMs when harmonizing from Site3 to Site4 (p < 0. 001) and a significant decrease when harmonizing from Site4 to Site3 (p < 0. 001). In both cases, the 2D CycleGAN significantly reduced the SSIMs (p < 0. 001). However, the SSIMs obtained with our 3D model were significantly higher in both harmonization directions (p < 0. 001).
Discussion In the present study, we developed a 3D domain transfer model for the inter-site harmonization of T1w brain images. In view of the known concerns about the technical requirements of deep learning model and their reliability with miscellaneous MR data   [70], we were looking for the efficient use of computation resources and robustness to various datasets. Based on a pool of MR images acquired with six different machines, our experiments showed that the model could reduce inter-site variabilities in intensity distributions, brain volumetry, quality metrics and radiomic features. Harmonization was also associated with significantly more accurate brain age prediction (whether with small single-site or large multicenter training sets) and mitigated site effects on changes in GM volume with age.
The radiologic scores also confirmed the consistency of biological information at the individual level before and after harmonization. Additionally, validation with a traveling-subject dataset indicated a superiority over a well-known 2D domain transfer model. In contrast to earlier research on models that processed parts of images only (slices or 3D patches), our model was designed to operate on whole 3D T1w brain images. We compensated for the additional computing costs by using a Unet architecture for the generators, which is less greedy than the standard Resnet architectures used in CycleGAN approaches  [17, 25, 18, 19]. We opted for transposed convolutions because resize convolutions  [71, 19]  lost more of the anatomic information in the input MR images (data not shown) and produced more cumbersome networks.
When compared with the removal of MR images outside a specific age range  [13] , our sampling strategy better conserved the diversity in the training data. Style transfer methods have been used to tackle biological differences between sites, by ignoring site information  [21]  or by introducing biological conservation modules  [23]. However, in  The results of our paired-site experiments (section 3. 1) generally confirmed the model's efficiency with various source and target datasets with differences in sample size, age distribution, scanner used and acquisition parameters (section 2. 1. 1). The data at Site5 and Site6 came from different studies (i. e. NKI-RS and NMorphCH) and differed markedly. The data at Site1 and Site2 came from the same database (IXI) and were more homogeneous. However, our results suggested the presence of clear site-related variabilitiesprobably because of the difference in field strength (i.
e. 1. 5 vs. 3 Tesla). The differences between the Site3 and Site4 datasets (both from the OASIS-3 study) were smaller but our model still managed to correct them significantly. This is a strong point, given that some harmonization methods harmed datasets with small or no site effects  [73]. The multisite experiment enabled us to evaluate the model in a more common situation, i. e. data from more than two sites, and harmonization into a common space. Although some site-related differences were still present after harmonization (mainly among the participants in the 60-70 age range), our analysis of the intensity distributions (section 3. 2. 1) and the MRI features (section 3. 2. 2) indicated that the datasets had indeed been uniformized. Harmonization significantly improved the performance of the brain age prediction model trained on Site1's MR images for all sites except Site5 (section 3.
2. 3. 1). When considering the predicted age differences and the TMD, RTTM appeared to be greater after harmonization of Site5 MR images. We speculate that site-related variabilities led to age underestimation in these images, and this fortuitously compensated for RTTM, resulting in age overestimation in younger participants. To validate this hypothesis, we included older participants in the dataset (section 9 of the Supplementary Materials). Thus, the age prediction results highlighted the advantages of our method for domain adaptation, i. e. when a predictive model is trained with data from one site and applied to others (sections 3. 1. 3 and 3. 2. 3. 1). In a similar manner, Bashyam et al. [27]  achieved significant improvements in predicting age with harmonization. We further demonstrated the predictive value of our model with a large multicenter training set (section 3.
The radiologic scores (sections 3. 1. 4 and 3. 2. 5) showed that our model was able to conserve precise radiologic information related to brain atrophy, perivascular spaces, and ventricle size. In some previous studies of medical image translations, generated images were radiologically assessed for realism  [74, 75]. In the present study, however, we wanted to investigate the conservation of individual features with harmonization. Indeed, it is known that the application of CycleGAN approaches may result in the loss of valuable information from the input images  [72]. Our validation on the traveling-subject dataset (section 3. 3) shows that our 3D CycleGAN model is more effective than the classical 2D CycleGAN in harmonizing Site3 and Site4 MR images. With our harmonization from Site3 to Site4, we achieved a significant increase in the SSIM compared to MR images after preprocessing.
Conversely, SSIM decreased when harmonizing from Site4 to Site3. This can be partially explained by the limited inter-site variability between these two sets, as indicated by the results of the corresponding paired-site experiment (section 3. 1). However, this mitigated outcome of harmonization from Site4 to Site3 is counterbalanced by the substantial improvement it enabled in the brain age prediction of the model trained on Site3 MR images and applied to Site4 MR images (section 3. 1. 3). The present study has some limitations. Firstly, the CycleGAN approach requires an independent training for each site. Nonetheless, the great diversity of input datasets and the large number of evaluations performed in the present study (relative to the typical literature findings) show the interest of our model for datasets comprising a relatively large number of MR images; Gebre et al.
[20]  V. Roca, G. Kuchcinski, J. -P. Pruvo et al. also showed the value of CycleGAN against ComBat approaches, conditional GAN and style transfer methods for the harmonization of large cross-sectional datasets. Secondly, we opt for a comparison between our 3D method and 2D CycleGAN due to its established status and the availability of implementation details  [16]. However, we acknowledge that other recent methods  [23, 22]  could also be evaluated. Nevertheless, performing a fair comparison would pose a significant challenge due to various factors, including the difficulty in accessing and using the implementations, as highlighted by Hu et al. [5] , the heterogeneity of the data used for development, and differences in the preprocessing (e. g. skull-stripping and rescaling). Thirdly, many of the features that we used to analyze site effects are dependent on the segmentation method (FSL-FAST here), and so our results might have been different if another method had been applied.
Acknowledgements This work was funded by  Philips France Commercial (Suresnes, France). Data were provided in part by OASIS-3:  Longitudinal Multimodal Neuroimaging : Principal Investigators:  T. Benzinger ,  D. Marcus ,  J. Morris ;  NIH   P30 AG066444 ,  P50 AG00561 ,  P30 NS09857781 ,  P01 AG026276 ,  P01 AG003991 ,  R01 AG043434 ,  UL1 TR000448 ,  R01 EB009352. In the  OASIS-3  study, AV-45 doses were provided by  Avid Radiopharmaceuticals , a wholly owned subsidiary of  Eli Lilly. We acknowledge the  In-vivo Imaging & Functions core facility (LIIFE)  for the data management of the study.
Data availability All the MR images used in this study come from public databases (section 2. 1. 1). The selection of participants is detailed in the Supplementary Materials (section 1). The features extracted to generate the results will be made available on request. The Python code for the harmonization model, the age prediction model, the IQM extraction, the computation of tissue volumes and the function used for balancing the age distributions is available in an online repository: https://gitlab. com /RocaV /3d _cyclegan _ mri _harmonization.
Conclusions In this article, we propose an unsupervised model for the inter-site harmonization of T1w MR images of the brain. This optimized 3D deep learning approach processes whole brain images and is robust to diverse MRI datasets. A range of experiences on various cohorts at different scales attests to the model's ability to eliminate diverse inter-site variabilities, conserve radiologic information and reinforce biological patterns. Despite the presence of major biological differences between sites, our choice of an appropriate training strategy helped to make the multisite harmonization a success. Our extensive validation of the harmonization results is promising for various future applications in multicenter studies.
Pictures of faces. Pictures of faces were selected from the Chicago Face Database  (Ma et al. , 2015)  (https://chicagofaces. org/default/). They were composed of the faces of nine women and nine men from various origins (Asian, Caucasian, African American, Latin American; Figure  1C ). All faces were presented on a white background, their relative position and dimensions were similar, and all persons wore a grey sweatshirt. To limit their distinctiveness, they were turned into black and white, and the most prominent details of each face were erased (spots, freckles; Photoshop®, Adobe, Dublin, Ireland). In the Episodor software, the 2863 x 1718 pixels pictures were presented in jpeg format inside the box cover, on the centre of the screen. The duration of their presentation was adjusted to the music duration when presented simultaneously and was 6 s otherwise.