dataesr/scientific-paragraphs-categorization huggingface dataset:
Statistical analyses Sample size and power analysis. We conducted an a priori sample size estimation based on 1) preliminary results of the device in internal tests and the available scientific literature that allowed us to expect a reliability (intraclass correlation coefficient, ICC) of 0. 85 (ρ1), and 2) the recommendations of Terwee et al. [30] , who proposed a minimal acceptable reliability (ICC) of 0. 70 (ρ0). This power analysis revealed that 53 participants were needed to reach 80% power and a two-sided type I error at 0. 05. Data analysis. Criterion validity was assessed by calculating the Person's correlation coefficient (the normality of the distribution was checked with a Shapiro-Wilk test) between the scores given by the app and the score measured via the gold standard. The scoring system for correlation coefficients as described by McCall  [40]  was used: 0.
The coefficient of variation (calculated as standard deviation / mean × 100) was also computed for both tests. Bland-Altman plots were performed to visualize agreement. Finally, for interpretability, the MDC was calculated as 1. 96 × p 2 × SEM. Sensitivity analyses were carried out on the basis of gender and age, with comparisons between men and women and between <30 and �30 years old (we dichotomized in this order to obtain two groups of similar size; 30 years being the median age of our sample). All analyses were conducted using R (R Foundation for Statistical Computing). The data and code for the statistical analyses used in the present study are available on Open Science Framework (https://osf. io/4hzke/).
The work of AM is supported by an  ANRT grant  ( Cifre PhD Thesis ) and by the company  Kiplin. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. AM's PhD grant is funded by the  French National Association for Research and Technology (ANRT)  and Kiplin. MB is employed by  Kiplin. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Deep Learning (DL) has greatly highlighted the potential impact of optimized machine learning in both the scientific and clinical communities. The advent of open-source DL libraries from major industrial entities, such as TensorFlow (Google), PyTorch (Facebook), and MXNet (Apache), further contributes to DL promises on the democratization of computational analytics. However, increased technical and specialized background is required to develop DL algorithms, and the variability of implementation details hinders their reproducibility. Towards lowering the barrier and making the mechanism of DL development, training, and inference more stable, reproducible, and scalable, without requiring an extensive technical background, this manuscript proposes the Generally Nuanced Deep Learning Framework (GaNDLF).
However, all these applications, toolkits, and frameworks (i) describe developer-focused tools targeting members of the advanced computational research community, (ii) can be difficult to grasp by researchers without sufficient experience in DL, (iii) do not make it easy for DL scientific developers to write their architectures in a generalizable way, allowing their application on problems spanning across domains, (iv) make it difficult to write training pipelines for different problem domains, (v) they put the onus of training robust and generalizable models to the user's knowledge of the training mechanism and the dataset in question, and (vi) they lack a single end-to-end application programming interface (API) for training and inference that can span across various problem domains. In this manuscript, we introduce GaNDLF (a GenerAlly Nuanced Deep Learning Framework) to enable researchers to solve problems involving segmentation, regression, classification, and synthesis, while producing robust DL models without requiring much knowledge of DL or coding experience.
We have developed GaNDLF in PyTorch/Python, as an abstraction layer that incorporates widely used open source libraries (such as Insight Toolkit (ITK) 8  [46]  for handling most I/O operations and basic image processing routines, and TorchIO 9  [47]  for data augmentation) that can help researchers generate robust DL models quickly and reliably, facilitating reproducibility  [48, 49]  and being consistent with the criteria of findability, accessibility, interoperability, and reusability (FAIR)  [50]. Furthermore, the flexibility of its codebase allows GaNDLF to be used across medical imaging modalities (e. g. , 3D radiology scans, and 2D histology whole slide images), with scope for integrating other clinical data (such as genomics and electronic health records) in the future.
Methods The intention behind the development of GaNDLF is to provide an end-to-end solution for training robust DL models to tackle a variety of tasks, such as segmentation, regression, and classification, in both 2D and 3D image datasets. It is also designed to work across radiology data (e. g. , Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Positron Emission Tomography (PET)) and digitized histology whole slide images (WSI) (e. g. , Hematoxylin and Eosin (H&E) stained tissue sections), including specialized image pre-processing functionalities for both. The notable difference between these images is the relatively small resolution and size of radiology images (typically occupying a few megabytes of disk space), compared with the histology WSI that are images of relatively large resolution (150K × 150K pixels), where a single image can occupy 30-40 gigabytes.
Data Augmentation DL methods are well-known for being extremely data hungry  [54, 55]  and in medical imaging, data is scarce because of various technical, privacy, cultural/ownership concerns, as well as data protection regulatory requirements, such as those set by the Health Insurance Portability and Accountability Act (HIPAA) of the United States  [56]  and the European General Data Protection Regulation (GDPR)  [57]. This necessitates the addition of robust data augmentation techniques  [58]  into the training data, so that models can gain knowledge from larger datasets and hence be more generalizable to unseen data  [59]. GaNDLF leverages an existing robust data augmentation package, namely TorchIO  [47]   11  , which provides augmentation transformations in a PyTorch-based mechanism (Table  1 ).
GaNDLF specifically focuses on TorchIO over other packages (such as Batch Generators  [60]  and Albumentations  [61] ), because TorchIO provides an easier API for storing extraneous image information. Such information include the affine transform of the image that is critical for maintaining correct physical definition of radiology scans. Furthermore, TorchIO also maintains a strong tie-in with PyTorch's Tensor data structure, thereby making it relatively easier to extend the data loading with different data formats. More details on the available types of augmentations through GaNDLF are shown in Table  1 , and examples of their effects are illustrated in Figure  2 , using a brain tumor T2-weighted-Fluid-Attenuated Inversion Recovery (T2-FLAIR) MRI scan from the BraTS challenge's dataset  [9, 11, [62] [63] [64].
k-fold cross validation  [65, 66]  is a useful technique in ML that ensures reporting unbiased performance estimates and helps capture information from an entire given dataset, by training k different models on corresponding folds of the complete training data. Specifically, to ensure robust model training, and providing unbiased performance estimates and quantitative validation of the generalization performance of the implemented algorithms (i. e. , by evaluating results on new unseen data), GaNDLF offers a nested k-fold cross validation schema  [67] , i. e. , a well-established way of evaluating algorithms on new datasets  [68]. Specifically, during this nested k-fold cross validation, all data are combined into a single cohort with a model configuration of three sets. Initially cases of the complete cohort are proportionally and randomly divided into k non-overlapping equally-sized subsets and during each fold, k -1 of these subsets are considered as the retrospective/discovery cohort and 1 as the prospective/replication cohort, which is unseen for this specific fold.
Training Mechanism GaNDLF provides a flexible mechanism for training DL models, with support for multiple imaging modalities and output classes, along with support for both 2D and 3D datasets. The main entry point of GaNDLF's training mechanism is a CSV file provided by the user, through the command line interface. The expected CSV file should comprise the subject identifiers along with the corresponding full paths of all required input images and masks (i. e. , for segmentation tasks) and the values required for training and follow up predictions (i. e. , for regression and classification tasks). The subject identifiers are used to randomly split the entire dataset into training, validation, and testing subsets, using k-fold cross-validation  [69]  (see Section 2. 3 for more details). Furthermore, a YAML-based configuration file is used to control and parameterize all aspects of the training, such as the proportional split of the cross-validation, data pre-processing, data augmentations (e.
for Radiology scans As soon as the data is read into memory, GaNDLF applies the pre-processing steps defined in the configuration file to each input dataset (see Section 2. 1 for examples of these steps). Then TorchIO's  [47]  inference mechanism is used to enable patch-based inference for radiology images. This entails patch extraction, usually of the same size as the one that the corresponding model has been trained on, from the image(s) on which the model needs to infer on. The forward pass of the model is then applied and the result is stored in the corresponding location (Figure  4 ). This enables models to be trained and inferred on varied patch sizes based on the available hardware resources. Overlapping patches can be stitched by either cropping or taking an average of the predictions at the overlapping area, and the amount of overlap can be specified to ensure that dense inference can occur  [47].
Dimensions To ensure maximum flexibility and applicability across various types of medical data, GaNDLF supports both 3D and 2D datasets. Using the same codebase, GaNDLF has the ability to apply various architectures across diverse modalities such as MRI, CT, retinal, and digitized histology images, including both immunohistochemical (IHC), Hematoxylin and Eosin (H&E) stained tissue sections.
Our implementation follows the principle of the UNet architecture, while including skip connections in every convolution block. The residual connections utilize additional information from previous layers (across the encoder and decoder) that enables a boost in the performance. • Inception UNet (UInc): The Inception module  [2, 79]  can be used to substitute the standard convolutional block (which is a simple series of convolutional layers) of the UNet to create the UInc architecture (Figure  7 (d), Figure  12 ). This module describes parallel pathways of convolutional layers of different kernel sizes, to improve the representation of multi-scale features. UInc has been applied towards semantic segmentation tasks  [80]. • Fully Convolutional Network (FCN): The FCN architecture  [81]  (Figure  7 (b), Figure  10 ) introduced in 2017, utilizes hierarchical feature extraction with an encoder recognizing both imaging patterns and spatial information of each input image, with varying receptive fields.
Applications As already mentioned, GaNDLF can train DL models for various tasks, including segmentation, regression, and classification. Depending on available resources, most models can be extended for all these tasks (such as UNet) and there are specialized models that perform specific tasks, such as the brain age prediction model  [88] , which starts from a VGG-16 model pre-trained on ImageNet weights and is only defined for regression. The flexibility of GaNDLF's framework makes it possible for all these models to co-exist and to leverage the robustly designed data loading and augmentation mechanisms for future extensions of studies. Having a common API for all these tasks also makes it relatively easy for researchers to start applying well-defined network architectures towards various problems and datasets, thereby helping to get DL-based pipelines into clinical workflows.
Experimental Design and Evaluation For each application, multiple models are trained in accordance with the cross-validation schema described in Section 2. 3. For performance evaluation, we use the model with the best validation score as defined in the applicationspecific evaluation criteria and apply this model to the test dataset for each fold, giving us the average performance of an architecture for the specific problem. To maintain reproducibility and prevent overfitting, we have trained each architecture with a 20/16/64 split, which results in the training of 25 models in total, for each architecture. Specifically, the 20/16/64 split comprises 5 non-overlapping splits (i. e. , each containing 20%) of the complete dataset. Each one of these splits is set aside as the testing cohort for each fold.
Segmentation of Brain in MRI Brain extraction is an essential pre-processing step in the realm of neuroimaging, and has an immediate impact on the quality of all subsequent processing and analyses steps. We have used a multi-institutional dataset of n = 2520 MRI scans along with their corresponding manually annotated brain masks. We trained on n = 1320 scans in a modality-agnostic manner (i. e. , each structural MRI scan was treated as a separate input) as described in  [7]  and setting a internal validation set of n = 180 scans, with an independent testing cohort of n = 360 scans to ascertain the model performance. We trained by resampling the data from an isotropic resolution of 1mm 3 with a shape of 240 × 240 × 160 to a anisotropic resolution of 1. 825 × 1. 825 × 1. 25 mm 3 with a shape of 128 × 128 × 128  [7].
Segmentation of Brain Tumor Sub-regions in MRI Gliomas are among the most common and aggressive brain malignancies and accurate delineation of these regions can provide valuable clinical insights. We have used the publicly available MRI data from the International Brain Tumor Segmentation (BraTS) challenge of 2020  [9, 11, [62] [63] [64]  to train multiple models to segment the various brain tumor sub-regions. Specifically, we used the full cohort of n = 371 training subjects, which we iteratively split it into n = 74 testing, n = 60 validation, and n = 237 training subjects following the k-fold cross-validation schema mentioned in Section 2. 3, with all the 4 structural MRI sequences making up a single input data-point  [7]. In total, 25 models are trained for each architecture (UNet, ResUNet, UInc, and FCN).
Segmentation of Retinal Fundus We used the dataset from the PALM challenge  [97] , which consists of segmentation of lesions in retinal fundus images and replicated the results for a ResUNet architecture from  [98]. Additionally, we trained on FCN, UNet, and UInc to show results from a diversified set of architectures from the same dataset. We used the full cohort of n = 400 training subjects, and iteratively split into n = 80 testing, n = 64 validation, and n = 256 training subjects following the k-fold cross-validation schema mentioned in Section 2. 3. In total, 25 models are trained for each architecture (UNet, ResUNet, UInc, and FCN). For each model, we used a set of common hyperparameters options that runs in a GPU with 11GB of memory, namely, patch size of 2048 × 1024, 30 base filters, Dice loss, with stochastic gradient descent as the optimizer.
Segmentation of Lung field in CT An accurate volumetric estimation of the lung field would be crucial towards furthering the clinical goals of tackling respiratory illnesses, such as influenza, pneumonia, and speciality pathologies such as COVID-19. We have used an internal dataset to extract the Lung Field from chest CT images, where we identified n = 488 scans images with their corresponding ground truth. The ground truth annotations were generated under a semi-automatic procedure leveraging 2-cluster k-means, followed by manual qualitative refinements. We then trained on n = 244 scans and internally validated with n = 60 cases. We performed windowed pre-processing and clipped the intensities from -900 to -300 Hounsfield Units (HU). We also resampled the data down to 128 × 128 × 128 in order to consider the entire chest region and to ensure that the trained model remained agnostic to the original image resolution.
Segmentation of Colorectal Cancer in WSI Colonoscopy pathology examination can find cells of early-stage colon tumor from small tissue slices, and pathologists need to examine hundreds of tissue slices on a day-to-day basis, which is an extremely time consuming and tedious work. The DigestPath challenge  [100]  motivated participants to automate this process and thereby contribute to potentially improved diagnostics. The data provided in the DigestPath challenge includes slides containing colorectal cancer in JPEG format. The dimensions of the provided images range from 3000 × 3000 to 30000 × 30000. 180000 patches of the shape 512 × 512 at 10× resolution were extracted for training and 30000 for validation, with a set of n = 30 WSIs being kept separate as independent testing dataset. We trained the ResUNet architecture, and prior to training we normalized the training values to 0 -1 by dividing each pixel by the maximum possible intensity, i.
Brain Age Prediction from MRI The human brain ages differently because of various environment factors. Quantifying the difference between actual age and predicted age can provide a useful insight into the overlap of aging signatures with various neurodegenerative pathologies. A 2020 study by Bashyam et al. [88]  has used common 2D CNN architectures, borrowed from the computer vision community, to predict brain age from T1-weighted MRI scans across a wide age range. Methodologically, the original fully connected layers of the VGG-Net was replaced by a global average pooling, followed by a new fully connected layer of size 1024 with 80% dropout, and then a single output node with a linear activation was added. The network was then trained with the Adam optimizer, while using MSE. This study was evaluated on 10, 000 diverse structural brain MRI scans, pooling data from various studies, including the UK Biobank  [101]  and a multisite schizophrenia consortium  [102] , thereby representing various subject populations and acquisition protocols.
This inherent variability of the collective dataset allowed to successfully learn a regression model generalizable across sites. The Bashyam el al. study  [88]  goes on to examine using the learned age prediction weights as a starting point for transfer learning to other neuroimaging tasks. It is shown that the age prediction weights serve as a superior basis for transfer learning compared to ImageNet, particularly in neuroimaging tasks where the new training data is limited  [88]. Leveraging the modular nature of GaNDLF, we replicated the age prediction results of that study  [88]  using the same model architecture, training schema, and dataset as in the original study, while following GaNDLF's procedures. Using the VGG-16 model architecture and GaNDLF's built-in cross-validation functionality, we trained regression models using the intermediate n = 80 axial slices of each subject, with input data being split on the subject level.
The central premise of GaNDLF is to enable researchers to conduct DL research across problem domains (segmentation, regression, classification, synthesis) and modalities (radiology, histology) without worrying about the details such as robust data splitting for training, validation, and testing, and implementing various loss functions. For DL method/architecture developers, GaNDLF provides a mechanism for robust evaluation of their architecture across a wide array of medical datasets that span across various dimensions, channels/modalities, and prediction classes, as well as to compare their algorithm's performance with well-established built-in architectures, including UNet  [75, 78] , UNet with residual connections  [76] , and Inception modules (UInc)  [2, 79] , as well as a fully convolutional network  [81].
Although GaNDLF has been evaluated across imaging modalities (on radiology and histology images), so far it has been limited to tasks of segmentation and regression, and not for classification or synthesis. Expanding the application areas would bolster the applicability of the framework. Additionally, application to datasets representing analysis of longitudinal images (such as perfusion imaging) has not yet been evaluated. Also, a mechanism to ensure cascading of models (i. e. , train/infer different models of same/different architectures sequentially) or aggregation (i. e. , train/infer models of different architectures concurrently) is not present, which have generally shown to produce better results  [103, 104]. Mechanisms that enable AutoML  [105] [106] [107]  and other network architecture search (NAS) techniques  [108]  are tremendously powerful tools that create robust models, but are currently not supported in GaNDLF.
Acknowledgements Research reported in this publication was partly supported by the  National Cancer Institute (NCI) , the  National Institute of Neurological Disorders and Stroke (NINDS) , the  National Institute on Aging (NIA) , the  National Institute of Mental Health (NIMH) , and the  National Institute of Biomedical Imaging and Bioengineering (NIBIB)  of the  National Institutes of Health (NIH) , under award numbers  NCI : U01CA242871 ,  NCI : U24CA189523 ,  NINDS : R01NS042645 ,  NIA : RF1AG054409 ,  NIA : U01AG068057 ,  NIMH : R01MH112070 , and  NIBIB : R01EB022573. The content of this publication is solely the responsibility of the authors and does not represent the official views of the  NIH.
The data were calibrated by the standard pipeline contained in the GILDAS package 1. In the images cleaned with default recipes with MAPPING  (Högbom 1974) , the final beam sizes are 1. 93 × 1. 48 (PA = 62 • ) and 3. 19 × 2. 50 (PA = 63 • ) at Band3/1. 3 mm and Band 2/2 mm, respectively. The spectra were extracted with circular Gaussian profiles centred at the location of the CO (3-2) emission and with a fixed full width at half maximum (FWHM) of 0. 6 (= 4. 8 kpc at z = 2. 84) to be consistent 1 https://www. iram. fr/IRAMFR/GILDAS with the results in  Magdis et al. (2017). The extraction was performed in the u-v space with the task UV_FIT. The continuum emission was extracted in a similar way after averaging the linefree spectral windows. Given the spatial resolution, we did not resolve the source.
Acknowledgements. The authors warmly thank the anonymous referee for the insightful comments that improved the presentation of our work and  Isabella Cortzen  for her contribution to the data reduction. The Cosmic Dawn Center (DAWN) is funded by the  Danish National Research Foundation  under grant No. 140. SJ is supported by the  European Union 's  Horizon Europe research and innovation program  under the  Marie Skłodowska-Curie grant  agreement No. 101060888.
Data The ASR experiments are based on the work of  Macaire et al. (2022) , who used a 60-minute-long speech corpus of spontaneous speech in Kréyòl gwadloupéyen for training. The testing data consist of several hours of raw, unsegmented, and untranscribed speech recorded during a 2022 fieldwork. The speech is spontaneous and sparse across the recording, with overlapping speech, laughters, silences, and random noises spread across the collection. The speech segments are also not necessarily in Kréyòl, and even if the limit between French and Kréyòl gwadloupéyen is not clear, some segments are clearly in French and even English. One 1-hour-long recording was selected, which, after some verification, contains a majority of segments in Kréyòl.