dataesr/scientific-paragraphs-categorization huggingface dataset:
Introduction Deep Learning (DL) is a growing trend in image analysis. Recent breakthroughs, notably the explosion of the available computer power and the availability of large datasets, have allowed artificial neural networks to obtain state-of-the art performances in several computer vision challenges (i. e. ILSVRC2012http://image-net. org). These developments have also been exploited successfully in several medical applications  (Esteva et al. , 2017; Poplin et al. , 2018; Titano et al. , 2018). A particular interest in DL methods has raised with the hope to create support tools for radiologists to analyze images, segment lesions and detect subtle pathological changes that even an expert eye can miss  (Hosny et al. , 2018; Topol, 2019; Bernal et al. , 2019). DL methods are hungry data consumers.
Over the years, the MRI community has curated several public databases with annotated images to test lesion segmentation algorithms. Recently DL approaches have defeated all alternative methods in challenges such as MSSEG, for multiple sclerosis lesion segmentation (http://portal. fli-iam. irisa. fr/msseg-challenge); BRATS, for brain tumor segmentation (www. braintumor segmentation. org); ISLES, for ischemic stroke lesion segmentation (www. isles-challenge. org); and mTOP for mild traumatic injury outcome prediction  (www. tbichallenge. wordpress. com). Moreover, the recent development of unsupervised anomaly detection techniques has opened the door to many more applications of lesion identification, specially for pathologies where annotated data is scarce and other, like Parkinson's Disease (PD), where lesions are not visible to the naked eye.
Indeed, at this stage PD affects only a few neighboring voxels, mainly located in subcortical structures, such as the Substantia Nigra (SN), as opposed to larger brain lesions such as brain tumors, stroke, or T2-hyperintense lesions in MS, clearly visible in MRI. A last, to eliminate additional sources of bias, it is wise to consider scans with homogeneous acquisition parameters, notably the same magnetic field and scanner manufacturer. As a consequence most medical studies tend to provide training data of reasonable size in a clinical context but limited with respect to the usual deep learning requirement. The neuroimaging database available for this study contains MRI volumes from 56 healthy controls (see section 3. 1). For all these reasons, the task of finding subtle anomalies in PD brain scans is not the most straightforward for DL techniques.
This paper is organized as follows. Section 2 reviews state-of-the-art approaches to detect and characterize abnormal regions in MR brain scans of PD patients, as well as examines recently published DL frameworks for anomaly detection in other medical applications. Section 3 describes the datasets we used and the implemented models. The experimental setup and procedures are described in Section 4 while the obtained results are featured in Section 5. Finally, Section 6 explores the significance of our results and Section 7 summarizes the impact of this study and gives directions for future work.
However, no significant association was detected between disease severity and FA values. The first studies were carried out on small cohorts (around 50 individuals) and presented opposing results. We can notably cite the work of  Schwarz et al. , 2013  where, in contrast to the work of  Du et al. , 2011 , no differences were found in SN for FA values between PD patients and controls but a significant increase of MD in the SN (P < 0. 005) was reported. A longitudinal study that follows de novo PD patients of 35 centers for five years has been carried out by the Parkinson's Progression Markers Initiative (PPMI)  (Marek et al. , 2018). The corresponding database is openly available for researchers and contains, among other clinical and behavioral assessments, MR structural and diffusion imaging scans.
Their architecture, named AnoGAN, consisted on a convolutional GAN architecture trained with 2D patches that achieved a ROC-AUC of 89%. This work demonstrated that unsupervised learning on large-scale imaging data could lead to the detection of relevant anomalies. As presented by  Alex et al. (2017) , another possibility is to use the discriminator of the trained GAN architecture to output, for each query image, a probability map that gives an indication of the likelihood for every point of belonging to the learnt "normal" data distribution. The authors applied the proposed method to detect brain tumors from the multimodal MR images (FLAIR, T2, T1, T1 post contrast) of the BRATS dataset. The model was trained with 2D patches and obtained a DICE score of 69 % for the whole tumor segmentation task.
, 2018). Using such an anomaly detection approach for the investigation of alterations in the brain of PD patients constitutes a great challenge as early symptoms in PD have not yet been translated to identifiable characteristics in MR imaging. Regardless, in the last years some innovative methods have been devised to apply unsupervised deep learning to PD anomaly detection in MR imaging. Li et al. (2019)  set out to employ a stacked sparse autoencoder (SSAE) to discriminate between controls and PD patients from longitudinal data pooled from the PPMI database. T1 and DTI maps were collected for every subject at baseline, after 12 months and over 24 months. Gray matter, white matter and mean diffusivity features were extracted for every one of the 116 ROIs in the AAL2 atlas and serve as input for the SSAE.
The outputs of the architecture at the three time-points were classified with an SVM. The proposed method achieved a ROC AUC score of 86% at baseline and 97% at 24 months. They compared their results with a simple sAE, a CNN and a DBN architecture. The sAE achieved the second best results with a ROC AUC of 82% at baseline and 92% at 24 months. Other studies preferred to employ CNNs to discriminate de novo PD patients from HC scans. Shinde et al. (2019)  adapted the well know ResNet50 design to study 2D NMS-MRI images of the SN from a local database. They achieved a good classification performance (ROC-AUC = 90%) taking into account clinical laterality. Sivaranjini and Sujatha (2020) proposed to employ the popular AlexNet CNN to discriminate T2-weighted MR scans of HC and PD patients from the PPMI at various stages of the disease.
Data description The dataset used in this work consists of DTI MR scans coming from the PPMI database. DTI data are particularly sensitive to various sources of bias  (Jones and Cercignani, 2010). To moderate this effect, we selected DTI scans from 56 healthy controls and 129 de novo PD patients acquired with the same MR scanner model (3T Siemens Trio Tim) and configured with the same acquisition parameters. From these images, two measures, mean diffusivity (MD) and fractional anisotropy (FA), per voxel were computed using MRtrix3. 0. Values of FA and MD were normalized into the range [0, 1]. Each MRI volume was obtained in the NIfTI format as a 3dimensional, 121 × 145 × 121, array with a voxel size of 1. 5 × 1. 5 × 1. 5mm 3. Only 21% ± 1. 9% of this volume was effectively occupied by the brain and cropped images of 40 axial slices around the center of the brain were used to train the models.
In addition, we considered the right and the left hemispheres images as two different samples. The left hemisphere images were flipped to resemble the right hemisphere images. This increased the number of images available for training and allowed us to study the laterality of PD. Hemispheres were preferred over smaller regions to preserve spatial information as advised in the work of  Baur et al. (2019) , which demonstrated the usefulness of spatial AE and VAE models for anomaly detection in high-resolution MRI. Finally, all 80 individual axial hemisphere slices were cropped to a 75×145 resolution keeping the brain at the center of the images and thus ensuring consistency for our models. The control dataset was divided into 41 training controls and 15 testing controls to avoid data leakage.
As a result the models were trained with a duo (FA & MD) of 1680 images (2 hemispheres × 40 slices × 41 controls). As stated in  Poldrack et al. (2019) , it is dangerous to infer predictive accuracy based on a model that is constructed for a specific population. To assess the influence of the training and test populations on the final results, a bootstrap procedure was carried out. We divided the control dataset into training and testing samples in 10 different manners, taking special care to maintain an age average around 61 years old for all the training and test population as well as a 40-60 proportion of females and males. Table  1  shows the age and gender characteristics of the training and test sets of the ten sub-populations. Once the models were trained with one of the 10 training sets, they were evaluated with the corresponding healthy control test set and the PD dataset.
The former is referred as a dense bottleneck and the latter as a spatial bottleneck. For a VAE, the difference is that the encoder generates the parameters of the approximate posterior of the latent variable given the input, constrained to follow a multivariate normal distribution, and a sampling operation is needed to obtain an actual value for z: z ∼ f φ (x) , x = g θ (z) (2) We constructed and evaluated three autoencoder-based models: a spatial variational auto-encoder (sVAE), a spatial auto-encoder (sAE) and a dense variational autoencoder (dVAE). All models were implemented using Python 3. 6. 8, Py-Torch 1. 0. 1, CUDA 10. 0. 130 and trained on a NVIDIA GeForce RTX 2080 Ti GPU with batches of 32 images. After each convolutional layer, batch normalization  (Ioffe and Szegedy, 2015)  was applied for its regularization properties and to avoid vanishing or exploding gradients.
Evaluation procedure After training, the models were fed with the images from the test controls and PD patients datasets. As already mentioned (Table  1 ), this was done 10 times using different training sets. The idea was to identify diffusion anomalies in PD patients by observing for which brain regions the models provided an inaccurate reconstruction while providing accurate ones for healthy test controls. We expected the reconstructions of healthy subjects be superior to those of patients because the models were trained solely with healthy scans. That being said, we also expected to reveal, if any, subtle anomalies since all patients were only recently diagnosed. For this purpose we computed a joint reconstruction error map for all outputs where the value for each voxel i was calculated as: (FA i -F A i ) 2 + (MD i -MD i ) 2 (5) FA i and MD i denote the original FA and MD values while F A i and MD i , the reconstructed values.
1. We also investigated the classification performances obtained with different sub-types of patients depending their clinical history, mainly concentrating on cognitive compliance and motor symptoms. Some studies have found that at the early stages of PD, anomalies were often detected in patients that present mild cognitive impairment (MCI) versus patients without cognitive symptoms  (Tessa et al. , 2014; Zeighami et al. , 2017; Malek et al. ,   2015;  Guimarães et al. , 2018). To test for this possibility in our dataset, we divided the patients into two categories based on their MoCA test scores that reflect their cognitive skills. The patients with a score below 26 points were classified as MCI. PD patients often present unilateral motor symptoms at the early stages of the disease. In this case, we expect a brain alteration in the contralateral hemisphere.
An original evaluation procedure was designed to compare the reconstruction error profiles of healthy individuals versus PD patients. As a result, thresholds of abnormality were established for the whole brain and also for specific subcortical structures preferentially impacted by the disease. With an unbalanced dataset (56 controls vs 110 PD patients) we took special care in employing convenient metrics such as the geometric mean between specificity and sensibility, but also in generalizing our results by performing a 10-fold cross-validation to train and test our models. This turned out to be a key part in the study. As a matter of fact, for no obvious reason the first sub-population achieved much better performances than the average of our ten populations, with an sAE g-mean score associated to the whole brain of 74.
To address this need, the prospect of engaging 'intelligent' technical approaches in the context of PV has previously been identified  [8] [9] [10] [11]. Currently, modern information technology (IT) approaches investigate new ways to support PV via data integration from medical and non-medical sources, such as scientific literature, biochemical databases (e. g. platforms with multiomics data, signalling pathway analysis and chemical properties of drug molecules)  [12] , electronic health records (EHRs), insurance claims or other observational databases, search engine logs and social media  [13, 14]. While these data sources could provide a vast amount of data that could, in principle, be used for PV purposes, they also come with their own limitations and challenges regarding procedural, regulatory and technical aspects.
e. knowledge graphs built upon the Resource Description Framework (RDF) 2  and ontologies built using Web Ontology Language (OWL),  3  aiming to provide knowledge-intensive analytics adapted on the special characteristics of each data source using well-defined terminologies and ontologies as reference concept hierarchies. While a detailed presentation of the PVClinical platform is out of the scope of this paper, a preliminary design of the PVClinical platform, i. e. a first set of 'user goals'/design objectives and its main information workflow has previously been presented  [21]. In this paper, we highlight the key challenges that impinge on the wide adoption of IS approaches in the clinical setting and beyond, based on the experience of the PVClinical design process. To this end, we emphasize on the need to focus on actionability, which emerged as a top priority during this process.
'Design Thinking' is a user-centred design approach that evolves through rapid, iterative cycles of ideation, prototyping and testing that is not yet widely adopted across healthcare software designs  [23, 24]. This approach entails the active engagement of end users in the overall system design, potentially using several approaches (e. g. storytelling, interviews, use of paper prototypes etc. ). In the context of the PVClinical project design process, several personal interviews and discussions with end users (more than 25 clinicians and 5 PV professionals) were conducted. These discussions/interviews also included the demonstration of prototypes, originally in terms of static mock-ups and progressively in the form of real interactive application prototypes. Furthermore, 'think aloud' sessions were conducted where end users navigated through the provided system functionality, expressing their thoughts (difficulties, challenges, etc.
Observational Data Workspace The observational data workspace enables the statistical analysis (e. g. based on pharmacoepidemiological metrics such as 'incidence rate') and visual representation of observational data (which could come from various sources, e. g. corporate data, data from the EHRs, claim databases and other SRS sources). Technically, the clinical data workspace is based on the Observational Medical Outcomes Partnership Common Data Model (OMOP-CDM) and the overall software stack provided by the Observational Health Data Sciences and Informatics (OHDSI) 7  initiative. Fig. 3  PVClinical platform design approach. UI user interface, RDF Resource Description Network, MedDRA Medical Dictionary for Regulatory Activities, ATC Anatomical Therapeutic Classification.
They intersect with practically all the BPs elaborated on, however they can be generalized when referring to the adoption of ISs in the healthcare domain as a whole, beyond PV. • Fragmented Medical Datasets: Typically, the available datasets in a hospital refer to mostly unstructured, incomplete, semantically unaligned, and 'siloed' data among the various departments of a healthcare facility, e. g. hospital  [33]. Moreover, when external datasets are available, they most frequently lack formal and computationally exploitable semantics. This special and semantic fragmentation of the available datasets prevents them from being aggregated and their integration could, in principle, be significantly facilitated by KE approaches (e. g. via the Linked Data paradigm and the use of Semantic Web technologies).
Obviously, these considerations overlap with ethics issues. For example, the concept of consent, one of the main legal and ethical cornerstones, needs to be adapted, as getting the concept of a patient to process his/her data using ML or KE methods when he/she does not really understand how these algorithms work is pointless and ethically questionable. • Information Security: IS outcomes depend heavily on datasets, either in order to train ML algorithms or to construct computationally exploitable knowledge structures (e. g. ontologies). Thus, major issues are raised regarding databased biases and potentially malicious data management. The above challenges have already been elaborated, to some extent, in various articles  [18] [19] [20] 36]. In this paper, we describe the user-centred design approach applied in the PVClinical platform design, based on a 'design thinking' approach and 'think aloud' sessions.
Throughput and Robustness Bounds Data throughput is the most relevant metric to compare block ciphers in a Trojan-resilient setting: increased throughput enables both to speed up the testing phase (potentially improving the robustness bounds) and to encrypt data at a higher rate during the online phase. For two algorithms running on the same physical support (and in particular, relying on the same communication performances), the data throughput mainly depends on the communication complexity and the majority vote circuit  [BDFS18]. Hereunder, the performances of MOE are compared to the ones of AES and Mysterion. We first evaluate the communication complexity, which is the key factor influencing the time spent in data transfer for a single encryption. For AES and Mysterion, the communication complexity comes from the multiplicative depth of the algorithm to implement.
Acknowledgments François-Xavier Standaert is a senior research associate of the  Belgian Fund for Scientific Research (FNRS-F. R. S. ). This work has been funded in parts by the  ERC  project  724725  (acronym SWORD). Sebastian Faust was partly funded by the  German Research Foundation (DFG)  through the  Emmy Noether Program   FA 1320/1-1. This work was initiated while  Virginie Lallemand  was with the  Horst Görtz Institute for IT Security at the Ruhr-Universität Bochum  and was funded by the  DFG  through  LE 3372/4-1. Gregor Leander 's work is partially funded by the  DFG , under Germany's Excellence Strategy -EXC 2092 CASA -390781972. Finally, we thank the ToSC reviewers for their comments, as well as  Pierre Karpman  and  Gaëtan Leurent  for pointing out a flaw in an earlier version of our security arguments.
, 2010). Over the long term, droughts can lead to changes in plant species composition  (Liu et al. , 2018)  or large-scale forest mortality  (Mcdowell et al. , 2008) , sometimes causing a transient situation where large ecosystems switch from being a sink of carbon dioxide to a source  (Ciais et al. , 2005; Gatti et al. , 2015). There is a spectrum of mechanisms through which species tolerate or acclimate to drought, meaning a "one-size-fits-all" approach to modeling can be inadequate. Explicit model representations of the xylem hydraulics are complex and difficult to parameterize globally. The emergence of plant trait databases has enabled early models to represent the hydraulic properties of the soil-plant-atmosphere continuum  (Sperry et al. , 2016; Eller et al. , 2018 Eller et al. , , 2020;; De Kauwe et al.
To do this, we chose a subset of sites in the FLUXNET2015 database and from the Large Scale Biosphere-Atmosphere Experiment in Amazonia (LBA) experiment based on availability of data. Where possible we prescribed soil moisture and LAI from site measurements to differentiate the roles of SMC, β parameterization, or modeled phenology in model biases. We used the GPP calculated before soil moisture stress is applied to understand seasons and locations where the β parameterization was contributing to model errors. We also reviewed other commonly used approaches for modeling soil moisture stress, presented in Sect. 2. 2, to motivate the representations evaluated in the remainder of the paper. This work is one of the first published results from a JULES community-wide focus group (called a JULES Process Evaluation Group, or JPEG) on understand-ing soil moisture stress impacts on vegetation, which began in 2016.
Model set up and evaluation We evaluated JULES at 40 sites covering eight general biome types from the tropics to the Arctic (Fig. 3 , Table  S1 ). Each JULES simulation was run with meteorological measurements taken at each site (i. e. , point-scale runs rather than simulating the entire grid box). The meteorological and flux tower observations were obtained from the LBA Model Intercomparison Project (sites with "LBA" in the name) or FLUXNET2015 dataset  (Pastorello et al. , 2020). We selected sites with soil moisture measurements at the time of our original data request (26 July 2016). At each site, we extracted temperature, precipitation, wind speed, surface pressure, specific humidity, and longwave and shortwave radiation for running JULES at either half-hourly or hourly resolution, depending on the data available.
We then used measured LE and calculated GPP as supplied in both datasets (for the FLUXNET2015 data, these are variables LE_F_MDS and GPP_NT_VUT_REF, respectively). Details of the data preprocessing are provided in the Supplement. We individually contacted site principal investigators (PIs) to gather details on LAI; the depth of soil moisture measurements (where available); and other details on soil texture, physical properties, and root depth. Based on the responses, this resulted in a subset of 21 sites with soil moisture measurements plus the additional information necessary for prescribing soil moisture in JULES. Of these sites, 14 also had the information necessary for prescribing LAI (these are listed in Table  S1 ). Often the time period of LAI/SM mea-surements was shorter than the full record, and we only ran JULES for the time periods with the most data to avoid the need for gap-filling.
The time periods of the simulations and soil layers for prescribing data are provided in Table  S1. The default plant parameter set was taken from  Harper et al. (2016). When LAI was not prescribed, we used the JULES phenology scheme to predict LAI. This scheme predicts leaf growth and senescence based on temperature alone. Fractions of each PFT (or bare soil) present at the site were determined from the vegetation class (Tables  S1, S2 ). We calculated soil properties from information supplied by site PIs where possible; otherwise, we used the grid box sand, silt, clay fractions of the Met Office Central Ancillary Program (CAP) high-resolution input file  (Dharssi et al. , 2009)  to derive the  Brooks and Corey (1964)  parameters, along with the approximations of the parameters (via pedotransfer function) required for the soil hydraulic properties as detailed in  Cosby et al.