dataesr/scientific-paragraphs-categorization huggingface dataset:
Study design and population This study was part of the Molecular Diagnosis and Risk Stratification of Sepsis (MARS) project, a prospective observational study conducted between January 2011 and December 2013 in the mixed ICUs of two tertiary teaching hospitals in the Netherlands (Academic Medical Center, Amsterdam and University Medical Center Utrecht, Utrecht; ClinicalTrials. gov identifier NCT01905033)  [28, 29]. For the current analysis, all consecutive patients with sepsis older than 18 years of age and with an expected length of stay greater than 24 h were included via an optout consent method approved by the institutional review boards of both hospitals (IRB No. 10-056C). The plausibility of an infection was assessed daily using a four-point scale (ascending from none, possible, probable, to definite)  [28].
Our data are in agreement with findings of previous investigations based on retrospective calculation of AKI severity scores  [3, 47, 48] , using different definitions for renal recovery ranging from 24 h to hospital discharge  [7, 12, [47] [48] [49] [50] [51] , or using ICU or hospital mortality as primary outcomes  [47] [48] [49]. The various patterns of renal recovery after AKI thus far have been mainly attributed to the severity of the underlying disease or of the AKI itself  [13]. Except knowledge derived from experimental studies and postmortem observations, little is known about the pathophysiological processes responsible for the onset and persistence of kidney dysfunction during sepsis. In a subgroup of patients, we measured 16 host response biomarkers reflective of important pathways involved in the pathogenesis of sepsis, during the first 4 days after ICU admission.
Acknowledgements The authors acknowledge all members of the  Molecular Diagnosis and Risk Stratification of Sepsis consortium  for the participation in data collection and especially acknowledge:  Friso M. de Beer, M. D. ,  Lieuwe D. J. Bos, Ph. D. ,  Gerie J. Glas ,  M. D. ,  Roosmarijn T. M. van Hooijdonk, M. D. , Ph. D. ,  Janneke Horn, M. D. , Ph. D. , Laura R. A. Schouten ,  M. D. ,  Marleen Straat, M. D. ,  Luuk Wieske, M. D. , Ph. D. ,  Esther Witteveen , M. D. ,  Department of Intensive Care ,  Academic Medical Center, University of Amsterdam , The Netherlands;  Arie J. Hoogendijk, PhD ,  Mischa A. Huson ,  M. D. ,  Maryse A. Wiewel ,  M. D. ,  Center for Experimental and Molecular Medicine, Amsterdam University Medical Centers , location  Academic Medical Center ,  University of Amsterdam, Amsterdam, The Netherlands , and  Center of Infection and Immunity Amsterdam (CINIMA) ,  Amsterdam University Medical Centers , location  Academic Medical Center ,  University of Amsterdam, Amsterdam, The Netherlands ;  Peter M.
The ICAReB platform has been established in 2008 to provide for access of pasteurian research teams and beyond, to high-quality human bio-resources. Its main fields of interest are infectious diseases, immunology and more recently neurosciences. Two activities were developped, hence its name ICAReB, both certified following the ISO 9001 and NF S96-900 standards, respectively. More than 134,000 samples and their associated bio-clinical data (including vaccination status, ethnicity, medical history such as chronic inflammatory diseases and allergies) are currently collected, as well as environmental factors (lifestyle, international travels, microbial environment, dietary habits, exposition to stress and pollutants …) The Diagmicoll main cohort (Clinical trials reference: NTC03912246) affords bio-resources from healthy subjects primarily, for the National Reference Centers, so as to optimize and develop new diagnostic methods  [1, 3, 4, 5, 8, 10, 13, 20, 21, 28].
Steps Investigation methods The main asset of our platform is represented by two large volunteers' cohorts. The healthy individuals are recruited during ambulatory clinical consultations organized at the ICAReB platform by our staff. Those volunteers actively participate in research protocols on human beings (including pathophysiology, immunology, genetics, epide-miology…), and their samples and associated data presently compose the biorepository available for research purposes. The Diagmicoll cohort includes approximatively 250 volunteers in the active file whereas the CoSImmGEn cohort is comprised of around 180 volunteers regularly followed up, from which genetic and environmental data are available. Besides the recurrent work with pasteurian teams, ICAReB also builds partnerships with numerous external public and private institutions (research teams, hospitalbased staffs and clinical investigation centres, other bioresources centres, and companies) and is also developing a collaborative framework with European and international organizations.
Data management The ICAReB platform uses a BRC-dedicated LIMS for the bioresources management (MBioLIMS, ModulBio, Marseille France). In addition, in-house bioclinical databases have been constructed within a pre-formated, web-based framework (Voozano, Epiconcept company, Paris). More recently we have introduced an alternative web-based framework (RedCap, Vanderbuilt university, Nashville, USA) hosted by the informatic center, IP. Only authorized staff members can manage the bioresources, samples and data.
Our aim was to determine whether the application of advanced machine learning methods on neural data revealed states that had reliable associations with an individual's self-reported experience during that period. In our study, therefore, we measured neural function during wakeful rest in a large cohort of healthy individuals using functional magnetic resonance imaging (fMRI). These participants completed in the scanner a set of questions related to their experiences during the scan, and a subset completed questionnaire measures of physical and mental health at a later session. We applied hidden Markov modelling (HMM) to the neural data to reveal distinct brain states corresponding to specific periods of temporally reoccurring patterns of neural mean activity and functional connectivity  19.
To that end, our study aimed to examine the possibility that these hierarchies represent a functional landscape that describes how organised transient neurocognitive states emerge. The study workflow is presented in Fig. 1. In a subset of the individuals, we also acquired measures of well-being. We then reduced the dimensionality of both data sets and applied hidden Markov modelling to the neuroimaging time series. We performed a multiple regression analysis in which we calculated the association between the mean dwell-time of each state to the patterns of experiences calculated at rest, as well as to trait measures of well-being. Finally, we calculated the association of the identified brain network states to the first three neurocognitive hierarchies identified by  20  , as well as to a distribution of synthetic states generated through permutation.
Results Identifying reoccurring neural states at rest. We applied hidden Markov modelling to the resting state fMRI data (see Methods). This method uses Bayesian inference to identify reoccurring patterns of brain network activity; see  19, 26  for an empirical validation of this approach. Here, we generated HMM solutions capturing mean activity and functional connectivity patterns within the neural data recorded from our sample. We opted for a 7-state solution, however, our results highlighting associations with behaviour remained broadly similar for a decomposition of 9 states as well. We ran the algorithm 10 times in each case and found that the 7-state solution produced the same decomposition across all iterations. For this reason, we did not ask for a solution with a smaller number of states, so as not to unnecessarily reduce the algorithm's ability to provide a detailed description of the dataset.
5b. The contour plot and histogram on the left show where the real data fall on a subset of the dimensions, while the three-dimensional scatter plot show the same data in a combined manner. In these plots, the null distribution generated through permutation is represented as the shaded area in grey. We also generated a video that shows the data in a 3-d form (https ://githu b. com/tkara pan/hmm-large -scale -hiera rchie s/blob/maste r/7S_Null. mp4). The synthetic states clustered towards the middle of the three dimensions, indicating that states generated in a random manner tend to not be very similar in spatial terms to the three pre-established neurocognitive hierarchies  20. In contrast, the states generated based on real data fall, on average, away from the centre and towards the outer edge of the distribution of synthetic states.
Methods and materials Participants. 277 healthy participants were recruited from the University of York. Informed consent was obtained for all participants, the study was approved by the York Neuroimaging Centre Ethics Committee, and all research was performed in accordance with relevant guidelines and regulations. 21 participants were excluded from analyses, 1 due to technical issues during the neuroimaging data acquisition and 20 for excessive movement during the fMRI scan (mean framewise displacement 61 > 0. 3 mm and/or more than 15% of their data affected by motion), resulting in a final cohort of n = 256 (169 females, µ age = 20. 7 years, σ age = 2. 4). Behavioural methods. We sampled participants' experience during the resting state fMRI scan by asking them to retrospectively report their thoughts at the end of the scan.
Experience was measured using a 4-scale Likert scale, with the question order randomised (all 25 questions are shown in Table  S1 ). In a subset (n = 168) of the final cohort, we also assessed their physical and mental health by administering well-established questionnaire measures at a later separate session outside of the scanner. Details about each questionnaire are presented in SI. MRI data acquisition. MRI data were acquired on a GE 3 Tesla Signa Excite HDxMRI scanner, equipped with an eight-channel phased array head coil at York Neuroimaging Centre, University of York. For each participant, we acquired a sagittal isotropic 3D fast spoiled gradient-recalled echo T1-weighted structural scan (TR = 7. 8 ms, TE = minimum full, flip angle = 20 • , matrix = 256 × 256, voxel size = 1. 13 × 1.
13 × 1 mm 3 , FOV = 289 × 289 mm 2 ). Resting-state functional MRI data based on blood oxygen level-dependent (BOLD) contrast images with fat saturation were acquired using a gradient single-shot echo-planar imaging sequence with the following parameters; TE = minimum full ( ≈ 19 ms), flip angle = 90 • , matrix = 64 × 64, FOV = 192 × 192 mm 2 , voxel size = 3 × 3 × 3 mm 3 , TR = 3000 ms, 60 axial slices with no gap and slice thickness of 3 mm. Scan duration was 9 minutes, which allowed us to collect 180 whole-brain volumes. fMRI data pre-processing. Functional MRI data pre-processing was performed using SPM12 (http://www. fil. ion. ucl. ac. uk/spm) and the CONN toolbox (v. 18b) (https ://www. nitrc. org/proje cts/conn)  62  implemented in Matlab (R2018a) (https ://uk. mathw orks. com/produ cts/matla b).
Dimensionality reduction. In order to reduce the dimensionality structure of the neuroimaging data, we used the 17 functional network parcellation from  30  , slightly eroded the parcels to avoid signal leakage from neighbouring parcels, masked them with subject specific grey matter masks, and calculated the average signal from the unsmoothed data within each parcel for each volume per participant. The acquired time series were then standardised and concatenated to form a "(256 participants × 180 volumes) × 17 parcels" matrix that was used as input to the HMM algorithm. For the behavioural data, we applied a principal component analysis (PCA) with varimax rotation to the scores describing the participants' experience at the end of the resting state scan. 7] [68]  ), using this kind of decomposition and rotation improves the identification and interpretation of distinct patterns of covariance in our multi-dimensional datasets.
Acknowledgements JS was supported by  European Research Council  ( WANDERINGMINDS -646927 ). RL received support from the  Wellcome/EPSRC Centre for Medical Engineering  (Ref:  WT 203148/Z/16/Z ) and would also like to acknowledge support from the Data to  Early Diagnosis  and  Precision Medicine Industrial Strategy Challenge Fund, UK Research and Innovation (UKRI). The work was also part-funded by a  European Research Council  grant to EJ ( FLEXSEM -771863 ). The authors would like to thank  Mladen Sormaz ,  Charlotte Murphy  and  Hao-Ting Wang  for their contribution to data acquisition.
grounded with the same references as databases (e. g. sequence banks, disease control records), improving data interoperability. Normalization can be seen as a multi-class classification problem where the entities are the examples to be classified and the categories or concepts are the classes. The reference vocabularies can be formal semantic resources such as ontologies  [1, 2]. An ontology concept can be labelled by several terms (e. g. synonyms). Each concept can be used as a semantic reference for textual entity mentions. For instance, the concept identified by OBT:001623 in the OntoBiotope ontology  [3, 4] , and labelled by the term "lymphocyte", could normalize textual entity mentions such as "lymphocytes", "lymphocytic", "t-cell" or "monoclonal B cells". In specialized domains, this classification problem is frequently characterized by a large number of classes but few manually annotated training examples.
The method uses word embeddings as input, which overcomes the limitation of TF-IDF bag-of-words representations. CON-TES uses the ordinary least squares method to estimate the linear projection parameters, which struggles to scale up on large training datasets and large ontologies. In the general domain, the state-of-the-art methods to address entity normalization in that way are large neural nets  [28]. The reason may hold in their capacity to model hard nonlinear phenomena. From a computational point of view, neural networks have the asset of being online algorithms. Nevertheless, large neural nets seem to have difficulty addressing the few-shot learning problem. In fact, in specialized domains, shallow convolutional neural networks (CNNs) have been used with some success in normalization tasks with small training data.
Bacteria Biotope dataset and task We used the BB-norm dataset of the Bacteria Biotope 2019 Task  [33]  to evaluate our method. Bacteria Biotope 2019 is part of BioNLP Open Shared Task 2019  [34]  and consists of several information extraction subtasks in the microbiology domain. The BB-norm subtask aims at normalizing Microorganism, Habitat, and Phenotype entities using taxonomies and ontologies. In our study, we focused on the normalization of Habitat and Phenotype entities. We chose to focus on these entities because they are expressed with noun and adjectival phrases that can take varied and complex forms. In contrast, microorganisms are mostly expressed in a codified way, based on proper nouns including a few orthographical variations. This type of named entities is typically easier to predict and classic rule-based methods usually yield high performance.
Habitat and Phenotype entities have to be mapped to concepts of the OntoBiotope ontology  [4]. The ontology consists of two main branches, the microbial habitat branch, which contains 3172 concepts and is used to normalize Habitat entities, and the microbial phenotype branch, which contains 429 concepts and is used to normalize Phenotype entities. The dataset is divided into a training, a development and a test set. Habitat and Phenotype entity annotations are provided in all sets and normalization annotations are provided in the training and development sets only. Figure  1  shows an example of the normalization task for Habitat and Phenotype entities. Each entity mention in the example text is mapped to an ontology concept. Statistics of the dataset are given in Table  1. Its size is rather small, especially the Phenotype part of the dataset, in comparison with the size of the ontology.
The main consequence is that some ontology concepts have few examples in the training data (if any). Indeed, the average number of examples per concept of the ontology is 0. 6 with a standard deviation of 4. 3 for habitats, and 1. 2 with a standard deviation of 6. 6. 4 with a standard deviation of 12. 3 for habitats, and 6. 0 with a standard deviation of 13. 5 for phenotypes, which is still very low for a classification task. Thus, the Bacteria Biotope corpus is a good example of the few-shot learning problem. Moreover, the task is also a good example of the zero-shot learning problem, since the proportion of concepts for which there is no example at all is quite high: 92. 7% for habitats and 84. 4% for phenotypes.
Overview of the method Our approach draws on findings from previous work, in particular that of Ferré et al. [27], Deng et al. [29]  and Limsopatham and Collier  [30]. We observe from Ferré et al. [35]  that the CONTES method is able to predict concepts that are relatively close to the target concepts in the semantic space, but has difficulty pinpointing the exact ones. We come to this conclusion by looking at the high non-strict performance score (where close matches are taken into account) obtained by CONTES in comparison with its lower strict performance score (where an exact match is required). In contrast, shallow CNNs obtain a good overall score on datasets with strict evaluation  [30] , and a state-of-the-art score on datasets with non-strict evaluation  [29] , suggesting that they can make a significant number of exact predictions, but when they fail, the prediction is often semantically very far from the correct one in the ontology graph.
Word embeddings We perform sentence and word segmentation on the corpus used to train Word2Vec, and to avoid out-of-vocabulary words, we apply the same word segmenter (see Availability of data and materials) on the labels of ontology concepts and on the BB-norm dataset. To obtain word embeddings adapted to the microbiology domain tackled by the BB-norm task, we use a dedicated corpus derived from PubMed, the same as in Ferré et al. [36]. In all these resources, we filtered stop-words (non-content words such as prepositions and determiners). All the initial embeddings have then been normalized to unit length.
Component methods: single layer feedforward neural network and shallow CNN Single Layer Feedforward Neural Network (SLFNN) Inspired by the CONTES method  [27] , the SLFNN aims to find a linear projection which globally aligns textual mention embeddings with the vectors of the concepts normalizing them. Embeddings of the words composing the mentions are averaged to obtain mention embeddings, which are then given to the dense layer of the SLFNN. We replaced the ordinary least squares method used in the original CONTES by a neural method to learn the projection. It allows the SLFNN to scale up to large training datasets and simplifies its combination with other neural architectures. The architecture of the SLFNN is shown in Fig. 2. Shallow CNN the shallow CNN we used is inspired by Limsopatham and Collier  [30] , with no final softmax layer and with concept representations as in Ferré et al.
Training sets To offset the small size of the manually annotated datasets, we followed a weak supervision strategy to increase the size of the training set with non-manually annotated examples. We used the labels of the ontology concepts as examples of normalized mentions, i. e. each concept label is handled as an entity mention normalized with that same Fig. 5  The C-Norm architecture combining the SLFNN and shallow CNN components. The input is the matrix of word embeddings of the tokens of the mentions concept. This strategy has shown good results in previous work on the 2016 edition of the Bacteria Biotope task  [36]. When tuning the models, we trained on the manually annotated training set augmented with ontology labels and used the development set as validation data. When evaluating on the test set, we also added the development set to the training data.
Most hyperparameters were chosen empirically by training multiple runs of the models for Habitat normalization and were re-used for models normalizing Phenotypes. We did so because the Phenotype part of the dataset is much smaller than the Habitat part and thus it is harder to observe clear tendencies on this dataset. For all models, we chose the Nadam optimizer  [37]  and the log_cosh loss function, after experimenting with a number of options-Stochastic Gradient Descent, Adam  [38] , Adadelta  [38] , Amsgrad  [39]  and Nadam  [37]  for the optimizer and mse, cos, huber, mae, and log_cosh for the loss function. The advantage of the log_cosh function, besides showing good performance (either the highest or similar performance compared to other loss functions), was to yield more stable results across repeated runs.
The strict exact match score is equal to 1. 0 if the predicted concept is equal to the reference concept, 0. 0 otherwise. This score can be relatively harsh for tasks where the references are semantically related (e. g. two semantically close subsumed concepts). Therefore, the BB-norm task uses a smoother similarity score based on the semantic distance between two concepts in an ontology, as defined by Wang et al. [41]. For a prediction, this similarity score is calculated between the reference concept and the predicted concept, having the value of 1. 0 if both concepts are equal, and tending towards 0. 0 if the concepts are farther in the hierarchical graph of the ontology. The overall score of a dataset prediction is the average of all scores for each mention. We compared our approach to those of the official participants of the BB-norm 2019 shared task and to the baseline provided by the task organizers.
Performance of the different methods Table  2  shows performance obtained on the BB-norm development sets for each neural network component and each combination architecture. Results are averaged over 10 runs to account for variations in the initialization of the neural networks and standard deviation is provided. The shallow CNN component performed better than the single-layer feedforward NN for Habitats. The SLFNN performed almost 20 points lower in terms of strict score, but its similarity score remains quite high (0. 654 vs. 0. 696). This suggests that the two methods are complementary: the S-CNN does better at pinpointing the exact concept, but the SLFNN is able to position the mentions quite well in the concept vector space (which explains the high Wang score). On the Phenotype normalization task, the two methods obtained similar results measured by both scores, the SLFNN performing slightly higher, although the very small size of the Phenotype dataset makes it hard to draw any conclusion.
Evaluation results Table  4  gives a summary of the characteristics of the methods evaluated on the test set of the BB-norm task, including the baseline, methods from official teams who participated in the BB-norm shared task (AmritaCen Healthcare, BOUN-ISIK, BLAIR GMU and PADIA BacReader), methods having obtained high performance on a similar dataset (CONTES and HONOR), and our best performing method C-Norm. Performance of the methods is shown in Table  5. C-Norm significantly outperforms all existing methods according to both measures, non-strict evaluation (Wang similarity score) and strict evaluation (strict exact score): respectively + 4 and + 7 points compared to the best approach (HONOR) for Habitat normalization, and + 9 points (for both scores) compared to the best approach (CONTES) for Phenotype normalization.