dataesr/scientific-paragraphs-categorization huggingface dataset:
Q. is funded by  Post-Doctoral Guarantors of Brain Clinical Fellowship. The  EU-GEI  Project is funded by the  European Community 's  Seventh Framework Programme  under grant agreement No. HEALTH-F2-2010-241909  (Project  EU-GEI ). The Brazilian study was funded by the  São Paulo Research Foundation  under grant number  2012/0417-0. J. B. K. was supported by  National Institute for Health Research, University College London Hospital, Biomedical Research Centre. The views expressed in this publication are those of the authors and not necessarily those of the  NHS , the  National Institute for Health Research  or the  Department of Health and Social Care. Funders were not involved in the design and conduct of the study; collection, management, analysis and interpretation of the data; preparation, review or approval of the manuscript and decision to submit the manuscript for publication.
Background UK Biobank (UKB) is a prospective population-based cohort study with extensive phenotypic and genotypic information on > 500,000 participants (www. ukbio bank. ac. uk). It is an open access resource, established to facilitate research into the determinants of a wide range of health outcomes  [1]. Disease outcomes are ascertained primarily via linkages to routinely collected coded national administrative health datasets  [2] , enabling the identification of a broad range of disease phenotypes with sufficient accuracy for many research studies  [2] [3] [4]. However, these coded data are often incomplete and less accurate when it comes to identifying specific disease subtypes  [2, 3]. For example, up to 40% of participants with a stroke code in hospital, death Open Access *Correspondence: kristiina.
Similarly, while coded data can be used to identify all-cause dementia, accuracy in identifying dementia subtypes, in particular vascular dementia, is much lower  [4, 5]. This may be a limitation for researchers studying genetic and environmental associations specific to disease subtypes, and hence automated, scalable methods are urgently needed to improve disease subtyping. Possible solutions to enhance the accuracy of coded data and improve the ability to deep-phenotype (e. g. subtype) all participants at scale include linkage to national disease-specific audit and registry datasets and/or the development of automated tools to extract data from participants' detailed electronic medical records (EMR). While linkage to disease-specific datasets is promising, these data are limited to select diseases, may not cover all regions or nations of the UK, may cover limited time periods, do not always capture primary care and outpatient as well as inpatient encounters, and may have unknown accuracy.
For example, inferring that the combination of the two entities "bleeding" and "intracerebral" signifies intracerebral hemorrhage (ICH) requires clinical knowledge. While deep learning methods have great potential to learn such associations, large datasets would be required to train them. At the same time, many disease subtypes are rare by nature, which is a limitation for supervised learning. By combining natural language processing (NLP) with clinical knowledge inference, this work aimed to investigate the feasibility and added value of automated methods applied to clinical radiology reports in ascertaining accurate disease subtype information for participants with any stroke code in a regional UKB subpopulation. We used stroke as an exemplar disease, specifically looking to improve hemorrhagic stroke identification.
Study population We conducted the study in a sub-population of 17,249 UKB participants in the Lothian region of southeast Scotland. All participants' records were linked to national administrative health datasets, providing hospital, death record and primary care administrative coded data. Within this UKB sub-population, we identified participants with ≥ 1 stroke code in their linked health data, indicating a stroke diagnosis after their recruitment to UKB. ICD-10 and Read v2 codes that were used to identify stroke cases are available in Additional file 1: Table  S1. The follow-up period was from the participant's date of recruitment up to the end of September 2015, the date at which data were complete for all sources at the time of this study. Clinicians screened the participants' EMR and extracted all clinical brain scan reports (MRI, CT, CTA, MRA, DSA) available relating to the respective codes (Fig.
Automated methods applied to clinical brain scan reports to derive stroke subtypes The automated method pipeline was composed of five steps (detailed in Fig. 2 ), including NLP (named entity recognition followed by machine learning) followed by applying clinical knowledge inference. We used an offthe-shelf tool-SemEHR  [10] , developed and trained on EMR in UK National Health Service (NHS) Trusts in London, and further extended on Scottish imaging datasets  [11]. We derived entity labels for each scan report (full list is available in the first column of Table  3  in the "Results" section). Steps 1-4 in Fig. 2  illustrate the process of identifying relevant entity labels for each scan report. Essentially, it comprised two phases. First, it used a baseline NLP model from SemEHR to obtain an initial set of annotation results, which were contextualized (positive/ negated/hypothetical/history mention/not a phenotype) mentions of Unified Medical Language System [12] concepts (detailed in  [10] ).
2 , full list of rules in Table  1 ). If a participant had more than one scan report, and the inferred diagnostic stroke subtype labels were different across these reports, the participant was classified into all inferred stroke subtype categories for the subsequent analyses. Such rules encode clinical knowledge to infer participant-level stroke subtypes in a computable format and are directly reusable in new settings. Although we considered the more common reasons for a false-positive stroke/stroke subtype diagnosis in our specific dataset when developing the domain-expert rules, the underlying principles are not unique to this dataset and hence we expect them to be generalizable across other stroke datasets. The methods and rules are publicly available  [13].
Precision and recall of automated methods in assigning a stroke subtype among all UKB participants with any stroke code For automated stroke subtype diagnoses at a participant level, we calculated precision (or positive predictive value, i. e. the percentage of participants allocated a particular stroke subtype by the automated method who were true positives for that subtype) and recall (or sensitivity, i. e. the percentage of participants who truly had a particular stroke subtype detected by the automated method). We used stroke physician adjudications derived based on the participants' complete EMR (including, but not limited to the clinical brain scan reports) as the ground-truth. We calculated 95% confidence intervals (CIs) using the exact Clopper-Pearson method in StatsDirect  [14]. We compared the results of the automated method to the results of using administrative disease codes alone based on our earlier work  [2].
Discussion Our results demonstrate the potential for significant added value and feasibility of using automated methods on clinical brain scan reports to improve stroke subtyping in UKB. While the automated method assigned a correct stroke subtype diagnosis to only 55% cases overall, its main benefit came from markedly improving the precision of hemorrhagic stroke codes. As expected, ischemic stroke code accuracy remained similar. This approach of combining NLP and clinical knowledge inference is potentially scalable across the UK and may also scale well in other settings. It may also be relevant to disease subtyping for other conditions, where information from images is important in the diagnosis of disease subtypes. Furthermore, the SemEHR tool used in this project can be easily adapted for research into other phenotypes by adopting transfer learning technologies  [15].
The dataset used for the precision and recall calculation from automated method in this study includes a total of 207 participants with a stroke code. The 207 are a subset of the 225 participants with a stroke code who also had a relevant clinical brain scan report available. 18 participants among the 225 participants did not have a brain scan available and were hence excluded from this study   Compared to coded data alone, for hemorrhagic stroke, the automated method improved precision at the expense of slightly poorer recall. Depending on the study design, more importance can be attributed to either estimate, however our achieved trade-off is likely to be preferrable for many research studies. For ischemic stroke, the effect was the opposite, resulting in a lower precision at the expense of improved recall.
However, to date most of the work has focused on named entity recognition tasks, such as semantics in domain terminologies (e. g. ontology-driven inferences)  [10, 28]  and identification of contextual mentions (e. g. negation, temporality and the person to whom the information refers to)  [15, 18]. Very few studies  [17]  have investigated methods to help derive disease sub-phenotypes from free text, where the information to derive these exists but additional clinical knowledge is needed to derive it. Our work addressed this gap by combining NLP with clinical knowledge inference. Advantages of this approach are that it does not require the very large datasets required to train machine learning methods, along with the potential both to transfer knowledge to new datasets from external sources and to apply the approach in other languages.
This is currently a relatively understudied area, with very few sharable resources available. Previous studies applying NLP and machine learning to classify stroke into subtypes have focused on automating ischemic stroke subtyping into specific sub-categories using the EMR  [30, 31]  or a selection of available features  [32]. Others, such as the Edinburgh Information Extraction for Radiology reports (EdIE-R)  [33]  have shown good performance of text mining systems in subtyping already expert-validated stroke cases into the three main subtypes (IS, ICH and SAH) based on radiology scan reports. Our study differs from these in two main ways. Firstly, it is nested in a population-based cohort study rather than a disease specific cohort. We combine existing information from national administrative health datasets with automated methods by identifying participants with a high prior probability of having had a truepositive stroke diagnosis (represented by them having a stroke code in the administrative data) followed by the application of automated methods to subtype stroke into the three main types (IS, ICH and SAH).
This approach means that the results are applicable to other populationbased studies and large biobanks using administrative data for disease identification (e. g. the UK-based Generation Scotland  [34]  study and SAIL Dataset  [35] ). Secondly, we use expert stroke physician adjudications based on the complete EMR to derive ground truth diagnoses. This step is important, since while in a large number of cases the correct hemorrhagic stroke subtype diagnosis can be reached by the expert based only on the brain scan report, in a proportion of cases, additional information from the complete EMR is required in addition. One example of this would be a case where a patient's brain scan report describes a brain hemorrhage, which could be secondary to head injury (i. e. a traumatic hemorrhage, not a stroke) or it could be a primary hemorrhage (i.
e. a stroke), and additional medical history regarding any mention of a relevant traumatic event prior to symptom onset in the EMR will help make the correct final diagnosis. We are not aware of any previous studies combining these two features in order to automate stroke subtyping. Our results show that in large population-based cohorts, the ascertainment of cases via codes indicating stroke combined with subsequent automated methods applied to the free text of brain scan reports is a feasible and potentially scalable approach for enhancing the accuracy of stroke subtyping. Our primary approach was to first identify participants with a high prior probability of having had a stroke of any subtype (defined as participants with any stroke code in administrative datasets) and then apply automated methods to enhance the accuracy of specific subtype diagnosis (IS vs ICH vs SAH).
We also explored the benefit of identifying participants with a high prior probability of having had a hemorrhagic stroke subtype (defined as participants with a hemorrhagic stroke specific code in administrative datasets) before applying automated methods. This improved the precision of SAH subtyping slightly, but would need to be validated in larger datasets. The strengths of our study include the application and testing of existing methods on a real-world dataset. In addition, we tested the performance of the methods against robust ground-truth diagnoses made by specialist physicians based on the complete EMR. To maximize the reusability of our work, we deliberately decoupled the NLP component from the clinical knowledge inference component in our pipeline, so that the latter can be reused in different settings.
We have also made the model and inference rules publicly available  [13]  to facilitate future similar studies by others. The imaging reports however are currently only available for UKB sub-cohorts via individual data linkage projects. Our study also has some limitations. The relatively wide confidence intervals for precision and recall suggest a high variability of these estimates, which could be due to the small sample size and heterogenicity of the sample, particularly for the SAH and ICH cases. Also, this work didn't have a replication or an external validation cohort for evaluating the pipeline. Furthermore, in our study, we included only participants' first-ever stroke events and their relevant clinical brain scans were selected manually by experts, whereas this step would also need to be automated to make the approach scalable in large datasets.
We envisage this may involve including all brain scans within a certain timeframe from the stroke code. Finally, we did not apply a rule-based approach to tackle the issue of some participants having multiple brain scans with competing disease subtypes. Developing methods to address this may improve the performance of automated methods further. Further work to build on these results is now needed and should focus on: (1) validating our automated methods in further datasets, which could include additional UK Biobank sub-cohorts as well as data from other population-based cohorts;  (2)  investigating the time interval between the code(s) and clinical scan reports to enable inclusion of the most relevant data; (3) investigating the usefulness of the automated methods in identifying recurrent stroke events; (4) developing rules for disease subtype adjudication based on multiple reports per participant; and (5) expanding this work to investigate disease subtyping of other conditions beyond stroke.
Acknowledgements V. H. N. and A. J. contributed equally to this work. A. H. A. would like to acknowledge  Prince Sattam bin Abdulaziz University, Alkharj ,  Saudi Arabia , for financial support. D. M. -R. acknowledges support from the  European Union 's  Horizon 2020 FETOPEN-1-2016-2017 research and innovation program  under Grant Agreement 801464 and through the  Marie Curie Actions  ( FP7/2007-2013 , Grant Agreement No. 63111 ). The  Agence Nationale de la Recherche  ( ANR, France)  is also acknowledged for funding via the programs  ANR-16-CE05-0021  (DESPATCH). V. H. N acknowledges a PhD grant from '' ARC Energies Auvergne-Rhône Alpes ''. This project was also financially supported by '' Carnot Energies du Futur''  ( ALDASH  project). K. P. M. acknowledges funding from the  NSERC   Discovery Program  ( RGPIN-2017-04212 ,  RGPAS-2017-507977 ),  Canada Foundation for Innovation John R.
Environmental samples In order to compare the protist diversity in the main ecosystems that compose the earth biosphere, we investigated the diversity of marine plankton, freshwater plankton and soil. Our meta-analysis took advantage of published data as well as new samples to complete a dataset composed of 122 sampling sites: 28 marine plankton, 21 freshwater plankton and 73 soil samples (Table  G. 1-2). Marine plankton samples were retrieved from the Tara Oceans project (http://taraoceans. sb -roscoff. fr/EukDiv/) and covered two depths (surface and Deep-Chlorophyll Maximum: DCM) from the world tropical to temperate oceans. For each sample, 80 to 100 L (until filters were clogged with biomass) of seawater were collected and the plankton was deposited on 0. 8 µm mesh size filters. The filter membranes were preserved into liquid nitrogen until DNA extraction  (de Vargas et al.
Generation of protist operational taxonomic units (OTUs) Environmental rRNA SSU V9 reads from all samples were first merged with the program flash v1. 2. 11  (Magoc and Salzberg, 2011)  and trimmed to remove the eukaryotic primers with a custom script (https ://github. com/cseppey/bin_src_my_prog/blob/master/perl/trim_prime r. pl). Sequences from all samples were then dereplicated with a custom script (https://github. com/cseppey/bin_src_my_prog/blob/master/ cpp/derep. cpp) prior to OTU clustering performed with swarm v2. 1. 13 with the fastidious option  (Mahé et al. , 2015). OTUs were finally assigned using the global alignment option on vsearch v2. 4. 4  (Rognes et al. , 2016)  against the PR 2 database v4. 12. 0  (Guillou et al. , 2012). Chimerical sequences were removed by comparing the reads within the dataset with the program vsearch v2.
4. 4  (Rognes et al. , 2016). Certain OTUs were then sorted out according to three criteria, as originating from putative non-protists and macroscopic organisms. 1) OTUs belonging to Metazoa, Embryophyceae, Fungi and Rhodophyta were removed, as well as 2) the ones having a sequence with a length below 100 nucleotides, and 3) the ones having a percentage of identity below 60% with the database as putative prokaryotes.
2. 5-2  (Oksanen et al. , 2018) ). We also estimated the α diversity (Shannon index, function diversity, package vegan v. 2. 5-2,  (Oksanen et al. , 2018) ) of each of the 55 major lineages that represent the bulk of the known protist diversity in each type of ecosystem, respectively, following the same procedure as described above. We assessed the similarity patterns among protist communities (β-diversity) by non-metric multidimensional scaling (NMDS). NMDS was calculated on Bray-Curtis dissimilarities retrieved from the sequence relative abundance. The significance of differences between pairs of ecosystems were measured by permutation tests (10 ′ 000 permutations; functions vegdist, metaMDS and envfit, package vegan v. 2. 5-2  (Oksanen et al. , 2018) ). To estimate which ecosystems were showing the highest β diversity, we compared the distributions of Bray-Curtis distances within ecosystem.
We finally tested for differences among ecosystems for α and β diversity indices by pairwise tests for multiple comparison of mean rank sums (Nemenyi test, P < 0. 05; function posthoc. kruskal. nemenyi. test, package PMCMR v. 4. 3  (Pohlert, 2014) ) in each bootstrap. We also use this approach to test the differences between ecosystems for the log of the number of reads, the log of the richness and the Shannon diversity within each of the 55 dominant taxonomic groups, as well as for the functional groups relative abundance.
Dataset consistency This study is, to a certain extent, a meta-analysis because part of the data (concretely, the marine data) was derived from other studies. Thus, most samples were obtained with distinct methods. Thus, despite targeting the same rRNA region (V9) and using the same sequencing technology (Illumina sequencing) the sampling and laboratory procedures used were not homogeneous, thus potentially producing biases. We therefore tested how robust our data was to methodological biases, including (1) different filtration and sample preservation protocols (2) use of different extraction kits and (3) use of different primer sets. (1). Pore filter sizes varied between freshwater and marine samples. In both cases, filter mesh sizes are below 1 μm, which corresponds to the size of the smallest eukaryotes known, such as for instance Ostreococcus tauri  (Chrétiennot-Dinet et al.
25 ± 0. 22 SE and 5. 05 ± 0. 08 SE) (Fig. 1B , Table  H. 1). In line, 20. 67 ± 0. 28 out of 55 broad taxonomic groups targeted showed significantly higher α diversity in soils while 9. 15 ± 0. 8 were more diverse in marine samples and 0. 90 ± 0. 05 were more diverse in freshwater (Fig. B. 2). This probably higher α diversity in soils followed our prediction based on the co-existence of several dormant and active organisms that differ in their ecological requirements but reactivate under suitable conditions. The high α and γ diversity in soils contrast with the low number of described soil protist species (ca. 21 ′ 000 species  (Mora et al. , 2011) ), while estimations suggest the existence of millions  (Geisen et al. , 2018). We compared the level of knowledge for all three ecosystems based on the percentage of identity between the OTU reads and the best match with the PR 2 eukaryotic ribosomal database  (Guillou et al.
Conclusions A key result of our study is that soils host the richest protist communities on Earth. Thus, while all ecosystems contain a wealth of undescribed protist species, this is especially true in soils. In line with our predictions, spatial and ecological heterogeneity are correlated with diversity, as shown by the higher β diversity observed in freshwater and soil environments. This calls for an increased exploration of less prospected ecosystem types to find new diversity followed by the expansion of the existing curated and annotated databases such as UniEuk  (Berney et al. , 2017). Our estimations for functional diversity were only partially fulfilled. On one hand, our study underlines the importance of phagocytosis through the relative abundance and diversity of consumers in all ecosystems, and especially in soils.
2012) (Fig. C. 1, TableH. 2). Soil OTUs had, on average, a significantly lower percentage of similarity with known sequences (87. 28% ± 0. 08 SE) than freshwater systems OTUs (90. 56% ± 0. 08 SE), with the best percentage of identity with the database for the OTUs found in marine ecosystems (91. 39% ± 0. 09 SE, Nemenyi test: P < 0. 05).
Acknowledgements This work was supported by the  Swiss National Science Foundation   31003A_143960 & 31003A_163254  to E. L. ,  P2NEP3-178543  to D. S. and  31003A_182531  to E. A. D. M. ; internal funding of the  University of Neuchâtel  to E. A. D. M. ; the 'Atracción de talentos' from the  Community of Madrid , project  2017-T1/AMB-5210  and the project  MYXOTROPIC VI  ( PGC2018-094660-B-I00 ) from the  Spanish Government  to E. L. ;  NWO-VENI  grant from the  Netherlands Organisation for Scientific Research  ( 016. Veni. 181. 078 ) to S. G. ; the  Norwegian Research Council  projects  270252  ( BiodivERsA-Climate change impacts on Arctic soil and lake microbiomes ) and 256132 (ERAnet-LAC,  METHAnogenic Biodiversity and activity in Arctic and Sub-Antarctic Ecosystems affected by climate change ) to C.
V. W. S. ; the  French Government 'Investissements d'Avenir' program OCEANOMICS  ( ANR-11-BTBR-0008 ) to C. d. V. , and the  Gordon and Betty Moore Foundation  through grant  GBMF5257   UniEuk. The authors would like to thank all the peoples involved in the sampling and in the laboratory and the  Calculations Center of the Faculty of Science of the University of Neuchâtel  as well as UNINETT Sigma2 -the  National Infrastructure for High Performance Computing  (project  NN9549K ) and  Data Storage  (project  NS9593K ) in Norway for the computing facilities.