dataesr/scientific-paragraphs-categorization huggingface dataset:
Description of Task 1b Task 1b, titled "Biomedical Semantic Question Answering", examines systems' ability to annotate questions with concepts from relevant ontologies and depending on the type of the question, return "exact" or paragraph-sized "ideal" answers. The benchmark dataset comprises the questions, their answers, and the related documents, concepts, triples, and statements from designated repositories. It was created by the BIOASQ biomedical expert team, using the BIOASQ annotation tool that will be described later. The task was organised in two phases; (a) Phase A: The BIOASQ team released questions from the benchmark datasets. The participating systems had to respond with relevant concepts from designated terminologies and ontologies, relevant articles in English from designated article repositories, relevant snippets from the relevant articles, and relevant RDF triples (statements) from designated ontologies.
The participating systems were allowed to return at most 100 concepts, 100 documents, 100 snippets and 1,000 RDF triples per question; (b) Phase B: The BIOASQ team released questions and gold (correct) relevant concepts, articles, snippets, and RDF triples (statements) from the benchmark datasets. The participating systems had to respond with "exact" answers, i. e. , "yes" or "no" in the case of yes/no questions, named entities in the case of factoid questions, list of named entities in the case of list questions and nothing in the case of summary questions, and "ideal" answers (i. e. , paragraph-sized summaries) for all types of questions. We call them "ideal" because it is what a human would expect as an answer by a peer biomedical scientist. For the synthesis of the answers, the systems were allowed to use the provided gold documents, concepts, snippets and statements which were found related by the experts.
Creation of benchmark datasets for Task 1b In Task 1b, the benchmark datasets contain development and test questions, in English, along with golden standard (reference) answers. The benchmark datasets have been constructed by the BIOASQ team of biomedical experts. As in Task 1a, the datasets follow the JSON format. More specifically, each dataset (development and test sets) contains an array of questions where each question (represented as an object in the JSON format) is constructed as shown in 'The format of the training data of Task1b' subsection.
Results in Task 1a According to  [34]  the appropriate way to compare multiple classification systems over multiple datasets is based on their average rank across all the datasets. On each dataset the system with the best performance gets rank 1. 0, the second best rank 2. 0 and so on. In the case that two or more systems tie, they all receive the average rank. Table  6  presents the average rank (according to MiF and LCaF) of each system over all the test sets for the corresponding batches. For comparison, the MTIFL, MTI and bioasq_baseline baseline systems used throughout the challenge are shown as well. MTIFL and MTI refer to the NLM Medical Text Indexer system  [35]. Note, that the average ranks are calculated for the 4 best results of each system in the batch according to the rules of the challenge  [36].
Finally, we had a very good cooperation with the team of biomedical experts, providing them with all the help and tools they needed for the creation of the benchmark datasets for Task 1B and the evaluation of the systems' responses. The complete evaluation of the challenge, including the distributed questionnaires and the analysis of responses can be found in Deliverable D5. 2 of the BIOASQ project  [39]. e. In the remaining of the section we present the highlights and discuss the main problems faced. With regards to the quantitative measures' results, there were 117 users registered on the BIOASQ's participants area, with a total of 697 benchmark dataset downloads. In total, 11 different teams participated in the BIOASQ challenge. Given that one important objective of the challenge is to establish BIOASQ as a reference point for the biomedical community, these statistics constitute a good basis that indicates that the challenge is in the right direction.
The qualitative evaluation was conducted using questionnaires distributed to the participants and to the biomedical experts that created the benchmark questions and answers. The questionnaires distributed to the participating teams included questions targeting several aspects of the individual tasks, like the quality of the datasets, the technical support etc. Additionally, more general questions were provided to capture the overall impression of the participants for the BIOASQ challenge. The analysis of the questionnaires from the participants showed that they were satisfied with the challenge and as a consequence, not only are they willing to participate in the next cycle of the challenge, but they are willing to recommend BIOASQ to other research groups as well. However, there were some difficulties in the general understanding of the challenge.
In order to support the challenge, BIOASQ has built powerful and agile infrastructure for developing benchmark data sets for biomedical semantic indexing and question answering, as well as for using these data to evaluate participating systems, either automatically or manually. Most of the software produced by BIOASQ is provided as open source and the data are provided free of charge for future research use. The basic tool provided for benchmark data generation is the BIOASQ annotation tool, which can be used by biomedical experts to create questions and answering material of the form used in task 1b of the first BIOASQ challenge. The tool is publicly available as an open-source project at https://github. com/AKSW/ BioASQ-AT. In addition to the tools provided for the biomedical experts, BIOASQ has constructed a platform for setting up and managing the evaluation campaign.
The principal task of the biomedical expert team is the composition of the Question/Answer benchmark dataset which is used during the BIOASQ challenge Task 1b. Moreover, it is envisaged to be further enriched in the future mainly by voluntary contributions from the community. In this direction, the BIOASQ social network is expected to serve that purpose. For the first year, the benchmark dataset contains 29 development and 282 test questions used in a dry-run test and in the challenge respectively, along with gold standard (reference) answers. For each benchmark question composed by a biomedical expert, relevant material is also provided. This includes: (a) documents from specific article and abstract repositories, (b) concepts from designated ontologies, and, (c) statements (RDF triples), from selected life science triple stores.
Selection of data sources For the task 1b the BIOASQ challenge aimed to cover a wide range of biomedical concepts, through the use of ontologies and linked data that describe several facets of the domain. The selection of resources for these tasks follows the triangle drug-target-disease which defines the prime information axes for any medical investigation. This "knowledge-triangle" supports the conceptual linking of biomedical knowledge databases and the processing of the related resources. The idea behind the drug-target-disease triangle is based on the tight links that exist between these three entities. Any disease is associated with research on prognosis, diagnosis, symptoms, co-morbidities and incidence/prevalence. Towards investigating these aspects, there is a tight relation between a disease and targets, e.
Any of the selected documents, concepts or triples from the previous step (Search screen) are listed here in the left of the user's screen. Besides typing the ideal answer (top of the screen), the user can also click on selected documents (left of the screen) to annotate snippets from these documents that are relevant and important for the formulation of the answer (middle of the screen). The process is considered completed when the user clicks on the "Save" button (top right of the screen), in which case, both the answer to the question and all of the associated annotations are stored in a back-end database. This allows the users to continue working on any questions they have started in any previous sessions. The data storage implementation of the BIOASQ annotation tool relies on the NoSQL database MongoDB  [42].
The BIOASQ participants area and evaluation platform The series of the BIOASQ challenges require frequent exchange of data between the challenge participants and the organisers. The developed machanisms that cover those needs are integrated in an online platform f , the BIOASQ Participants Area. The BIOASQ Participants Area provides mechanisms for the participants to find information and support regarding the challenge and participate in the tasks. On the other hand, the BIOASQ team using the platform can administrate the challenge, release the benchmark datasets and provide the necessary mechanisms that will allow the evaluation of the participating systems. The platform was designed to be user-friendly. A key concept was to make it simple enough so that participants would not need much time to exchange data, receive support or check the performance of their systems by browsing the evaluation measures.
Another key concept was to provide ways to automate the process of downloading and submitting results. For this reason, web services were developed so that users could programmatically download the test sets and submit results saving time and effort. The platform provides a set of mechanisms to the registered users of the challenge. After subscribing to the platform, they gain access to the following: • the BIOASQ benchmark datasets; they consist of training and test datasets which are available for downloading after their release, • detailed guidelines describing the BIOASQ tasks, • tools that have been developed to help participants process the datasets e. g. tokenizers for Task 1a, • mechanisms for submitting results; they include: -HTML forms available as long as there are active tests, and -Web services for submitting results in an automated way, Figure  5  Screenshot of the answer formulation and annotation with document snippets.
In both approaches the selected examples are fixed in order to create manageable datasets especially in the upper levels of the hierarchy. The second system (Rebayct) that has participated in the challenge was based on a Bayesian network which models the hierarchical relations as well as the training data (that is the terms in the abstracts and titles). A major drawback of this system is that it cannot scale well to large classification problems with thousands of classes and millions of documents. For this reason, the authors reduced drastically the training data keeping only 10% of the data split in 5 disjoint parts in order to train five different models. During the testing phase, the models are aggregated through a simple majority voting. In  [32]  (AUTH) a flat classification approach has been employed (participating systems system1-5) which trains a binary SVM for each label in the training data  [44].
All the abstracts are ordered (first N -empirically set to 100) according to their distance and the top M (empirically set to 10) labels are retained. For the final prediction, the two systems are combined by keeping the common predicted labels; the other labels are ordered according to their confidence scores. The system predicts 10-15 labels for each test article. A learning to rank method has been used in the NCBI team  [49]. More specifically, the systems (all RMAI participating systems) follow a three stage approach: (i) first the k-nearest neighbours of the test article are retrieved from the MEDLINE database, (ii) next the labels are ordered using a learning to rank algorithm and (iii) finally a cutoff method prunes the ordered list. It is interesting to note that in the definition of the features for the learning to rank problem, the authors use the results of the MTIFL baseline system (see next paragraph).
Description of Task 1b systems In the second task of the BIOASQ challenge a total of three teams participated in both phases with 11 systems. Only two descriptions were available from these systems  [45, 48]. For phase A of Task 1b the Wishart system  [48]  makes use of query processing and document ranking techniques. More specifically, each test question in natural language form is converted by extracting the noun phrases and referencing them using a thesaurus of biomedical entities. Then the question is expanded by adding synonyms and relevant biomedical entities using the Poly-Search tool  [50]. The entities found by PolySearch are used to rank the retrieved set of concepts, articles, triples and snippets. In phase B of the task a similar approach to phase A is used in order to augment the set of given concepts.
Detailed description of the selected data sources In this appendix, the selected data sources (documents, databases, ontologies) that were used in the BIOASQ challenge are described. Ontologies such as the Medical Subject Headings (MESH) and the Gene Ontology (GENE ONTOLOGY) play a major role in biology and medicine since they facilitate data integration and the consistent exchange of information between different entities. They can also be used to index and annotate data and literature, thus enabling efficient search and analysis. In the past few years, the volume of the biomedical literature has been growing very fast, expanding by almost 1 million new scientific papers per year, indexed by MEDLINE. This fact makes the task of monitoring the knowledge and the changes in the biomedical domain extremely difficult.
Hierarchical classification algorithms  [3]  can classify documents and questions onto the concepts of these inventories, facilitating the matching of questions, documents, and also structured data (e. g. , RDF triples) that already have explicit semantics based on the same concepts. More specifically, for the two BIOASQ challenge tasks, the selection of resources follows the notion of the triangle drug-target-disease which defines the prime information axes for any medical investigation. This "knowledgetriangle" supports the conceptual linking of biomedical knowledge databases and the processing of the related resources. Based on this notion, systems can address questions that combine any path connecting the vertices of the triangle, provided that they can also annotate with accuracy the natural language questions with ontology concepts.
UniProt The Universal Protein Resource  [56]  (UNIPROT) provides the scientific community with a comprehensive, high-quality and freely accessible resource of protein sequence and functional information. Its protein knowledge base consists of two sections: Swiss-Prot, which is manually annotated and reviewed, and contains approximately 500 thousand sequences, and TrEMBL, which is automatically annotated and is not reviewed, and contains approximately 23 million sequences. The primary mission of UNIPROT is to support biological research by maintaining a stable, comprehensive, fully classified, richly and accurately annotated protein sequence knowledge base, with extensive cross-references and querying interfaces freely accessible to the scientific community. In particular the Swiss-Prot component of UniProt is a high-quality, manually annotated, non-redundant protein sequence database which combines information extracted from scientific literature and biocurator-evaluated computational analysis.
Disease ontology The DISEASE ONTOLOGY  [57]  contains data associating genes with human diseases, using established disease codes and terminologies. Approximately 8,000 inherited, developmental and acquired human diseases are included in the resource. The DISEASE ONTOLOGY semantically integrates disease and medical vocabularies through extensive cross-mapping and integration of MESH, ICD, NCI's thesaurus, SNOMED CT and OMIM disease-specific terms and identifiers. The DISEASE ONTOLOGY is utilized for disease annotation by major biomedical databases (e. g. , ARRAY EXPRESS, NIF, IEDB), as a standard representation of human diseases in biomedical ontologies and as an ontological crossmappings resource between MESH and OMIM. The DIS-EASE ONTOLOGY has been incorporated into open source tools (e.
g. , Gene Answers, FunDO) to connect gene and disease biomedical data through the lens of human diseases. The medical subject headings hierarchy Medical Subject Headings [58] (MESH) is a hierarchy of terms maintained by the United States National Library of Medicine (NLM) and its purpose is to provide headings (terms) which can be used to index scientific publications in the life sciences, e. g. , journal articles, books, and articles in conference proceedings. The indexed publications may be then searched through popular search engines, such as PUBMED or GOPUBMED, using the MESH headings to filter semantically the results. This retrieval methodology seems to be in some cases beneficial, especially when precision of the retrieved results is important (  [2] ). MESH includes three types of data: (i) descriptors, also known as subject headings, (ii) qualifiers, and, (iii) supplementary concept records.
PUBMED, a service provided by the National Library of Medicine (NLM), under the U. S. National Institutes of Health (NIH), contains over 23 million citations from MEDLINE, a bibliographic database (DB) of biomedical literature, and other biomedical and life science journals dating back to the 1950s. It is accessible through the National Center for Biotechnology Information (NCBI). PUBMED CENTRAL is a digital archive of full-text biomedical and life science articles. The full text of all PubMed Central articles is freely available, but not for bulk download. As of July 2011, the archive contains approximately 2. 2 million items, including articles, editorials and letters. Linked life dataThe LLF project[61]provides the LinkedLifeData platform. LinkedLifeData is a data warehouse that syndicates large volumes of heterogeneous biomedical knowledge in a common data model.
Acknowledgements The research leading to these results has received funding from the  European Commission 's  Seventh Framework Programme  (FP7/2007-2013, ICT-2011. 4. 4(d), Intelligent Information Management, Targeted Competition Framework)  under grant agreement n. 318652. The authors would also like to gratefully acknowledge the support of  James Mork , computer scientist at the  Cognitive Science Branch of the Lister Hill Center for Biomedical Communications (NLM) , and of  Dr. Alan Aronson , Senior Researcher at the same center. They both provided valuable feedback during the BIOASQ challenge, and they kindly accepted to allow their center's systems, namely  MTI  and MTIFL, be two of the systems that acted as very competitive baselines in Task 1a of the challenge.
The geometry simulated in this paper is presented in the figure  2  and is around 2m length and 100mm in the spanwise direction. The inlet Mach number is 0. 5 and the outlet pressure has been calibrated to have a Mach number of 1. 22 in front of the first blade. The grid was generated by NUMECA/Autogrid and the y + is lower than 1 in all the regions of interest. The size of the 3D grid is 9. 6×10 6 cells. The highest Reynolds number corresponding to the sea level altitude, Re = 4. 4 × 10 6 , and an inlet turbulence intensity of 4% are used. In this study the compressible NSMB flow solver  [17]  has been used. The NSMB code solves the compressible Navier-Stokes equations using a finite volume formulation on structured Multi-Block grids and is parallelized using the Message Passing Interface.
The percentage of Y-bearing cells was determined as the ZFY / RNase P ratio. For CDH1 and SNAI1 gene expression analyses, digital droplet PCR assays were performed using Hs01023894_m1 (NM_004360. 3) and Hs00195591_m1 (NM_005985. 3) primers (Applied Biosystems). Droplet digital PCR assays were performed in triplicate. Controls were no template, male, and female DNA. Reaction mixes were prepared using standard Taqman primer/probe chemistry with a 2 X ddPCR Mastermix (BioRad, Laboratories, CA), a 20 X primer/ probe (900/250 nM), and 5µl sample DNA template in a final volume of 20 μl. The PCR products were analysed using QuantaSoft software (BioRad Laboratories, CA). Raw data were collected on the Bio-Rad QX100 instrument. CD133, CD45, CDH1 and SNAI1 mRNA expressions were expressed as the copy number of each gene / TBP copy number ratio.
Overview Sordariomycetes phylogeny First of 260 equally most parsimonious trees obtained from the LSU alignment (213 parsimony-informative, 115 parsimony-uninformative and 453 constant characters) based on a heuristic analysis with simple taxon additions and tree-bisection-reconnection algorithm using PAUP v. 4. 0b10 (TL = 939, CI = 0. 520, RI = 0. 891, RC = 0. 463). GenBank accession numbers are indicated behind the species names. Bootstrap support values > 50 % from 100 000 fast replicates are shown at the nodes and the scale bar indicates the number of changes. Families, orders and classes are indicated with coloured blocks to the right of the tree. Strict consensus branches are indicated with thickened lines. The tree was rooted to Saccharomyces cerevisiae (GenBank Z73326) and the novel species described in this study for which LSU sequence data were available are indicated in bold face.
Overview Dothideomycetes and other classes phylogeny First of 1 000 equally most parsimonious trees obtained from the LSU alignment (308 parsimony-informative, 44 parsimonyuninformative and 417 constant characters) based on a heuristic analysis with simple taxon additions and tree-bisection-reconnection algorithm using PAUP v. 4. 0b10 (TL = 1 536, CI = 0. 414, RI = 0. 844, RC = 0. 349). GenBank accession numbers are indicated behind the species names. Bootstrap support values > 50 % from 100 000 fast replicates are shown at the nodes and the scale bar indicates the number of changes. Families, orders and classes are indicated with coloured blocks to the right of the tree. Strict consensus branches are indicated with thickened lines. The tree was rooted to Saccharomyces cerevisiae (GenBank Z73326) and the novel species described in this study for which LSU sequence data were available are indicated in bold face.
Roussoella mexicana First of five equally most parsimonious trees obtained from the ITS alignment based on a heuristic analysis with 100 random taxon additions and tree-bisection-reconnection algorithm using PAUP v. 4. 0b10 (TL = 481, CI = 0. 644, RI = 0. 594, RC = 0. 383). GenBank accession numbers are indicated behind the species names. Bootstrap support values > 50 % from 1 000 replicates are shown at the node and the scale bar indicates the number of changes. The tree was rooted to Massarina lacustris (GenBank AF250831) and the novel species described in this study is indicated in bold face. The alignment and tree were deposited in TreeBASE (S18408).
The actA sequence confirmed that there are no matches on GenBank, closest matches being with P. podocarpi (GenBank KF289235), P. pseudotsugae (GenBank KF289236) and P. owaniana (GenBank FJ538484). The gapdh and tef1 sequences confirmed this association. No Phyllosticta species are known on Carissa  (Crous et al. 2000) , and P. carissicola also differs morphologically from P. podocarpi in that the latter species has somewhat longer and narrower conidia, (10-)14(-17) × (8-)9(-10) μm, and longer appendages, 10-40 μm  (Crous et al. 1996). Fungal Planet 375 -4 December 2015  Crous & M. J. Wingf. , sp. nov. Etymology. Name refers to the host genus Pandanus from which this species was collected.