dataesr/scientific-paragraphs-categorization huggingface dataset:
A typical dataset available in the protein structure repositories contains, in addition, a list of genes and a list of research articles'. Thus, a global pharmaceutical company, for instance, may need close to 30 different databases to complete a clinical study. These sources of data require recording provenance for datasets and data curation. Moreover, the data resulting from biomedical experiments often possess an implicit hierarchy  (1). In terms of granularity needed for specific databases, a PubMed article needs to be decomposed into snippets which describe structured data markup. Snippets may be organized using a comprehensive data type ontology which will provide definitions of types of data (Protein, Phenotype, Gene Expression, Nucleotide Sequence, Clinical Trials, Imaging Data, Morphology, Proteomics Data, Physiological Signals, Epigenetic Data, Data from Papers, Omics Data, Survey Data, Cell Signalling and Unspecified).
Snippets in different databases may often be found at different levels of a database schema. Since different types of metadata are of importance for given specialized databases, historically their schemas were developed independently, and do not conform to any standardized pattern. Since datasets are combination of structured and unstructured data, often presented in incompatible ways (e. g. the same information with different tags), using them in a complex processing can be quite difficult. Futhermore, a significant percentage of specific data that had been reported in clinical reports does not made its way into journals  (2). Nevertheless, data needs to be compared and verified. Often, cost and utility considerations make it necessary to try a multi-sponsored clinical development approach termed Portfolio of Innovative Platform Engines, Longitudinal Investigations and Novel Effectiveness to generate a new hypothesis.
In such environment  (3) , this need for shared collaborative data governance forces a use of integrated data-therefore, improving the effectiveness of retrieval is paramount to finding state-of-the-art methods of diagnosis, testing and treatment for individual patients. Existing platforms such as Google and PubMed serve their purpose providing an up-to-date sources of information with various additional functionalities but it is difficult to assess their effectiveness. Thus, the crucial aspect for addressing this complexity is the availability of annotated distributed datasets created by the scientific community, with which researchers can test the effectiveness of various approaches. That in turn leads to better data structures and indexes of various granularities. This can be achieved only within a shared task environment, which enables researchers from many different institutions to work together at solving important scientific problems.
bioASQ Task on Biomedical Semantic query answering (QA) related to information retrieval and query answering-one of the most complex semantic tasks in natural language processing (NLP). Previous TREC CDS and earlier medical tracks and bioASQ challenges had many specific task orientations, data sources and retrieval conditions. For example, some TREC sources were either full publications or abstracts. The topics of a question could be electronic health records (EHR) admission notes curated by physicians. Notes could be of Diagnosis, Test and Treatment type. Notes could be much longer compared with concise bioCADDIE questions. Currently, the format for run submissions of TREC and bioCADDIE is the standard trec_eval format. The bioASQ contest shares a deep semantic approach to answer questions with bioCADDIE when word embeddings (WEs) are used for query expansion or within document vector framework.
Based on these tasks, the Biomedical and healthCAre Data Discovery Index Ecosystem (bioCADDIE) consortium, funded by the US National Institute of Health Big Data to Knowledge program, aims to empower researchers to find data the most efficient way and expand sources and types of data. These would include opinion on research on non-scientific portals (i. e. conversations about scholarly content) together with monitoring attention surrounding particular work (altmetric). BioCADDIE  (6)  has developed DataMed, a search engine prototype of Data Discovery Index (DDI), using the data tag suite (DATS) model to support the DataMed discovery index  (7). This enables searching data of various types and formats (while maintaining a core set of elements), curated by separate institutions. DataMed based on ISA formatted metadata aims to facilitate the discovery of a digital object.
At this time, DataMed has indexed close to 1 400 000 datasets drawn from 66 repositories  (8). The bioCADDIE challenge concerned finding most relevant docnos (elements of datasets) in response to 15 questions provided by bioCADDIE experts. The structure of the questions followed the DataMed prototype idea of the rdf type of relations between entities ('data type' ¼ w, 'biological process' ¼ x, 'species/organism' ¼ y and 'phenotype' ¼ z)  (9). The graph structure of a query suggests that if we also transformed documents into graph structure the matching process would be at the level of relations and not keywords. The aim of the 2016 bioCADDIE Challenge (9) was the retrieval of datasets from a collection that is relevant to the needs of biomedical researchers; the purpose was to facilitate the reutilization of collected data and enable the replication of published results.
Such work is the focus of WG4 of the bioCADDIE consortium: Use Cases and Testing Benchmarks. The goal is to develop usability specifications/requirements and appropriate benchmarks with associated testing content for DataMed. To address this goal sections, later discuss the following aspects: • The Related work section discusses the content of already published bioCADDIE articles • The Methodology section presents the methods, algorithms and solutions prepared by our team, divided into following subsections: • The Overview, describing the model of our information retrieval system • The Collection, with information on the bioCADDIE datasets • An Analysis of document structure and content, presenting the differences among various repositories • A Selection of documents with valuable data for indexing, with the description of our algorithm evaluating whether a document is worth indexing • An Index of data, including information of corpus preparation for indexation • Query preprocessing • Query expansion, describing the methods chosen to expand the query • Information retrieval and evaluation, with information on the retrieval platform • The Results and discussion section is divided into the following subsections: • Selection of the optimal baseline system • Query expansion.
Related work At present, details of bioCADDIE Challenge systems exist for selected contributions. Apart from standard similar preprocessing similar to that presented in this work, processing can be divided into advanced preprocessing, retrieval and re-ranking. The University of California San Diego (UCSD) team that obtained the top infNDCG result (9) implemented a two-step 'retrieval plus re-ranking' strategy  (10). Based on this idea, they developed a method to find the Google top 10 returned documents and then transformed these documents into queries for relevant datasets. This strategy was used by East China University in their winning contribution to TREC CDS 2015  (11). Their baseline was Elastic search (a Lucene-based search engine that is part of a DataMed technology). The Elastic search top 5000 retrieved datasets were reranked based on the concatenated documents using the pseudo sequential dependence (PSD) model  (12).
The best run used the PSD-allwords model. UCSD used the concept matching formula with Dirichlet smoothing, with weights based on the annotated dataset repository. In contrast to the original algorithm in  (12) , an actual term frequency was increased by a constant ¼ 5. UCSD found (as we do) that neither ordered nor unordered bigrams have improved performance. We would like to point out that the UCSD results presented in  (10)  do not exactly match the official results  (9). Elsevier  (13)  used two approaches: word embeddings and ontology-based indexing (queries and data sources were tagged with named entities from MeSH and Entrez Gene) with indexing and search platform Apache Solr. For WEs, fastText  (14)  gave better results than word2Vec  (15)  and GloVe (  16 ) that we both used. FastText, based on a skip-gram model, uses character n-grams and smaller windows that translate to better WEs for query expansion.
804), (glycogenolysis, 0. 797), (glyconeogenesis, 0. 771), (gluconeogenic, 0. 751), (glycogen, 0. 7405), (lipogenesis, 0. 738), (ureogenesis, 0. 738), (glycogenic, 0. 738), (ketogenesis, 0. 737) and (glycogenolytic, 0. 734)]. Elsevier obtained the best result with Elsevier four run modified queries (all additional modifications) þ concept expansion þ multi-phase execution; Search: Apache Solr, stemmed index) but only 2% better than their baseline. SIBTex  (17)  divided query terms into non-relevant, relevant and key, assigning larger weights to key relevant terms compared with relevant terms. This is the same strategy that we used for expanded terms. Universal protein resource (UniProt) was used to constrain query and datasets to a set of 14 biomedical topics  (18). They used the Gensim word2vec library (as we did) for finding expansion candidates.
Their best run SIBTex 3 was achieved with a baseline þ query expansion with weighted terms þ results categorization in the post-processing phase. OHSU assumed a variable number and relative weighting of MeSH terms for query expansion in the work after the challenge. Additional runs determined the optimal number of MeSH terms and weighting. Their best overall score used five MeSH terms with a 1:5 terms: words weighting ratio  (19). This is the same ratio we used in our best run when query expanded terms are derived from word2vec. The University of Melbourne, UM  (20)  provided useful determination of appearance of most important metadata in bioCADDIE used repositories. This information could be helpful for determination whether a query term belongs to a concept expressed by metadata or using weights for answers coming from different repositories.
Methodology The overview The information retrieval process, we used was divided into four steps: 1. Analysis of repositories structure and their information content. 2. Selection of the optimal baseline system. 3. Selection of optimal possible system extension. 4. Optimization of parameters of the complete system. The model of the system developed to generate information retrieval for the bioCADDIE challenge includes the following elements: 1. Preparation of database with valuable information from datasets 2. Indexing of data collection 3. Query preprocessing 4. Preparation of two vector space models based on data from bioCADDIE datasets and PubMed abstracts 5. Query expansion with the use of prepared vector space models and pseudo-relevance feedback (PRF) (provided by Terrier) 6. Information retrieval by the Terrier engine 7.
The collection The bioCADDIE corpus was a collection of metadata (structured and unstructured) from biomedical datasets generated from a set of 20 individual repositories (Table  1 ). A total of 794 992 XML documents were made available for use from the set of indices that was frozen from the DataMed backend on 24 March 2016  (21). Data in each document was organized into the following tags: <DOCNO>: document number, <TITLE>: document title, <REPOSITORY>: biomedical repository used to generate document, <METADATA>: various data from the repository presented in json format.
Analysis of document structures and their information content Each repository uses different json schema to organize data. Moreover, in some cases a variation was noted within the same repository (Table  1 ). To prepare a text corpus for indexing, tags and keys with potentially valuable information were selected and their values were exported to the SQL database. The data was then assigned to one of three categories: Title, Keywords or generalized Description. For one of the repositories (geo), the generalized description contained additional text data, obtained from geo database online resources, based on the 'geo_accesion' code found in the metadata (Table  2 ).
Selection of documents with valuable data for indexing Because documents from some repositories (e. g. dryad, geo) contained very little useful information (see examples in Table  3 ), we decided to assess if a document's content is worth indexing using MeSH. MeSH, which stands for 'Medical Subject Headings', is a vocabulary thesaurus used by the National Library of Medicine (NLM) to index articles stored in PubMed  (22). At this point it does matter whether we use words or lemmatized words, so we chose to remain with the former. Terrier tokenises a query and documents so various word forms are treated exactly the same. In WE methodology various words forms represent different elements of space but when these words became expanded terms only the token form count. For each category (Title, Keywords or Description) of each record (from the previously prepared SQL database), a score was calculated according to the following heuristic algorithm that removes meaningless records before indexing (i.
e. shown in Table  3   Items in the table represent column names from the SQL database (prepared on the basis of documents' JSON keys). In most cases, more than one column was used to prepare text categorized as Description. Thirteen repositories did not provide any keywords. MeSH terms (descriptors and their synonyms in the MeSH database). 4. If there are no words (e. g. the document is lacking keywords) set the Score to À1. 5. Calculate the Score ¼ (Y1 þ Y2)/X. 6. For Title and Description categories, if the Score is >0 and X is >2, take the record for indexing. 7. For the Keywords category, if the Score is >0 (as follows from the condition at Step 4), take the record for indexing. The Descriptor/Concept/Term structure makes it possible to attach various data elements in MeSH to the appropriate object.
This sentence is directly taken from https://www. nlm. nih. gov/mesh/concept_structure. html. Word (linguistic notion), term (appears in a query), data element (part of a taxonomy structure) differ in contexthere they are used in the meaning of word. MeSH terms used in the previous algorithm were prepared according to the following procedure: 1. 'Descriptor', 'Substances with pharmacologic action', 'Qualifiers' and 'Supplementary records' files were downloaded from the MeSH database website (20). 2. Words were collected from specific tags depending on the file (<DescriptorName> and <ConceptList> tree from 'desc2017. xml'; <DescriptorName> and <Phar macologicalActionSubstanceList> tree from 'pa2017. xml'; <QualifierName> and <ConceptList> tree from 'qual2017. xml'; <SupplementalRecordName> and <ConceptList> tree from 'suppl2017.
Similarity was calculated for each dataset using word2vec, an efficient model allowing for learning vector representations of words from unstructured text data  (15)  with the following parameters: • PubMed collection: number of dimensions ¼ 100; window size ¼ 5; minimum word count ¼ 10; this resulted in the collection of 1 498 219 words; • BioCaddie collection: number of dimensions ¼ 100; window size ¼ 20; minimum word count ¼ 5; this resulted in the collection of 296 503 words. A similarity threshold was set to 0. 9 for vectors generated from PubMed abstracts and 0. 8 for vectors calculated on the basis of bioCADDIE datasets (lower values resulted in dissimilar query terms). As in (  29 ) and (  31 ), if queries are expanded with WE obtained terms and added to a list of query terms with the same weight as the original terms, the results, in general, get worse, because a query drift is introduced.
In Question 9 (question pertains to 'ob' and Mus musculus), adding terms such as 'mouse' or 'mice' to a question does not improve the result. The most important result of this work is observation that the results improve if query expanded terms are given a much smaller weight than the original terms. The weight of original query terms was set to 100, terms obtained from PubMed to 20 and terms provided with bioCADDIE embeddings to 1. This is justified by the relative smallness of the bioCADDIE dataset. In  (26) , we used MeSH not only for filtering but also for query expansion, with positive results. For the purpose of this work, we use MeSH only for filtering because the free access interface was discontinued. We tried query expansion with WE using two approaches: 1. The skip-gram method (15) on abstracts of the entire PubMed using Gensim library (33).
Further analysis To better understand the results, we did evaluation for individual questions (Table  7 ) for our best result: LGD with query expanded with word2vec and Terrier PRF. Strikingly, the highest value of measure is for Question 15 (for which, similar to Question 7 no Score 2 of evaluation was assigned). Further analysis of which particular databases carry information gain is required. For example, neuromorpho provided 11% of the contribution to infNDCG measure, although it constitutes <5% of data volume. Table  8  presents the details of run options for the LGD algorithm using the same or different weights for original and expanded terms and shows that expansion terms should not have the same weight as original terms. We evaluated the results using the query relevance file with partially relevant documents denoted as non-relevant.
Conclusions and future work Shared tasks bioCADDIE challenge fulfilled an important role in the advancement of biomedical Information retrieval methods using data snippets as datasets. Our postchallenge analysis indicates that bioCADDIE data is quite different from continuous biomedical data. There are quite a number of documents that basically present the same information duplicated in NML databases. Manual expansion, in general, makes the results worse. Word2vec based query expansion improves the results but expansion term weights have to be much smaller than the original weights. For effectiveness of word2vec, a method for calculating the similarity of candidate expansion terms to the original query terms is crucial. In this work, we use the pure word2vec. Several recently proposed theoretical approaches to query expansion reporting positive results  (34) (35) (36) (37) (38) (39) (40) (41) (42)  deserve to be applied in a bioCADDIE context [including word2vec  (38) ].
Such a method (enhanced to relatedness) could allow direct comparison of query and target terms. Other query expansion schemes are based on WE exist  (41) (42). Terrier provides a state-of-the-art baseline system but our perspective is that PRF and phrase query expansion could be significantly improved within Terrier. Direct comparison of this work results with original bioCADDIE results is not warranted. Nevertheless, our results are strong. They are close to the top in most measures, and the best in infAP measure. To summarize, the main conclusions of this article are the following: 1. Use of language models created on the basis of distribution semantics to expand the query (using WE) has the potential to significantly improve WE results in the near future. 2. Assigning different weights to words in a query, depending on whether the words were added in the expansion process or originating from the original content of the query significantly improves the result.
Characteristics of antiproton annihilations in matter An antiproton annihilation with an atomic nucleon creates on average in total 5 pions (charged and neutral) per annihilation  [11]. Some of these pions are likely to penetrate the a ected nucleus, and nuclear fragments can break o. The multiplicity and energy distribution of these annihilation products is not fully known. Since simulations are based upon theoretical models and sparse data, they are not able to completely reproduce experimental results. Studies of antiproton annihilations in copper, silver and gold  [12]  show that the simulation software FLUKA performs better than other available software such as Geant4, and FLUKA was therefore used for this study. The annihilation products acquire kinetic energy from the mass-to-energy conversion of the annihilation, and travel away from the annihilation point.
Analysis of experimental data Two data samples were collected during the test beams in 2015 and 2016, and consisted of one main data sample and one reference data sample. The main sample was taken using the optimal GRACE settings described in section 1. 1, and consists of 560 bunches from the AD, each containing a mixture of annihilation clusters and secondary clusters. In this data set 44 bunches are reserved as training data to develop the algorithms for clustering and characterization. The reference data sample was taken with the detector rotated such that its backs was facing the beam. In this configuration no antiprotons annihilate directly on the detector. The reference data sample therefore consists of 1561 bunches only containing secondary clusters.
This phenomenon is called the plasma e ect  [24] , and its impact increases with increasing energy deposition. The simple model in equation (3. 2) does not consider the plasma e ect, however it is included in the device simulation software TCAD  [25]. Therefore this software was used to find the collection time for di erent energy depositions. The simulation was done for three di erent depths, corresponding to the three di erent depths of the center of the voxels. The results from the TCAD simulations are found in table 1. For comparison the collection time from the simple model in equation (3. 2) is also indicated. There is good agreement between the collection time given by the two models for energy depositions of 1 keV and 10 keV, giving evidence that the plasma e ect is not present here.
Test of the detector response model The detector response model was verified by comparing the distributions of cluster observables in data and simulation. The observables studied are the cluster size, the number of prongs, the energy deposited in the center of the cluster and the energy deposited in the whole cluster. The main data sample contains a mixture of secondary clusters and annihilation clusters, while the reference sample contains only secondary clusters. Therefore the main data sample was compared to a linear combination of the reference data sample and the sample of simulated annihilation clusters. This combined sample of reference data and simulated annihilation clusters was normalized to the total number of clusters in the main data sample. The fraction of the reference data sample in the combined sample, from here on referred to as r, is a free parameter that was optimized to minimize the 2 value of the di erence between the histogram bins.
Estimate of the tagging e ciency This section shows how cuts on cluster size and number of prongs are set in order to tag the antiprotons. These two cluster observables were chosen because observables relating to the shape of the clusters are the most accurately modeled ones. The false tagging rate, from here on referred to as p f , was estimated by the fraction of clusters passing the cuts in the reference data sample. The tagging e ciency, from here on referred to as p t , was found by applying the cuts on the sample of simulated clusters. Figure  10. Histograms comparing the cluster observables in data and simulations. Observables relating to the shape of the clusters seem to be modeled better. The fraction of secondary clusters is estimated to be r = 0. 83.