dataesr/scientific-paragraphs-categorization huggingface dataset:
Polystores process a query assuming that each of its input datasets is available in one store (chosen for its support of a dataset data model). We identify two limitations of such architectures. First, they do not exploit possible data redundancy: the same data could be stored in several stores, some of which may support a query operation much more efficiently than others. Second, they are unable to take advantage of the presence of previously computed query results, which may be available in one or several stores (in the style of materialized views), when the data model of the queried dataset differs from the data model of the store hosting the view. To overcome these limitations, we have described in  [4]  ESTOCADA, a novel approach for allowing an application to transparently exploit data stored in a set of heterogeneous stores, as a set of (potentially overlapping) data fragments; if fragments store results of partial computations applied on the data, we show how to speed up queries using them as materialized views, which reduces query processing effort and seeks to take maximum advantage of the efficient query processing features of each store.
Motivating Scenario Dataset. The Medical Information Mart for Intensive Care III (MIMIC-III)  [11]  dataset comprises health data for more than 40000 Intensive Care Unit (ICU) patients. The total size is 46. 6 GB, and it consists of: (i) all charted data for all patients and their hospital admission information, ICU stays, laboratory measurements, caregivers' notes, and prescriptions; (ii) the role of caregivers and (iii) diagnosis related groups' codes descriptions. Our motivation query Q1 is: "for the patients transferred into the ICU due to "coronary artery" issues, with abnormal blood test results, find the date/time of their admission, previous location (e. g. , clinic referral), and the drugs of type "additive" prescribed to them". Evaluating this query through AsterixDB  [1]  (v0. 9.
4) JSON store took more than 25 minutes; this is because the system does not support fulltext search by an index if the text occurs within a JSON array. In SparkSQL (v2. 3. 2), the query took more than an hour, due to its lack of indexes for selective data access. In the MongoDB JSON store (v4. 0. 2), it took more than 17 minutes due to its limited join capability. Finally, in Post-greSQL with JSON support (v9. 6), Q1 took 12. 6 minutes. Now, consider we had at our disposal three materialized views, which pre-compute partial results for Q1. Consider a SOLR (well-known full-text search engine) sever stores a view V1 storing the IDs of patients and the caregivers' notes on the patients' stay. Full-text search on V1 for "coronary artery" allows to efficiently retrieve the respective patients' IDs.
r. t. plain JSON query evaluation in MongoDB and Postgres, respectively. Lessons learned. We can draw the following conclusions from the above example. (1. ) Unsurprisingly, materialized views improve query performance since they pre-compute partial query results. (2. ) More interestingly: materialized views can strongly improve performance even when stored across several data stores, although such a hybrid scenario incurs a certain performance penalty due to the marshaling of data from one data model/store to another. In fact, exploiting the different strengths of each system (e. g. , SOLR's text search capability) is the second reason (together with materialized view usage) for the performance gains. (3. ) Different system combinations work best for different queries. Thus it must be easy to add/remove a view in one system, without disrupting other queries that may be currently well-served.
Thus, their view-based query rewriting algorithms are designed specifically within the bounds of that model, e. g. , relational or XML. Different from these, some LAV systems  [12, 7]  allow different data models for the stored views, but consider only the case when the application data model is XML. As a consequence, the query answering approach adopted in these systems is tailored toward the XML data model and query language. In contrast, we aim at a unified approach, supporting any data model both at the application and at the view level. The core technical question to be answered in order to attain such performance benefits without disrupting applications is view-based query rewriting across an arbitrary set   [6]. This demonstration improves over it through several contributions at the level of: (i) modeling: support for the array (matrix), JSON, and graph models; (ii) algorithms: optimizing the stateof-the-art algorithm for relational view-based rewriting under constraints to make it scale to a polytsore setting  [4] , and (iii) systems, by deploying ESTOCADA on top of three existing polystore systems: BigDAWG  [8] , SparkSQL, and Tatooine  [5]  and showing the performance benefits that our approach can bring to these systems.
REWRITING PROBLEM STATEMENT We assume a set of data model-query language pairs P = {(M1, L1), (M2, L2),. } such that for each 1 ≤ i, an Li query evaluated against an Mi instance returns an answer which is also an Mi instance. The same model may be associated to several query languages (e. g. , AsterixDB and MongoDB have different query languages for JSON). We consider expressive languages for realistic settings, supporting conjunctive querying, nesting, aggregation, and object creation. Without loss of generality, we consider that a language is paired with one data model only. We consider a polystore setting comprising a set of stores S = {S1, S2,. } such that each store S ∈ S is characterized by a pair (MS, LS) ∈ P, indicating that S stores instances of the model MS and evaluates queries expressed in LS.
We consider a set of datasets D = {D1, D2,. }, such that each dataset D ∈ D is an instance of a data model MD. A dataset D is stored as a set of (potentially overlapping) materialized views {V 1  D , V 2 D. }, such that for every 1 ≤ j, V j D is stored within the storage system S j D ∈ S. Thus, V j D is an instance of a data model supported by S j D 1. We consider a Polystore integration language IL  [8, 3] , capable of expressing computations to be made within each store and across all of them, as follows: • For every store S and query qS ∈ LS, IL allows specifying that qS should be evaluated over S; • Further, IL allows expressing powerful processing over the results of one or several such source queries. View V 1 : FOR AJ :{ SELECT M. patientID AS patientID , A. admissionID AS admissionID , A.
report AS report FROM MIMIC M , M. admissions A } RETURN SJ :{ " patientID " : patientID , " admissionID " : admissionID , " report " : report } We assume available a cost model which, given an IL expression e, returns an estimation of the cost of evaluating e (including the cost of its source sub-queries). We outline the concrete model we use in  [4]. We term a rewriting of a query q an integration query expressed in IL, which is equivalent to q. We consider the following rewriting problem: Definition 1 (Cross-model rewriting problem). Given a query q ∈ IL over several datasets Di, 1 ≤ i ≤ n, and a set of views V materialized over these datasets, find the equivalent rewriting r of q using the views, which minimizes the cost estimation c(r).
ESTOCADA OVERVIEW We describe below our approach for solving the above rewriting problem and the technical challenges this raises. We refer readers to our paper  [4]  for details. Polystore Integration Language. We designed QBT XM ∈ IL, a concrete polystore integration language. QBT XM follows a block-based design, with blocks organized into a tree, each block can be expressed in a different query language and carry over data of a different data model. Other existing polystore integration languages, e. g. ,  [8]  resemble QBT XM. Cross-Model Views. Each materialized view V is defined by an IL query; it may draw data from one or several data sources, of the same or different data models. Each view returns (holds) data following one data model, and is stored in a data store supporting that model.
An incoming query Q ∈ IL is encoded as a relational query enc(Q) over the relational encoding of the datasets it refers to. The reformulation problem is thus reduced to a purely relational setting: given a relational query enc(Q), a set of relational integrity constraints encoding the views, enc(V1)∪. ∪ enc(Vn), and the set of relational constraints obtained by encoding the data models M1,. , M l , find the queries RW i r expressed over the relational views, for some integer k and 1 ≤ i ≤ k, such that each RW i r is equivalent to enc(Q) under these constraints. The challenge in coming up with the reduction consists in designing a faithful encoding, i. e. , one in which rewritings found by (i) encoding relationally, (ii) solving the resulting relational reformulation problem, and (iii) decoding each reformulation RW i r into a IL query R = dec(RW i r ) over the views in the polystore, correspond to rewritings found by solving the original problem.
DEMO WALKTHROUGH The ESTOCADA approach can be combined with (applied on top of) any polystore, to enhance its performance through exploiting materialized views. We propose to demonstrate it on three existing polystores: BigDAWG  [8] , Spark-SQL, and Tatooine  [5] , with underlying data stores: Post-greSQL, PostgresSQL with JSON support, Apache Solr, and Apache SystemML. We extended ESTOCADA to support array(matrix) and graph data models in addition to the supported data models in  [4]. We propose to show ESTOCADA in action on a set of scenarios similar to the one described in Section 1. We use three datasets: (i) MIMIC-III  [11] , described in Section 1. (ii) GDELT  [2] , which captures worldwide events (e. g. , protests, peace appeals) and consists of the events' structural information (e.
g. , event's id and date), events' actors, geography, and various links to articles describing each event. The dataset is in a relational form, and 26GB (2015 events) in file size. (iii) GENOMICS  [13] , comprising patients, gene metadata and microarray data (matrix form-dense). The size of the dataset is 32GB. Demo attendees can interact with the systems through the following interfaces: Datasets Interface. This allows to browse, for each data source, the schema (or its structural summary, for semistructured data such as JSON lacking a schema), and sample of the data, to get familiar with its structure and help formulate queries (see Figure  3(a) ). Query Rewriting Interface. The user can interact with this interface by selecting a dataset, a polystore system query language (QBT XM , SparkSQL or BigDAWG) to use, and a query (expressed in the chosen polystore's system query language) from a workload we prepared, as shown in Figure  3(b).
Each dataset comes with a previously specified workload, and a set of stored overlapping/redundant views (their definitions, storage model, and location will be shown to the user); the users also have the option to write their queries and views. On the same interface, the user can inspect the cross-model query rewriting, its translation in the pivot model, and its translated form in the chosen language. Understanding Constraints Interface. This interface enables the users to select one of the supported data models, inspect the relational integrity constraints that capture the properties of that model, disable/enable/add some constraints, and observe the query rewritings thus obtained. In the same interface, we provide a real-time visualization of which constraints are exploited by the rewriting engine.
CONCLUSION The goal of our demonstration is to show that query evaluation performance in polystore architectures can be significantly enhanced by exploiting materialized views. In such a setting, operating over multiple data stores, with potentially different data models, dramatic performance enhancements can be attained by materializing the right view in the right store, that is: the one which can provide the best performance for the operations that need to be applied on it. ESTOCADA supports this functionality by enabling localas-view query rewriting over a large range of data models and heterogeneous stores. Its immediate benefit is flexibility since it enables taking advantage in an optimal way of any combination of underlying data stores. At the technical core of ESTOCADA lies the constraint-based query rewriting.
targeting with associated target). It is then matched to Computed Endorsed facts stemming from accessing an endorsed knowledge source. This comparison leads to a Computed Class {0,6,7,8} for each nucleotide sequence. The benchmark aims to compute quality metrics by comparing system outputs to the Ground Truth 1 3 and the reagent-gene homology can be assessed using the BLASTN software  (Altschul et al. 1997). A reagent is said to be: • targeting gene X when it exhibits a significant level of homology or identity with gene X. • non-targeting when it exhibits no significant homology with any gene or sequence of the stated genome. The rule-based algorithm illustrated in Fig. 2  of the "Appendix" (Reagent-gene assessment of nucleotide sequence homology) assesses the significance of homology, leading to predictions of whether reagents are likely to be targeting or non-targeting.
These rules have been determined and validated on the base of laboratory practices (BF, JAB, and TG) and on common siRNA and shRNA rule design  (Yu et al. 2002). Deciding on the (non-)targeting nature of a reagent requires feeding this rule-based algorithm with the output of the BLASTN software (run with the reagent given as input). An endorsed fact (Computed Endorsed fact in Fig. 1 ) reflects the current knowledge provided from BLASTN and refined by the rule-based algorithm. In biomedical publications, nucleotide sequence reagents are claimed to be (non-) targeting a gene. For various reasons a claim may be wrong  (Labbé et al. 2019) , such as through typographical errors, copy and paste errors or through limited understanding of the experiments described. Invalidating a claim requires a comparison of an automatically extracted claimed fact with the BLASTN endorsed fact.
Deriving the ERC_H_v2 test collection The benchmarking of fact-checking software requires the assembly of a test collection that represents a variety of errors. This is challenging as most of the literature is supposed to be free from such errors! We therefore designed a threefold strategy to identify error-prone papers: 1. Two of the authors (JAB and CL) reported 48 highly similar publications that commonly described nucleotide sequence reagents  (Byrne and Labbé 2017). JAB factchecked a proportion of the nucleotide sequence reagents described in these papers using BLASTN. 2. The list of aforementioned dubious sequences was used as query to Google Scholar to retrieve additional, potentially questionable, papers. We hypothesised that some erroneous papers may stem from paper mills  (Liu and Chen 2018)  that are known to produce papers with similar textual contents.
All 48 papers flagged in Step 1 were used as seeds (i. e. , starting points) to collect 1,664 similar papers using the PubMed Similar articles feature. We manually screened these error-prone papers to constitute the ground truth illustrated in Fig. 1. In a preliminary analysis, four experts (BF, JAB, TG, and Ms Natalie Grima  (Labbé et al. 2019 )) annotated 44 papers. Each paper was presented in PDF format while the markup-language counterpart was also downloaded in XML-JATS or HTML for further processing. Experts performed manual fact-checking, involving paper reading to identify nucleotide sequences, delineation of the purpose of each nucleotide sequence in the reported experiments, as well as the analysis of the BLASTN results they obtained. Among the annotated 44 papers mentioning 77 error-prone sequences, 21 sequences from 12 papers triggered discussions and all conflicting cases were resolved after reaching consensus.
This preliminary analysis stressed two kinds of issues. First, the wording of some 1 3 papers was either ambiguous or difficult to understand. Second, all papers mentioning any nucleotide sequence not related to the human genome or transcriptome required the identification of the appropriate nucleotide sequence database for reagent fact-checking. Following this preliminary analysis, we decided to focus on papers where all nucleotide sequence reagents relate to the human genome or transcriptome. The ground truth capturing expert knowledge was acquired as follows. Biomedical researchers (JAB, RAW, and Natalie Grima) annotated a subset of 161 papers among the previously collected ones. The annotators' assessment of each nucleotide sequence was recorded in the ground truth pictured in Fig. 1 , namely the claimed characteristic targeting or non-targeting with gene name or target where applicable, the endorsed fact, and the output Class {0,6,7,8}.
Eventually, the ground truth resulted from the analysis of 161 papers con- taining 1679 nucleotide sequences annotated by at least one expert (JAB, RAW, or Natalie Grima). All papers were processed by Grobid and pdftotext to extract two plain text versions. This resulted in three different input formats denoted as: 1. ML: markup-language (either XML or HTML). 2. GB: plain text extracted using Grobi d v. 0. 5. 2 released October 17, 2018. 3. PT: plain text extracted using pdfto text v. 0. 78. 0 released June 26, 2019. These constitute the test collection of the proposed benchmark, called ERC_H_v2 for Error Reagent Collection, Human, version 2. We used ERC_H_v2 to report the results of benchmarking the Seek&Blastn tool in the next section. Note that these results do not generalise to the entire biomedical literature, as the ground truth is biased towards error-prone papers.
Assessing the efficiency of the fact-checking Seek&Blastn tool Running the ERC benchmark allowed us to examine the extent to which input format impedes the flagging of errors in the biomedical literature. All metrics reported in Table  2  were computed for the three different input formats: ML, GB, and PT. Overall, the share of correctly classified sequences in Class {0,6,7,8} reflects the performance of the fact-check- ing on a given input format (see the aforementioned Fact-check_OK definition). ML is the most appropriate format for Seek&Blastn with a 79% success rate (Fact-check_OK) when processing text in HTML or XML formats. Grobid (GB) appears to be a viable alternative with a 69% success rate (Fact-check_OK) on the text extracted from PDFs with this software. Fact-checking effectiveness decreases to 58% of all sequences correctly classified based on the text produced with pdftotext (PT).
The variability and structural ambiguity of either natural language or table layouts makes CP2 particularly challenging. The tied performance of ML and GB suggests that text extraction with GB managed to preserve text structure. In contrast, PT produced a less suitable text for Seek&Blastn to identify sequence status correctly. • Gene identification (CP3) with ML or GB inputs (66% and 64% Gene_OK) outperforms PT (49% Gene_OK). This task first identifies gene names out of the tokenized sentence, where there can be variability in naming genes and spelling gene identifiers  (Giorgi and Bader 2020). Then the associated nucleotide sequence must be identified. Results reflect that CP3 is the most complex task. Seek&Blastn clearly needs to be improved in this direction. Again, the tied performance of ML and GB suggests that text extraction with GB is a suitable alternative to ML.
We expect this latter component to help fact-checking designers to create and tune their systems to reach higher accuracy. In closing, let us stress the sensitivity of most classification approaches to class imbalance which is a well-known problem in machine learning  (Japkowicz and Stephen 2002). This 1 3 for the textual passages of the papers in the ERC_H_v2 test collection 1 3 is also characteristic of the fact-checking ground truth contributed in this paper, revealed by the sizes of each class for the text in ML format ( Class 0 : 1361, Class 6 : 74, Class 7 : 35, Class 8 : 139, as seen in Table  3 ) While the rule-based classification implemented in Seek&Blastn  (Labbé et al. 2019 ) makes this software resilient to data imbalance, using other techniques (e. g. , Support Vector Machines) on such imbalanced classes may require advanced over/under-sampling.
Discussion Let us now discuss the ERC_H_v2 test collection we contributed in this paper comprising: 1. The input given to the fact-checking system under study: a textual corpus. 2. The ground truth: the expected output of the system under study. 3. Metrics to assess the quality of a given output regarding the expected output. First, we reported the results of Seek&Blastn  (Labbé et al. 2019) , which is the first biomedical reagent fact-checking software to our knowledge. By releasing the benchmark with the associated ERC_H_v2 test collection, 4  we encourage other researchers to tackle the error detection problem. Running the benchmark on the outputs of different systems will inform on the accuracy of the benchmarked approaches (e. g. , machine learning vs. rule-based decision making). As such, Seek&Blastn contributes a baseline for such other systems.
Conclusion The introduction of bibliographic databases fostered the indexing of scientific literature (De Bellis 2009). These products have been indexing metadata only: title, abstract, keywords, and authors. Current endeavours aim to collect the entire literature in plain text. For instance,  Pulla (2019)  reported that the Indian JNU initiative has indexed 73 million journal articles in PDF. Specific indexing of genes and chemicals for knowledge extraction is one of the reported purposes of such a giant resource. This endeavour is in the spirit of literature-based discovery of new knowledge  (Bruza and Weeber 2008). Collecting nearly all scientific publications in a single place offers another opportunity: one could screen the entire literature for errors. Retracting dubious papers is critical for human readers as well as for automated text-mining processes: misleading reasoning and inference must be prevented to improve the integrity of science and the trust that people place in science and the scientific method.
Previous work has flagged plagiarism  (Citron and Ginsparg 2015) , nonsensical papers  (Labbé and Labbé 2013) , and erroneous papers  (Byrne and Labbé 2017). Thousands of papers have been retracted from the literature (Van Noorden 2014;  Ledford et al. 2017). In the present study, we have screened published papers. Triggering the fact-checker earlier, during the peer review process, could save time and effort by desk-rejecting the offending submission. As is currently done to combat plagiarism  (Smart and Gaston 2019) , fact-checkers such as Seek&Blastn could be used to screen submitted manuscripts ahead of referee assignment. The ERC benchmark proposed in this paper contributes to enhance the quality of such systems. Designers can measure the failures related to each subcomponent of the factchecker, make changes to improve the subcomponents, and then check that no regressions in performance have occurred.
Acknowledgements We gratefully acknowledge the assistance of  Ms Natalie Grima (Children's Cancer Research Unit) , and funding from the  Post-Truth Initiative , a  Sydney University Research Excellence Initiative  ( SREI 2020 ) (to JAB), and from the  US Office of Research Integrity  Grant  ORIIR 18003 8-01-00  (to JAB and CL). This work was supported by donations to the  Children's Cancer Research Unit of the Chil-.
Methods In this section, the improvements for each step of the workflow of the algorithm published in  Sorzano et al. (2009)  are detailed. They have been included in the ImageJ  (Abramoff et al. , 2004; Schneider et al. , 2012)  plugin TomoJ  (Messaoudi et al. , 2007)  and are freely available from http://www. cmib. fr/en/download/softwares/TomoJ. html. These modifications affect the method used to generate landmark chains (Section 2. 1), the deformation model (Section 2. 2), and the 3D reconstruction algorithm that compensates for these deformations (Section 2. 3).