dataesr/scientific-paragraphs-categorization huggingface dataset:
Automated Dataset Generation Pipeline From a technical perspective, the CORD-19 corpus essentially consists of one JSON document per scientific article. Consequently, yielding the Covid-on-the-Web RDF dataset involves two main steps: process each document of the corpus to extract the NEs and arguments, and translate the output of both treatments into a unified, consistent RDF dataset. The whole pipeline is sketched in Fig. 1. Named entities extraction. The extraction of NEs with DBpedia Spotlight, Entity-fishing and BioPortal Annotator produced approximately 150,000 JSON documents ranging from 100KB to 50 MB each. These documents were loaded into a MongoDB database, and pre-processed to filter out unneeded or invalid data (e. g. , invalid characters) as well as to remove NEs that are less than three characters long.
Then, each document was translated into the RDF model described in Section 2. 1 using Morph-xR2RML,  21  an implementation of the xR2RML mapping language  [15]  for MongoDB databases. The three NE extractors were deployed on a Precision Tower 5810 equipped with a 3. 7GHz CPU and 64GB RAM. We used Spotlight with a pre-trained model  22  and Annotator's online API 23  with the Annotator+ features to benefit from the whole set of ontologies in BioPortal. To keep the files generated by Annotator+ of a manageable size, we disabled the options negation, experiencer, temporality, display links and display context. We enabled the longest only option, as well as the lemmatization option to improve detection capabilities. Processing the CORD-19 corpus with the NEs extractors took approximately three days.
Publishing and Querying Covid-on-the-Web Dataset The Covid-on-the-Web dataset consists of two main RDF graphs, namely the CORD-19 Named Entities Knowledge Graph and the CORD-19 Argumentative Knowledge Graph. A third, transversal graph describes the metadata and content of the CORD-19 articles. Table  1  synthesizes the amount of data at stake in terms of JSON documents and RDF triples produced. Table  2  reports some statistics against the different vocabularies used. Dataset Description. In line with common data publication best practices  [8] , we paid particular attention to the thorough description of the Covidon-the-Web dataset itself. This notably comprises (1) licensing, authorship and provenance information described with DCAT 25  , and (2) vocabularies, interlinking and access information described with VOID 26.
The interested reader may look up the dataset URI 27  to visualize this information. Dataset Accessibility. The dataset is made available by means of a DOIidentified RDF dump downloadable from Zenodo, and a public SPARQL endpoint. All URIs can be dereferenced with content negotiation. A Github repository provides a comprehensive documentation (including licensing, modeling, named graphs and third-party vocabularies loaded in the SPARQL endpoint). This information is summarized in Table  3. Reproducibility. In compliance with the open science principles, all the scripts, configuration and mapping files involved in the pipeline are provided in the project's Github repository under the terms of the Apache License 2. 0, so that anyone may rerun the whole processing pipeline (from articles mining to loading RDF files into Virtuoso OS).
Dataset Licensing. Being derived from the CORD-19 dataset, different licences apply to the different subsets of the Covid-on-the-Web dataset. The subset corresponding to the CORD-19 dataset translated into RDF (including articles metadata and textual content) is published under the terms of the CORD-19 license. 28 In particular, this license respects the sources that are copyrighted. The subset produced by mining the articles, either the NEs (CORD19-NEKG) or argumentative components (CORD19-AKG) is published under the terms of the Open Data Commons Attribution License 1. 0 (ODC-By). 29 Sustainability Plan. In today's context, where new research is published weekly about the COVID-19 topic, the value of Covid-on-the-Web, as well as other related datasets, lies in the ability to keep up with the latest advances and ingest new data as it is being published.
Visualization and Current Usage of the Dataset Beyond the production of the Covid-on-the-Web dataset, our project has set out to explore ways of visualizing and interacting with the data. We have developed a tool named Covid Linked Data Visualizer 30 comprising a query web interface hosted by a node. js server, a transformation engine based on the Corese Semantic Web factory  [5] , and the MGExplorer graphic library  [4]. The web interface enables users to load predefined SPARQL queries or edit their own queries, and execute them against our public SPARQL endpoint. The queries are parameterized by HTML forms by means of which the user can specify search criterion, e. g. , the publication date. The transformation engine converts the JSON-based SPARQL results into the JSON format expected by the graphic library.
The proposed use of information visualization techniques is original in that it provides users with interaction modes that can help them explore, classify and analyse the importance of publications. This is a key point for making the tools usable and accessible, and get adoption. During a meeting with some health and medical research organisations (i. e. , Inserm and INCa), an expert provided us with an example query that researchers would be interested in solving against a dataset like the one we generated: "find the articles that mention both a type of cancer and a virus of the corona family". Taking that query as a first competency question, we used Covid Linked Data Visualizer whose results are visualized with the MGExplorer library (Fig. 2 ). We also created several Python and R Jupyter notebooks 31  to demonstrate the transformation of the result into structures such as Dataframes 32  for further analysis (Fig.
Potential Impact and Reusability To the best of our knowledge, the Covid-on-the-Web dataset is the first one integrating NEs, arguments and PICO elements into a single, coherent whole. We are confident that it will serve as a foundation for Semantic Web applications as well as for benchmarking algorithms and will be used in challenges. The resources and services that we offer on the COVID-19 literature are of interest for health organisations and institutions to extract and intelligently analyse information on a disease which is still relatively unknown and for which research is constantly evolving. To a certain extent, it is possible to cross-reference information to have a better understanding of this matter and, in particular, to initiate research into unexplored paths. We also hope that the openness of the data and code will allow contributors to advance the current state of knowledge on this disease which is impacting the worldwide society.
In addition to being interoperable with central knowledge graphs used within the Semantic Web community, the visualizations we offer through MGExplorer and Notebooks show the potential of these technologies in other fields, e. g. , the biomedical and medical ones. Interest of communities in using the Dataset and Services. Several biomedical institutions have shown interest in using our resources, eithet direct project partners (French Institute of Medical Research -Inserm, French National Cancer Institute -INCa) or indirect (Antibes Hospital, Nice Hospital). For now, these institutions act as potential users of the resources, and as co-designers. Furthermore, given the importance of the issues at stake and the strong support that they can provide in dealing with them, we believe that other similar institutions could be interested in using the resources.
Answering these complex queries requires exploration of the CORD-19 corpus, and for that we offer a variety of analysis and visualization tools. These queries and the generic typology shall be reused in further extensions and other projects. The Covid Linked Data Visualizer (presented in section 4) supports the visual exploration of the Covid-on-the-Web dataset. Users can inspect the attributes of elements in the graph resulting from a query (by positioning the mouse over elements) or launch a chained visualization using any of the interaction techniques available (ex. IRIS, ClusterVis, etc). These visualization techniques are meant to help users understand the relationships available in the results. For example, users can run a query to visualize a co-authorship network; then use IRIS and ClusterVis to understand who is working together and on which topics.
Related Works Since the first release of the CORD-19 corpus, multiple initiatives, ranging from quick-and-dirty data releases to the repurposing of existing large projects, have started analyzing and mining it with different tools and for different purposes. Entity linking is usually the first step to further processing or enriching. Hence, not surprisingly, several initiatives have already applied these techniques to the CORD-19 corpus. CORD-19-on-FHIR 35  results of the translation of the CORD-19 corpus in RDF following the HL7-FHIR interchange format, and the annotation of articles with concepts related to conditions, medications and procedures. The authors also used Pubtator  [21]  to further enrich the corpus with concepts such as gene, disease, chemical, species, mutation and cell line.
KG-COVID-19 36 seeks the lightweight construction of KGs for COVID-19 drug repurposing efforts. The KG is built by processing the CORD-19 corpus and adding NEs extracted from COVIDScholar. org and mapped to terms from biomedical ontologies. Covid9-PubAnnotation 37 is a repository of text annotations concerning CORD-19 as well as LitCovid and others. Annotations are aggregated from multiple sources and aligned to the canonical text that is taken from PubMed and PMC. The Machine Reading for COVID-19 and Alzheimer's 38 project aims at producing a KG representing causal inference extracted from semantic relationships between entities such as drugs, biomarkers or comorbidities. The relationships were extracted from the Semantic MEDLINE database enriched with CORD-19. CKG-COVID-19 39 seeks the discovery of drug repurposing hypothesis through link prediction.
Finally, to the best our knowledge, our approach is the only one to integrate argumentation structures and named entities in a coherent dataset. Argument(ation) Mining (AM)  [3]  is the research area aiming at extracting and classifying argumentative structures from text. AM methods have been applied to heterogeneous types of textual documents. However, only few approaches  [22, 11, 14]  focused on automatically detecting argumentative structures from textual documents in the medical domain, e. g. , clinical trials, guidelines, Electronic Health Records. Recently, transformer-based contextualized word embeddings have been applied to AM tasks  [18, 14]. To the best of our knowledge, Covid-on-the-Web is the first attempt to apply AM to the COVID-19 literature.
Conclusion and Future Works In this paper, we described the data and software resources provided by the Covid-on-the-Web project. We adapted and combined tools to process, analyze and enrich the CORD-19 corpus, to make it easier for biomedical researchers to access, query and make sense of COVID-19 related literature. We designed and published a linked data knowledge graph describing the named entities mentioned in the CORD-19 articles and the argumentative graphs they include. We also published the pipeline we set up to generate this knowledge graph, in order to (1) continue enriching it and (2) spur and facilitate reuse and adaptation of both the dataset and the pipeline. On top of this knowledge graph, we developed, adapted and deployed several tools providing Linked Data visualizations, exploration methods and notebooks for data scientists.
Through active interactions (interviews, observations, user tests) with institutes in healthcare and medical research, we are ensuring that our approach is guided by and aligned with the actual needs of the biomedical community. We have shown that with our dataset, we can perform documentary research and provide visualizations suited to the needs of experts. Great care has been taken to produce datasets and software that meet the open and reproducible science goals and the FAIR principles. We identified that, since the emergence of the COVID-19, the unusual pace at which new research has been published and knowledge bases have evolved raises critical challenges. For instance, a new release of CORD-19 is published weekly, which challenges the ability to keep up with the latest advances. Also, the extraction and disambiguation of NEs was achieved with pre-trained models produced before the pandemic, typically before the SARS-Cov-2 entity was even created in Wikidata.
7 https://github. com/Wimmics/covidontheweb/dataset 8 https://www. dublincore. org/specifications/dublin-core/dcmi-terms/ 9 https://sparontologies. github. io/fabio/current/fabio. html 10 http://bibliontology. com/specification. html 11 http://xmlns. com/foaf/spec/ 12 https://schema. org/ 13 https://www. w3. org/TR/annotation-vocab/ 14 https://www. w3. org/TR/prov-o/ 15 PICO is a framework to answer health-care questions in evidence-based practice that comprises patients/population (P), intervention (I), control/comparison (C) and outcome (O).
Mathematical reference parameter values are calculated for a series of fifteen predefined surfaces and ten parametric surfaces to assess the performance of the software under test for a given dataset resolution. Assessment of the number of significant figures of the software-obtained values that agree with the reference values is used as a performance metric that enables comparison between different third-party software applications for a given dataset resolution. An assessment of the sampling methods used to create discrete datasets of a mathematical surface function for use with numerical third-party software is performed. Two implementations of surface height sampling are used to create datasets that are input into four third-party surface texture parameter calculation software packages, and the results compared, showing a significant variation in the performance metric values for different sampling methods.
Introduction Surface texture parameters are quantitative descriptors of a surface topography measurement that serve as a crucial aspect of precision engineering  [1, 2] , enabling greater control of functional part properties such as friction, wear and lubricant retention  [3] [4] [5]. Surface texture parameter values are often calculated using third-party software, and it is important that the results obtained by these software packages agree with the parameter definitions laid out in the ISO specification standards  [6, 7]. Validation of surface texture parameter calculation software is performed using software measurement standards, typically developed by national measurement institutes (NMIs)  [8] [9] [10] [11] [12] [13]. Software measurement standards are given in two forms: type F1/S1 reference datasets with corresponding, well-defined parameter values, and type F2/S2 reference software, which calculates parameter values for any given input dataset with high accuracy  [14, 15].
While, theoretically, type F2/S2 reference software is designed to use high accuracy methods to obtain reference parameter values for a given dataset, in practice they are still discrete software implementations utilising finite precision arithmetic and numerical algorithms operating on a discretely sampled dataset, and are still subject to the same sources of approximation as commercial software  [16]. Previous work has shown variation in the parameter values obtained by reference software developed by different NMIs  [17]. The development of mathematical references for the validation of surface texture parameter calculation software is a valuable step towards providing users with a traceable assessment of the parameter values they obtain. Using this new approach, reference surfaces are defined and evaluated analytically, ensuring mathematical traceability and increased accuracy compared to the current state of the art.
By moving away from reference software, which is subject to differing interpretations of algorithms, users can have more confidence in the performance of the software under test and deliver more accurate specification and analysis of high-precision parts. Previous work introduced the use of mathematical references for functional areal surface texture parameters  [18] , showcasing this new mathematical approach. As all functional surface texture parameters are derived from the material ratio curve and not the surface heights directly, it was logical to define the material ratio curve analytically for direct calculation of the parameters, and obtain surface datasets by sampling the curve. For field surface texture parameters, the direct surface height information is needed, and so analytical definitions of the surface height, z(x, y), are required.
Mathematical parameter evaluation 2. 1 Parameter evaluation To ensure mathematical traceability to analytical surface representations, the surface texture parameter calculations must also be performed analytically. To perform analytical, or symbolic, calculations computationally, a computer algebra system (CAS) is required. A CAS is a mathematical software package or module that can manipulate symbolic mathematical expressions directly, typically using a series of rules or look-up tables dictating mathematical procedures, without relying on numerical approximation. Such an approach delivers seemingly infinite precision results, providing the system is successfully able to obtain a mathematical expression, and ensures mathematical traceability throughout the operation. There are a variety of CASs available, and while some are open-source such as SageMath  [20] , the systems that are most popular are closed-source and commercial products.
Sp, Sv and Sz The next three parameters, Sp, Sv and Sz, are conceptually simple in that they relate to the largest peak/valley height values on the surface  [7]. Sp corresponds to the largest peak height within the defined area, Sv corresponds to the magnitude of largest valley depth, and Sz is the total distance between the largest peak height and largest valley depth. For a discrete dataset, maximum peak/valley heights are obtained trivially by sorting the discrete heights and obtaining the highest and lowest values. For a continuous analytical surface, obtaining maximum peak/valley heights symbolically becomes more complex. The parameters require calculating the global maxima and minima of the function within the specified region. Several optimisation techniques can potentially obtain values corresponding to only local minima, not global minima, and so care must be taken to avoid this.
Each software package was given a 700 × 700 high resolution SF surface. SDF dataset sampled from the surface expressions defined in above, where resolution is defined as the number of individual height values included in the surface dataset. This resolution was chosen because software D suffered from an upper limit on the file size of datasets of 10 MB, and a resolution of 700 × 700 was the highest resolution that reliably came under this limit, ensuring comparisons to be possible across all four software packages. No additional form removal or filtration operations were performed to enable sole assessment of the parameter value calculation methods. The results of each of the third-party software tests are normalised relative to the mathematicallyobtained parameter values to allow for more direct comparisons.
Discrete-based software can obtain the largest height value in a surface dataset easily with simple sorting algorithms for the height values (although this may not correspond to the true peak, depending on where the continuous surface was sampled). In contrast, the Sdq parameter values show more significant differences from the mathematically-defined values, although these variations are still within a tenth of a percent of the mathemtically-obtained value. Despite these deviations, three of the four software packages perform similarly. The definition of the Sdq parameter is based on the average gradient of the surface, which can be highly dependent on the scale at which the surface is measured  [29]. This effect can also be affected by the resolution of the surface, and so it should be considered that the 700 × 700 dataset sampling resolution of the analytical surfaces is a primary reason for the deviations.
By identifying which parameters deviate the most from a mathematical reference, in comparison to other software packages, resources can be allocated more efficiently, ensuring the weakest areas of the software are addressed first. The amount of information given to the software via a discrete dataset input can put a limit on the software's ability to obtain a parameter value close to the mathematical reference value. However, by performing an assessment at several resolutions of input surface dataset, software developers can identify to what extent the deviation from the mathematical parameter value is due to discretisation, and what is due to shortcomings in the software. This idea is addressed further in section 5.
Chebfun To perform the extended precision calculations, the MATLAB plug-in Chebfun was used  [30, 31]. Chebfun is an open source software package that utilises the idea that smooth functions can be effectively represented by expansions of Chebyshev polynomials (which are a series of recursively defined polynomials whose roots are used in polynomial interpolation techniques). Analytical functions are stored using the roots of Chebyshev polynomials, and are able to be represented accurately to machine precision, that is, fifteen to sixteen significant decimal digits. The use of Chebyshev polynomials to represent smooth non-periodic functions has been seen as an equivalent to Fourier series and its ability to represent smooth periodic functions and is a major component of the branch of mathematics known as approximation theory  [32, 33].
The nature of the process is stable for even very large numbers of Chebyshev points, corresponding to complex analytical surfaces. The aim of the Chebfun software is to achieve for functions what floating-point arithmetic achieves for numbers: rapid computation in which each successive operation is carried out exactly apart from a rounding error that is very small in relative terms  [34]. This precise, numerical approach to computation with functions is an effective solution for overcoming the current computational limitations of purely symbolic calculations using CASs.
Comparison with mathematical evaluation Before applying the Chebfun methods to new applications, such as complex surfaces and analytically challenging parameters, it is first worthwhile to perform a test to gain confidence in Chebfun's performance. This can be achieved by using the Chebfun method to calculate the same parameters as can be obtained symbolically using a CAS. A selection of seven parameters were calculated for eight analytical surfaces mathematically, using the methods described in section 2. 1. Each of the analytical results were then evaluated to fifteen significant decimal figures and stored numerically. Each analytical surface expression was then input into MATLAB and converted into a two-dimensional Chebfun object over the required evaluation area. Algorithms for the calculation of each surface texture parameter were performed, utilising the relevant Chebfun-specific functions.