dataesr/scientific-paragraphs-categorization huggingface dataset:
However, CompuCell3D specifies models using a custom XML format (CC3DML) and Python scripts, thereby limiting model portability. While recent versions of Compu-Cell3D adopt a modular software architecture API, they currently lack integrated backing for model annotation, parameter constraints, provenance and validation data integration as part of model specification. This is also the case for other similar tools such as PhysiCell  [67] , Chaste  [68]  and CellSys  [69]. An effort to promote best practices regarding model development and annotation workflows across multiple virtual-tissue platforms will be essential to making virtual-tissue models truly reusable and shareable.
Use of scorecards and simple rules to enhance model reproducibility The reproducibility of scientific results is of utmost importance in research. The team developing the BioModels database of mathematical models of biological systems  [2]  has attempted to systematically reproduce 455 kinetic models published in peer-reviewed research articles from 152 journals  [70]. Unfortunately, about half (49%) of the models could not be reproduced using the information provided in the published manuscripts. With further effort, an additional 12% of the models could be reproduced either by empirical correction or feedback from authors. The main reasons why reproduction failed were missing parameter values, followed by missing initial conditions and inconsistencies in the model structure. As Along the same lines, a recent publication suggests 10 simple rules for credible modelling practices in healthcare  [71].
In parallel, the logical modelling community proposed an adaptation of SED-ML to cover the needs of logical models. The community also incorporated more tools into CoLoMoTo notebooks  [56]  to promote reproducibility and interoperability. An adapted SED-ML for logic-based models would promote the use of COMBINE archives, harmonising best practices of model development across communities. Computational biology models should also follow the FAIR principles. Most of the good-practice guidelines followed and suggested by individual systems biology communities fulfil the FAIR criteria, even when not explicitly mentioning the FAIR facets. For example, FAIR research objects should be findable  [42]. The BioModels database, an open repository for simulation studies of biomedical systems  [2] , provides each model with a persistent Identifier (ID) that can be represented as a uniform resource identifier to guarantee global uniqueness  [75].
Computational biology models have long been reused and provide many examples for adopting FAIR principles. For example, the HealthEcco project (https:// healthecco. org/) integrates health-related data for several diseases in a graph database at the intersection between health and systems medicine. HealthEcco relies solely on standardised interfaces to access data from open repositories. For their data section on COVID-19, HealthEcco integrated computational biology models from the COVID collection of Biomodels (https://www. ebi. ac. uk/biomodels/covid-19) with patents, biomedical ontologies, PubMed entries and clinical trials  [76]. More recently, a workgroup at COMBINE 2021 (https:// combine-org. github. io/events/) investigated the FAIR metrics for computational models to determine how FAIRification can lead to higher quality, more reliability and, ultimately, more frequent reuse of model-based results in biomedical applications.
Many scientific journals have updated their policies to include hosting the code, datasets and resources necessary to reproduce the research detailed in submitted manuscripts. It would be helpful if this support could extend to computational models and simulations, as very often published models are not annotated at all, include no references to the sources used to infer reactions and the simulation settings are vaguely described, hampering the reproducibility of the results and the reusability of the models. Support for and use of public, shared repositories can improve the findability and reusability of models. As these issues affect all computational models, modelling communities should improve communication to collectively develop solutions to these challenges.
The purpose here is to emphasise that, even if extensive experimental evaluation is reported in IR papers, the reports are mainly under the form or tables and curves, which are low level data analysis representations that we do not discuss in the rest of this paper. Here, a deep learning-based model was used and comparisons are reported on the different training types, encoders, and batch sizes; using different effectiveness measures (nDCG@10, MRR@10, and R@1K), on different collections (here TREC DL'19, TREC-DL'20, and MSMARCO DEV). The best results are highlighted in bold font. the bottom part is a typical graph to compare different variants or hyper-parameters effect on effectiveness. Here, the lines represent different combination of hyperparameters, effectiveness is measured in terms of recall (Y-axis) for different cut-off of the retrieved document list.
Data Analysis Methods System effectiveness analyses rely on different statistical analysis methods, including, but not limited to, machine learning. Boxplot is a graphical representation of a series of numerical values that shows their locality, spread, and skewness based on their quartiles. Whiskers extend the Q1-Q3 box, indicating variability outside the upper and lower quartiles. Beyond the whiskers, outliers that differ significantly from the rest of the dataset are plotted as individual points. Effectiveness under different conditions (different queries, different values of a component parameter) is a typical series that can be represented under the form of a boxplot. Correlation is a family of analysis that measures the relationship between two variables, its strength and direction. Correlation calculation results in a value that ranges between -1 (strong negative correlation) and 1 (strong positive correlation); 0 indicating that the two variables are not correlated.
Data and Data Structures for System Effectiveness Analysis There are different international challenges in IR where participants use the same data collections to answer shared tasks and, thus, that can be used to deeply analyse system effectiveness, its factors and parameters. In this paper, the studied papers focused on the pioneering TREC challenge. TREC considered many different languages, but when it began and nowadays it is mainly focused on English. TREC encompasses various tasks; the most popular and running from the largest number of years is ad hoc retrieval where the task is to retrieve the relevant documents, given a query. It was also the first and unique task introduced in TREC in 1992  [40]. This paper focuses on ad hoc retrieval. System performance analyses (presented in Sections 4-6) share the same type of data structures, namely matrices.
System Performance Analysis Based on Their Participation to Evaluation Challenges International evaluation forums such as TREC. TREC started in 1992 and supports research in the information retrieval and provides the framework for large-scale evaluation of information retrieval have provided shared collections consisting of queries, documents, and relevance judgements, but also consisting of participants' results. For a given collection, we have different systems that ran the queries over the same document set and are evaluated the same way. Indeed, each participant to TREC tracks receives a detailed report where their runs or systems are scored according to various effectiveness measures obtained by the open evaluation tool trec_eval described in https://github. com/topics/trec-eval accessed on 15 May 2022.
It was made possible because of the valuable work that has been performed in Glasgow on Terrier  [5]  to implement IR components from the literature. Other platforms can also serve this purpose, such as Lemur/Indri (https://www. lemurproject. org/ accessed on 15 May 2022); https://sourceforge. net/p/lemur/wiki/ Indri%20Retrieval%20Model/ accessed on 15 May 2022 although more centred on language models or Cherche (https://github. com/raphaelsty/cherche accessed on 15 May 2022) for neural models. Compared to using participants' systems, generated query processing chains gives the ability to know the exact components and hyper-parameters used and thus make deeper analysis possible. One of the pioneer studies that analysed a huge number of automatically generated systems is Ayter et al. [7]. They used about 80,000 combinations of components in query processing chains with: 4 stemmers, 7 weighting models, 6 query processing, 7 query expansion models, and various numbers of query terms and documents to consider in query expansion.
a b s t r a c t Knowledge Graphs (KGs) integrate heterogeneous data, but one challenge is the development of efficient tools for allowing end users to extract useful insights from these sources of knowledge. In such a context, reducing the size of a Resource Description Framework (RDF) graph while preserving all information can speed up query engines by limiting data shuffle, especially in a distributed setting. This paper presents two algorithms for RDF graph summarization: Grouping Based Summarization (GBS) and Query Based Summarization (QBS). The latter is an optimized and lossless approach for the former method. We empirically study the effectiveness of the proposed lossless RDF graph summarization to retrieve complete data, by rewriting an RDF Query Language called SPARQL query with fewer triple patterns using a semantic similarity.
Introduction During the past decades, the number of linked datasets -known as knowledge graphs (KGs)-has rapidly increased as evidenced in the current state of the Linked Open Data cloud 1. These datasets are structured following the W3C's standard Resource Description Framework, RDF  ( Manola, Miller, McBride et al. , 2004 ) , and share knowledge on various domains, from a more general purpose KGs such as DBpedia  ( Lehmann et al. , 2015 )  or WikiData  ( Vrandecic & Krötzsch, 2014 )  to specialized ones, e. g. , SemanGit  ( Kubitza, Böckmann, & Graux, 2019 ). Real-world applications over these types of sources demand the development of optimized techniques to extract meaningful information. The Semantic Web community has actively contributed to RDF management and has proposed formalisms, e.
Graph summarization in RDF Graph databases use graph structures for representing entities as nodes and their relationships as edges of a graph  Bourbakis (1998). The increment of data in graph databases makes the query processing complicated. Summarization technique is a way to overcome the complexity of search query in graph databases  ( LeFevre & Terzi, 2010 ). Graph summarization has been studied for semi-structured graph data models such as XML  ( Qun, Lim, & Ong, 2003  ) and RDF graphs. There are many existing works in this area to compact RDF graphs whose structure is computed from the original RDF graph, such that all the paths present in the original graph are also present in the summary graph  ( Bonifati, Dumbrava, & Kondylakis, 2020 ). These techniques can be classified into four categories, as the following  ( Cebiric et al.
g. ,  Singh, Devi, Devi, & Mahanta (2022)  propose a method to group the terms with similar semantic meaning by evaluating the similarity between words using GloVe  ( Pennington, Socher, & Manning, 2014 ). The results of applying this method to the RDF graph in Fig. 1   A portion of DBpedia that consists of 2047 RDF triples is presented in Fig. 2  a. Nodes and edges are represented as ovals and rectangles, respectively. The dataset has 1000 edges and 510 nodes. This figure also presents a SPARQL query comprising four triple patterns. The evaluation of this query retrieves five answers that correspond to the names of people in Germany, or with German nationality, or born or die in Germany. The Sparklify query engine 3  produces these answers in 19 seconds. Fig. 2  b and c depict the execution time over summarized RDF graphs computed following the two graph summarization methods previously described.
Similarity values can guide the summary of properties in RDF graphs and allow for the transformation of SPARQL queries. Fig. 3  illustrates a transformed SPARQL query. Albeit simpler, the transformed SPARQL query can retrieve the same results as the original query, but in less time. Meanwhile, there are a number of applications where data are represented in the form of graphs, which required graph embedding. RDF2Vec presented by Ristoski, Rosati, Noia, Leone, & Paulheim (2019) as a graph embedding model is applied to learn the context of the relations. In the case of RDF knowledge graphs, entities and relations between entities are considered instead of word sequences. First, the graph data is converted into sequences of entities; it can be considered as sentences using two different approaches, i.
e. , graph walks and Weisfeiler-Lehman (WL) subtree RDF graph kernels. Using those sen-tences, RDF2Vec trains the same neural language models to represent each entity in the RDF graph as a vector of numerical values in a latent feature space. Built on existing results on graph embeddings and summarization, we propose two approaches for summarizing RDF graphs. The first called Grouping Based Summarization (GBS) approach; it summarizes the RDF graph based on grouping subjects with the same predicates and objects. The second one, optimized for the first one, called Query Based Summarization (QBS) considers only the part of the RDF graph which is related to the SPARQL query. In the next, GBS and QBS are defined in detail.
A naive approach for summarizing RDF graphs A Grouping Based Summarization (GBS) approach able to reduce size of RDF graph represents our naive method. GBS works into phases: Offline Phase and Online Phase. In Fig. 4  both phases are shown. The  input of the offline phase is an RDF dataset that has been loaded as an RDF graph, and the output is a semantic summary graph. For the online phase, the input is a generated summary graph from the offline phase, a SPARQL query, and a semantic similarity metric and the output is a transformation of the query to the one with fewer triple patterns with final answers. In the offline phase presented in Algorithm 1 , data should be preprocessed. All edges and vertices are read from the RDF graph. As discussed before, the aim is to provide a semantic summary graph without losing the needed information.
After receiving a dataset as an input in (step 1) , the RDF graph is loaded (step 2). Then, the RDF graph is expanded to find new relations which are already not available in the original RDF graph, but they have similar semantic meanings. This extension is performed by computing the transitive closure of the properties in the RDF graph. Inference layer is used in order to extract new knowledge. For inferring the new facts from the current knowledge bases, inference rules are applied. Transitive Closure (TC) is one of the inference rules which is exerted in this work to infer more facts. It is deployed in the graph in order to find more facts which not exist in the original one. For example, in the given original RDF graph, there is no relation between entities Germany and Country. As seen in Fig.
5  , by applying TC inference rule, the new triple ⟨Germany, type, Country ⟩ has been inferred out of existing triples ⟨Germany, type, EuropeanCountry ⟩ and ⟨EuropeanCountry, subClassOf, Country ⟩. Since the original RDF graph has expanded by applying inference rules with more triples, queries over Country and Germany can be equally answered. The Transitive Closure (TC) inference rule is deployed in the graph in order to find more facts (step 3) and for expanding the RDF graph. In (step 4) , the inferred RDF graph is generated with the new properties. Subjects with similar predicates and objects need to be identified to find Super-Nodes (SN). In order to store the results from massive data, in-memory Spark Resilient Distributed Dataset (RDD) is used (step 5). The Resilient Distributed Dataset (RDD) is the core of Apache Spark.
6  As it comes from the name, RDD is a resilient, distributed, and immutable collection of data that are partitioned over a cluster of machines. In Spark RDD, a cluster of workers is connected to a driver  or master node. A master node will take care of work execution while worker nodes execute the jobs which are split and then distributed to them. In Sparklify, RDF graphs are stored and modeled based on Spark RDD through fast processing for efficient evaluation of SPARQL queries over distributed RDF datasets. All subjects which have common predicates and objects are grouped. RDDs of (key,value) pairs are used like ( Predicate, Object, List ( Subjects )); this resembles the technique proposed by  Consens, Fionda, Khatchadourian, & Pirrò (2015)  ; it also finds the same nodes by clustering entities with the same type.
This idea is inspired by  Zhang et al. (2014). In this way, not only the execution time is reduced, but also all the necessary information is preserved. The input of this algorithm is an RDF dataset that has been loaded as an RDF graph, a SPARQL query, and a semantic similarity metric. The output is a transformation of the query to the simple one with fewer triple patterns and a semantic summary graph with final answers; it is presented in Algorithm 3. After receiving an RDF graph as input and load it (step 1-2) , the query of the user should be collected to extract predicates and objects related to the query as Super-Predicates and Super-Objects (step 3). In this approach, contrary to GBS, the inferred graph is not generated due to the time-consuming and considering small part of graph related to the query.
Empirical evaluation The effectiveness of the approaches described in Section 4 is analyzed based on the size of the RDF graph, the number of retrieved data, scalability, and query processing. We aim to answer the following research questions: RQ1) What is the impact of predicate relatedness by analyzing similarity measures in different size of datasets using diverse embedding models on finding complete answers? RQ2) Are the proposed RDF summary graph techniques able to reduce the size of the original RDF graph by considering only required triples for querying? RQ3) How is the effectiveness of the proposed summarized techniques on answer completeness and cardinality compared with the original RDF graph? RQ4) How is the impact of the complexity of query on query processing and execution time in the proposed RDF summary graph compared with the original RDF graph?.
Data preparation and methods GBS and QBS have been evaluated in terms of compactness, completeness, and execution time. The summarization ratio is calculated by comparing the size of original RDF graphs with the size of summary graphs generated by two summarization approaches. The number of retrieved answers over original graphs is compared with the number of answers by querying over summarized graphs. Finally, the speed of query processing is computed by total time of returning answers by original and summary graphs using Sparklify. One of the most important aspects of our evaluation is to collect and process the dataset to ensure that it is being worked by our approach. Therefore, the first step is preparing datasets to meet our conditions as an input. One condition is the dataset does not contain any literals.
Since literal values and properties coming with literal values cannot be embedded by knowledge graph embeddings techniques. Then, RDF dataset should be loaded as an RDF graph. Later, the complex query is evaluated against this graph, and the answers are returned by Sparklify. The number of results and the running time are measured. For the proposed approaches, after loading the RDF dataset as an original RDF graph, the summarization technique is applied to generate a summary RDF graph. The number of answers retrieved by querying transformed queries over the summarized graph and query processing time are compared with the results produced over the RDF graph.
Datasets and queries Four datasets with different sizes are applied to realize the effectiveness of the mentioned techniques. One is a small part of DBpedia dataset consists of 2,047 triples wherein a bunch of companies, persons, and places with some of their information are stored. The second one is an Entity Summarization BenchMark (ESBM) with 6,584 triples which are sample entities from two datasets, DBpedia and LinkedMDB  ( Hassanzadeh & Consens, 2009 )  a popular movie database. The other selected datasets are Waterloo SPARQL Diversity Test Suite (WatDiv) with 10,916,457 triples (WatDiv. 10M) and with 108,997,714 triples (WatDiv. 100M) as medium and large datasets, respectively. Our evaluation is composed of 15 queries selected from QALD-3 7  for DBpedia dataset, from ESBM Benchmark v1.
Impact of predicate relatedness by analyzing similarity measures The distribution of similar predicates in each RDF graph using diverse embedding models is different. Referring to the research question RQ1 , the predicate relatedness in each dataset has an impact on finding complete answers. For rewriting queries, the embedding techniques such as Word2Vec and RDF2Vec find similar predicates to the given predicate in the query to transform it. If the most similar and semantically relevant predicates are found, the transformation of a query would be more efficient in terms of finding the complete answers. Since finding the most similar predicates can affect verifying the cardinality property; it becomes an important issue on rewriting queries. During experiments, it is discovered that not only different datasets with various sizes have a different distribution of similarity values, but also the different distribution can be seen in diverse embedding models.
It means not only the number of triples in RDF datasets can change the result of predicate relatedness, but also the techniques used for embedding have an impact. Figs. 10 and 11 show the distribution of cosine similarity to find the most similar predicates between the original RDF graph and summarized RDF graphs provided by Grouping Based Summarization(GBS) approach and Query Based Summarization (QBS) approach using word embedding model and graph embedding model, respectively. The results suggest that the way to find similar predicates depends on the size of the RDF graph and the technique for embedding. As seen, the probability of cosine similarity close to 1 in QBS approach is higher than the probability in the original graph and summary graph by GBS approach. The reason is, summary graphs in QBS have much fewer and more similar triples compared with the original ones.
In QBS approach, only a part of the RDF graph related to the query has been considered, not the whole graph. Hence, by considering the small part of RDF graph includes triples with the same objects, similar and relevant predicates are close to each other and the distance between them is less. Therefore, by decreasing the number of triples in the dataset and keeping relevant triples to the query, the probability of finding the most similar predicates to a given one in its neighborhood is higher. In GBS approach, the probability of having a cosine similarity close to 0. 5 using word embedding and close to 0. 7 using graph embedding is higher. The reason is, after summarizing RDF graphs based on an explanation in Section 4. 3 , many relevant triples are lost. So, the embedding model finds the properties which are not completely relevant and similar.