dataesr/scientific-paragraphs-categorization huggingface dataset:
SP 2 Bench. The SP 2 Bench 12 is a language-specific benchmark  [Schmidt et al. 2010]  which is based on the DBLP dataset. The generated datasets follow the key characteristics of the original DBLP dataset. In particular, the data mimics the correlations between entities. All random functions of the generator use a fixed seed that ensures that the data generation process is deterministic. SP 2 Bench is accompanied by 12 queries covering the various types of operators such as RDF access paths in addition to typical RDF constructs. DBPSB. DBpedia SPARQL Benchmark (DBPSB) 13 proposed at the University of Leipzig has been designed using workloads that have been generated by applications and humans  [Morsey et al. 2011; Morsey et al. 2012 ]. In addition, the authors argue that benchmarks like LUBM, BSBM, or SP 2 Bench resemble relational database benchmarks involving relational-like data which is structured using a small amount of homogeneous classes, whereas, in reality, RDF datasets are tending to be more heterogeneous.
For example, DBpedia 3. 6 consists of 289,016 classes, whereas 275 of them are defined based on the DBpedia ontology. In addition, in property values different data types as well as references to objects of the various types are used. Hence, they presented a universal SPARQL benchmark generation approach which uses a flexible data production mechanism that mimics the input data source. This dataset generation process begins using an input dataset; then multiple datasets with different sizes are generated by duplicating all the RDF triples with changing their namespaces. For generating smaller datasets, an adequate selection of all triples is selected randomly or using a sampling mechanism over the various classes in the dataset. The methodology is applied on the DBpedia SPARQL endpoint and a set of 25 templates of SPARQL queries is derived to cover frequent SPARQL features.
LODIB. The Linked Open Data Integration Benchmark (LODIB) 14 has been designed with the aim of reflecting the real-world heterogeneities that exist on the Web of Data in order to enable testing of Linked Data translation systems  [Rivero et al. 2012]. It provides a catalogue of 15 data translation patterns (e. g. , rename class, re-11 http://wifo5-03. informatik. uni-mannheim. de/bizer/berlinsparqlbenchmark/ 12 http://dbis. informatik. uni-freiburg. de/forschung/projekte/SP2B/ 13 http://aksw. org/Projects/DBPSB. html 14 http://lodib. wbsg. de/ move language tag etc. ), each of which is a common data translation problem in the context of Linked Data. The benchmark provides a data generator that produces three different synthetic data sets that need to be translated by the system under test into a single target vocabulary.
They reflect the pattern distribution in analyzed 84 data translation examples from the LOD Cloud. The data sets reflect the same e-commerce scenario used for BSBM. Geographica. The Geographica benchmark 15  has been designed to target the area of geospatial data  [Garbis et al. 2013 ] and respective SPARQL extensions GeoSPARQL  [Battle and Kolas 2012]  and stSPARQL  [Koubarakis and Kyzirakos 2010]. The benchmark involves a real-world workload that uses openly available datasets that cover various geometry elements (such as, e. g. , lines, points, polygons, etc. ) and a synthetic workload. In the former case there is a (1) a micro benchmark that evaluates primitive spatial functions (involving 29 queries) and (  2 ) macro benchmark that tests the performance of RDF engines in various application scenarios such as map exploring and search (consisting of 11 queries).
In the latter case of a synthetic workload the generator produces synthetic datasets of different sizes that corresponds to an ontology based on OpenStreetMap and instantiates query templates. The generated SPARQL query workload is corresponding to spatial joins and selection using 2 query templates. WatDiv. The Waterloo SPARQL Diversity Test Suite (WatDiv)  16  has been designed at the University of Waterloo. It implements stress testing tools that focus on addressing the observation that the state-of-the-art SPARQL benchmarks do not fully cover the variety of queries and workloads  [Alucet al. 2014 ]. The benchmark focuses on two types of query aspects -structural and data-driven -and performs a detailed analysis on existing SPARQL benchmarks (LUBM, BSBM, DBPSB, and SP 2 Bench) using these two properties of queries.
The structural features involve triple pattern count, join vertex degree, and join vertex count. The data-driven features involve result cardinality and several types of selectivity. The analysis of the four benchmarks reveals that their diversity is insufficient for evaluation of the weaknesses/strengths of the distinct design alternatives implemented by the different RDF systems. In particular, WatDiv, provides (1) a data generator which generates scalable datasets according to the WatDiv schema, (2) a query template generator which produces a set of query templates according to the WatDiv schema, and (3) a query generator that uses the generated templates and instantiates them with real RDF values from the dataset, and (4) a feature extractor which extracts the structural features of the generated data and workload.
RBench. RBench [Qiao and Özsoyo glu 2015] is an application-specific benchmark which receives any RDF dataset as an input and produces a set of datasets, that have similar characteristics of the input dataset, using size scaling factor s and (node) degree scaling factor d. These factors ensure that the original RDF graph G and the synthetic graph G ′ are similar and the average node degree and the number of edges of G ′ are changed by s and d respectively. A query generation process has been implemented to produce 5 different types of queries (edge-based queries, node-based queries, path queries, star queries, subgraph queries) for any generated data. The benchmark project FEASIBLE  [Saleem et al. 2015 ] is also an application-specific benchmark; however, contrary to RBench, it is designed to produce benchmarks from the set of sample input queries of a user-defined size.
In practice, one way for handling big RDF graphs is to process them using the streaming mode where the data stream could consist of the edges of the graph. In this mode, the RDF processing algorithms can process the input stream in the order it arrives while using only a limited amount of memory  [McGregor 2014 ]. The streaming mode has mainly attracted the attention of the RDF and Semantic Web community. S2Gen. Phuoc et al. [Le-Phuoc et al. 2012 ] presented an evaluation framework for linked stream data processing engines. The framework uses a dataset generated with the Stream Social network data Generator (S2Gen), which simulates streams of user interactions (or events) in social networks (e. g. , posts) in addition to the user metadata such as users' profile information, social network relationships, posts, photos and GPS information.
The data generator of this framework provides the users the flexibility to control the characteristics of the generated stream by tuning a range of parameters, which includes the frequency at which interactions are generated, limits such as the maximum number of messages per user and week, and the correlation probabilities between the different objects (e. g. , users) in the social network. RSPLab. Tommasini et al. [Tommasini et al. 2017 ] introduced another framework for benchmarking RDF Stream Processing systems, RSPLab. The Streamer component of this framework is designed to publish RDF streams from the various existing RDF benchmarks (e. g. , BSBM, LUBM). In particular, the Streamer component uses TripleWave  17  , an open-source framework which enables to share RDF streams on the Web  [Mauri et al.
2016]. TripleWave acts as a means for plugging-in and combining streams from multiple Web data sources using either pull or push mode. LDBC. The Linked Data Benchmark Council 18  (LDBC)  [Angles et al. 2014 ] had the goal of developing open source, yet industrial grade benchmarks for RDF and graph databases. In the Semantic Web domain, it released the Semantic Publishing Benchmark (SPB)  [LDBC 2015 ] that has been inspired by the Media/Publishing industry (namely BBC 19 ). The application scenario of this benchmark simulates a media or a publishing organization that handles large amount of streaming content (e. g. , news, articles). The data generator mimics three types of relations in the generated synthetic data: correlations of entities, data clustering, and random tagging of entities. Two workloads are provided: (1) basic, involving an interactive query-mix querying the relations between entities in reference data, and (2) advanced, focusing on interactive and analytical query-mixes.
The LDBC has designed two other benchmarks: the Social Network Benchmark (SNB)  [Erling et al. 2015]  for the social network domain (see Section 3. 4) and  Graphalytics [Iosup et al. 2016]  for the analytics domain. LinkGen. LinkGen is a synthetic linked data generator that has been designed to generate RDF datasets for a given vocabulary  [Joshi et al. 2016 ]. The generator is designed to receive a vocabulary as an input and supports two statistical distributions for generating entities: Zipf 's power-law distribution and Gaussian distribution. LinkGen can augment the generated data with inconsistent and noisy data such as updating a given datatype property with two conflicting values or adding triples with syntactic errors. The generator also provides a feature to inter-link the generated objects with real-world ones from user-provided real-world datasets.
The datasets can be generated in any of of two modes: on-disk and streaming. Strengths and Weaknesses of Semantic Web Graph Generators. Graphs are intuitive and standard representation for the RDF model that form the basis for the Semantic Web community which has been very active on building several benchmarks, associated with graph generators that had various design principles. A comparison of 4 RDF benchmarks (namely TPC-H [Solutions 2016] data expressed in RDF, LUBM, BSBM, and SP 2 Bench) and 6 real-wold data sets (such as, e. g. , DBpedia, the Barton Libraries  Dataset [Abadi et al. 2007]  or WordNet  [Miller 1995] ) has been reported by  [Duan et al. 2011]. The authors focus mainly on the structuredness (coherence) of each benchmark dataset claiming that a primitive metric (e. g. , the number of triples or the average in/outdegree) quantifies only some target characteristics of each dataset.
With respect to a type T the degree of structuredness of a dataset D is based on the regularity of instance data in D in conforming to type T. The type system is extracted from the data set by finding the RDF triples that have property 20  and extract type T from their object. Properties of T are determined as the union of all properties of type T. The structuredness is then expressed as a weighted sum of share of set properties of each type, whereas higher weights are assigned to types with more instances. The authors show that the structuredness of the chosen benchmarks is fixed, whereas real-world RDF datasets are belonging to the non-tested area of the spectrum. As a consequence, they introduce a new benchmark that receives as input any dataset associated with a required level of structuredness and size (smaller than the size of the original data), and exploits the input documents as a seed to produce a subset of the original data with the target structuredness and size.
Graph Databases Currently there exists a number of papers which compare the efficiency of graph databases with regards to distinct use cases, such as the community detection problem  [Beis et al. 2015] , social tagging systems  [Giatsoglou et al. 2011] , graph traversal  [Ciglan et al. 2012] , graph pattern matching  [Pobiedina et al. 2014] , data provenance  [Vicknair et al. 2010] , or even several distinct use cases  [Grossniklaus et al. 2013 ]. However, the number of graph data generators and benchmarks that have been designed specifically for graph data management systems (Graph DBMS) is relatively small. Either a general graph generator is used for benchmarking graph databases, such as, e. g. , the HPC Scalable Graph Analysis Benchmark  [Dominguez-Sal et al. 2010]  or the graph DBMS benchmarking tools are designed while having in mind a more general scope.
Hence it is questionable whether a benchmark that is targeted specifically for graph databases is necessary. [Dominguez-Sal et al. 2011 ] discussed this question and related topics. On the basis of a review of applications of graph databases (namely, social network analysis, genetic interactions and recommendation systems), the authors analyzed and discussed the features of the graphs for these types of applications and how such features can affect the benchmarking process, various types of operations used in these applications and the characteristics of the evaluation setup of the benchmark. In this section, we focus on graph data generators and benchmarks that have been primarily targeting graph DBMSs. XGDBench. XGDBench [Dayarathna and Suzumura 2014] is an extensible benchmarking platform for graph databases used in cloud-based systems.
Its intent is to automate benchmarking of graph databases in the cloud by focusing on the domain social networking services. It extends the Yahoo! Cloud Multiplicative Attribute (MAG) Graph Serving Benchmark (YCSB)  [Cooper et al. 2010 ] and provides a set of standard workloads representing various performance issues. In particular, the workload of XGDBench involves basic operations such as read / insert / update / delete an attribute, loading of the list of neighbours and BFS traversal. Using the generators, 7 workloads are created, such as update heavy, read mostly, short range scan, traverse heavy etc. The data model of XGDBench is a simplified version of the Multiplicative Attribute Graph (MAG)  [Kim and Leskovec 2010 ] model, a synthetic graph model which models the interactions between node attributes and graph structure.
Strengths and Weaknesses of Graph Database Generators. The graph DBMS generators discussed in this section have in common the fact that they can generate semantically rich labeled graphs with properties (ranging from properties values in GraphGen to MAG structures in XGDBench). They are also capable of generating graph instances and query workloads in concrete syntaxes (among which OpenCypher in GraphGen and gMark) and one of them (XGDBench) can also handle update operations on both graph structure and content. However, more comprehensive graph DBMS generators that also produce data manipulation operations (such as updates for graph databases) are urgently needed. Additionally, none of these generators is enabled to work on corresponding query languages for property graphs, such as the newly emerging standard GQL  [Chafi et al.
They can be categorized into statistical and agent-based. Statistical approaches  [Lancichinetti et al. 2008; Yao et al. 2011; Armstrong et al. 2013; Pham et al. 2013; Ali et al. 2014; Erling et al. 2015; Nettleton 2016]  focused on reproducing aspects of the network. In agent-based approaches  [Barrett et al. 2009; Bernstein and O'Brien 2013]  the networks are constructed by directly simulating the agents' social choices. Realistic Social Network. [Barrett et al. 2009 ] focused on the construction of realistic social networks. For this purpose the authors combine both private and public data sets with large-scale agent-based techniques. The process works as follows: In the first step it generates synthetic data by combining public and commercial databases. In the second step, it determines a set of activity templates.
2005b ] with link symmetry. It has two parameters: the symmetry probability P s and the burning probability P b. P b ensures a forward burning process based on BFS in which fire burns strongly with P b approaching 1. P s ensures backward linking from old nodes to new nodes and "adds fuel to the fire as it brings more links". LinkBench. The LinkBench benchmark  [Armstrong et al. 2013 ] has been designed for the purpose of analysis of efficiency of a database storing Facebook's production data. The benchmark considers true Big Data and related problems with sharding, replication etc. The social graph at Facebook comprises objects (nodes with IDs, version, timestamp and data) and associations (directed edges, pairs of node IDs, with visibility, timestamp and data). The size of the target graph is the number of nodes.
A heuristic observation that "the probability that two nodes are connected is typically skewed with respect to some similarity between the nodes" enables to focus only on sliding window of most probable candidates. The core idea of the framework is demonstrated using an example of a social network (consisting of persons and social activities). The dictionaries for property values are inspired by DBpedia and provided with 20 property value correlations. The edges are generated according to 3 correlation dimensions. SIB. The developers of the Social Network Intelligence BenchMark (SIB) 27  based the design of their benchmark on the claim that the state-of-the-art benchmarks are limited in reflecting the characteristics of the real RDF databases and are mostly focusing on the relational style aspects.
Hence, they proposed a benchmark for query evaluation using real graphs  [Boncz et al. 2013 ]. The proposed benchmark mimics using an RDF store for a social network. The distribution of the generated data for each type follows the distribution of the associated type inferred from real-world social networks. Additionally, association rules are exploited for representing the real-world data correlation in the generated synthetic data. The generated data is linked with the RDF datasets from DBpedia. The benchmark specification contains 3 query mixes -interactive, update, and analysis -expressed in SPARQL 1. 1 Working Draft. Cloning of Social Networks. [Ali et al. 2014]  introduces two synthetic generators to reproduce two characteristics typically observed in social networks: node features and multiple link types.
As new nodes are created, their labels are assigned based on the prior label distribution. After the network has reached the same number of nodes as the original social media dataset, each node initially receives a random attribute assignment. Then a stochastic optimization process is used to move the initial assignments closer to the target distribution extracted from social media dataset using the Particle Swarm Optimization algorithm. The tuned attributes are then used to add additional links to the network based on the feature similarity parameter -a source node is selected randomly and connected to the most similar node. The second proposed generator, so-called Multi-Link Generator (MLG), further uses link co-occurrence statistics from the original dataset to create a multiplex network.
MLG uses the same network growth process as ASG. Based on the link density parameter, either a new node is generated with a label based on the label distribution of the target dataset or a new link is created between two existing nodes. LDBC SNB. Despite having a common Facebook-like dataset, thanks to three distinct workloads the Social Network Benchmark (SNB)  [Erling et al. 2015 ] provided by LDBC represents three distinct benchmarks. The network nodes correspond to people and the edges represent their friendship and messages they post in discussion trees on their forums. The three query workloads involve: (1) SNB-Interactive, i. e. , complex read-only queries accessing a high portion of data, (2) SNB-BI, i. e. , queries accessing a high percentage of entities and grouping them in various dimensions, and (3) SNB-Algorithms, involving graph analysis algorithms, such as community detection, PageRank, BFS, and clustering.
Generating Noisy Graphs and Graphs with Anomalies Injecting noise and/or anomalies and errors into graphs is crucial for testing both machine learning algorithms working on this complex data and data quality techniques aiming at detecting anomalies and repairing graph data. Concerning the former, analyzing and labeling structural networks is deemed to be more difficult for graph datasets in the presence of noise. Since de-noising graph data is difficult to achieve, various machine learning-based approaches have been adapted to work with noise (i. e. , mislabeled samples) or outliers, such as imbalanced graph classification [Pan and Zhu 2013] and binary graph classification with positive and negative weights  [Cheung et al. 2016]. Synthetic graph generators that take into account noisy and missing data have been studied in  [Namata Jr.
Multi-Model Data With the dawn of Big Data and especially its Variety, another key 3V characteristic, new types of database management systems have emerged. One of the most interesting ones are the so-called multi-model databases  [Lu and Holubov á 2019]  that enable to store and thus query across structurally different data, including unstructured, semi-structured, and structured. There exist various types of multi-model systems combining distinct subsets of Big Data structures including graph data. For example, OrientDB 29 which has been mainly designed as an object DBMS currently supports graph, document, key/value, and object models. Such type of DBMSs also needs a specific data generator that would enable to test new features and analyze efficiency of operations. However, since the multi-model systems are in the context of Big Data rather new, there exist only a few benchmarks targeting multi-model DBMSs (such as Bigframe  [Kunjir et al.
Big data applications increasingly involve diverse datasets, conforming to different data models. Such datasets are routinely hosted in heterogeneous stores, each capable of handling one or a few data models, and each efficient for some, but not all, kinds of data processing. Systems capable of exploiting disparate data in this fashion are usually termed polystores. A current limitation of polystores is that applications are written taking into account which part of the data is stored in which store and how. This fails to take advantage of (i) possible redundancy, when the same data may be accessible (with different performance) from distinct data stores; (ii) previous query results (in the style of materialized views), which may be available in the stores. We propose to demonstrate ESTOCADA [4], a novel approach that can be used in a polystore setting to transparently enable each query to benefit from the best combination of stored data and available processing capabilities.
INTRODUCTION Big Data applications increasingly involve diverse data sources, such as flat or nested relations, structured or unstructured documents, data graphs, etc. Such datasets are routinely hosted in heterogeneous stores. One reason is that the fast pace of application development prevents consolidating all the sources into a single data format and loading them into a single store. Instead, the data model often dictates the choice of the store, e. g. , JSON documents get loaded in a JSON store. Systems capable of exploiting diverse data in this fashion are termed polystores  [8, 3]. Query evaluation in a polystore recalls to some extent mediator systems; in both cases, sub-queries are delegated to the underlying stores when possible, while the remaining operations are applied in the integration layer.