dataesr/scientific-paragraphs-categorization huggingface dataset:
Other functionalities include data cleaning and integration, through tools like Kettle 3  , Talend 4  , and Scribe  (Lee et al. , 2012). Analytic Processing Module: It constitutes the hard core studied within this paper and can be defined as the chosen approach for the analysis phase implementation in terms of data storage (i. e. , disk or memory) and the data model used (e. g. , relational, NoSQL). Then, several criteria, such as parallel programming model (e. g. , MapReduce, PACT) and scalability allow comparing approaches in the same class. Analytical Requests: This module considers Visualization, Reporting, and BI (Business Intelligent) Dashboards Generation functionalities. Based on the analytic data and through different query languages (e. g. , procedural, SQL-like), this module can generate outputs in several formats with pictorial or graphical representation (dashboards, graphics, reports, etc), which enables decision makers to see analytics presented visually.
Analytic Processing Classification Concerning the Analytic Processing component, several architectures in Big Data context exist. We clas-sify the approaches according to their data storage support: (i) those based on disk data storage, which we divide into two groups according the data model, NoSQL databases (class A in Fig. 1 ) and relational databases (class B in Fig. 1 ); and (ii) those that support in-memory databases (class C in Fig. 1 ), in which the data is kept in RAM reducing the I/O operations. We detail each class as follows: This type of architecture attempts to satisfy commercial demand on real-time and scalable data warehousing criteria, based on a distributed multidimensional in-memory DBMS and the support of OLAP operations over a large volume of data for some of the systems.
Criteria of Comparison To compare the different approaches on each class, we establish a set of criteria related to the interaction facilities (e. g. , OLAP-like support, query languages used, Cloud services support) , as well as implementation criteria related to performance aspects (e. g. , scalability, programming model, fault tolerance support). OLAP Support: Some analytical solutions integrate OLAP in their system allowing OLAP operators for mining and analyzing data directly over the data warehouse support (i. e. , without extraction from it). Since most solutions classified in our C class are characterized by in-memory databases support, they satisfy this criterion. However solutions implementing the architectures A or B can optionally consider an OLAP phase to support queries from the Analytical Requests module (for Visualization, Reporting, and Analysis).
Thus, this property is fixed to integrated when it is the case or not-integrated otherwise. Query Languages: This criterion specifies the language on which the user relies on for querying the system. It could be procedural, language extension, or declarative (see Section 3). Cloud Services Support: The three types of architectures described above can be offered as a product (with no cloud support) or as a service deployed in the cloud. In the case of cloud support, it can be done partially, such as Infrastructure as a Service (IaaS, which provides the hardware support and basic storage and Programming Model: It precises for each item which programming model is adopted. We consider MapReduce, PACT, RDD-Spark models and we mention ad-hoc for specific implementations.
NoSQL based Architectures Several tools which are NoSQL based architecture exist. We cite as examples: • Avatara: It leverages an offline elastic computing infrastructure, such as Hadoop, to precompute its cubes and perform joins outside of the serving system  (Wu et al. , 2012). These cubes are bulk loaded into a serving system periodically (e. g. , every couple of hours). It uses Hadoop as its batch computing infrastructure and Voldemort (Sumbaly et al. , 2012), a key-value storage, as its cube analytical service. The Hadoop batch engine can handle terabytes of input data with a turnaround time of hours. While Voldemort, as the key-value store behind the query engine, responds to client queries in real-time. Avatara works well while cubes are small, for far bigger cubes, there will be significant network overhead in moving the cubes from Voldemort for each query.
• Apache Kylin 5  : It is a distributed analytics engine supporting definition of cubes, dimensions, and metrics. It provides SQL interface and OLAP capability based on Hadoop Ecosystem and HDFS environment. It is based on Hive data warehouse to answer mid-latency analytic queries and on materialized OLAP views stored on a HBase cluster to answer low-latency queries. • Hive: It is an open-source project that aims at providing data warehouse solutions and has been built by the Facebook Data Infrastructure Team on top of the Hadoop environment. It supports ad-hoc queries with a SQL-like query language called HiveQL  (Thusoo et al. , 2009; Thusoo et al. , 2010). These queries are compiled into MapReduce jobs that are executed using Hadoop. The HiveQL includes its own system type and Data Definition Language (DDL) which can be used to create, drop, and alter tables in a database.
It also contains a system catalog which stores metadata about the underlying table, containing schema information and statistics, much like DBMS engines. Hive currently provides only a simple, naive rule-based optimizer. • Cloudera Impala 6  : It is a parallel query engine that runs on Apache Hadoop. It enables users to issue low-latency SQL queries to data stored in HDFS and Apache HBase without requiring data movement or transformation. Impala uses the same file and data formats, metadata, security, and resource management frameworks used by MapReduce, Apache Hive, Apache Pig, and other Hadoop software. Impala allows to perform analytics on data stored in Hadoop via SQL or business intelligence tools. It gives a good performance while retaining a familiar user experience. By using Impala, a large-scale data processing and interactive queries can be done on the same system using the same data and metadata without the need to migrate data sets into specialized systems or proprietary formats.
Discussion and Comparison: NoSQL-based systems have shown to have superior performance than other systems (such as parallel databases) in minimizing the amount of work that is lost when a hardware failure occurs  (Pavlo et al. , 2009; Stonebraker et al. , 2010). In addition, Hadoop (which is the open source implementations of MapReduce) represents a very cheap solution. However, for certain cases, MapReduce is not a suitable choice, specially when intermediate processes need to interact, when lot of data is required in a processing, and in real time scenarios, in which other architectures are more appropriate.
Relational Parallel Databases based Architectures Several projects aim to provide low-latency engines, whose architectures resemble shared-nothing parallel databases, such as projects which embed MapReduce and related concepts into traditional parallel DBMSs. We cite as examples the following projects: • Google's Dremel: It is a system that supports interactive analysis of very large datasets over shared clusters of commodity machines  (Melnik et al. , 2011). Dremel is based on a nested column-oriented storage that is designed to complement MapReduce. It is used in conjunction with MapReduce to analyze outputs of MapReduce pipelines or rapidly prototype larger computations. It has the capability of running aggregation queries over trillion-row tables in seconds by combining multi-level execution trees and columnar data layout.
PowerDrill is constrained by the available memory for maintaining the necessary data structures. • Amazon Redshift: It is an SQL-compliant, massively-parallel, query processing, and database management system designed to support analytics workload  (Gupta et al. , 2015). Redshift has a query execution engine based on ParAccel 7  , a parallel relational database system using a shared-nothing architecture with a columnar orientation, adaptive compression, memory-centric design. However, it is mostly PostgreSQL-like, which means it has rich connectivity via both JDBC and ODBC and hence Business Intelligence tools. Based on EC2, Amazon Redshift solution competes with traditional data warehouse solutions by offering DWaaS, that it is translated on easy deployment and hardware procurement, the automated patching provisioning, scaling, backup, and security.
• AsterData System: It is a nCluster sharednothing relational database 8  , that uses SQL/MapReduce (SQL/MR) UDF framework, which is designed to facilitate parallel computation of procedural functions across hundreds of servers working together as a single relational database  (Friedman et al. , 2009). The framework leverages ideas from the MapReduce programming paradigm to provide users with a straightforward API through which they can implement a UDF in the language of their choice. Moreover, it allows maximum flexibility, since the output schema of the UDF is specified by the function itself at query plan-time. • Microsoft Polybase: It is a feature of Microsoft SQL Server Parallel Data Warehouse (PDW), which allows directly reading HDFS data into databases using SQL  (DeWitt et al. , 2013).
It allows HDFS data to be referenced through external PDW tables and joined with native PDW tables using SQL queries. Polybase employs a split query processing paradigm in which SQL operators on HDFS-resident data are translated into MapReduce jobs by the PDW query optimizer and then executed on the Hadoop cluster. • Vertica: It is a distributed relational DBMS that commercializes the ideas of the C-Store project  (Lamb et al. , 2012). The Vertica model uses data as tables of columns (attributes), though the data is not physically arranged in this manner. It supports the full range of standard INSERT, UP-DATE, DELETE constructs for logically inserting and modifying data as well as a bulk loader and full SQL support for querying. Vertica supports both dynamic updates and real-time querying of transactional data.
• Microsoft Azure 9 : It is a solution provided by Microsoft for developing scalable applications for the cloud. It uses Windows Azure Hypervisor (WAH) as the underlying cloud infrastructure and. NET as the application container. It also offers services including Binary Large OBject (BLOB) storage and SQL service based on SQL Azure relational storage. The analytics related services provide distributed analytics and storage, as well as real-time analytics, big data analytics, data lakes, machine learning, and data warehousing. Discussion and Comparison: Although these parallel database systems serve some of the Big Data analysis needs and are highly optimized for storing and querying relational data, they are expensive, difficult to administer, and lack fault-tolerance for long-running queries  (Pavlo et al.
, 2009; Stonebraker et al. , 2010). None of these systems have been designed to manage replicated data across multiple datacenters (they are mostly centralized and do not scale to thousands of nodes). While more nodes are added into the system, hardware failures become more common and frequent, causing a limited scalability. In contrast, most relational databases assume that hardware failure is a rare event. They employ a coarser-grained recovery model, where an entire query has to be resubmitted if a machine fails. This works well for short queries where a retry is inexpensive, but faces significant challenges in long queries as clusters scale up  (Abouzeid et al. , 2009). Hence, these systems do not support fine-grained fault tolerance. Also, it is difficult for these systems to process non-relational data.
In-memory Structure based Architectures Some type of architectures are designed to support complex, multi-dimensional, and multi-level on-line analysis of large volumes of data stored in RAM. We present the following ones: • Cubrick: It is a new architecture that enables real-time data analysis of large dynamic datasets  (Pedro et al. , 2015). It is an in-memory distributed multidimensional database that can execute OLAP operations such as slice and dice, roll up and drill down over terabytes of data. Data in a Cubrick cube is range partitioned in every dimension, composing a set of data containers called bricks where data is stored sparsely and in an unordered and append-only fashion, providing high data ingestion ratios and indexed access through every dimension. Unlike traditional data cubes, Cubrick does not rely on any pre-calculations, rather it distributes the data and executes queries on-the-fly leveraging MPP architectures.
Cubrick is implemented at Facebook from the ground up. A good result is obtained in a first Cubrick de-ployment inside Facebook. • Druid: It is a distributed and column-oriented database designed for efficiently support OLAP queries over real-time data  (Yang et al. , 2014). It supports streaming data ingestion and faulttolerance. Druid cluster is composed of different types of nodes, each type having a specific role (real-time node, historical nodes, broker nodes, and coordinator nodes) that operate independently. Real-time nodes ingest and query event streams using an in-memory index to buffer events. The in-memory index is regularly persisted to disk. Persisted indexes are then merged together periodically before getting handed off. Both, in-memory and persisted indexes are hit by the queries.
Historical nodes are the main workers of a Druid cluster, they load and serve the immutable blocks of data (segments) created by the real-time nodes. Broker nodes route queries to real-time and historical nodes and merge partial results returned from the two types of nodes. Coordinator nodes are essentially responsible for management and distribution of data on historical nodes: loading new data, dropping outdated data, replicating. A query API is provided in Druid, with a new proposed query language based on JSON Objects in input and output. • SAP HANA 10  : SAP HANA is an in-memory and column-oriented relational DBMS. It provides both transactional and real-time analytics processing on a single system with one copy of the data. The SAP HANA in-memory DBMS provides interfaces to relational data (through SQL and Multidimensional expressions, or MDX), as well as interfaces to column-based relational DBMS, which allows to support geospatial, graph, streaming, and textual/unstructured data.
• Nanocubes 11 : It is an in-memory data cube engine offering efficient storage and querying over spatio-temporal multidimensional datasets. It enables real-time exploratory visualization of those datasets. The approach is based on the construction of a data cube (i. e. , a nanocube), which fits in a modern laptop's main memory, and then computes queries over that nanocube using OLAP operations like roll up and drill down. • Stratosphere: It is a data analytics stack allowing in situ data analysis by connecting to external data sources going from structured relational data to unstructured text data and semi-structured data  (Alexandrov et al. , 2014). It executes the analytic process directly from the data source (i. e. , data is not stored before the analytical processing), by converting data to binary formats after the initial scans.
CONCLUSIONS In this paper, we present a generic architecture for Big Data analytical approaches allowing to classify them according to the data storage layer, the distributed parallel model, and the type of database used. We focus on the three most recently used architectures, as far as we know: MapReduce-based, Parallel databases based, and in-memory based architectures. We compare several implementations based on criteria such as: OLAP support, scalability as the capacity to adapt the volume of data and business needs, type of language supported, and fault tolerance in terms of the need of restarting a query if one of the node involved in the query processing fails. Besides the proposed classification, the main aim of this paper is to clarify the concepts dealing with analytical approaches on Big Data, thus allowing non-expert users to decide which technology is better depending on their requirements and on the type of workload.
computing services) and Data Warehouse as a Service (DWaaS), or as a whole analytical solution in the cloud, i. e. , Platform as a Service (PaaS), which provides the whole computing platform to develop Big Data analytical applications. Scalability: This criterion measures the availability on demand of compute and storage capacity, according to the volume of data and business needs. For this property we fix the values large-scale or medium-scale (for the case where systems do not scale to thousands of nodes). Fault Tolerance: This criterion establishes the capability of the system of providing recovery from failures in a transparent way. In the context of analytical workloads, we consider the following fault tolerance techniques: the classical Log-based (i. e. , writeahead log), which is used for almost every database management system and Hadoop-based techniques, which provides data replication for fault tolerance.
based Architecture: It is based on NoSQL databases as storage model, relying or not on DFS (Distributed File System). It relies also on a parallel-processing models such as MapReduce, in which the processing workload is spread across many CPUs on commodity compute nodes. The data is partitioned among the compute nodes at run time and the underlined framework handles inter-machine communication and machine failures. The most famous embodiment of a MapReduce cluster is Hadoop. It was designed to run on many machines that do not share memory or disks (the shared-nothing model). This kind of architecture is designed by A in Fig. 1.
based Architecture: It is based on NoSQL databases as storage model, relying or not on DFS (Distributed File System). It relies also on a parallel-processing models such as MapReduce, in which the processing workload is spread across many CPUs on commodity compute nodes. The data is partitioned among the compute nodes at run time and the underlined framework handles inter-machine communication and machine failures. The most famous embodiment of a MapReduce cluster is Hadoop. It was designed to run on many machines that do not share memory or disks (the shared-nothing model). This kind of architecture is designed by A in Fig. 1. multiple autonomous nodes, each owning its own persis-tent storage devices and running separate copies of the DBMS), shared-memory or shared anything (a global memory address space is shared and one DBMS is present), and shared-disk (based on multiple loosely coupled processing nodes similar to shared-nothing, but a global disk subsystem is accessible to the DBMS of any processing node).
However, most of current analytical DBMS sys-tems deploy a shared-nothing architecture paral-lel database. In this architecture, the analytical solution is based on the conjunction of the par-allel (sharing-nothing) databases with a parallel programming model. We can see this type of ar-chitecture designed by B in Fig. 1. C-The in-memory Structures based Architecture: B-Relational Parallel Database based Architecture: It is based, as classical databases, on relational tables stored on disk. It implements features like indexing, compression, materialized views, I/O sharing, caching results. Among these architectures there are: shared-nothing (.
It should be noted that, due to the high consistency of the GNSS and levelling data at all four sites, even quite small problems in the ARP heights (below 1 cm) could be detected and corrected after on-site inspections and additional verification measurements. This strongly supports the recommendation of "Appendix 3" to have sufficient redundancy in the GNSS and levelling stations. The mean differences are 23 mm or below for the three German sites, whereas 106 mm is obtained for the OBSPARIS site. The magnitude of the mean values for the German sites is excellent and proves the low uncertainty of both the geometric levelling and the GNSS/quasigeoid results in Germany as well as the correct implementation of the corresponding height system bias terms (δW (i) 0 ). The somewhat larger value for OBSPARIS is believed to be mainly related to accumulated systematic levelling errors (see Sect.
(  17 ) or (  24 ) and (  25 ), respectively. Regarding the geometric levelling approach, the above-mentioned value W (EVRF2007) 0 = 62,636,857. 86 m 2 s -2 based on the European EUVN_DA GNSS/levelling data set from  Kenyeres et al. (2010)  is utilized in Eq. (  17 ). The GNSS/geoid approach according to Eqs. (  24 ) and (  25 ) is based on the disturbing potential T or the corresponding height anomaly values ζ from the EGG2015 model (see above and Eq. (  27 )), as well as the normal potential U 0 = 62,636,860. 850 m 2 s -2 , associated with the surface of the underlying GRS80 (Geodetic Reference System 1980; see  Moritz 2000)  level ellipsoid, and furthermore, the mean normal gravity values γ are also based on the GRS80 level ellipsoid; for further details, see  Torge and Müller (2012). Besides the geopotential numbers derived by the two independent approaches, Table  2  also shows the differences between the two approaches as well as corresponding ITRF2008 coordinates and normal heights referring to EVRF2007.
10 m (0. 094 m for the difference to PTB, 0. 084 m for LUH, and 0. 125 m for MPQ) over distances between about 650 and 700 km. Regarding the significance of the 1 dm discrepancies in the potential differences between the two geodetic approaches for the German stations and OBSPARIS, these have to be discussed in relation to the corresponding uncertainties of levelling, GNSS, and the quasigeoid model (EGG2015). The geometric levelling results are based on the EVRF2007 adjustment, where a posteriori standard deviations of about 1 mm × √ d (d is the distance in km) are reported for the German levelling data and 2 mm × √ d for the French data  (Sacher et al. 2008). Starting with a rough uncertainty estimation for the levelling connections between the German stations and OBSPARIS by assuming that the levelling is half in Germany and half in France, and considering that the levelling lines will be longer than the linear distances 123 ) and GNSS/geoid approach (C (GNSS/geoid) ) relative to the IERS2010 conventional reference potential W 0 = 62,636,856.
2016). This was mainly limited by the uncertainty and instability of the participating clocks, which is likely to improve in the near future. For clocks with performance at the 10 -17 level 123 and below, time-variable effects in the gravity potential, especially solid Earth and ocean tides, have to be considered and can also serve as a method of evaluating the performance of the optical clocks (i. e. a detectability test). Then, after further improvements in the optical clock performance, conclusive geodetic results can be anticipated in the future, and clock networks may also contribute to the establishment of the International Height Reference System (IHRS). Acknowledgements The authors would like to thank Thomas Udem, Ronald Holzwarth, and Arthur Matveev (Max-Planck-Institut für Quantenoptik, MPQ, Garching, Germany) for their support and facilitating the access to the MPQ site, Christof Völksen and Torsten Spohnholtz (Bayerische Akademie der Wissenschaften, Kommission für Erdmessung und Glaziologie, Munich, Germany) for carrying out the GNSS observations and the data processing for the MPQ site, Nico Lindenthal and Tobias Kersten (Leibniz Universität Hannover, LUH, Institut für Erdmessung, Hannover, Germany), Cord-Hinrich Jahn and Peter Lembrecht (Landesamt für Geoinformation und Landesvermessung Niedersachsen, LGLN, Landesvermessung und Geobasisinformation, Hannover, Germany) for corresponding GNSS work at LUH and PTB, and Martina Sacher (Bundesamt für Kartographie und Geodäsie, BKG, Leipzig, Germany) for providing information on the EVRF2007 heights and uncertainties, the associated height transformations, and a new adjustment of the UELN from 2017.