dataesr/scientific-paragraphs-categorization huggingface dataset:
INTRODUCTION The blooming of different cloud data management infrastructures, specialized for different kinds of data and tasks, has led to a wide diversification of DBMS interfaces and the loss of a common programming paradigm. This makes it very hard for a user to integrate and analyze her data sitting in different data stores, e. g. RDBMS, NoSQL, and HDFS. For example, a media planning application, which needs to find top influencers inside social media communities for a list of topics, has to search for communities by keywords from a key-value store, then analyze the impact of influencers for each community using complex graph database traversals, and finally retrieve the influencers' profiles from an RDBMS and an excerpt of their blog posts from a document database. The CoherentPaaS project  (CoherentPaaS, 2013)  addresses this problem, by providing a rich platform integrating different data management systems specialized for particular tasks, data and workloads.
The platform is designed to provide a common programming model and language to query multiple data stores, which we herewith present. The problem of accessing heterogeneous data sources has long been studied in the context of multidatabase and data integration systems  (Özsu and Valduriez, 2011). More recently, with the advent of cloud databases and big data processing frameworks, the solution has evolved towards multistore systems that provide integrated access to a number of RDBMS, NoSQL and HDFS data stores through a common query engine. Data mediation SQL engines, such as Apache Drill, Spark SQL  (Armbrust et al. , 2015) , and SQL++ provide common interfaces that allow different data sources to be plugged in (through the use of wrappers) and queried using SQL. The polystore BigDAWG  (Duggan et al.
, 2015)  goes one step further by enabling queries across "islands of information", where each island corresponds to a specific data model and its language and provides transparent access to a subset of the underlying data stores through the island's data model. Another family of multistore systems  (DeWitt et al. , 2013)  has been introduced with the goal of tightly integrating big data analytics frameworks (e. g. Hadoop MapReduce) with traditional RDBMS, by sacrificing the extensibility with other data sources. However, since none of these approaches supports the ad-hoc usage of native queries, they do not preserve the full expressivity of an arbitrary data store's query language. But what we want to give the user is the ability to express powerful ad-hoc queries that exploit the full power of the different data store languages, e.
g. directly express a path traversal in a graph database. Therefore, the current multistore solutions do not directly apply to solve our problem. In this paper, we give an overview of the design of a Cloud multidatastore query language (CloudMdsQL) and the implementation of its query engine. CloudMdsQL is a functional SQL-like language, capable of querying multiple heterogeneous databases (e. g. relational, NoSQL and HDFS) within a single query containing nested subqueries  (Kolev et al. , 2015). Thus, the major innovation is that a CloudMdsQL query can exploit the full power of local data stores, by simply allowing some local data store native queries (e. g. a breadth-first search query against a graph database) to be called as functions, and at the same time be optimized based on a simple cost model, e.
Thus, each query, although agnostic to the underlying data stores' schemas, is executed in the context of an ad-hoc schema, formed by all named table expressions within the query. A named table expression can be defined by means of either an SQL SELECT statement (that the query compiler is able to analyze and possibly rewrite) or a native expression (that the query engine considers as a black box and delegates its processing directly to the data store). For example, the following simple CloudMdsQL query contains two subqueries, defined by the named table expressions T1 and T2, and addressed respectively against the data stores rdb (an SQL database) and mongo (a MongoDB database): T1(x int, y int)@rdb = (SELECT x, y FROM A) T2(x int, z array)@mongo = {* db. B. find({$lt: {x, 10}}, {x:1, z:1}) *} SELECT T1.
x, T2. z FROM T1, T2 WHERE T1. x = T2. x AND T1. y <= 3 The purpose of this query is to perform relational algebra operations (expressed in the main SELECT statement) on two datasets retrieved from a relational and a document database. The two subqueries are sent independently for execution against their data stores in order the retrieved relations to be joined by the common query engine. The SQL table expression T1 is defined by an SQL subquery, while T2 is a native expression (identified by the special bracket symbols {* *}) expressed as a native MongoDB call. The subquery of expression T1 is subject to rewriting by pushing into it the filter condition y <= 3, specified in the main SELECT statement, thus reducing the amount of the retrieved data by increasing the subquery selectivity. Note that subqueries to some NoSQL data stores can also be expressed as SQL statements; in such cases, the wrapper must provide the translation from relational operators to native calls.
CloudMdsQL allows named table expressions to be defined as Python functions, which is useful for querying data stores that have only API-based query interface. A Python expression yields tuples to its result set much like a user-defined Note that parametrization and nesting is also available in SQL and native named tables. For example, we validated the query engine with a use case that involves the Sparksee graph database and we use its Python API to express subqueries that benefit from all of the features described above  (Kolev et al. , 2015). In fact, our initial query engine implementation enables Python integration; however support for other languages (e. g. JavaScript) for user-defined operations can be easily added. In order to also address distributed processing frameworks (such as Apache Spark) as data stores, we introduce a formal notation that enables the adhoc usage of user-defined Map/Filter/Reduce (MFR) operators as subqueries in CloudMdsQL to request data processing in an underlying big data processing framework (DPF)  (Bondiombouy et al.
, 2015). An MFR statement represents a sequence of MFR operations on datasets. In terms of Apache Spark, a dataset corresponds to an RDD (Resilient Distributed Dataset -the basic programming unit of Spark). Each of the three major MFR operations (MAP, FILTER and REDUCE) takes as input a dataset and produces another dataset by performing the corresponding transformation. Therefore, for each operation there should be specified the transformation that needs to be applied on tuples from the input dataset to produce the output tuples. Normally, a transformation is expressed with an SQL-like expression that involves special variables; however, more specific transformations may be defined through the use of lambda functions. Let us consider the following simple example inspired by the popular MapReduce tutorial application "word count".
This worker can subcontract parts of the query plan to other workers and integrate the intermediate results to produce the final result. Each worker node acts as a lightweight runtime database processor atop a data store and is composed of three generic modules (i. e. same code library)query execution controller, operator engine, and table storage -and one wrapper module that is specific to a data store. These modules provide the following capabilities: • Query execution controller: initiates and controls the execution of a query plan (received from a master or worker) by interacting with the operator engine for local execution or with one or more workers (through communication processors). In addition to the generic modules, which are valid for all the workers, the components of a worker collocated with a data processing framework as data store, have some specifics.
Operator Engine The main reasons to choose Derby database to implement the operator engine are because Derby: • Allows extending the set of SQL operations by means of CREATE FUNCTION statements. This type of statements creates an alias, which an optional set of parameters, to invoke a specific Java component as part of an execution plan. • Has all the relational algebra operations fully implemented and tested. • Has a complete implementation of the JDBC API. • Allows extending the set of SQL types by means of CREATE TYPE statements. It allows working with dictionaries and arrays. Having a way to extend the available Derby SQL operations allows designing the resolution of the named table expressions. In fact, the query engine requires three different components to resolve the result sets retrieved from the named table expressions: • WrapperFunction: To send the partial execution plan to a specific data store using the wrappers interfaces and retrieve the results.
As a second step, the execution engine evaluates which named expressions are queried more than once and must be cached into the temporary table storage, which will be always queried and updated from the specified Java functions to reduce the query execution time. Finally, the last step consists of translating all operation nodes that appear in the execution plan into a Derby specific SQL execution plan. Once the SQL execution plan is valid, the Derby core (which acts as the operator engine) produces a dynamic byte code that resolves the query that can be executed as many times as the application needs. Derby implements the JDBC interface and an application can send queries though the Statement class. So, when the user has processed the query result and closed the statement, the query execution controller drops the previously created functions and cleans the temporary table storage.
Wrappers The wrappers are Java classes implementing a common interface used by the operator engine to interact with them. A wrapper may store locally catalog information and capabilities, which it provides to the query planner periodically or on demand. Each wrapper also implements a finalizer, which translates a CloudMdsQL sub-plan to a native data store query. We have validated the query engine using four data stores -Sparksee (a graph database with Python API), Derby (a relational database accessed through its Java Database Connectivity (JDBC) driver), MongoDB (a document database with a Java API), and unstructured data stored in an HDFS cluster and processed using Apache Spark as big data processing framework (DPF). To be able to embed subqueries against these data stores, we developed wrappers for each of them as follows.
The wrapper for Sparksee accepts as raw text the Python code that needs to be executed against the graph database using its Python client API in the environment of a Python interpreter embedded within the wrapper. The wrapper for Derby executes SQL statements against the relational database using its JDBC driver. It exports an explain() function that the query planner invokes to get an estimation of the cost of a subquery. It can also be queried by the query planner about the existence of certain indexes on table columns and their types. The query planner may then cache this metadata information in the catalog. The wrapper for MongoDB is implemented as a wrapper to an SQL compatible data store, i. e. it performs native MongoDB query invocations according to their SQL equivalent. The wrapper maintains the catalog information by running probing queries such as db.
collection. count() to keep actual database statistics, e. g. cardinalities of document collections. Similarly to the Derby wrapper, it also provides information about available indexes on document attributes. The MFR wrapper implements an MFR planner to optimize MFR expressions in accordance with any pushed down selections. The wrapper uses Spark's Python API, and thus translates each transformation to Python lambda functions. Besides, it also accepts raw Python lambda functions as transformation definitions. The wrapper executes the dynamically built Python code using the reflection capabilities of Python by means of the eval() function. Then, it transforms the resulting RDD into a Spark DataFrame.
The architecture of the CloudMdsQL query engine is fully distributed, so that nodes can directly communicate with each other, by exchanging code (query plans) and data. Thus, the query engine does not follow the traditional mediator/wrapper architectural model where mediator and wrappers are centralized. This distributed architecture yields important optimization opportunities, e. g. minimizing data transfers by moving the smallest intermediate data for subsequent processing by one particular node. The wrappers are designed to be transparent, making the heterogeneity explicit in the query in favor of preserving the expressivity of local data stores' query languages. CloudMdsQL sticks to the relational data model, because of its intuitive data representation, wide acceptance and ability to integrate datasets by applying joins, unions and other relational algebra operations.
The CloudMdsQL query engine has been validated with four different database management systems -Sparksee (a graph database with Python API), Derby (a relational database accessed through its JDBC driver), MongoDB (a document database with a Java API) and Apache Spark (a parallel framework processing distributed data stored in HDFS, accessed by Apache Spark API). The experiments show the impact of optimization techniques on overall query execution performance  (Kolev et al. , 2015; Bondiombouy et al. , 2015).
AKNOWLEGMENTS This research has been partially funded by the  European Commission  under projects  CoherentPaaS and LeanBigData  (grants  FP7-611068 ,  FP7-619606 ), the  Madrid Regional Council, FSE  and  FEDER , project  Cloud4BigData  (grant  S2013TIC-2894 ), and the  Spanish Research Agency MICIN project BigDataPaaS  (grant  TIN2013-46883 ).
Our framework is intended to span this space, with the exception that ours is intended to enable only synchronous (real-time) interactions. Weinberg  [25]  provides a historical overview of interconnected musical networks. Weinberg's examples, as well as his descriptions of intra-player interdependencies were influential on our development of the BOEUF framework. Several protocols and software tools have been created to deal with the sharing of musical data for both single instruments and within networked orchestras. For example, Jamoma  [20]  and libMapper  [14]  both give access to the structure and parameters of networked DMIs, sometimes with features for watching and grabbing parameters. An interesting example is the Digital Orchestra Toolbox  [15]  which simplifies the collaborative creation and mapping of DMIs.
g. tempo or scale. An instrument represents a bounded set of music-generating processes (i. e. modules) and a user interface. It has parameters, outputs, meters and it can receive and send messages. We presume that each musician in the orchestra is in control of at least one instrument. Thus, in our modeling an instrument often acts as a proxy for the musician. A module is a software component that produces musical data, of audio or control type. It is composed of several parameters and outputs. Modules have a type, possibly from a common set of types (each with a predefined number of parameters and data). This way a module can be copied by another instrument if this instrument handles modules of the same type. For example, a module of type LowPassFilter might have Cutoff Frequency and Q as parameters, whereas a module of type MidiPattern will hold an array of MIDI events.
Many instruments have an internal structure that is more complex than a simple chain of modules. They may have complex audio graphs with many hierarchical levels or feedback. However, from the perspective of user interaction these can usually be flattened to a set of modules and associated parameters. A parameter is an attribute of a module or instrument that influences its musical production. Parameters can be of various types such as MIDI events, float or integer values, input audio streams, and so on. Parameters can be: -Retrieved : the current value of the parameter is returned once. -Watched : the value of the parameter is sent every time it changes, until it is not watched anymore. -Indicated : a new value is proposed for the parameter but not set. -Set: the parameter is set to a new value.
Integration In order to ensure portability, the low level layer of the BOEUF implementation should rely on a commonly used network protocol such as Open Sound Control. Depending on the type of DMI, we envisage four levels of integration of BOEUF, each with progressively greater functionality. Hardware Box: In the case of black-box instruments (i. e. hardware synthesisers and instruments with fixed software structure), BOEUF integration would require a dedicated hardware device. This device would connect to the instrument via MIDI, audio and OSC input and output ports. The device would in turn connect to the BOEUF session via a network connection. It would provide a limited set of modes of collaboration with a limited number of instruments. This limitation depends on the number of ports, controls and displays available on the device.
Software Plugin / Standalone: In the case of DMIs implemented as nonreprogrammable software components, a BOEUF software plugin or standalone application can be connected to the virtual audio and control ports of the DMI. The BOEUF software would in turn connect to the BOEUF session. Compared to a BOEUF hardware device, the software application can enable more modes of collaboration. In particular, thanks to the software's graphical interface, awareness and indication modes of communication, as well as cooperation and organisation modes with a flexible number of other instruments, are now possible. Open Sound Control Namespace: In the case of an instrument with programmable input and output layers, a BOEUF session can be accessed directly via Open Sound Control messages following the BOEUF protocol.
The input layer must parse incoming messages, send them to the appropriate components of the instrument, and perform any required actions (e. g. set, grab, or retrieve the value of a parameter). The output layer must emit messages to other instruments, for example when the value of a watched parameter is changed, or when the user wants to send a message to another instrument or group of instruments. This integration would resemble that of the Jamoma framework in Max/MSP. Software Library: Finally, in the case of open-source software instruments, a BOEUF software library can be integrated into the source code of the instrument. This library would manage the network connection to the session internally. The instrument must register its components with the BOEUF library and implement callback functions that provide access to its components.
Interfaces The new modes of collaboration enabled by digital orchestras and described in our framework generate new streams of information that a musician must apprehend and respond to, and they create new opportunities for action. These lead to two challenges in the development of a DMI interface that would give access to these collaboration modes. The first concerns the visualisation of information available through the BOEUF framework. In order to enable all modes of collaboration, an interfaces should provide ways of displaying information from various components in the framework, such as meters for awareness, messages for indication, modules for exchange modes, and so on. However, we also want to preserve non-mediated modes of collaboration such as the non-verbal communication the occurs between musicians who can see each other.
This paper introduces Ocelet, a domain specific language and simulation tool for modelling changes in geographical landscapes. It is characterised by the use of interaction graphs (graphs with interaction functions on their edges) to represent the system as composed of processes, each involving several entities distributed in space that are in interaction with each other. Entities are the vertices of the graphs, and interactions are the edges on which (interaction) functions can be applied to make the system change through time. Examples are given to illustrate the generic disposition of the simulation approach to model and study changing geographical setups.
Examples of software based on these principles include STELLA  [8]  and Vensim  [9]. But when a system is distributed in a geographical space, aggregated system variables become inadequate. The solution proposed by the Spatial Modelling Environment (SME-  [26] ) was to disaggregate the system space into cells. Stock-flow models could then be included in each of the cells, with neighbouring cells able to exchange flows. Referred to as "individualbased modelling approaches", CA and ABM are inherently different in that aggregate patterns emerge from the sum of individual behaviour  [10]. With CA, geographical space is represented by grid cells that can take a finite number of states. The state of a given cell changes following transition rules that depend on the states of the neighbouring cells.
Urban dynamics is a field where CA application has been particularly successful (e. g. DEUM-  [11] ; SLEUTH-  [12] , and more recently O'Sullivan, 2001;  [13] ). When the system to be modelled involves heterogeneous entities in more complex situations such as those in social systems, ABM is generally preferred. Agents are defined by their behaviour, can be reactive or cognitive, and interact with other agents and their environment  [14]. A review of the use of ABM in ecosystem management can be found in  [15]. Software for multi-agent simulations includes CORMAS  (Bousquet et al. , 1998) , NetLogo  [16]  and GAMA  [17]. All three modelling approaches have specific characteristics that can be considered merits or weaknesses depending on the objectives sought. In particular, modellers often need to study a system as a whole, and at the same time decipher how local and intermediate level processes sum up to form the whole system.
The generally spatial entities used in Ocelet models are represented with data types that are commonly used in Geographical Information Systems (GIS): points, lines, polygons, multi-points, multilines and multi-polygons. Interactions between entities result in changes in the state and (spatial) configuration of the entities. The key difference is that, once imported from a GIS data file (e. g. shapefile) into the model, the entities no longer belong to a "GIS layer", and can be interconnected individually through several interaction graphs. Spatial dynamics models are built within a software environment called the "Ocelet Modelling Platform" (OMP). After a few years of practice and the transition phase between the initial prototype and the current stabilised version of Ocelet (version 1. 1), we hereby (Section 2) present the main concepts and features of the software.
Software description The OMP software environment is built around the Ocelet DSL in order to facilitate model creation and maintenance, code edition, compilation, simulation run, display and exportation of simulation results. The Ocelet language is used to describe the geographical setup and the processes relevant for a given study. It incorporates specific language elements required for implementing dynamic interaction graphs that are at the heart of the Ocelet modelling approach. The main concepts of the language are given in Section 2. 1. Section 2. 2 describes how interaction graphs are built and used, and Section 2. 3 gives a brief presentation of the Ocelet Modelling Platform.