dataesr/scientific-paragraphs-categorization huggingface dataset:
The platform is, however, designed in an open manner, with a clearly defined and separated API for database access, so that any project can write its own backend translating its data into the Arkheia database format. This both allows any current private ad-hoc project to use Arkheia as its graphical book-keeping and publishing frontend, as well as ensuring Arkheia can be used with any future workflow tools that may be developed. Arkheia does not currently offer an internally implemented fine-grain access control and is not meant to be used as a centralized platform. It rather allows any user (or group of users) to set up a separate repository, either publicly for access by the general population, as well as locally for internal use. Overall, Arkheia provides users with an automatic means to communicate not only their models but also individual simulation results and the entire experimental context in an approachable graphical manner to a wider audience.
COMPARISON TO OTHER TOOLS Several recent software projects have overlapping goals with Arkheia. Lancet  (Stevens et al. , 2013)  and Pypet  (Meyer and Obermayer, 2016)  are Python simulation workflow libraries that provide users with a means to organize and automate their numerical simulation workflow, automate exploration of model parameter space and manage the resulting data. Similarly to Arkheia, they provide users with structured access to the data produced in the simulations, enriched with a limited set of metadata tracked during the simulation workflow, mostly comprising the parametric configuration of the simulated models. Arkheia does not provide direct handling of the simulation workflow or the exploration parameter space, these are instead expected to be performed by the source of the data handled by Arkheia, and the Mozaik toolkit for which the data-import backend is currently provided offers both these services.
Crucially, the Lancet and Pypet toolkits do not provide a graphical interactive representation of the data to the user, which is the primary goal of Arkheia. Furthermore, Lancet and Pypet are agnostic as to the nature of the simulations they handle, which makes them more general, but at the cost of explicitly exposing only a very limited set of information about the simulations to the user. In contrast, Arkheia is focused exclusively on neural simulations allowing it to provide the user with much richer and deeper introspection of information about the simulations held in the repository and to do it via a clear and convenient web-based graphical user interface. Probably the most similar tool currently available to the computational neuroscience community is the Open Source Brain (OSB) (www.
ARCHITECTURE Arkheia follows the standard database-server-client architecture. To facilitate efficient, but at the same time flexible, storage of complex highly structured data that describe the makeup and results of neural simulations we have selected a modern document based database  MongoDB (www. mongodb. com). The nature of the data describing a simulation run and its results are straightforwardly described by a hierarchical document that can be efficiently represented and retrieved in a documentbased database. Presently MongoDB is an industrial standard for document based databases and is particularly frequently used in web based software solutions. It is well-supported and accessible to new users, overall making it a suitable choice for this project. The data stored in the database is served to the client via a thin server layer developed in the the asynchronous event driven JavaScript runtime Node.
In addition a more complex web application for interaction exploration of parameter search results is offered. The insertion of data into Arkheia is expected to be done by an arbitrary set of backends, which should automatically export data from a given simulation framework and insert it into the MongoDB database in the format expected by Arkheia, which is described in the following section. The backends connect directly to the database, and thus no assumptions about their behavior beyond the insertion of the data in the correct format are made. The interaction of Arkheia with external tools is thus fully specified by the format of data stored in the database. While the backends are primarily meant as automatic exporters from simulation environments, in principle one could create an interactive GUI based application (which from the point of view of Arkheia would behave as any other backend) that would allow manual insertion of data into Arkheia.
API The Arkheia API is essentially a description of how the shared data about individual simulation runs should be stored in the MongoDB database used by Arkheia (see Figure  2 ). The data is stored in three MongoDB collections, one storing the individual simulation runs, one storing the parameter searches and one storing any binary files (e. g. , images and movies) that are referenced from the documents in the other two collections (see Figure  2 ). Thus, with the exception of the mechanisms for storing of image and movie data, this storage description reduces to the description of the format of the hierarchical document that will be stored for each simulation run. This specification covers the storage of model specification, sensory stimuli, experimental protocols, resulting data analysis, and visualization outputs.
Simulation Run Representation Arkheia data specification follows a document based design, which conveniently maps onto the document based MongoDB database used by Arkheia. Each simulation run is represented by a single JSON hierarchical data-structure which corresponds to the document inserted into the database. Thus the Arkheia input data specification reduces to the expected format of this JSON data-structure. At the root level the SimulationRun datastructure contains the following entries with the indicated value types: The submission_data and run_date entries are expected to be strings representing time in "YYYY/MM/DDHH:MM:SS" format. The simulation_run_name is an arbitrary name given to this specific simulation run (not the model). The model_name is the name of the model that was simulated.
The results, stimuli, recorders, and experimental_protocols are each a list of JSON data structures, the format of which will be described below. Finally the parameters variable should describe the full parametrization of the model used in this run, and as all parametrization throughout Arkheia API, it should follow the ParameterSet format. ParameterSet is a nested dictionary (see schema below) where each value associated with a key (that corresponds to the name of the parameter) is a tuple (a, b, c), where a corresponds to the value of the parameter and can either be a scalar or ParameterSet itself, b is the type of the parameter, and c is a short description of the parameter's meaning. The results entry should contain a list of Result JSON datastructures, each describing one result produced during the simulation (presumably after analysis and visualization of the raw data recorded during the simulation).
The recorders entry should thus contain a list of Recorder JSON data structures, each corresponding to one such recording configuration entity. The format of the Recorder data structure is as follows: The code, parameters, short_description, and long_description entries are analogous to those in the Stimulus data structure. The variables entry should contain a list of strings, each identifying the variable that the given recording configuration recorded in the selected neurons, and the source should contain the name of the population of neurons to which this recording configuration was applied. Finally the experimental_protocols root entry is expected to contain a list of Protocol JSON data structures characterizing the experimental protocols that were performed during the simulation of the model.
Parameter Search Representation Parameter search is essentially a collection of simulation runs with a systematically varying subset of parameters. In principle, it would be possible to add the simulation runs originating from parameter searches into the same database as the individual simulation runs, and keep a separate record of their membership to the parameter search group. We have decided against such an organization, as parameter searches can consist of hundreds or more runs and would thus clog the presentation of the individual runs, as well as potentially make access to the individual run database slow. The parameter searches are thus stored in a separate database (within the same instance of the MongoDB server). Each parameter search is a JSON document of the following format: Where submission_date is expected to be a string containing the data of the submission to the repository, the name entry is the name given to the simulation run.
API Design Discussion First we want to emphasize that the goal of Arkheia is not to provide storage for the full raw data associated with a given simulation, but rather a structured description of the simulation context and results, sufficient for replication or reuse of the various aspect of the given simulation, and to facilitate a convenient human readable comparison and browsing of models and their results. The data format specification described above broadly follows the organization of data in the Mozaik framework, facilitating seamless integration with the workflow toolkit, but is designed with sufficient flexibility to allow integration with other modeling frameworks. The specification makes a number of assumptions about the nature of data describing the simulation run and its results, which, however, we believe are mappable onto the majority of use cases in systems neuroscience.
g. , experimental protocol) with an explanation of how the associated parameters configure the entity. An elegant solution to this is to keep this description as a code "docstring" of the given code block, where it fulfils the good practice of well documented code, and can be automatically harvested into the documentation of the given simulation framework while at the same time exported for publication in Arkheia. The final source of flexibility is that most of the entries of the specification do not actually have to hold data (e. g. , all the string values can be set to empty strings, or list values to empty lists). Arkheia can thus be used in projects which either do not use, or do not yet expose all the data so far covered in our specification. It is the intention that this specification will be developed further with the main goal of increasing the coverage of the information that can be exposed about simulation runs.
We particularly hope that some of the ongoing efforts in the wider community will generate welldesigned and popular specification standards for some of the aspects of neural simulations covered by Arkheia, such as standardization of experimental protocols, higher level model specifications or stimulus definitions, which we would eagerly seek to incorporate into Arkheia  (Hucka et al. , 2003). To this end Arkheia represents both a sketch of how such specifications could look and an example of how they could be used to facilitate communication and comparison across different models, thus motivating the development of these technologies, which we believe are key to the future computational neuroscience software infrastructure.
Individual Run Inspection Most views of the Arkheia web app follow a tabular design, and the main simulation run view is a typical example. Each line of the table corresponds to one simulation that has been added to the Arkheia database. The first four columns offer basic information about the given simulation run, specifically the time of its submission to the repository (Figure  3. 1), the approximate time when the simulation was executed (Figure  3. 2), the label the user gave to the specific simulation run (Figure  3. 3), and finally the name of the model that was simulated (Figure  3. 4). The last five columns then contain links to other views that show more detailed information about the given simulations, specifically a pop-up that contains brief description of the model (Figure  3   Each row of the table corresponds to one simulation run which was added to the Arkheia repository.
DEPLOYMENT We envision two main types of deployment for Arkheia. First local, where a user utilizes Arkheia on a daily basis as a local store for his/her simulation results with the advantage of GUI access to them, and as a tool for interactive exploration of parameter search results. Second, public online deployment, where the Arkheia instance is used as a publishing platform for a user's models and results, for example accompanying publication with a much more detailed view of the data as well as a more thorough and usable specification of the model and entire experimental context. To provide an easy solution for both scenarios we offer several options for deployment of Arkheia. In the simplest case, a user can install the two dependencies: Node. js and MongoDB. Clone the Git repository (https://github.
com/antolikjan/Arkheia) of the project, run the MongoDB client, install the remaining dependencies by issuing npm install and bower install commands. Upon completing this user can issue the gulp serve command in the root directory, at which point user has running a local Arkheia instance at localhost:3000. New results can be added by pointing the given backend to the database. This setup is suitable for development, whereupon any changes to the Arkheia code will be automatically reflected in the web-browser. This is a very straightforward installation procedure, with narrow margin for error. To facilitate even simpler deployment, especially in the cloud, we also provide a Docker (www. docker. com) image of the latest stable version of the system, with accompanying dockercompose specification which is configured for deploying the system on a server for public access.
Introduction Born with big data in the 2010's, data lakes propose a way to store in their native format voluminous and diversely structured data, in an evolutionary storage place allowing later analyses (reporting, visualization, data mining. )  [3]. The concept of data lake opposes that of data warehousing, which outputs an integrated, highly structured, decision-oriented and subject-oriented database, but has the disadvantage of dividing data into silos  [26]. Yet, everyone agrees that a data lake must be well designed. Otherwise, it is doomed to quickly become an inoperable data swamp  [1, 9]. That is, a data lake must allow querying data (selection/restriction) with good response times, and not only storing data and handling a "key-value" access. However, actual solutions are more or less non-existent in the literature and pertain to undisclosed industrial practices.
This is why  [21]  proposed a conceptual metadata model that allows indexing and efficiently querying an industrial heritage data lake constituted of XML data, texts, floorplans and pictures. This multidimensional model is similar to snowflake models used in data warehouses or datamarts, but only stores metadata, not the document corpus itself. This metadata model has been instantiated physically in several NoSQL database management systems (DBMSs) to enable scaling. However, such a data warehouse-like schema cannot evolve easily when data sources evolve themselves, or when new sources pop up, whereas this is a crucial point in the management of a data lake. Consequently, we propose in this paper: 1. to replace the multidimensional model of  [21]  by an ensemble model, more precisely a data vault  [16, 8, 4, 18] , which is a data model allowing easy schema evolutions and has, to the best of our knowledge, never been used in the context of metadata management; 2.
to evaluate, the feasibility, on one hand, and the effectiveness of this model in terms of metadata query response (as an index), on the other hand, because it induces many joints. For this sake, we translate our conceptual metadata vault into different logical (i. e. , relational and document-oriented) and physical (i. e. , PostgreSQL 1  and MongoDB 2  ) models, which also helps compare the respective efficiency of the two physical models. The remainder of this paper is organized as follows. Section 2 is devoted to a state of the art concerning data lakes and metadata management, on one hand, and ensemble modeling and data vaults in particular, on the other hand. Section 3 presents our conceptual metadata vault model, as well as its logical and physical translations. Section 4 details the experiments we conducted to validate the feasibility of our approach and compare the PostgreSQL and MongoDB physical models in terms of storage and metadata query response time.
Data lakes are typically subdivided into three components  [26, 1] : 1. a storage system: very often HDFS, though NoSQL DBMSs are also an option [9, 14]; 2. a metadata system, which we focus on below; 3. an access and analysis system generally relying on MapReduce or Spark  [10]. By contrast, [9] proposes an architecture with respect to data types, made of so-called data ponds for raw data, analog (stream) data, application data (i. e. , a data warehouse), textual data and archived data, respectively. In such a context, database research on querying multistore systems  [2, 27]  looks highly relevant. Data lakes mostly bear a "flat" architecture, where each data element has a unique identifier and a set of characterizing tags  [19]. Such tags are actually essential metadata to help comprehend data and access data  [1] , not to mention query effectiveness.
• A hub is a basic entity that represents a business concept (a dimension hierarchy level in a classical data warehouse, e. g. , customer or product). A hub mainly contains a business key. • A link materializes an association between two or more hubs. It would correspond to a fact entity in a classical data warehouse. • A satellite contains the various descriptive data related to a hub or link. Finally, note that, although they are defined at the logical level by  [16] , these concepts can easily be transposed to the conceptual [11] and physical  [15]  levels. An example of conceptual data vault model is shown in Figure  1.
Thus, we select title (Hub Title), location (Hub Location), date (Hub Date) and document category (Hub Category). Then, we associate with each of these hubs a satellite that contains the hub's descriptive attributes. For example, the satellite connected to Hub Title, Sat Title, contains the Title, Authors, Description and Keywords attributes. Hub Category is associated with four satellites that form a classification corresponding to each of the data sources we have. The descriptive attributes of these satellites are specific to each source. Link Document allows to associate all hubs. Finally, since our model concerns only metadata, each entity (hub, link, satellite) is described by a direct reference (Source) to a document (physical file) in the data lake. All data access are made through this reference.
Logical and Physical Models To show that our conceptual model can adapt to different contexts, and to compare them, we translate it into two types of logical and physical models: a relational model with PostgreSQL and a document-oriented NoSQL model with MongoDB. The relational metadata model (Table  2 ) 3  is classically translated from the conceptual model (Figure  1 ). Let us only note that, in Table  2 , the elements of the attributes column that are postfixed by id are foreign keys, and that satellite primary keys are also foreign keys that reference the corresponding hub's primary key. Eventually, the Datetime and Source attributes featured in all tables are not repeated for clarity. The document-oriented metadata model (Table  3 ) exploits the notion of collection. Each hub, link and satellite is translated into a collection comprising various documents, each having an identifier (ObjectID).
In the MongoDB physical model, this identifier is generated automatically when the document is created. As in the relational case, the Datetime and Source fields present in all collections are not repeated for clarity. Each hub is associated with both Link Document and its satellite(s) through its ObjectID. For example, Hub Title is associated with Link Document and Sat Title through Title id. Finally, Figure  2  presents an example of translation from the document-oriented logical model into the MongoDB physical model in JavaScript Object Notation (JSON) format. { "Sat_Book": { "00000C842_001": { "dc:relation": "http://www. sudoc. fr/122661389", "dc:creator": [ "Petit, Jules", "Chambre de commerce et d'industrie (Boulogne-sur-Mer, Pas-de-Calais). ", "Commission du projet de chemin de fer direct de Calais a Marseille" ], "dc:subject": "Calais-Marseille, Chemin de fer (France).
Experimental Validation The objective of this section is to show the feasibility of our metadata vault model and to compare the physical models of Section 3. 3 in terms of storage space and query response time. We conduct our experiments on an Intel Core i5-5300U 2. 30 GHz PC with 8 GB of memory, running Windows 64-bit. The DBMSs we use are PostgreSQL 9. 6. 2-1 and MongoDB ssl 3. 4. 3. The data corpus in this test set is the one presented in Table  1.
Storage Volume After inserting metadata in the chosen DBMSs' native format, we measure the storage volume required by PostgreSQL and MongoDB (Table  4 ). Our measurements take into account indexes automatically created by the DBMSs (e. g. , on primary keys in PostgreSQL; no other index is created), unused memory space, and the space that is released when deleting or moving data. 5. Retrieve all titles containing the word "factory", and then the corresponding documents, whatever their category (data sources: Hub Title, Sat Title, Hub Category, Sat IHRIS and/or Sat VoixDuNord and/or Sat Inventory and/or Sat Book, Link Document). The translation of all queries in SQL and MongoDB's query language are provided in Appendices A and B, respectively. Figure  3  features the average execution time of queries #1 to #4 under PostgreSQL and MongoDB, respectively.
Conclusion From the modeling of data lake metadata as a multidimensional schema, noticing that schema evolution is not guaranteed, we instead propose in this paper an ensemble model, more precisely a metadata vault. The translation of our conceptual metadata model into logical and physical models, as well as experiments carried out on the TECTONIQ corpus, help demonstrate the feasibility of our modeling choice in terms of metadata storage volume and response time to queries formulated on the metadata. The comparison of two physical models (PostgreSQL and MongoDB) also reveals the superiority of document-based models for storing this type of metadata. The perspectives that this preliminary work is opening are numerous. In particular, it would be necessary to test the robustness of the metadata model when source data scale up, add data sources to verify the relevance of data vault modeling and test the model with queries more complex than projections/restrictions.
Moreover, although the supporters of anchor modeling argue that 6NF modeling allows good response times thanks to the join eliminating technique used in modern optimizers  [22] , it is also important to compare query efficiency over data vault and anchor models vs. classical star-like schemas, with a benchmark that allows significant database scale-up. Finally, various alternative metadata models, independently from the modeling techniques used, could be considered and compared. Schemas of the TECTONIQ documents could also be automatically extracted to enrich the metadata. Introduction To The Concept Of Data Lake And Its Benefits. https://www. esds. co. in/blog/ introduction-to-the-concept-of-data-lake-and-its-benefits/, February 2015. [7] H. Giménez. PostgreSQL Performance Considerations.