dataesr/scientific-paragraphs-categorization huggingface dataset:
Use Case 3: Selection of Datasets Data of an experiment can usually be used to address multiple scientific questions. In this context, it is necessary to select datasets according to certain criteria which are defined by the requirements and constraints of the scientific question. These criteria are usually represented by metadata related to one recording session and generated during or after the recording period (see Table  2 ). Use case 3 demonstrates how a comprehensive metadata collection with a standard file format supports the automatic selection of datasets according to defined selection criteria. Alice reported that she had the feeling that the weekend break influences the monkey's performance on Mondays and asked Bob to investigate her suspicion. To solve this new task, Bob first wants to identify the recording sessions for each weekday (criterion 1) which were recorded with standard task settings (criterion 2) and under the standard experimental paradigm (instructed delayed reach-to-grasp task with two cues; criterion 3).
To automatize the data selection, Bob writes a Python program where he loops through all available recording sessions, uses criterion 1 to identify the weekday, and adds each session name to a corresponding list if in the session criteria 2 and 3 were fulfilled. To write this program, Bob uses in scenario 1 the knowledge he gained from the manual inspection in use case 2 to identify for the chosen selection criteria the different metadata source files. Criterion 1 was stored in the. nev data file (label 5 in Figure  1  and Table  2 ) which Bob can access via the data loading routine. Criteria 2 and 3 are instead stored in the recording-specific spreadsheet (label 7 in Figure  1  and Table  2 ). To automatize their extraction, Bob additionally needs to write a spreadsheet loading routine, because there is no standard loading routine available for the homemade structure of the spreadsheet.
In contrast, in scenario 2, Bob extracts all three criteria for each recording from one comprehensive metadata file using an available loading routine of the chosen standard file format. In summary, compared to scenario 1, scenario 2 improved the workflow of selecting datasets according to certain criteria in two aspects: (i) to check for the selection criteria only one metadata file per recording needs to be loaded, and (ii) to extract metadata stored in a standardized format a loading routine is already available. Thus, in scenario 2 the scripts for automatized data selection are less complicated, which improved the reproducibility of the operation.
Use Case 5: Metadata Queries for Data Selection It is common that datasets are analyzed not only by members of the experimenter's lab, but also collaborators. Two difficulties may arise in this context. First, the partners will often base their work on different workflow strategies and software technologies, making it difficult to share their code, in particular code that is used to access data objects. Second, the geographical separation represents a communication barrier resulting from infrequent and impersonal communication by telephone, chat, or email, requiring extra care in conveying relevant information to the partner in a precise way. Use case 5 demonstrates how a standard format used to save the comprehensive metadata collection improves cross-lab collaborations by formalizing the communication process through the use of queries on the metadata.
Alice has detected a systematic noise artifact in the LFP signals of some channels in recording sessions performed in July (possibly due to an additional air-conditioning). As a consequence, Alice decided to exclude recordings performed in July from her LFP spectral analyses. Alice collaborates with Carol to perform complementary spike correlation analyses on an identical subset of recordings in order to find out if the network correlation structure is affected by task performance. To ensure that they analyze exactly the same datasets, the best data selection solution is to rely on metadata information that is located in the data files, rather than error-prone measures such interpreting the file name or file creation date. In scenario 1, since Alice and Carol use different programming languages (MATLAB and Python), they need to ensure that their routines extract the same recording date.
This procedure will require to provide corresponding crossvalidations between their MATLAB and Python routines to extract identical dates. In scenario 2, Alice and Carol agree on a concrete formal specification of the dataset selection (Figure  4 ) via the comprehensive metadata collection. In such a specification, metadata are stored in a defined format that reflects the structure of key-value pairs: in our example, Alice would specify the data selection by telling Carol to allow for only those recording sessions where the key Month has the exact value July. For the chosen standardized format of the collection, the metadata query can be handled by an application program interface (API) available both at the MATLAB and Python levels for Alice and Carol, respectively. Therefore, the query is guaranteed to produce the same result for both scientists.
GUIDELINES FOR CREATING A COMPREHENSIVE METADATA COLLECTION The five use cases illustrate the importance and usefulness of a comprehensive metadata collection in a standardized format. One effort to develop such a standardized format is the odML project, which implements a metadata model proposed by  Grewe et al. (2011). The odML project supports a software library (the odML library) for reading, writing, and handling odML files through an API with language support for Python, Java and MATLAB. The remainder of this paper complements the original technical publication of the software  (Grewe et al. , 2011)  by illustrating in a tutorial like style its practical use in creating a comprehensive metadata collection. We demonstrate this process using the metadata of the described experiment as a practical example.
g. , description of an experimental subject) are grouped in so called Sections which are by their name, type and a Sections can further be nested, i. e. , contain sub-sections and thus form a tree-like structure. At the root of the tree is the Document which contains information about the author, the date of the file and a version. Figure  5  illustrates how a subset of the experimental metadata is organized in an odML file. The design concepts presented here can be transferred to many other software solutions for metadata handling. We chose the odML library because it is comparatively generic and flexible, which makes it well-suited for a broad variety of experimental scenarios, and which enabled us to introduce it as metadata framework in all our collaborations. For extensive details on the odML metadata model in addition to  Grewe et al.
Metadata Strategy: Distribution of Information In preparing a metadata strategy for an experiment, one has to decide if and how information should be distributed over 2 https://github. com/G-Node/python-odml multiple files. For example, one could decide to either generate (i) several files for each recording, (ii) a single file per recording, or (iii) a single file that comprises a series of recordings. The appropriate approach depends on both, the complexity of metadata and the user's specific needs for accessing them. In the following, we will exemplify situations which could lead to one of the three different approaches: (i) If the metadata of a preprocessing step are complex, combining them with the metadata related to the initial data acquisition could be confusing. One could instead generate separate files for the preprocessing and the recording.
To keep the structure as generic as possible, we registered, besides the total number of electrodes references and grounds, the number of active electrodes, and the number of grids as Properties of the (level-2-) Section "Array" (guideline ii). Within the "Array" Section, (level-3-) Sections named "Electrode_XXX" FIGURE 6 | Schematic view of the tree structure of the odML template for a Utah array. Color code of boxes matches color code of odML objects in Figure  5. To simplify the schema, odML Properties and their values are listed in blocks for each odML section and additional attributes of the odML objects (e. g. , definitions) are not displayed. Likewise, the remaining 94 odML Sections of the Utah Array electrodes ("Electrode_002" to "Electrode_095") are left out and only indicated by the dashed line.
Metadata Strategy: Workflow For most experiments it is unavoidable that metadata are distributed across various files and formats (Figure  1  and Table  2 ). To combine them into one or multiple file(s) of one standard format, one not only needs to generate a meaningful structure for organizing the metadata of the experiment, but also to write routines that load and integrate metadata into the corresponding file(s). For reasons of clarity and comprehensibility we argue to separate these processes. Figure  7  illustrates and summarizes the corresponding workflow for the example experiment using the odML library. As a first step, it is best to write a template in order to develop and maintain the structure of the files of a comprehensive metadata collection. In the odML metadata model, a template is defined as an "empty" odML file in which the user determined the structure and attributes of Sections and Properties to organize the metadata, but filled it with dummy Values.
To create an odML template one can use (i) the odML Editor, a graphical user interface that is part of the odML Python library (see supplement), (ii) a custom-written program based on the odML library (Figure  8 ), or (iii) a spread-sheet software (see Figure  9 ), e. g. , Excel (Microsoft Corporation). Especially for large and complex structures, (ii) or (iii) are the more flexible approaches to generate a template. For (iii) we developed the odML-tables package 3  which provides a framework to convert between odML and spreadsheets saved in the Excel or CSV format. To simplify editing templates it is advisable to create multiple smaller templates (e. g. , into the top-level Sections). These parts can then be handled independently and are later easily merged back into the final structure. This approach facilitates the development and editing even of large odML structures.
The top of Figure  7  illustrates how three templates of the example experiment (Project_Temp, UtahArray_Temp, and Subject_Temp) representing the three possible template formats (i-iii) are merged into one large "empty" odML file via the Python odML library. Once a template is created, as a second step, the user has to write or reuse a set of routines which compile metadata from the various sources and integrate them into a copy of the template and save them as files of the comprehensive metadata collection (see Figure  7 , left part). Although, it would be desirable if the complete workflow of loading and integrating metadata into one or multiple file(s) of a comprehensive metadata collection would be automatized, in most experiments it cannot be avoided that some metadata need to be entered by hand (see Section 2).
Importantly, we here avoid direct manual modification of FIGURE 7 | Schematic workflow for generating odML files in the reach-to-grasp experiment. The Templates box (top): illustrates three out of six template parts which are used to build the complete, but (mostly) empty odML Document combining metadata of one recording session in the experiment. The three possible template formats (script, odML file, and spreadsheet) are indicated within the gray shaded boxes (left to right, respectively). The Metadata collection box (left) illustrates all metadata sources of a recording session (black labeled small boxes) ordered according to file formats (white boxes). Labels of metadata sources are listed in Table  2. The central large white box shows the workflow of the odML generator routine (odMLGenerator.
py). Code snippets used at the different steps of this workflow are illustrated in the smaller gray scaled boxes whose colors represent the software used for the code (see legend right of center). The black colored boxes indicate files or file stages. The workflow consists of 5 steps: (i) loading all template parts, (ii) merging all template parts into one empty odML Document, (iii) loading all metadata sources of a session, partially with custom-made routines (metadata_io), (iv) integrating all metadata sources into the empty odML Document using custom-made routines (metadata_io), and (v) saving the filled odML Document as odML file of the corresponding recording session. The Metadata collection box (right) illustrates the reduction of all metadata sources of one recording session to one metadata source (odML file).
the comprehensive metadata files. Instead, the final metadata collection is always created from a separate file containing manually entered metadata. This approach has the advantage that, if at one point the structure needs to be changed, these manual entries will remain intact. Practically, manual metadata entries are collected into a source file that is in a machinereadable format, such as text files, CSV format, Excel, HDF5 4  , JSON 5  , or directly in the format of the chosen metadata management software (e. g. , odML), and use the corresponding libraries for file access to load and integrate them into the file(s) of the comprehensive metadata collection. In the example experiment we stored manual metadata in spreadsheets that are accessible by the odML-tables package. In the special case that these manual metadata entries are constant across the collection (e.
g. , project information, cf. Table  2 ), one can even reduce the number of metadata sources by entering the corresponding metadata directly into the templates (e. g. , Project_Temp in Figure  7 ). This avoids unnecessary clutter in the later compilation of metadata and facilitates consistency across odML files. Metadata that are not constant across odML files should be preferably stored in source files that adhere to standard file formats which can be loaded via generic routines. For this reason, all spreadsheet source files in the presented experiment were designed to be compatible with the odML-tables package (see Subject_Temp in Figure  7  and as example in Figure  9 ). Furthermore, the files generated by the Blackrock data acquisition system as well as the results of the LFP quality assessment are loadable via the file interfaces of the Neo Python library  (Garcia et al.
, 2014)  which provides standardized access to electrophysiological data. Nevertheless, for the example experiment it was still necessary to write also custom loading and integration routines from scratch to cover all metadata from the various sources (e. g. ,. hdf5 and. mat files for results of the various preprocessing steps in Figure  7 ). In summary, this two stage workflow of first generating templates, and then filling them from multiple source files guarantees flexibility and consistency of the metadata collection over time. In particular, in a situation where the structure needs to be changed at a later time (e. g. , if new metadata sources need to be integrated) one only needs to adapt or extend the template as well as the code that fills the template with metadata in order to generate a consistent, updated metadata collection from scratch.
Default values which should be manually changed to actual metadata are marked in red. To be able to directly use such a. xls file as template and later directly translate it to the odML format one should at least include the Section Definition as additional column [e. g. , for "Path: /Subject" add "Section Definition: Information on the investigated experimental subject (animal or person)"]. of sets of related metadata in a fast manner in order to screen data across the experiment, and (v) to formalize communication in collaborations by means of metadata queries. We illustrated how to practically create a metadata collection from data using the odML framework as example, and how to utilize existing metadata collections in the context of the five use cases. In light of the increasing volume of data generated in complex experiments, neuroscientists, in particular in the field of electrophysiology, are facing the need to improve the workflows of the daily scientific life  (Denker and Grün, in press).
de/survey. While the storage of metadata to improve the overall experimental and analysis workflow is technically feasible, it cannot solve the intrinsic problem of identifying the metadata in an experiment, collecting the individual metadata, and pooling them in an automatic way. All of these steps are not trivial and are time-consuming by nature, in particular when organizing metadata for a particular experiment for the first time. Currently, no funding is granted for such tasks, and software tools that support the automated generation and usage of metadata are largely missing. Therefore, rightly the question may arise whether it is truly worth the effort. Through the illustration of use cases, we hope to have convinced the reader that indeed numerous advantages are associated with a well-maintained metadata collection that accompanies the data.
For the individual researcher these can possibly be best summarized by the ease of organizing, searching and selecting datasets based on the metadata information. Even more advantages are gained for collaborative work. First, an easily accessible central source for metadata information ensures that all researchers access the exact same information. This is particularly important if metadata are difficult to access from the source files, hand-written notebooks or via a specialized program code, or if the metadata need to be collected from different locations. Second, the communication between collaborators becomes more precise by the strict use of defined property-value pairs, lowering the probability for unintended confusion. Last but not least, once the metadata collection structure, content and method of creation are defined by the experimenter, metadata entry and compilation will in general run very smoothly, be less error prone, and will even save time in comparison to traditional methods.
As a result of our experience with complex analysis of experiments in systems neuroscience as reported in this paper, one of our recommendations is to record as much information about the experiment as possible. This may seem in contrast to efforts specifying minimal information guidelines in the life sciences (MIBBI;  Taylor et al. , 2008, MINI; Gibson et al. , 2009). However, those initiatives target the use case where data are uploaded to a public database, and their goal is to achieve a balance of information detail such that the minimally sufficient information is provided to make the data potentially useful for the community. They are not meant as guidelines for procedures in the laboratory. To ensure reproducibility of the primary analysis, but also in the interest of future re-use of the data, we argue that it is highly desirable to store all potentially relevant information about an experiment.
We have exemplified the practical issues of metadata management using the odML metadata framework as one particular method. Similar results could be obtained using other formats. RDF 6  is a powerful standard approach specifically designed for semantic annotation of data. Many libraries and tools are available for this format, and in combination with ontologies it is highly suitable for standardization. However, efficiently utilizing this format requires elaborate technology that is not easy to use. Simpler formats like JSON 7  or YAML 8  are in many respects similar to the XML schema used for odML, and we would expect that they have been used in individual labs to realize approaches similar to the one presented here. We are, however, not aware of specific tools available for the collection of metadata that use these formats.
Nevertheless, the approach we have described here must ultimately be combined with such systems, as the development of automated workflows depends critically on the availability of a metadata collection. Likewise, our approach is suitable for combination with provenance tracking solutions like Sumatra  (Davison et al. , 2014). Indeed, while we believe that using a metadata framework, such as odML, represents an important step toward better data and metadata management, it is also clear that this approach has a number of potential improvements. A natural step would be for odML to become an intrinsic file format for the commercially available data acquisition systems, such that their metadata are instantly available for inclusion in the user's hierarchical tree. In this context, the odML format so far has been adopted in the connectome file format 15  , the Relacs data acquisition and stimulation software 16  , and the EEGBase database for EEG/ERP data 17.
Likewise, the availability of interfaces for easy odML export in popular experiment control suites, vendor-specific hardware or generic software, such as LabVIEW, would greatly speed up the metadata generation. In addition, a repository for popular terminologies and hardware devices has been initiated by the German Neuroinformatics Node 18  , which is open for any extensions by the community. More importantly, in conditions where the details of recording or post-processing steps may change over time, it is essential to keep track also of the versions of the code that generated a certain post-processing result, including the corresponding libraries used by the code and installed on the computer executing the code. Popular solutions to version control (e. g. , git) or provenance tracking (e.
g. , Sumatra;  Davison et al. , 2014)  offer mechanisms to keep track of this information. However, it is up to the user to make sure that information about the correct version numbers or hash values provided by these systems are saved to the file containing the metadata collection to guarantee that metadata can be linked to its provenance trail. The more direct integration of support for metadata recording into the various tools performing the postprocessing would allow to automatize this process, leading to a more robust metadata collection, paired with enhanced usability. A perhaps more challenging problem is devising mechanisms that link the metadata to the actual data objects they refer or relate to. This issue becomes particularly clear in the context of the Neo library  (Garcia et al.
, 2014) , which is an open-source Python package that provides data objects for storing electrophysiological data, along with file input/output for common file formats. The Neo library could be used to read a particular spike train that relates to a certain unit ID in the recording, while the corresponding odML file contains for each unit ID the information about the assigned unit type (SUA, MUA, or noise) and the signal-to-noise ratio (SNR) obtained by the spike sorting preprocessing step. A common task would now be to link these two pieces of information in a generic way. Currently, it is up to the user to manually extract the SNR of each neuron ID from the metadata, and then annotate the spike train data with this particular piece of metadata. This is a procedure that is time-consuming, and again may be performed differently by partners in a collaboration causing incoherence in the workflow.