dataesr/scientific-paragraphs-categorization huggingface dataset:
ESAMP Description Overview ESAMP is a database architecture designed to store experimental materials science data. It aims to capture all three of the types of aforementioned data: 1) information about the samples in the database including storing provenance regarding how they were created and what processes they have undergone, 2) raw data from processes run on the samples, and 3) information derived from analyses of these raw data. Altogether, this architecture enables users to use simple SQL queries to answer questions like: • What is the complete history of a given sample and any other samples used to create this one? • How many samples have had XRD run on them both before and after an electrochemistry experiment? • What is the figure of merit resulting from a given set of raw data analyzed using different methods? Identification of data to evaluate any scientific question requires consideration of the context of the data, motivating our design of the ESAMP structure to intuitively specify contextual requirements of the data.
For example, if a researcher wishes to begin a machine learning project, creating a custom dataset for their project can be done by querying data in the ESAMP architecture. For example, training data for machine learning prediction of the overpotential in chronopotentiometry (CP) experiments from catalyst composition can be obtained via a query to answer questions such as • Which samples have undergone XPS then CP? • How diverse are the sample compositions in a dataset? The researcher may further restrict the results to create a balanced dataset or a dataset with specified heterogeneity with respect to provenance and experiment parameters. The query provides transparent self-documentation of the origins of such a dataset; any other researcher wondering how the dataset was created can look at the WHERE clause in the SQL query to see what data was included and excluded.
To enable these benefits, we must first track the state of samples and instruments involved in a laboratory to capture the ground truth completely. In this article, we focus mainly on the state of samples and note that the architecture could capture the state of instruments or other research entities. A sample provenance can be tracked by considering three key entities: Sample, Process, and Process_data, which are designed to provide intuitive ingestion of data from both traditional manual experiments and their automated or robotic analogues. Sample: A sample is a label that specifies a physicallyidentifiable representation of an entity that can undergo many processes (e. g. the liquid in that vial or the thin film on that substrate). Samples can be combined or split to form complex lineages, such as an anode and a cathode being joined in a battery or a vial of precursor used in multiple catalyst preparations.
Samples, Collections, and Lineage The trinity of sample, process, and process-data enable us to have a generalized framework that captures the ground truth associated with any given sample in an experimental dataset. However, interpretation of experimental data requires us to capture the provenance of a sample completely. That is, throughout the sample's lifetime, it is important to track three key things: • How was the sample created? • What processes occurred to the sample? • If the sample no longer exists, how was it consumed? The middle question is directly answered by the sequence of entries in the sample_process table wherein each record in sam-ple_process specifies the time that a sample underwent a process. This concept is complicated by processes that merge, split, or otherwise alter physical identification of samples.
Process Data & Analysis While tracks inputs to a , tracks the output of a. For reproducibility, transparency, and ability to continue experiments without reliance on an active database connection, it is prudent to store process outputs as raw files independent from the data management framework. Therefore, while may include relevant data parsed from the raw files, it should also always include a raw file path. Additionally, attributes can be added to specify the location to search for the file, such as an Amazon S3 bucket or local storage drive. A single file may also contain multiple pieces of data that each refers to different samples. This complexity motivates the inclusion of the start and end line numbers for a file identifying information for. If an entire file should be consumed as a single piece of process data, null values can be provided for those attributes.
As a significant amount of scientific data is stored as comma-separated values (CSV) files, it can also be beneficial to parse these files directly into values in the database utilizing flexible column data types, such as JavaScript Object Notation (JSON) that is supported by modern RDMS's. For large datasets, storing data using efficient binary serializations such as Messagepack could be beneficial. 19  The relationship between process outputs and their associated processes and samples can be complex. The most straightforward relationship is one piece of process data is generated for a single sample, which is typically the case for serial experimentation and traditional experimentation performed without automation. In parallel experimentation, a single process involves many samples, and if the resulting data is relevant to all samples, has a many-to-one relationship to.
In multi-modal experiments, multiple detectors can generate multiple pieces of data for a single sample in a single process, where has a one-to-many relationship to. Parallel, multi-model experimentation can result in manyto-many relationships. To model these different types of experimentation in a uniform manner, ESAMP manages many-to-many relationships between and. The raw output of scientific processes may require several iterative analytical steps before the desired results can be obtained. As the core tenet of this framework design is tracking the full provenance of scientific data, analytical steps must have their lineage tracked similarly to that of samples and processes. This is achieved by the analysis, analysis_details, and analysis_parent tables. The analysis table represents a single analytical step and, similar to , is identified by inputs, outputs, and associated parameters.
By designating only P 3 as state-changing, the sample can be considered to have only 2 unique states. SERs can be further parameterized by utilizing the 's of the process to determine state-changing behavior. For example, if P 2 is an anneal step, we might only consider it state-changing if the temperature rises above a certain level. By defining simple rules, merging equivalent states yields simpler state graphs that serve as the basis for dataset curation. This powerful concept of state is enabled by the core framework's ability to track the process provenance of samples throughout their lifetime.
Database Implementation The framework so far has been defined using standard entity relationship language. It is important to note that this framework can be instantiated in most or all RDMS's and is not tied to a specific implementation. However, the specific implementation details of the framework may change slightly depending on the RDMS used. These changes are vital in deciding the RDMS system that is appropriate for a particular use case. Figure  S1  shows the framework in its entirety. All double-sided arrows indicate a many-to-many relationship. The implementation of many-to-many relationships differs between SQL, NoSQL, and graph databases. In a SQL RDMS such as PostgreSQL, the standard practice uses a "mapping" table where a row is defined simply by its relationship to the two tables with the many-tomany relationship.
In graph databases, many-to-many relationships can be represented simply as an edge between two nodes. Additionally, entities that track lineages, such as , state, and anal-ysis_parent, can also be represented simply as edges between two nodes of the same type. The cost of this simplicity is the reduced constraints on column datatypes as well as a less standardized and, in some cases, less powerful query functionality. If complicated process provenance and lineages are expected to exist along with a need to query those lineages, then a graph database may be the right choice. However, if simpler lineages with large amounts of well-structured data are used, a standard SQL RDMS would be more advantageous. Data can even be mi-grated quite easily between implementations of this framework in two RDMS's if the slight differences noted above are carefully considered.
Results Implementation of the ESAMP framework is demonstrated via ingestion and modeling of MEAD, the database resulting from high throughput experimental investigation of solar fuels materials in the Joint Center for Artificial Photosynthesis (JCAP). 20  MEAD contains a breadth and depth of experiments that make it representative of a broad range of materials experiments. For example, the 51 types of processes include serial, parallel, and multi-modal experiments. Using the most conservative rule that every process is statechanging, the database contains approximately 17 million material states. This dataset contains many compositions in high-order composition spaces, particularly metal oxides with three or more cation elements. For electrocatalysis of the oxygen evolution reaction (OER), the high throughput experiments underlying MEAD have led to the discovery of catalysts with nanostructured mixtures of metal oxides in such high-order composition spaces.
Catalyst discovery use case To demonstrate the importance and utility of the management of process provenance and parameters, we consider a use case where data is curated to train a machine learning model and predict the catalytic activity of new catalyst compositions. We commence by considering all MEAD measurements of metal oxides synthesized by inkjet printing and evaluated as OER electrocatalysts, particularly the OER overpotential for an anodic electrochemical current density of 3 mA cm -2. This overpotential is the electrochemical potential above 1. 23 V vs RHE required to obtain the current density, so smaller values correspond to higher, desirable catalytic activity. Measurement of this overpotential can be made by cyclic voltammogram (CV) or chronopotentiometry (CP) measurements. Querying MEAD for all measurements of this overpotential and identifying the synthesis composition for each sample produces a dataset of composition and activity regardless of each sample's history prior to the CP experiment and the electrochemical conditions of the measurement.
This dataset is referred to as dataset A in Figure  5a  and contains 262,087 measurements of overpotential. Considering a provenance to be the ordered set of process types that occurred up to the overpotential measurement, this dataset contains 460 unique provenances. To increase the homogeneity in provenance and materials processing, the SERs can require that the catalyst samples have been annealed at 400 • C. Additionally, to generate a single activity metric for each sample, the SERs can also require only the most recent or "latest" measurement of activity, which results in a dataset B contain-ing 43,860 measurements, corresponding to 113 unique provenances. To further increase the homogeneity, the SERs can also require the electrolyte pH to be within 0. 5 of pH 13 and require those catalysts to have been operated for at least 100 minutes before catalyst activity measurement, resulting in dataset C containing 10,885 measurements.
This dataset contains only 26 unique provenances that differ in their sequence of electrochemical experiments that preceded the overpotential measurement. Dataset C contains 54 unique 4-cation composition spaces. To demonstrate machine learning prediction of catalyst activity in new composition spaces, each of these 54 combinations of 4cation elements is treated as an independent data instance in which the test set is taken to be all catalyst measurements from dataset C where the catalyst composition contains three or all four of the respective 4-cation elements. Keeping the test set consistent, three independent eXtreme Gradient Boosting (XGB) random forest regression models, one for each of the three datasets, were trained to predict over-potential from composition, wherein each case the composition spaces that comprise the test are held out from training.
Repeating these exercises for all 54 data instances enables calculation of the aggregate mean absolute error (MAE) for predicting catalyst activity, as shown in Figure  5a  for the three different datasets. The MAE improves considerably when increasing the homogeneity of provenance and experimental parameters from dataset A to B and from dataset B to C, demonstrating the value of using appropriate SERs to curate materials databases with specific provenance and property conditions to generate suitable training data for a specific prediction task. The parameters used for creating the SERs can also be considered as properties of the catalyst measurements, enabling the training of machine learning models that not only use composition as input but also additional parameters, in the present case the maximum annealing temperature, the number of previous measurements of the catalyst activity, the electrolyte pH, the duration of prior catalyst stability measurements, and whether the measurement occurred by CV or CP.
Automated ML pipelines In the catalyst discovery use case described above, we identified that the choice of state-changing processes had a significant effect in predicting OER overpotential. To avoid making such decisions a priori, which is often not possible in experimental research, all distinguishable processes should be reflected in the data management. For instance, a sample storage event is typically assumed to be non state-changing, which may not be the case. The simplest example is air-sensitive materials whose sample handling between experiments should be documented as "sample handling" processes. The ESAMP framework allows for every event in the laboratory to be defined as a process. However, in practice, capturing every event in a laboratory is infeasible in a typical laboratory setting.
There may always exist "hidden" processes that altered a material's state but were not tracked, which compounds the issues discussed above with human-made decisions about what processes are state-changing and whether that designation varies with either sample or process parameters. By liberally defining what constitutes a process and aggregating data from many experimental workflows, ESAMP will ultimately enable machine learning to identify hidden processes and which tracked processes are indeed state-changing. Recently, several research works have focused on developing closed-loop methods to identify optimal materials and processing conditions for several applications such as carbon nanotube synthesis  24  , halide perovskite synthesis  25  , and organic thin film synthesis  26. The workflows of these experiments are typically static.
Generality for modeling other experimental workflows While the breadth of process provenances and the dynamic range of depth within each type of provenance makes the MEAD database an excellent demonstrator of ESAMP, the provenance management and the database schema are intended to be general to all experimental and computational workflows. A given type of experiment may be considered equivalent when performed in 2 different labs, although differences in the process parameters and data management have created hurdles to universal materials data management. Such differences may require lab-specific ingestion scripts and tables, but custom development of these components of ESAMP comprise a low-overhead expansion of the database to accept data from new labs as well as new types of processes. One of the most widely used experimental inorganic crystal structural and diffraction databases (ICDD) was generated by manual curation and aggregation over several decades of x-ray diffraction data generated in many laboratories.
We anticipate that ESAMP's universal data management will result in a more facile generation of several large experimental datasets with full provenance that enables data-driven accelerated materials discoveries. In addition to the generation of new insights from provenance management and acceleration of research via more effective incorporation of machine learning, we envision materials provenance management to profoundly impact the integrity of experimental science. In the physical sciences, the complexity of mod-ern experimentation contributes to issues with reproducing published results. 27  However, the complexity itself is not the issue, but rather the inability of the Methods sections in journal articles to adequately describe the materials provenance, for example, via exclusion of parameters or processing steps that were assumed to be unimportant, which is exacerbated by complex, many-process workflows.
Generation of knowledge graphs and data networks As discussed above, we anticipate ESAMP to provide the framework that enables the curation of large and diverse datasets with full provenance. Such large datasets are a great starting point for machine learning applications. However, ESAMP is quite general, and adapting a more specific data framework to one's use case can make knowledge extraction easier. These frameworks may extract subsets of the data stored in the main framework and apply simplifying assumptions that apply to the specific use case. However, as long as a link exists between the higher-level framework and ESAMP, then the complete provenance information will still be preserved and queryable. Machine learning datasets, such as those described in datasets A, B, and C in the above use case, are examples of a practical higher-level extraction.
One example of a higher-level framework enabled by ESAMP is that of knowledge graphs. Knowledge graphs are a powerful abstraction for storing, accessing, and interpreting data about entities interlinked by an ontology of classes and relations  28. This allows for formal reasoning, with reasoning engines designed for queries like "Return all triples (x 1 , x 2 , x 3 ) where φ (x 1 , x 2 ) and φ (x 1 , x 3 ) and (ψ(x 2 , x 3 ) if and only if θ (x 3 ))". Beyond direct queries which produce tabular results suited for traditional machine learning applications, machine learning models can be applied directly to relational databases  29, 30  and knowledge graphs  31. Applications involving knowledge graphs and ontologies have been explored in the space of chemistry and materials science research  32, 33.
The population of knowledge graphs is mainly facilitated by ESAMP in two ways. Firstly, data within a relational database structure is straightforwardly mappable into the data structure of knowledge graph triples  34. Secondly, a solid grasp of how to resolve distinct entities can be achieved through ESAMP before populating the nodes of the knowledge graph. Alternative approaches of merging all samples with the same label or considering every possibly-distinct sample to be a unique material are too coarse-and fine-grained, respectively. Beyond knowledge graphs, other high-level frameworks specialize in the migration and merging of data between research groups that structure their experiments and analyses differently  35  , and these demand structured data such as ESAMP data as their initial input.
Better theory-experiment integration While the initial focus on experimental materials data is motivated by the historical lack of concerted effort to manage and track provenance compared to computational materials science, we also envision that ESAMP has the inherent flexibility and expressiveness to model computational materials provenance as well and assist in the holy grail of combining theory, experiment, and machine learning systematically. In the computational context, simulation details are recorded from automated workflows, input and output files, and code documentation, so the provenance and parameters involved in computations are simpler to track and ingest than experiments. To ingest computational data into ESAMP, we would consider the physically relevant aspects of a simulation, such as atomic positions, composition, microstructure, or device components and dimensions, to comprise a sample.
The simulation itself would be the process, with numerical parameters, theoretical approximations, and the compute hardware and software, potentially being relevant process details. Output files and logs would be the process data. Just as samples in experiments can undergo multiple processes, a simulation "sample" can start in a specific configuration, undergo optimization during a "process," and the new configuration, still associated with the same "sample," can be passed on for further processing. Computational samples could be combined -results of a simulation are mixed in a new simulation -or partitioned into new samples. The ESAMP framework allowing analyses to be built on multiple process data components also is relevant when post-processing simulation data. Integrating theoretical and experimental workflows has long been pursued in materials research.
If a computational simulation indicates a material has desirable properties, it is advantageous to directly query all of the experimental data associated with that material to validate the prediciton. Similarly, connecting a physical material to its computational counterpart can provide key insight into the fundamental source of its properties. ESAMP provides a framework for storing both computational and experimental workflows using the same relationships and entities, which creates a common language for querying the complex information associated with computational and experimental samples needed to complete this mapping. Not only will mapping experiments to theory be made easier through ESAMP, but more valid comparisons will also be enabled. Computational models are often benchmarked against experimentally obtained values.
Conclusions In this work, we present a database architecture, called ESAMP, designed for storing materials science research data. We demonstrate that the database architecture captures each material sample's provenance, the data derived from experiments run using each sample, and high-level results derived from the raw data. 6  An illustration of how process provenance can be queried to ensure uniformity within output datasets. If process data is populated by process P4 it may be necessary to group sample's based on the processes that preceded P4. Each of the four purple squares indicate different constraints on the processes that occurred before P4. A plain arrow is used to indicate a process is immediately followed by another, while a wildcard * is used to indicate that the any number of processes can intervene.
Acknowledgements The development and implementation of the architecture were supported by the  Toyota Research Institute  through the  Accelerated Materials Design and Discovery program. Generation of all experimental data was supported by the  Joint Center for Artificial Photosynthesis, a US Department of Energy (DOE) Energy Innovation Hub , supported through the  Office of Science of the DOE  under Award Number  DE-SC0004993. The development of the catalyst discovery use case was supported by the  U. S. Department of Energy, Office of Science, Office of Basic Energy Sciences , under Award  DESC0020383. The authors thank  Dr. Edwin Soedarmadji  for stewardship of MEAD and all members of the  JCAP High Throughput Experimentation group  for the generation of the data. The authors thank  Daniel Schweigert  for providing insights into standard database management practices.