dataesr/scientific-paragraphs-categorization huggingface dataset:
The preservation of electronic datasets, including software, in the scientific record therefore has reproducibility as its main goal. The second important goal is to permit reuse of both data and computational methods. This corresponds to the second role of the scientific record, which is to allow future research to build on work from the past. Both reproduction and reuse add an important requirement to the technical criterion of replicability: the original computational protocol must be understandable by the scientist who sets out to explore the failure of a reproduction attempt, or to apply the protocol to a different problem. Experience has shown that understanding a computational protocol is based both on inspection of the source code and on experimentation, i. e. observation of the programs' behavior for various input data.
The ActivePapers project The goal of the ActivePapers research and development project is to design a platform for publishing and archiving computer-aided research whose first priority is to ensure the utility of electronic artifacts as items in the scientific record. Such a platform should ideally meet all of the following requirements: 1. A published electronic dataset, in the following called an ActivePaper, should be allowed to contain a combination of data, code, and narrative related to a research project. 2. An ActivePaper can reuse data items from previously published ActivePapers via references. These references can also be used for attributing credit to everyone involved in producing the information that is reused. 3. The representation of executable code inside an Active-Paper should be well-defined, stable, and sufficiently simple to allow implementation on future computing systems with minimal effort.
The execution of any piece of code from an ActivePaper should always produce exactly the same results at the bit level. 4. Any code stored in an ActivePaper should be safe to execute, i. e. it should not be able to cause any harm to the computing environment it is executed on. 5. An ActivePaper should contain metadata for provenance tracking and reproducibility. In view of the planned use of this platform for biomolecular simulations, additional requirements follow from the needs of highperformance computing on big datasets: 6. An ActivePaper should support large datasets by ensuring compact storage and high-performance data access. 7. No data conversion should be required to prepare an ActivePaper for long-term preservation, or to reuse data from an archived ActivePaper. While the first requirement states that an ActivePaper should be able to combine data, code, and narrative in a single package, this does not preclude a more fine-grained organization of the results of a scientific study in the form of multiple ActivePapers.
On the contrary, it is to be expected that important software tools remain forever outside of the ActivePaper universe and work on ActivePapers as data. This includes everything requiring user interaction, from authoring tools to data visualization programs, and also highly machine-specific software such as batch execution managers. It is also possible to write external code accelerators that take code from an ActivePaper and execute it after optimization and/or parallelization, guaranteeing identical results. While the current state of the art does not provide techniques for making such code accelerators both general and efficient, it is possible and even straightforward to write problem-specific code accelerators, which are simply efficient reimplementations (in a language like Fortran or C) of algorithms stored in an ActivePaper, with the equivalence of the results verified by extensive tests.
The requirement of metadata for provenance tracking and replicability implies that an ActivePaper must contain sufficient information to reconstruct the full dependency graph of all datasets. Each dataset that is the result of a computation must have a reference to the program that generated it plus a reference to all other datasets that this program has used as input. The notion of an ActivePaper bears many resemblances with the concept of a Research Object that has recently been proposed  28. In fact, an ActivePaper can be considered a realization of the Research Object idea with particular emphasis on the computational aspects of research.
Tools for reproducible research The Reproducible Research movement addresses many of the needs of publishing computational science, with the notable exception of long-term preservation. It is therefore interesting to consider the tools that have been developed for making computational research more reproducible. The computational notebook approach 13 , pioneered by Mathematica 29 and recently popularized by the Jupyter project (formerly known as the IPython notebook)  30  , builds on earlier developments in literate programming 31 , which have also been applied to computational science directly  32, 33. It aims to integrate computational methods expressed as working code with input/output data and the scientific narrative. It permits a seamless transition from interactive exploratory work to a documented computational method that can be shared and published.
Compared to traditional scripts, computational notebooks represent an important advance in improving reproducibility through improving human understanding. However, none of the existing notebook implementations fulfill any of the requirements listed above for an ActivePaper. Like a script, a notebook depends on the computational environment in which it was generated. This environment is neither preserved nor even documented in the notebook. After a few years, a notebook still provides a human-readable and rather detailed description of the method, but re-running it is likely to be difficult or impossible. Moreover, the notebook approach does not take into account datasets, unless they are small enough to be included as literal data into the notebook itself. All other data is accessed by usually non-permanent references such as filenames or Universal Resource Locators (URLs).
Similar remarks apply to workflow management systems such as Kepler 34 , Taverna 35 , or VisTrails 36. In fact, workflows, scripts, and notebooks all refer to the same basic concept: the outer algorithmic layer that defines a specific computational study in terms of more generic components. The differences lie in the user interface and in the kind of components that can be used (libraries, executables, Web services, etc. ). Some workflow managers can archive these components partially, and also some kinds of datasets, but such support is neither complete nor exhaustive. The one aspect in which workflow managers improve on notebooks is the fifth ActivePapers requirement, i. e. provenance tracking. Attempts to record a computational environment in a form that can be restored on a different computer are still rare.
Ref. 37 describes how this goal can be achieved for programs in the R language, extending that language's package and repository mechanisms by package libraries, package cohorts, and validated repositories. There are many similarities with the ActivePapers approach to software preservation and deployment (requirement 3 in the list given above). It is also interesting to note the main difference: the tools switchr and GRANbase described in Ref. 37 start from the R language's existing mechanisms and extend it with functionality required by the scientific record, whereas ActivePapers starts from existing mechanisms for publishing electronic artifacts and extends them to include software. The most comprehensive approach to archiving scientific software in an executable form is based on virtual machine technology  [38] [39] [40] [41].
Virtual machine technology effectively creates a technical barrier to fulfilling requirement 2. The longevity of archived virtual machine images is uncertain, making requirement 3 impossible to fulfill in the long run. Executing such an image requires complex and sophisticated software, which for the moment is produced and maintained by non-scientific organizations for reasons completely unrelated to science. Once technological progress makes these efforts obsolete, it must be expected that computations archived as virtual machine images will become unusable. In summary, virtual machines are a good solution to facilitate deployment for replication attempts, but are not a suitable data format for preservation. Finally, the Exec&Share server 42 and Elsevier's Collage Authoring System 43 propose the creation of "companion sites" to journal articles, which contain both software and datasets.
A Web interface permits users to run the software on the authors' input data or on their own data sets. In terms of the ActivePapers requirement list, a companion site fulfills points 1, 3, and 4. The main advantage of this approach is the ease of deployment for replication or reuse, which requires no software installation. It is the server administrators' responsibility to maintain the computational environment for the deposited software. For this reason, there are severe restrictions on the kind of software that can be deposited, and similar restrictions apply to the format and size of the deposited datasets. Long-term preservation is technically possible, but at this time none of the operators of these services has made any promise of longterm engagement for keeping the contents available in a functional state.
Bytecode platforms The term bytecode (sometimes also called p-code) describes an instruction set for a hypothetical processor. A compiler that generates bytecode instead of machine code for a hardware processor typically produces smaller compiled code which moreover does not depend on a specific hardware platform. However, since bytecode cannot be executed directly by a physical processor, it requires an additional processing step, either interpretation or a second compilation phase, which tends to reduce overall performance. The principle of compilation to bytecode has been used for almost 50 years, initially with the goal of generating compact and portable code, the loss of performance being accepted as a trade-off. More recently, just-in-time (JIT) compilation techniques have significantly reduced the performance issues related to bytecode to the point that there is often no significant performance difference to programs compiled to native machine code.
The runtime system processing the bytecode can offer much more fine-grained access control to system resources than today's operating systems, making bytecode platforms a good choice wherever there is a risk of malicious code distribution (viruses etc. ). In summary, a well-designed bytecode platform implements points 3 and 4 of the ActivePapers requirements list. The notion of a platform as a precise and complete specification of the computational environment that a piece of software relies on is of particular importance for software preservation. In fact, software preservation requires preservation of the files containing the software plus preservation of the platform the software was created for. Most of the well-known difficulties with software distribution, deployment, and preservation have their roots in the lack of well-defined computational platforms, as does the frequent nonreplicability in computer-aided research.
This implies that all results can be obtained from pure computation. This fact is not obvious because in most of today's scientific software, the scientific aspects are closely entangled with technical aspects such as user interaction or resource management. As an example, a typical visualization tool entangles the mapping from input data to graphics primitives, which is a pure computation, with the code for displaying these primitives on a computer screen and letting the user interact with the display. A carefully designed software architecture can separate the scientific contents from the user interface aspects and ensure replicability and preservation for the scientific aspects. The limited portability of floating-point operations on the JVM is due to a deliberate decision to give up their precise initial specification in favor of a less rigid one that permits a more efficient implementation on today's most widely used hardware architecture, Intel's x86 series of processors.
g. 50-52). The second popular bytecode platform, the ECMA standard CLI  45  , can be considered a more modern implementation of the basic JVM idea, with a change in priorities. The CLI platform was designed by Microsoft and implemented as the. NET framework for Windows. The. NET framework, which also contains libraries that are not part of the CLI platform, remains the dominant CLI implementation, to the point that much software described as CLI-compliant has in fact never been tested with any other implementation. Portability of. NET/CLI applications is therefore more limited than with the JVM. Software developers use. NET because of its security features, ease of deployment, and safe interoperability between multiple programming languages. It is in particular the last point where the CLI platform has clear advantages over the JVM, which was originally designed for a single language, Java.
The ActivePapers JVM edition The original ActivePapers architecture 55 , which was subsequently implemented in the "ActivePapers JVM edition", was a proof-ofconcept design intended to show that it is possible with existing technology to meet all the requirements defined above. The key design and implementation choices were • An ActivePaper is a file in HDF5 format  56. The HDF5 format ensures flexibility, compactness, and high-performance data access (requirements 6 and 7). • Any data item inside a published ActivePaper can be referenced by the combination of the ActivePaper's DOI and the HDF5 path to the dataset (requirement 2). • Executable code is stored as JVM bytecode. Any other code representation, in particular human-readable source code in any language, is admissible if a compiler or interpreter exists in the form of JVM bytecode (requirement 3).
• The JVM security model is used to prevent executable code in an ActivePaper from accessing any data outside of the ActivePapers platform, enforcing security at the platform level (requirement 4). • HDF5 dataset attributes are used to store metadata, including a dataflow graph that records provenance (requirement 5), but also creation time stamps and a data type indicator distinguishing references and executable code from "plain" datasets. • Individual programs inside an ActivePaper are labeled as either pure computations ("calclets"), or as data importers ("importlets"). The latter have unrestricted read access to anything, including local files and network resources. They share the write restrictions of pure computations, meaning that they cannot modify anything outside of the ActivePapers universe.
Moreover, they are never run automatically, but only on explicit user request. The original design included a third kind of programs, "viewlets", intended for visualization or more generally for letting users inspect the results of computations, possibly interactively. This idea was abandoned, at least for the initial implementation, because of the difficulty of achieving efficient yet portable user interaction on the JVM platform. An implementation of the original ActivePapers platform is available from the ActivePapers Web site 14. Its only dependencies are (1) a Java Virtual Machine implementation, (2) the HDF5 library, and (3) JHDF5 57 , a Java interface to the HDF5 library. The Active-Papers software provides a command-line interface for creating ActivePapers, inspecting their contents and metadata, and for running the embedded executable code.
This is clearly a minimal working environment. For production use by a wide community of computational scientists, many convenience functions would have to be added: a code and data editor, data visualization, data management, etc. The use of the JVM bytecode platform by itself ensures that most of the requirements listed in the beginning of this section are met. In fact, many of these requirements could be met by using JAR files as ActivePapers, adding just a set of conventions for storing data, metadata, and references inside JAR files. The use of the HDF5 format rather than the JAR format for ActivePapers is motivated by the requirement of efficient handling of large datasets. The JAR format, which is a variant of the ZIP format, is designed to be compact and provide efficient read access to individual files, but it does not provide efficient write access nor efficient random access to parts of files.
The HDF5 format is much more general, and in particular allows performance tuning for various data access patterns. The HDF5 library also offers much useful functionality for structuring data access in scientific software. Finally, HDF5 is already widely used in scientific computing. One of the dataset types offered by HDF5 is a one-dimensional array of bytes, which is exactly the data model for files in today's operating systems. An ActivePaper can therefore store arbitrary file contents in a dataset, a feature which it uses to store JAR files for executable code. A disadvantage of the HDF5 format is the increased complexity of the ActivePapers platform, which includes HDF5 and JHDF5 as dependencies. It would be preferable to be able to access HDF5 files from pure JVM code, reducing the additional dependency to one more JAR file.
JVM bytecode is rather simple, and a working but inefficient implementation is not difficult to realize. As for bit-level replicability, it could be ensured in two ways: (1) allow only JVM bytecode in which all functions are compiled in strictfp mode, or (2) use a special JVM implementation that always uses the initial strict floating-point semantics. An important design decision is related to the management of the metadata that tracks dependencies and provenance. The ActivePapers platform creates and updates this metadata automatically during program execution. From the user's point of view, an ActivePaper is a collection of datasets and programs, of which the latter can be run individually just like traditional executables or scripts. The ActivePapers platform tracks all data accesses from programs and generates the dependency graph from them.
When a program is re-run, typically after modification, all the datasets it generated earlier are deleted automatically. Moreover, when a program reads data generated by another program which has been modified since it was last run, the modified program is re-run automatically to ensure coherence of all data. This automatic dependency handling has worked well in practice. It is the inverse of the approach taken by automation tools such as make 58 , which execute programs according to a manually prepared definition of the dependencies between their results. Note that the difference concerns only how the metadata is created. The resulting dependency graph is the same with both approaches, and the choice between them is not a fundamental one, but mainly a user interface design aspect. The ActivePapers implementation for the JVM provides an automatic replication tool.
The platform was never used widely enough to generate any experience with malicious code, but the security mechanisms have proven very helpful in detecting errors. A typical situation is the transfer of code written for stand-alone use into an ActivePaper. The original code thus accesses local files and network resources for input and output. These access must be modified to work exclusively on ActivePaper datasets. The JVM access control mechanisms were very helpful in locating these accesses and in ensuring that every single one was taken care of. While the ActivePapers JVM edition satisfies all the initial requirements, it is very difficult to use in real research projects, because of a critical lack of scientific software compatible with its constraints. All code running inside of the ActivePapers platform must exist as JVM bytecode.
The ActivePapers Python edition In order to gain experience with the ActivePapers approach in practice, a second implementation was developed for the Scientific Python ecosystem  59. Its dependencies are the Python language 60 , the HDF5 library, the h5py library 61 for interfacing HDF5 to Python, and the NumPy library 62 which is a dependency of h5py. For the Python edition of ActivePapers, all executable code must exist in the form of Python scripts, which access the datasets through the h5py library. Python libraries that contain extension modules (which is Python jargon for modules written in a compiled language such as C) cannot be placed inside an ActivePaper, but can be declared as an external dependency. This effectively means that the platform required for using an ActivePaper with such a dependency includes that library in addition to the packages listed above.
Adding external dependencies is clearly not desirable, but it provides a shortterm workaround to the fundamental problem that most scientific software is not ready for installation-free deployment and longterm preservation. The Scientific Python ecosystem provides a large choice of libraries that can be used within these constraints, and the Python language is already very popular for scientific computing, making the Active-Papers Python edition a good vehicle for testing the ActivePapers approach on real research projects. On the other hand, the Python edition cannot fulfill requirements 3 and 4 listed above. In particular, the Python language lacks sufficiently strong security mechanisms to implement a useful level of user protection (requirement 4). A more subtle problem is the stability of the platform itself.
The Python language has no formal specification and in fact evolves together with its principal implementation. The scientific libraries, in particular NumPy, also evolve rather rapidly, with only moderate efforts to maintain compatibility with older versions. For example, version 1. 9 of NumPy removed a compatibility layer for use with older Python code that was written for NumPy's predecessor Numeric. It is difficult to estimate how much published research code was affected by this change, but it includes everything I have published between 1997 and 2012. The ActivePapers platform records the version of all libraries that were used in the preparation of an ActivePaper, but the long-time usability of these versions is questionable, as in general only the current versions can be expected to work in current computing environments.
Requirement 3 is therefore only partially fulfilled. The ActivePapers Python edition has been used for several research projects in the field of biomolecular simulation, some of which have already been published  [63] [64] [65]. Each publication has one or more ActivePaper files deposited as supplementary material, but all the files are also available in digital repositories with DOIs. Among the published ActivePapers, there are software libraries 66,67 , a database of protein structures 68 , and combinations of datasets and code that document computational studies  [69] [70] [71]. Additional published Active-Papers contain obsolete versions of the pyMosaic library 72-74. These files remain permanently available because other ActivePapers depend on them. They also remain usable for as long as the underlying platform remains compatible.
Many of the published ActivePapers reuse code or data from other published ActivePapers through references. Each reference consists of the DOI of the ActivePaper file that contains the data plus the HDF5 path that identifies the dataset inside the file. The ActivePapers management software automatically downloads ActivePapers that are required for satisfying references. This download mechanism supports Zenodo 5 and figshare 4 at the moment. It would be desirable to support any DOI-granting digital repository, but unfortunately the DOI infrastructure does not yet provide the required functionality. DOI resolution produces a URL pointing to a humanreadable landing page. This page typically contains links to the data files, but its layout is not standardized, making it impossible to find these links automatically and reliably.