dataesr/scientific-paragraphs-categorization huggingface dataset:
The diversity of storage instances provides an excellent opportunity for the access and analysis of the data from the Grid (dCache  [26]  instance), or from an local desktops using ownCloud. 14 The data consists of about 35,000 files with a total size of 806 Gb. The preserved JADE software consists of the original codes designed to process the JADE data, some MC event generators from the 1980-2000, the detector simulation routines for the JADE experiment, some calibration codes, the event display, and the analysis routines which were used to create the ROOT/PAW n-tuples mentioned above. In addition to that some interface codes to modern MC event records were added. The original JADE software has evolved on many different platforms and after the first resurrection  [27]  in the middle of the 2000s consisted of approximately 50,000 lines of Fortran code running on IBM AIX4.
g. mixed endianess I/O), therefore only the GNU Fortran and Intel Fortran toolchains and runtimes are practically useable. Thanks to the portability of the CMake build system it became possible to compile the JADE codebase not only on Linux but also for the first time on MacOSX. The dependence of the original codes on the CERNLIB and HIGZ was undesirable, so to avoid these dependencies the required CERN-LIB and HIGZ functions were emulated with a help of the ROOT analysis framework, with an optional dependence on CERNLIB. With the performed code updates it became possible to create new MC generated event samples using modern MC event generators, and to re-reconstruct the original JADE data. Some functionality of the event display was also restored. The computing model of the JADE data re-analysis would include the processing of the data (and/or MC simulated events) into plain ROOT n-tuples and subsequent steps performed using the ROOT framework.
The JADE codes were publicly accessible from the dedicated JADE site for a long time. 15 After the updates, the codes were put in a public repository account in GitHub. 16 The usage of GitHub has allowed for regular automated builds 14 https://owncloud. com/. 15 https://www. mpp. mpg. de/en/research/data-preservation/jade. 16 https://github. com/andriish/JADE. of the JADE software on modern Linux and MacOSX platforms. All the dependencies needed for the software are available from open source projects. The physics papers of JADE are available on INSPIRE. Many of those are scanned copies of the hard copies from CERN or KEK libraries. The JADE web site and the GitHub site mentioned above include as well: the full list of JADE physics papers, technical notes, scanned logbooks, data and software preservation documentation.
DELPHI The DELPHI experiment has moved its software stack to modern technologies, e. g. GitLab at CERN for the sources and CVMFS for the binaries, while archiving older binaries on EOS. During 2022 the full stack was ported to 64-bit as support for 32-bit libraries is vanishing. This was possible thanks to the efforts to revive CERNLIB, see chapter 5. 8. The experiment software heavily relies on CERN-LIB thus a validated and complete 64-bit version of CERN-LIB is a pre-requisite for the future of the DELPHI software stack. The revised DELPHI software stack supports data analysis frameworks, simulation, reconstruction and event visualisation. Builds are available for the following operating systems: CentOS7, CentOS-Stream 8 and 9 and Alma 8 and 9 in both 32 bit and 64 bit, and Ubuntu 18, 20 and 22 (64 bit only).
OPAL OPAL has preserved its data and complete software stack, with the exception of the event display, and kept it working up to CentOS8. CERNLIB and in particular ZEBRA are needed for access to the data. A 32-bit SLC6 based VM has been defined, but creating new instances does require the option to boot and install SLC6 based nodes, showing the weakness of relying on VMs for data archiving. Keeping the whole software alive and porting it to newer OS versions appears to be a safer approach, but also requires porting the required external libraries (CERNLIB, ZEBRA, etc. ). For OS versions newer than SLC6 the availability of the revived CERNLIB, see chapter 5. 8, is essential. The data has been migrated to EOS, and the software environment is being migrated from AFS to CVMFS. HBOOK/PAW based n-tuples exist for some specific types of analysis.
H1 The H1 experiment  [32, 33]  recorded a unique data set of lepton-proton collisions at HERA in the years 1992 to 2007. The complete RAW collision data comprises around 75 TB, the set of compressed DST data amounts to about 20 TB and the analysis level files are about 4 TB. A sizeable and universally employed software stack for the processing and analysis of the H1 data also exists, which was initially developed in the years 1988-2012. This is comprised of a series of core software packages in Fortran and the object-oriented analysis core framework, H1oo  [35]. , which is written in C++ and based on ROOT. Following a DHPEP level 4 preservation policy, the H1 Collaboration continues to maintain these data, all related software, including simulation and reconstruction code, as well as all relevant documentation on the data, MCs, software, detector, operation, meetings and collaboration life.
A continuously updated webserver  17  provides access to all resources for the collaboration members and for external visitors. The H1 Collaboration works under a renewed collaboration agreement with individuals as the only entity, and it is coordinated by a spokesperson, two deputies and a scientific secretary, and DESY acts as host laboratory. Since 2012, an additional twenty H1 papers (9% of the total) have been published, and presently several data analyses are ongoing or started recently. In 2021, a new Monte Carlo production campaign with about 4×10 7 events was performed to support one of the ongoing analyses and the full capability of the present software environment was proven. Access to the data, software and internal documentation is granted through a DESY computing account, which also provides access to computing resources for data analysis activities.
The offline documentation is stored in 150 m of shelve space in the DESY library archive. Since 2012, several software migrations were performed including the migration of the entire software and environment to 64-bit operating systems, up to Scientific Linux 6 and CentOS7. Previous stable software releases are kept in a central repository and can be used for bit-level validations or for production within containerized workflows. A comprehensive update of the software and analysis frameworks was performed recently  [36] , and is briefly summarized in the following. Whilst the full data analysis capability and flexibility was retained, after one decade, the overall status of the H1 software was shown to have a few shortcomings from the present point of view. The programming languages and standards such as C++98 were found to be unattractive for young people to learn and liable to slow down new developments and data analysis efforts.
In addition, outdated dependencies such as ROOT 5 complicated the usage of modern data analysis techniques and tools. Furthermore, the risk of incompatibility due to different compiler standards or different interfaces such as MC event generators only increases with time. As no externally maintained package repository was used, new packages had to be provided manually and the compilation and maintenance still required a profound and detailed knowledge about the specific structure of the H1 software, as well as some insight into the historic develop-ment. The renewed structure of the H1 software stack and environment  [36]  allows for an easy transition to even newer platforms, like AlmaLinux 8 or CentOS8 or 9, and are kept up-to-date with DESY's central Linux platforms. Considering the above and further arguments, the status of the H1 software and the data analysis capabilities was revisited in 2020.
All core software packages have since been successfully migrated to a modern computing platform, based on amd64 (x86-64) CentOS7, using the GNU 9. 2 compiler. Remaining external dependencies were updated to the latest releases and are now obtained from the LCG/AA package repository  [37, 38] , greatly reducing the number of H1 specific solutions. The common object-oriented data analysis framework H1oo is now based on ROOT 6 and supports the C++17 standard. An online code documentation for H1oo is available, whilst interactive analysis in C++ through CLING is now also possible, as well as for the first time in Python (v3). Several benchmark analyses codes were migrated, like those of jet production  [39, 40]  or inclusive DIS cross section measurements  [41] , and serve as a valuable and validated starting point for new data analyses.
The programs and libraries are now provided to the members of the H1 Collaboration on shared global file systems for convenience. All H1 software packages are now maintained in git and new build instructions for a complete rebuild of the entire software stack have been prepared. Container solutions have also been implemented for backward compatibility and software tests. New Monte Carlo event production campaigns with the full detector simulations and run-dependent conditions can be, and are, performed at the DESY computing cluster(s) and using the modernized H1 software stack. See Fig. 4  for a historical summary of the H1 MC production campaigns. Many H1 data analysis activities are still ongoing to this day, and new analysis projects are beginning due to the uniqueness of this scientific data set.
123 low level basic objects (calorimeters deposits, tracks, lepton candidates,. ), as well as higher level composite objects (e. g. jets, missing ET, particular meson decays,. ) which can also be rebuilt from the available basic information. These data comprise so-called 'common n-tuples' created from the 360 M real events recorded by the ZEUS detector between 1996 and 2007. They are stored in a unified flat ROOT n-tuple format which can be read and processed with almost any past or future version of the ROOT software package, or any other package providing a ROOT data format interface such as e. g. root-numpy, pyroot, RDataFrame or uproot. Consequently so far essentially no software maintenance updates were needed since the original setup in 2006, there is no need for virtual machines or containerization, and in this respect the ZEUS analysis software is always as modern as ROOT or any future backwards compatible successor (note that ROOT itself is backwards compatible to earlier HBOOK/PAW formats dating all the way back to the late 1970s through a simple 'h2root' converter).
All simulated data sets available up to the time of the end of funding in 2014 are also stored in the same format. The relations between the MC generated events and the description of the simulated processes is documented in dedicated internal ZEUS web pages and the lists of files belonging to each generated MC sample are also stored in a standalone sqlite database. The generation of small additional simulated data sets including detector simulation (level 4, through an encapsulated and/or containerized approach) is possible  [49]  and has been used successfully. Dedicated codes needed for the zevis event display and for automated handling of the cninfo data base (both very useful but not crucial) are available as well in GitHub. 19  All basic real and simulated data, with a total size of about 250 Tb, are stored and made available in two different geographical locations (DESY/Hamburg and MPP/Garching) using dcache technology  [50] , and can be used through respective generic computing facilities, as detailed above for the DESY case.
Optionally, they can also be accessed through the ZEUS Grid instance. Currently, the data are accessible only to ZEUS members or individual ZEUS associates, but plans exist to release at least part of the data as Open Data through the recently funded German national PUNCH4NFDI project  [51]. In summary, the ZEUS data preservation project, in line with the HERA data preservation project as a whole and the general DPHEP strategy, is successful. Large parts of this program have been implemented as foreseen with very limited dedicated resources; (more resources would have allowed and would still allow the success to be even bigger). More than 30% of the total HERA physics results were pro- 19 https://github. com/. duced in the 14 years since the end of data taking, and more results are coming.
MINERvA MINERvA  [59]  is a neutrino scattering experiment at Fermilab that recorded data between 2009-2019. The MINERvA collaboration has published more than thirty neutrino interaction cross section measurements  [60]  that will inform and tune interaction models for future oscillation experiments such as DUNE and HyperKamiokande. To ensure that its data is usable into the 2030s, the MINERvA collaboration began a data preservation project in 2019  [61]. The project has three components: -Preservation of the high-and low-level reconstructed objects in a ROOT-based analysis n-tuples corresponding to the entire MINERvA dataset. -A software library known as the MINERvA Analysis Toolkit (MAT) with utilities for transforming the ROOTbased tuples into cross section measurements. -A second software library (MAT-MINERvA) based on MAT that can be used to reproduce existing MINERvA analyses and form the basis for new analyses.
As of 2022, the two software libraries have been developed and are available on GitHub  [62, 63] , and the datasets are available on Fermilab-hosted DCache servers. 4. 9 BES III BESIII  [64]  at the BEPCII accelerator is a major upgrade of BESII at the BEPC for the studies of hadron physics and τ -charm physics with the highest accuracy achieved until now. The peak luminosity of the double-ring e + e -collider, BEPCII, is 10 33 cm -2 s -1 at center-of-mass energy 3. 78 GeV. BESIII is an unique experiment currently operating in taucharm energy zone in the world. It started to collect data in May 2009, and its end of data acquisition will be extended to 2030. The data preservation of BESIII will follow LEVEL 4 of DPHEP. These data is expected to be preserved for another 5-10 years after the end of data acquisition.
More than half of ATLAS analyses currently include HEPData records. ATLAS also routinely releases collaboration-validated software implementations approximating the actual event selection as Rivet routines  [66]  and SimpleAnalysis classes  [67]. The current Rivet analysis coverage of ATLAS is 35% [68] (the highest of any tracked experiment). For SimpleAnalysis, ATLAS provides the SimpleAnalysis framework which is released as Open Source software  [69] , and more than 35 search analyses have been included already. As a key data product for re-use, ATLAS also releases the statistical models underlying the search or measurement (see e. g. Refs. [70, 71] ). This captures all systematic effects and allows external researchers to reproduce the statistical procedure at full fidelity or re-use ATLAS results in follow-up studies such as global statistical inference.
The focus of public Level 1 products is the use of long-term, stable human-and machine-readable formats. The Level 1 records are assigned stable identifiers (DOIs) and can be individually cited by third-party researchers. Augmenting the public Level 1 data, ATLAS preserves detailed internal publication-specific information through two complementary approaches. A comprehensive metadata record in its internal GLANCE  [72]  database captures the full analysis life cycle from inception to final journal publication and collects diverse metadata such as analysis team membership, presentation, approval reviews, code repositories, physics keywords, etc. In addition, a growing number of analyses are preserved as a software product for future reuse and reproducibility. Here, analyses are described using a declarative computational workflow language, using parameterized task templates and Linux Container Images.
The latter are able to capture not only the analysis software but its full set of dependencies (OS, language runtimes, compilers, etc). A particular use-case is the reinterpretation of analyses in terms of alternative signal models using RECAST  [73, 74]. Here a tight integration with the REANA platform developed by CERN IT  [75]  was developed. ATLAS does not host Level 1 data directly but relies on the continued development and maintenance of the archival cyberinfrastructure, most crucially HEPData, to provide this service to the community. At Level 2, ATLAS compiles special-purpose data sets targeted for outreach and education beyond the HEP community and releases them publicly on the CERN Open Data Portal (see e. g. Refs. [76, 77] ). The data sets are deliberately simplified in both content and format to facilitate broad usage in non-academic settings, minimizing technical boundaries.
As a consequence these data are not suitable for research usage. Alongside the data sets, a range of software examples such as interactive notebooks, data analysis macros, or visualization code are prepared and maintained (see e. g. Ref. [78] ). Reconstructed event-wise data sets beyond the scope of a single analysis are categorized as DPHEP Level 3. These represent the main data used by ATLAS members wishing to prepare a physics result and are available internally in a range of data formats. The data are produced from raw data sets using the ATLAS reconstruction software Athena, which is made available as Open Source and whose development is publicly accessible  [79]. In addition, Athena is citable through a corresponding Zenodo record  [80]. A given release of the software is packaged and preserved independently from the source repository using industry standard formats for global distribution.
In some cases, container images of the software are available  [81]. Both observed collision events and simulated events are processed through tightly controlled versions and configurations of the software, and for any given data set, detailed metadata and provenance information is stored in the ATLAS Metadata Interface (AMI). Based on these metadata, reproduction of derived data sets is straightforward if necessary. To ensure a coherent data set of all data produced during a data-taking period, the reconstruction software only undergoes performance improvements, while major changes and algorithmic improvements are incorporated during largescale reprocessing campaigns. Such reprocessing of old data with new software is a key step towards internal data preservation and allows ATLAS to maintain consistent multi-run data sets that may be processed by a single software release.
The provenance information records the exact parameters and software conditions used at the time of data-taking, for collision data, and in the event generation and simulation step for Monte Carlo data. The input parameters are recorded also for the subsequent processing steps. This information together with the open-source CMSSW software makes it possible to trace back all data handling, although doing so is certainly a tedious task. A virtualised computing environment, compatible with the software version with which the original data can be analysed, is provided and maintained. The CMS provides docker software containers  [91, 92] , regularly tested and updated, and Virtual Machine (VM) images, based on the CernVM software appliance  [93]. They are part of the data release, as well as any additional information necessary to perform a research-level physics analysis.
g. CERN GitLab 20 ) which are currently being set up. They will be made available and searchable to collaborators through CERN analysis preservation services  [87]. The open data releases can include selected analysis workflows, at an example level. The reproducibility 20 https://about. gitlab. com/. of the preserved workflows can be tested in systems like REANA  [75]. CMS open data are widely in use. The number of scientific publications by open data users external to the CMS collaboration is comparable to that of an internal working group in the collaboration, thus broadening the scientific value of these data. The usage can be monitored through citations to the data records that all have a digital object identifier. However, the counting mechanism for repositories such as Inspirehep is still imprecise.
LHCb The LHCb collaboration is committed to the community efforts to make LHC data available to the public. Its collaboration board has ratified the CERN Open Data policy  [9]  in 2020. Data releases are made available through the CERN Open Data portal  [90]  and trusted repositories, such as HEPData  [65]  or RIVET  [95] , and comprise data on three levels of complexity. Publications are open access and routinely accompanied with supporting materials and releases of machine readable results to HEPData, where appropriate. All plots and tables of LHCb papers are accessible in machine readable form through the LHCb published results pages. 21 For educational purposes and outreach selected data sets with various degrees of information depth are published on the open data portal. These datasets are usually intended for a specific application, such as the International Masterclasses, and contain heavily filtered information necessary for the respective purpose.
The software to analyse these data is available as open source  [96]  and documented through the LHCb starterkit. However, only limited support can be provided by LHCb to potential external users. The development of the open data curation tools, documentation and metadata schemes is organised within the LHCb Data Processing and Analysis (DPA) project. All related efforts invested by collaboration members count as service work. To improve the sustainability of these efforts two issues with the currently planned open data release have to be addressed. First, the data volumes scheduled for release by the LHCb collaboration are quite significant and would amount to higher storage requests on the open data portal than those of ATLAS and CMS combined. Second, despite the best documentation efforts, the access to the data provided by current tools, while appropriately flexible for experts, lacks an intuitive interface, that would facilitate the exploration of the data by external users.
In order to capture the user analysis parts, which typically deal with interpretation of the data, rather than data preparation, current efforts focus on the Snakemake workflow engine, which has become quite popular within LHCb and is now also supported by REANA. A future vision would be to enable the majority of LHCb analyses to preserve at least the workflow producing the central value of the analysis for deployment to REANA. For the capture of meta-information generated during the internal discussions and review a new database system, called Analysis Lifecycle Management (ALCM) is under construction in LHCb, which will eventually replace the current public results pages and offer the possibility to export public extracts of the information to systems like CAP.
ALICE Recently, ALICE together with the other CERN's Large Hadron Collider (LHC) collaborations has adopted a new policy  [9]  supporting a consistent approach towards the openness and preservation of experimental data. Essentially, all LHC experiments share a common strategy for the release of Published Results (Level 1), Outreach and Education Data (Level 2), and Reconstructed Data (Level 3), while Raw Data (Level 4) are not considered usable in a meaningful way outside the collaborations. In compliance with the CERN Open Access Policy, all ALICE publications are available with Open Access and data points together with additional information including the analysis code are made public at the time of publication using the HEPData portal  [65]  and Rivet toolkit  [95]. In addition, dedicated subsets of ALICE data for the purposes of education and outreach have been released in the TTree ROOT data format and are accessible through the CERN Open Data Portal (CODP)  [90]  together with the needed software to analyze them via virtual machine, container technology, and web-applications.