dataesr/scientific-paragraphs-categorization huggingface dataset:
On the contrary, the editorial team may rightly consider that the benefit for the journal of publishing an impactful paper will be larger than the cost of not fully enforcing the data policy. A second possibility is for the journal to appoint a dedicated editor in charge of implementing the verification policy, thus avoiding the conflict of objectives that arises when the same editor is in charge of selecting impactful papers and verifying reproducibility. 25 For instance, in 2018 the AEA appointed Lars Vilhuber from Cornell University as the data editor for all the journals operated by the Association. The AEA data editor oversees the Replication Lab at Cornell University, which checks the reproducibility of the results of all conditionally accepted papers. Since then, similar positions have been created at Review of Economics Studies, Economic Journal, and Management Science.
The Case of Non-Shareable Data The use of non-shareable data is often mentioned as a major impediment to the implementation of reproducible research  (Christensen and Miguel, 2018). To summarize the situation, we provide in Figure  3  a flowchart explaining which types of datasets can be shared. In particular, some data directly collected by the researchers and most data obtained from third parties (e. g. , confidential administrative data, proprietary data obtained from companies) cannot be put in the public domain without breaching contracts or violating the law. Without a solution to handle papers using non-shareable data, only two outcomes are possible. A first outcome is exclusion: the journal publishes only papers based on non-confidential data, and may have to pass on many interesting papers.
For instance, the DCAP of the review PLOS states that whenever the data cannot be accessed by other researchers, the manuscript must include an additional analysis based on public data that validates the A second outcome is exemption: papers using confidential data are exempted from the DCAP. Nowadays, exemption is the most common policy among economics journals. Among the 49 DCAP considered by  Vlaeminck and Herrmann (2015) , 34 offer exemptions to the policy for confidential datasets. Christensen and Miguel (2018)  show that the share of empirical papers published in the AER that fall under these exemptions rose sharply from 10% to around 40% between 2005 and 2015 (also see  Vilhuber (2020) ). These numbers show that exclusion is not a realistic possibility in economics. Confidential data on consumers or firms allow researchers to address new research questions or provide innovative answers to traditional ones.
To verify the reproducibility of all these articles, each of these 91 different journals would have had to go through a lengthy accreditation process to access the same original data. 27 A recent example is the EU General Data Protection Regulation enforced in 2018. Unlike the European Union, the US does not have a single law on data protection but instead a system of federal and state laws and regulations, including among others the Federal Trade Commission Act, the Financial Services Modernization Act, the Health Insurance Portability and Accountability Act, the Electronic Communications Privacy Act, etc.
A Quantification of Verification Costs An obvious way of favoring the move towards verified reproducibility is to decrease its cost, represented by C J (q, n A ) in our framework. In this section, we summarize some quantitative information regarding this cost and discuss the implications for how to best organize the verification of reproducibility. We consider a given level of reproducibility q, corresponding to Stage 3 on the reproducibility scale displayed in Figure  1. We take as given the number n A of articles to verify, and the number n D of distinct non-shareable databases to access. The total cost can be represented as: C(n A , n D ) = c F + n A (c L + c C ) + n D c D (15) We briefly discuss and give a tentative estimate of each cost. Our estimates are based on the actual experience of the cascad verification agency.
By looking at the actual time spent by the reproducibility reviewers in 2020, we set the average number of hours per verification to 10 hours. We now have estimates for all the parameters of the function C(n A , n D ). Given the large share of data costs in the total reproducibility costs, a critical issue is how many new databases become necessary as the number of papers to be verified grows. The marginal cost of a new paper using a dataset that is not currently available to reviewers is much higher than the cost of a paper using already available datasets. Furthermore, bringing a new data source enriches the data portfolio of the reviewing team, which makes less likely the need to access an additional data source for the next papers to be verified (as the pool of available data sources is now larger).
To estimate how the number of necessary datasets grows with the number of papers, we use the following model. Assume there is a maximum number n D of non-shareable datasets that exist at a given time and can be used in economic research. A fraction 1 -θ of papers use shareable data or no data, and a fraction θ pick one dataset at random among the n D of non-shareable datasets. We show in the Appendix that the expected number of distinct non-shareable datasets that will be used by n A papers is equal to: n * D (n A , n D ) = n D × 1 - n D -1 n D θn A. (16) In our quantification exercise below, we use θ = 0. 4 (estimate of  Christensen and Miguel (2018)  for the American Economic Review ) and tentatively set n D = 50. We end up with the following cost function: C J (q, n A ) = C(n A , n * D (n A , n D )) = c F + n A (c L + c C ) + c D n * D (n A , n D ).
Heckman and Moktan (2020)ata Sharing: Evidence from the Journal of Financial Economics. Panel A displays the summary statistics of the four explanatory variables. Panel B displays the estimated coefficients and the associated p-values of the logistic regression (8), estimated both with and without year fixed effects (FE). The explained variable Open is a binary variable taking the value of one for any paper with code and/or data available, and zero otherwise. Top10 takes the value of one if at least one coauthor is affiliated with a top-10 university (seeHeckman and Moktan (2020)) and International equals one if at least one coauthor is affiliated with a university outside North-America. Seniority is equal to the time between the year of a given JFE publication and the year when the author got his or her first citation in the Web of Science database.
28Given the salaries actually paid by cascad in 2020, we use an hourly rate of 15 EUR. Hence c L is approximately equal to 150 EUR on average. -Computing costs (c C ) need to be paid when the code is run on a commercial cloud. Our estimate of C C is 11 EUR per article. We estimate the average computing cost by multiplying the average computing time reported in the large-scale reproducibility study conducted by cascad on Menkveld et al. and conservative estimate. (2021) by the actual cost per hour of the same machine. The average com- puting time per paper is 660. 53 minutes, or approximately 11 hours, and the cost per hour for a 8vCPU, 64 GB RAM, 1388 GB Temporary storage virtual machine is 1. 112 USD, or approximately 1 EUR. 29 -Costs of accessing data (c D ) vary a lot across databases. Many commer- cial databases are already available "for free" via a campus license.
Acknowledgements Co-authors of this paper are U. S. Government employees, therefore: Any use of trade, product, or firm names in this publication is for descriptive purposes only and does not imply endorsement by the  U. S. Government. We thank  Nadav Gazit  for comments, technical edits, and graphic design,  Don Rosenberry  and  Collin Lawrence  for  U. S. Geological Survey technical reviews , and  Amanda Sigouin  for editing support. We appreciate the helpful comments from two outside reviewers and the thoughtful guidance from the editors.
In spite of that, these breeds are still largely supported by special policy mechanisms in order to ensure their preservation  [2]. This is one of the critical points for the future because most of the local breeds are presently not managed in a secure way and depend upon financial support from the governments for preservation programmes. The best conservation strategy is the one that makes the breed self-sustaining without the use of external subsidies  [3]. Theoretically, the self-sustainable condition of a local pig breed should be reached by the exploitation (sale) of products characterized by an extra added value, which in return assures breeding of a sufficient number of animals to have an adequate genetic diversity  [4]. Nevertheless, this condition is seldom attained in the local pig breeds, and the intervention of public bodies is often considered essential for preservation  [5].
The procedure of performing a reliability analysis by using the SPCE/GSA is given in Table  1. Post-processing on the MCS results to obtain failure probability and PDF of system response etc. It can be seen from Table  1  that the method SPEC/GSA mainly consists of 5 steps. The two first steps aim at reducing the input dimension by selecting the significant variables. Once the dimension is reduced, a higher-order SPCE can be created and an MCS can be performed then. Concerning the construction of SPCE meta-models, one can use Uncertainty Quantification toolboxes such as the UQlab  [16]  based on Matlab or the OpenTURNS  [17]  based on Python etc. There are a variety of reliability methods in the mentioned toolbox and one can easily obtain a meta-model (SPCE, kriging or Support vector machine etc.
The deterministic models Two deterministic models are established in this study for computing the dam factor of safety (FoS) under steady state flow conditions. The first one, based on the strength reduction method, is created within the finite difference program Flac2D  [27]  while the second one is constructed in Matlab using the limit equilibrium theory. This section firstly presents the studied academic embankment dam in terms of the geometry and soil properties. It is followed by a description of the numerical model. Then, the analytical model is presented.
The numerical model A 1. Initial equilibrium calculation. Once the mesh is realized and the boundary conditions fixed, a mechanical calculation of the initial stress state is performed under the loading of the soil's own weight. 2. Flow only calculation. A hydrostatic head is applied in the upstream and the pore pressure distribution is obtained under steady state flow condition. The obtained pore pressure distribution will also be used for the analytical model. 3. Assignment of the random fields to the mesh. Two random fields ( † # ‡‚ |) are generated within the Matlab code using the Karhunen-Loève expansion method. They are then assigned to each element of the mesh according to the coordinates. Figure  3  gives an example of two negatively correlated random fields ( † # ‡‚ |) mapped to the mesh.
The analytical model An analytical model based on the limit equilibrium theory is created using the Matlab software. This model combines the Morgenstern Price method  [31]  and the pore water pressure distribution determined by the numerical model for estimating the FoS of a given slip surface. Particularly, the algorithm proposed by  [32]  is adopted in this paper to perform the Morgenstern Price method. The slip shape is assumed to be circular and the critical slip surface is located by mapping: the entry point and the exit point of the slip surface (A and B in Figure  4 ) and the deepest passage of the slip arc (D in Figure  4 ). shows the high efficiency of the analytical model compared to the numerical analysis. Reduction of computational time for deterministic models is very significant for a reliability analysis since it usually needs a large number of calls to deterministic calculations.
Each of which can be run in parallel. We show how to bundle a number of tools used in evolutionary biology by using Debian, GNU Guix, and Bioconda software distributions, along with the use of container systems, such as Docker, GNU Guix, and Singularity. Together these distributions represent the overall majority of software packages relevant for biology, including PAML, Muscle, MAFFT, MrBayes, and BLAST. By bundling software in lightweight containers, they can be deployed on a desktop, in the cloud, and, increasingly, on compute clusters. By bundling software through these public software distributions, and by creating reproducible and shareable pipelines using these workflow engines, not only do bioinformaticians have to spend less time reinventing the wheel but also do we get closer to the ideal of making science reproducible.
Overview In this chapter, we show how to create a bioinformatics pipeline using four workflow systems: CWL, GWL, Snakemake, and Nextflow. We show how to put them together, so you can adapt it for your own purposes while discussing in the process the different approaches. All scripts and source code can be found on GitHub. The online material allows a direct comparison of how such workflows are assembled with their syntax. Due to large volumes, the analysis and integration of data generated by high-throughput technologies have become computationally intensive, and analysis can no longer happen on a typical desktop computer. Researchers therefore are faced with the need to scale analyses efficiently by using high-performance compute clusters or cloud platforms. At the same time, they have to make sure that these analyses run in a reproducible manner.
One example of legacy software requiring lengthy computation is Ziheng Yang's CodeML implementation of PAML  [3]. PAML finds amino acid sites that show evidence of positive selection using d N /d S ratios, i. e. , the ratio of nonsynonymous and synonymous substitution rate. For further discussion see also Chapter. 12. Executing PAML over an alignment of 100 sequences may take hours, sometimes days, even on a fast computer. PAML (version 4. x) is designed as a single-threaded process and can only exploit one central processing unit (CPU) to complete a calculation. To test hundreds of alignments, e. g. , different gene families, PAML is invoked hundreds of times in a serial fashion, possibly taking days on a single computer. Here, we use PAML as an example, but the idea holds for any software program that is CPU bound, i.
As an alternative, Arvados is an open-source product specifically aimed at bioinformatics applications that makes the cloud behave as if it is a local cluster of computers, e. g. ,  [8]. At an even higher level, MapReduce is a framework for distributed processing of huge datasets, and it is well suited for problems using large number of computers  [9]. The map step takes a dataset and splits it into parts and distributes them to worker nodes. Worker nodes can further split and distribute data. At the reduce step, data is combined into a result, i. e. , it is an evolved scatter and gather approach. An API is provided that allows programmers to access functionality. The Apache Hadoop project includes a MapReduce implementation and a distributed file system  [10]  that can be used with multiple cloud providers and also on private computer clusters.
Another similar example is the Apache Spark project based on resilient distributed datasets (RDD)-a fault-tolerant collection of elements that can be accessed and operated on in parallel. The advantage of such higher-level systems is that they go well beyond hardware virtualization: not only the hardware infrastructure but also the operating system, the job scheduler, and resource orchestration are abstracted away. This simplifies data processing, parallelization, and the deployment of virtual servers and/or containers. The downside is that users have less control over the full software stack and often needs to program and interact with an application programmers interface (API). Overall, in the last decade, both commercial and noncommercial software providers have made cloud computing possible.
Parallelization of Applications Using a Workflow In case of embarrassingly parallel applications, programs are run independently as separate processes which do not communicate with each other. This is also a scatter and gather approach, i. e. , inputs split into several jobs are fed into each process by the user. Job outputs are collected and collated. In bioinformatics, such tasks are often combined into computational pipelines. With the PAML example, each single job can be based on one alignment, potentially giving linear speed improvements by distributing jobs across multiple CPUs and computers. In other words, the PAML software, by itself, does not allow calculations in parallel, but it is possible to parallelize multiple runs of PAML by splitting the dataset. The downside of this approach is the deployment and configuration of pipeline software, as well as the management and complexity of splitting inputs and the collecting and collating of outputs.
Nextflow is routinely used in organizations and institutes, such as the Roche Sequencing, the Wellcome Trust Sanger Institute, and the Center for Genomic Regulation (CRG) Nextflow workshop. Forth there is Snakemake, another widely used workflow manager system, written in Python and inspired by GNU Make. It allows for the composition of workflows based on a graph of rules whose execution is triggered by the presence, absence, or modification of expected files and directories. It is interesting to note that all these workflow languages and systems originated in bioinformatics. It suggests that in this rapidly growing field, the increasing computational needs and moreover the diverse demands made more formal solutions a necessity. It also suggests that existing workflow engines used in astronomy and physics, for example, have different requirements.
MrBayes, for example, a program for calculating phylogenetic trees based on Bayesian analysis, comes with MPI support. MPI is a message-based abstraction of parallelization, in the form of a binary communication protocol implemented in a C programming library  [12]. In some cases the parallelized version is slower than the single CPU version. For example, the MPI version calculates each Markov chain in parallel, and the chains need to be synchronized with each other, in a "scatter and gather" pattern. The chains spend time waiting for each other in addition to the communication overheads introduced by MPI itself. Later MrBayes adopted a hybrid use of coarsegrained OpenMPI and fine-grained use of pthreads or OpenMP leading to improved scalability, e. g. ,  [13]. Another example of communication overhead is with the statistical programming language R  [14] , which does not have native threading support built into the language.
Where MPI and Actors are related to a concept of message passing and mail boxes, goroutines are more closely related to Unix named pipes. Goroutines also aim to make reasoning about parallelization easier, by providing a pipe where data goes in and results come out, and this processing happens concurrently without use of mutexes, making it easier to reason about linear code. Goroutines are related to communicating sequential processes (CSP), the original paper by Tony Hoare in 1978  [22]. Meanwhile, recent practical implementations are driven by the ubiquity of cheap multicore computers and the need for scaling up. A Java implementation of CSP exists, named JCSP  [23] , and a Scala alternative named CSO  [24]. Go made goroutines intuitive and a central part of the strongly typed compiled language.
Erlang, Elixir, Haskell, Scala, Julia, Go and D are languages offering useful abstractions and tools for multi-core programming It is important to note that the problems, ideas, and concepts of parallel programming are not recent. They have been an important part of computer science theory for decennia. We invite the reader interested in parallel programming to read up on the languages that have solid built-in support high-level parallelization abstractions, in particular, Scala  [17] , Go  [21] , and D  [20].
Package Software in a Container Container technologies, such as Docker and Singularity, have gained popularity because they have less overhead than full virtual machines (VMs) and are smaller in size  [24]. Containers are fully supported by the major cloud computing providers and play an important role for portability across different platforms. Adoption of container solutions on HPC has been problematic, mostly because of security concerns. Singularity  [26]  offers a decentralized environment encapsulation that works in user space and that can be deployed in a simpler way since no root privileges are required to execute tools provided with Singularity. That is, Singularity containers can be created on a system with root privileges but run on a system without root privileges-though it requires some special kernel support.
Docker containers can be imported directly in Singularity, so when we present how to build Docker container images in the following sections, the reader should be aware that the same images can also be used with Singularity. Singularity is slowly being introduced in HPC setups  [27]. GNU Guix also has support for creating and running Linux containers. One interesting benefit is that, because the software packaging system is read-only and provides perfect isolation, containers automatically can share specific software running on the underlying system, making running containers even lighter and extremely fast. In this section we discuss three popular software distribution systems for Linux: Debian GNU/Linux (Debian), GNU Guix, and Conda can be used together on a single system allowing access to most bioinformatics software packages in use today.
Debian Med Debian (http://www. debian. org) is the oldest software distribution (started 1993) mentioned here with the largest body of software packages. Debian targets a wide range of architectures and includes a kernel plus a large body of other user software including graphical desktop environments, server software, and specialist software for scientific data processing. Overall Debian represents millions of users and targets most platforms in use today, even though it is not the only packaging system around (RPM being a notable alternative, for RedHat, Fedora, OpenSuSE, and CentOS). Debian Med is a project within Debian that packages software for medical practice and biomedical research. The goal of Debian Med is a complete open system for all tasks in medical care and research  [28]. With Debian Med over 400 precompiled bioinformatics software programs are available for Linux, as well as some 400 R packages.
Create a Docker Image with Debian Using the bio packages already present in Debian, it is straightforward to build a Docker container that includes all the necessary software to run the example workflows. Here is the code for creating the Docker image (see also  [30] ). We created a pre-built Docker image which is available on Docker Hub  [31]. Essentially, write a Docker script: And build and run the container: docker build -t scalability_debian -f Dockerfile. debian.
GNU Guix GNU Guix (https://www. gnu. org/software/guix/) is a package manager of the GNU project that can be installed on top of other Linux distributions and represents a rigorous approach toward dependency management  [32]. GNU Guix software packages are uniquely isolated by a hash value computed over all inputs, including the source package, the configuration, and all dependencies. This means that it is possible to have multiple versions of the same software and even different variants or combinations of software, e. g. , Apache web server with SSL and without SSL compiled on a single system. As of November 2017, GNU Guix provides over 6500 soft ware packages, including a wide range of dedicated scientific software for bioinformatics, statistics, and machine learning 2. 2. 1 Create a Docker Image with GNU Guix GNU Guix has native support for creating Docker images.