dataesr/scientific-paragraphs-categorization huggingface dataset:
Numerical experiments In complement to our theoretical analysis, we developed a python package that implements Algorithm 2 and gives the choice of using the variant of Subroutine 3 or of Subroutine 4 to do the "update X()" function. This package relies on three python libraries: scipy and numpy for matrix operations, and numba to compile the python code. To facilitate its usage, this package can be installed by using pip install markovianbandit-pkg. All experiments were conducted on a laptop (Macbook Pro 2020) with an Intel Core i9 CPU at 2. 3 GHz with 16GB of Memory using Python 3. 6. 9 :: Anaconda custom (64-bit) under macOS Big Sur version 11. 6. 2. The version of the packages are scipy version 1. 5. 4, numpy version 1. 19. 5 and numba version 0. 53. 1. The code of all experiments is available at https://gitlab.
As expected, not doing the indexability test does improve the performance compared to doing the indexability test (here by a factor approximately 1/3 for Subroutine 3 and 1/5 for Subroutine 4). More importantly, this table shows that the time when using the subcubic variant, Subroutine 4, diminishes the computation time by about 40% to 50% compared to when using Subroutine 3. Note that for n = 100, using the Subroutine 3 is slightly faster than using the Subroutine 4 (while both takes only a few milliseconds). This indicates that the subcubic algorithm becomes interesting when n is large enough (say n ≥ 2000). To give a visual idea of how the various variants of the algorithms compare, we plot in Figure  6a  the runtime of the four variants along with two variants of the algorithms of  [29] : FPA-Matlab (the original matlab implementation), and FPA-Julia: a Julia's implementation of the algorithm provided by the authors.
We choose to compare to this algorithm as it was the one with the smallest complexity up to now. The numbers for FPA-Matlab are the ones reported in  [29]  and they are comparable to the ones that we obtained on our machine with the same algorithm. FPA-Julia is significantly faster. Hence, we plot in Figure  6b  the runtime of each variant divided by the runtime of FPA-Julia. For large n, our best implementation is about 4 to 6 times faster than the best one of  [29]. For instance, for n = 15000, our implementation takes about 7 minutes to compute the index (or 9 minutes when checking indexability) whereas FPA-Julia takes 33 minutes (and does not check indexability on the fly). In our implementation, not testing indexability reduces the computation time of 15 -20% for Subroutine 4 or 25 -30% for Subroutine 3 compared to the version that tests indexability.
It should be clear that the comparison of the computation times of our implementation versus the ones of FPA-Matlab or FPA-Julia has its limits, because we do not use the same programming language. The influence of the choice of programming language is clear when comparing FPA-Matlab and FPA-Julia: while both codes are similar, the compiled Julia code is about 5 times faster. To obtain a fairer comparison with our algorithm, we tried to Whittle Index rewrite the algorithm of  [29]  in Python with Numba but our implementation was significantly slower than the one of FPA-Julia. We do not know if this is by lack of optimization of our code or because Julia is indeed faster. Regardless of the choice programming langages, our implementation has nevertheless a few advantages that can explain why it is faster: our algorithm has some technical advantage (a simpler internal loop, and the use of Subroutine 4), and our implementation is optimized to improve data access pattern (which consists mostly in sorting on the fly the array according to the permutation σ, see Appendix C).
DIVERSITY OF PARTNERS AND SERVICES As an international project, ISIDORe's DMP needs to align with different federal and European requirements. While some of the RIs participating in ISIDORe had already collaborated in the frame of RI cluster projects, such as RI-VIS, CORBEL and EOSC-Life, the ISIDORe project represents a unique challenge, due to the diversity of participating RIs and networks, in terms of their respective sizes, expertise, legal structures and/or working practices. This required a new approach to coordination and collaboration among the different partners. It also made the elaboration of plans for data management, to favour the fast generation of FAIR policies for data sharing, reuse and interoperability, and open access to existing methods and software, such as online tools, workflows and registries, particularly challenging.
DIVERSITY OF PARTNER MATURITY In addition to complying with the various laws, ISIDORe's DMP must also be compatible with the consortium partners' diverse data handling procedures. While some, especially in Life Science (LS)-RI communities with extensive experience in data handling (e. g. structural biology, INSTRUCT; high-content screening, EU-OPENSCREEN; biological and biomedical imaging, Euro-BioImaging) have a solid set of internal procedures, implemented in close exchange with the relevant repositories (e. g. PDB, ECBD, BioImage Archive, EMPIAR), others still rely primarily on the diligence of individual researchers. The challenge is therefore to accommodate this high heterogeneity. It requires first determining precisely the current state-of-the-art in data management across the RIs and networks, to reconcile differences and avoid duplication of the efforts already in place.
The established DMP procedures, FAIR guidelines, data repositories and metadata will be collected through platforms and tools (such as those illustrated). A live representation of the standards and databases used within the DMP will be available as a FAIRsharing collection. While many LS-RI partners already have DMP workflows in place, the goal and challenge of the ISIDORe project will be to set up cross-disciplinary pipelines, onboarding additional communities and adapting DMP procedures to meet the needs of infectious diseases research (data and tools) for the different disciplines in the consortium. David et al. Data Science Journal DOI: 10. 5334/dsj-2023-035  To organise this homogenisation, ISIDORe MetaData mobilisation is based on an access management system (AMS). The main purpose of the AMS is to administer responses to calls, and requests for services, and to collect feedback from users in a standardised and centralised manner.
This AMS, currently in development, will allow the management of the catalogue of services, project submission information with its associated metadata, and create a community-validated registry of projects (both those submitted and accepted). For certain RIs, any data generated by the provision of a service to a user is channelled into existing registries and repositories. This will be indexed in the ISIDORe AMS and syndicated through metadata tools and registries (described in the Supplementary Material). Metadata mobilisation will be achieved through existing RI AMSs when they exist (Figure  1 ); associated syndication processes are described in the ISIDORe DMP. Since the project's contributors are numerous and diverse, the DMP should be sufficiently flexible to fit these broad needs, whilst retaining minimal common requirements in standards, protocols and rules for metadata, traceability and provenance of data.
This work on the ISIDORe DMP may introduce elements that could significantly impact the DMPs of individual project's members, making them more general and potentially more ambitious. FAIR-compliant DMPs covering service management and delivery have, up until now, generally been designed and implemented at the level of the RIs and networks. This type of DMP is distinct from the DMP covering the production of scientific data (Figure  2 ). For the latter, the onus to support users in DMP compliance has previously generally fallen on the individual service providers (e. g. national or regional core facilities). ISIDORe provides an opportunity to explore ways to harmonise user support. In this, as for all data challenges, ISIDORe is supported by BY-COVID. The BY-COVID project aims to make data and analysis tools related to infectious disease outbreaks more interoperable and accessible, not only to scientists but also to medical staff in hospitals, and policy makers, such as government officials, or indeed any other potential user.
The two projects are working together: BY-COVID provides support, tools, data discovery and dissemination strategies, to enable data from user projects implemented by ISIDORe to be as FAIR as possible, while ISIDORe promotes the adoption of FAIR (meta)data standards by partners, via its network of data stewards, and tries to ensure that data is made available in the public domain, as widely as possible, preferably via dedicated sites such as the COVID-19 Data Portal. As digital data needs to be actively managed over time to ensure continued availability and useability, depositing data resources with a trusted digital archive helps ensure that they are curated and handled according to good practices in digital preservation (ICPSR_DMP). Through the COVID-19 data portal, BY-COVID provides a platform for data discovery across a broad range of disciplines, with the flexibility to add any relevant data resource proposed by ISIDORe, through a lightweight metadata format jointly developed by both projects  (Hermjakob et al.
2022 ). The collaboration between the two projects will lay the groundwork for Europe's future pandemic preparedness with FAIR data pipelines, enabling rapid response to outbreak situations, and a broad framework for answering research questions across research disciplines and country borders. BY-COVID also provides standardised data management and analysis methods (partially based on the Galaxy Project), protocols, and training, to ensure FAIR is an integral part of the data pipeline. Building on existing infrastructure, data repositories are explorable through a single platform (COVID-19 data platform, and in the future, the EBI pathogen portal) allowing users to deposit, access and analyse pathogen data more easily. BY-COVID's data source catalogue of choice is FAIRsharing. It describes relationships among the data sources (databases, knowledge bases, repositories) and assists with the selection, documentation and visualisation of standards.
The use of RO-crate  (Eguinoa et al. 2022) , based on FAIR Digital Object (FDO) principles, guarantees efficient data sharing, including data and metadata compliance using community-adopted standards such as bioschemas and common workflow language. Workflow repositories such as WorkflowHub that include the use of persistent and unique identifiers (e. g. DOIs [cool URIs]) ensure findability and discoverability of computational workflows and pipelines. Data discovery is further enabled by the BY-COVID data discovery network, based on the principles of the Beacon Network and employing an Application Programming Interface (API) that allows, for instance, discovery of genomic, phenotypic and clinical data.
Indeed, in certain cases, these may have already paved the way for easy and fast submission of data. Projects that require the provision of services by two or more RIs, and whose datasets must therefore comply with the requirements and recommendations of those RIs, present a special challenge. Again, the introduction of a two-tier DMP will help to overcome this, although the participating RIs will still need to reach a consensus on the recommendations to be given to users regarding their specific data constraints. One can imagine that this will also be facilitated by the variability of knowledge models within the DSW. Ideally, the RI or network (preferably via its data steward) should be automatically alerted of possible conflicting recommendations or overlapping issues, and thus ISIDORe will be able to provide users with access to coherent and curated catalogues of data, resources and repositories.
Additional developments are planned for ARIA to allow for the output of an executed project, such as publications and datasets, to also be findable via the AMS. This would be a powerful tool to allow data traceability. EVA-GLOBAL's AMS is currently based on the Drupal opensource content management system (CMS) and also implements GDPR compliance. Similar to ARIA, the EVA portal manages many aspects of RI operations and service provision. Currently used by the EVA and Infravec networks, it has the advantage of being user-friendly and has demonstrated great adaptability in the context of rapid responses to epidemic and pandemic situations (including the Zika epidemic, COVID-19 pandemic and most recently, the 2022 Mpox outbreak). This flexibility would be a great asset, especially considering ISIDORe's expected role in rapid research response.
Introduction The availability of large dataset along with data transformation and analysis tools, has revolutionized how bioinformaticians conduct computational experiments  [1]. These data analyses are increasingly performed through pipelines implemented using scripting languages or notebooks  [2]. These pipelines link and intertwine data transformation and analysis tools to convert raw input data into results that allow scientists to gain insights and draw conclusions about the validity of a hypothesis or known fact. The above approach, mainly involving custom scripts, has been extensively used for small-scale experiments involving a small number of datasets and a small number of analysis and transformation tools  [3]. For instance, in the case a single user (bioinformatician) is responsible for the design and the execution of the pipeline as well as the analysis of its results, she or he has prior knowledge of the datasets involved and the analysis tools being used.
It is also fair to assume that the pipeline is being run on a single local and homogeneous computing environment. However, with the above approach, developing large and complex pipelines handling massive amounts of data becomes challenging  [4, 5] , especially when teams with different expertise and operating in a distributed (and potentially heterogeneous) computing execution environment are involved. In addition, it does not facilitate pipeline sharing and reuse  [3]. A moderately sized pipeline can quickly become difficult to understand and maintain, and even more difficult to reuse by third parties. This has become a major concern as bioinformaticians (and scientists in general) are expected to share their resources, including datasets, analysis tools, and also pipelines, in a form that can be readily understood and reused by peers.
Difficulties in the development of bioinformatics pipelines This section outlines why managing large-scale pipelines throughout all stages of their life cycle, from development to sharing and reuse (see Fig. 1 ) is a difficult task. To illustrate such difficulties, we use a few real examples encountered during the large scale biomedical project "understanding the pathophysiology of IntraCranial ANeurysm" (ICAN). ICAN aims to better understand and predict the development and rupture of intracranial aneurysms  [11, 12]. For this study, biologists and physicians have set up a large collection of biological samples from a population of 3000 individuals. This bio-collection has been used to generate and analyse various types of data, including genomic sequencing data, cerebral vascular organ neuroimaging data, and clinical data (like family history and lifestyle).
Tracing data for testing and reuse Once a first version of the pipeline has been developed, the testing phase may start and needs dedicated datasets. The "Data layer" defined in Fig. 2  represents the datasets consumed and produced during an execution of a given bioinformatics pipeline. Representing and tracing such data provenance in a uniform way is key, both for testing and documenting pipelines which will allow reuse. Elements of solutions include the W3C PROV recommendation 2  that offers a highly generic model for exchanging provenance data on the Web. As PROV does not explicitly provide all the concepts necessary for modeling pipelines or their executions, several PROVcompliant extension languages co-exist: wfprov, 3  prov-wf  [19]  and ProvONE. 4  recent initiative to annotate scientific datasets with lightweight provenance metadata has been introduced, namely, RO-crate.
Currently active repositories include the Galaxy repository  [36]  and WorkflowHub  [37]  (the successor of myExperiment). Other initiatives propose databases of curated workflows, such as nf-core  [31]  (42 workflows) and sequana  [38]  (11 workflows) for Nextflow and Snakemake respectively. However, GitHub remains the most important source of workflows, with several thousands of bioinformatics workflows available to users. We shall get back to this point in the next section.
Workflow systems and bioinformatics pipelines reuse In this section, we present the results of a comprehensive study we conducted on the reuse of scientific workflows. Our aim is to show how these workflow systems are being used in practice by the bioinformatics community and gain a better understanding of reuse practices. While previous studies (e. g. ,  [39] ) focused on Taverna  [10]  and Galaxy  [8]  workflow systems, our study is the first to consider the increasingly popular Snakemake  [6]  and Nextflow  [30]  workflow systems. To do so, we studied the reuse of Nextflow and Snakemake workflows available on GitHub. After presenting how we have collected workflows, we will introduce our results through three main points. First, workflow reuse through the perspective of repository owners and contributors is studied.
Collection of workflows Figure  5  illustrates our approach for extracting and collecting information about the workflows used in our study. We implemented and executed the three software components (CrawlWF, ParsWF, and WF2BT) to search, extract, parse and annotate Snakemake and Nextflow workflows from GitHub. After workflow collection and extraction from GitHub, CrawlWF harvests GitHub metadata to associate each workflow with its owner (who owns the repository) and contributors (who contributed to the repository). ParsWF then examines the content of the repository, and retains only the well-formed workflow files by syntactically checking Nextflow and Snakemake specifications. It then extracts information about the workflow steps (workflow processors). Finally, WF2BT annotates workflow steps with metadata retrieved from the bio.
We are thus not in a situation where workflows are systematically 13 Code available at https://github. com/r-blanchet/BAM_QC uploaded by a restricted number of owners. Interestingly, 31% (Snakemake) vs. 42% (Nextflow) authors have published at least two workflows. There exists a set of owners who provide several workflows to the community, and overall, workflow authors in general produce several workflows and are willing to share their workflows. While these results have been obtained on workflow owners, similar trends are observed on workflow contributors (in Nextflow, 37% of contributors have contributed to two workflows or more, and this figure reaches 44% for Snakemake).
Identifying reused tools We now focus on identifying the kind of operations users implemented in their workflows using the bioinformatics tools used as a proxy. In order to identify reuse of tools, we count the number of times each tool is used considering all workflow processors. When studying the tools used in processors, we found that some of them were widely used, as much as 2841 times, indicating that many users performed very similar tasks. When looking at the list of the most used tools (top tools) for Nextflow and Snakemake, we notice that 9 out of 10 top tools are common between Nextflow and Snakemake. In the following, we will focus on the 14 tools which are common between the top 20 tools of Nextflow and Snakemake. Such tools are presented in Table  2. While some tools are used for generic bioinformatics data processing (e.
g. Samtools), a number of them are domain-specific tools performing specific bioinformatic tasks and reflecting trends of tool usage in bioinformatics workflows (e. g. QIIME for microbiome analysis). Not surprisingly, toolkits, mappers and quality control tools are widely used, as they are involved in many workflows (even several times in some workflows) for different types of analysis: BWA and FastQC are used prior to most sequence data analyses, and Samtools is used for multiple tasks (e. g. , converting between mapping formats, filtering reads) involved in different kinds of analyses (e. g. , RNA-Seq, SNP calling). Generally, we find that (i) a few tools are widely re-used, and (ii) widely used tools depend little on the workflow system used, as the top used tools are almost the same in Nextflow and Snakemake.
| INTRODUC TI ON Across scientific disciplines, researchers increasingly rely on code written in open-source software, such as R and Python, to clean, manipulate, visualize, and analyse data  (Lai et al. , 2019; Mislan et al. , 2016; Peikert & Brandmaier, 2021; Peikert et al. , 2021). Such software allows for increased transparency and reproducibility compared with software that operates through point-and-click interfaces ('User Interface' or 'UI-based'), such as Minitab and SPSS  (Obels et al. , 2020). One of the key benefits of this code-based software is flexibility, because researchers can tailor analyses to their specific research needs which would otherwise be unavailable. However, the flexibility of code comes at a cost, as it means that it can be more error-prone  (Budd et al. , 1998).
These errors may be conceptual (e. g. , implementing the wrong function for a given task), programmatic (e. g. , indexing the wrong column of a data frame), or syntactic (e. g. , the incorrect spelling of a statement or function). Although UI-based software is also prone to conceptual errors, programmatic and syntactic errors are more common in code-based software. These errors can contribute to a lack of reproducibility or to the propagation of incorrect results (see  Obels et al. , 2020  for a review of code and data in psychology). Indeed, several high-profile retractions have centred on these types of mistakes  (Bolnick & Paull, 2009; Huijgen et al. , 2012; Ma & Chang, 2007; Miller, 2006; Williams & Bürkner, 2020). One way to minimize potential errors, besides carefully annotating code and following best coding practices, is to undergo a process of code review.
However, unlike in some disciplines (such as in computer science and software development) where code review is routinely implemented  (Badampudi et al. , 2019; Nelson & Schumann, 2004) , it is noticeably absent from the research and publication processes in other academic disciplines that rely on code to make inferences and predictions  (Indriasari et al. , 2020) , including ecology and evolutionary biology. To address this, we advocate for a fundamental shift in research culture that brings code review into all stages of the research process, as reviewing of code is necessary to facilitate error correction and to confirm the reproducibility and reliability of reported results. This is particularly important as analyses are becoming ever more complicated, especially in the fields of ecology and evolutionary biology  (Touchon & McCoy, 2016).
But how can we implement code review? By whom, when, and how can it take place? In this paper, we provide some suggestions about how to conduct a code review and how to produce code that facilitates this form of review. Finally, we discuss the application of code review throughout the entire process of publication, from the early stages of pre-publishing right through to after work is published. Although we focus mainly on issues and techniques related to the R and Python coding languages due to their popularity in the fields of ecology and evolutionary biology  (Lai et al. , 2019; Mislan et al. , 2016) , the concepts and principles we discuss are widely applicable.