# Towards an open analysis ecosystem for Plasmodium genomic epidemiology

Shazia Ruybal-Pesántez, Jorge Amaya-Romero, Sophie Bérubé, Nicholas Brazeau, Mouhamadou Fadel Diop, Nicholas Hathaway, Jason Hendry, Kirsty Mccann, Kathryn Murie, Maxwell Murphy, et al.

► To cite this version:

Shazia Ruybal-Pesántez, Jorge Amaya-Romero, Sophie Bérubé, Nicholas Brazeau, Mouhamadou Fadel Diop, et al. Towards an open analysis ecosystem for Plasmodium genomic epidemiology. 2025. pasteur-05266791

HAL Id: pasteur-05266791

https://pasteur.hal.science/pasteur-05266791v1

Preprint submitted on 18 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
1 Towards an open analysis ecosystem for *Plasmodium* genomic epidemiology

3

4 Shazia Ruybal-Pesántez¹,², Jorge Amaya-Romero³, Sophie Bérubé⁴, Nicholas F.

5 Brazeau⁵, Mouhamadou Fadel Diop⁶, Nicholas Hathaway⁷, Jason Hendry⁸, Kirsty

6 McCann⁹, Kathryn Murie¹⁰, Maxwell Murphy¹⁰, Karamoko Niaré¹¹, Jody Phelan¹²,

7 Stephen F. Schaffner³,¹³, Alfred Simkin¹¹, Aimee R. Taylor¹⁴, Bryan Greenhouse¹⁰, Amy

8 Wesolowski¹⁵, Robert Verity¹

9

10 ¹ MRC Centre for Global Infectious Disease Analysis, School of Public Health, Imperial

11 College London, UK

12 ² Instituto de Microbiología, Universidad San Francisco de Quito, Ecuador

13 ³ Department of Immunology and Infectious Disease, Harvard T.H. Chan School of

14 Public Health, Harvard University, USA

15 ⁴ Department of Biostatistics, University of Florida, USA

16 ⁵ Physician-Scientist Training Program, Duke University Hospital, Durham USA

17 ⁶ Medical Research Council Unit The Gambia at London School of Hygiene and

18 Tropical Medicine, The Gambia

19 ⁷ Department of Medicine, University of Massachusetts Chan Medical School, USA

20 ⁸ Max Planck Institute for Infection Biology, Germany

21 ⁹ Centre for Innovation in Infectious Disease and Immunology Research (CIIDIR), The

22 Institute for Mental and Physical Health and Clinical Translation (IMPACT), School of

23 Medicine, Deakin University, Australia

24 ¹⁰ EPPIcenter program, Division of HIV, ID, and Global Medicine, University of California

25 San Francisco, USA

26 ¹¹ Department of Pathology and Laboratory Medicine, Brown University, USA

27 ¹² London School of Hygiene and Tropical Medicine, UK

28 ¹³ Infectious Disease and Microbiome Program, Broad Institute of Harvard and MIT,

29 USA

30 ¹⁴ Institut Pasteur, Université Paris Cité, France

31 ¹⁵ Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health,

32 USA

33

34
## 35 Abstract

Major advances in *Plasmodium* sequencing approaches, bioinformatic pipelines, and data analysis tools have provided valuable insights into malaria epidemiology from parasite genomic data. However, translating genetic data into actionable information for decision-makers remains a challenge. Significant barriers limit the integration of these advances into a functional data analysis ecosystem that produces standardized, interpretable results for use by national malaria control programs. The *Plasmodium* Genomic Epidemiology (PlasmoGenEpi) network convened 18 subject matter experts across 15 institutions at the Reproducibility, Accessibility, Documentation, and Interoperability Standards Hackathon in 2023 (RADISH23) to identify available analysis tools, evaluate software standards, improve documentation, and outline workflows. Eight use cases for genomic data were identified, and a subset were developed into analysis workflows in terms of a series of connected functionalities. Software tools were then mapped against functionalities to outline a modular approach to data analysis for these use cases. In addition to outlining workflows, a set of objective criteria were developed for evaluating software standards. Forty *Plasmodium* genomic analysis tools were identified, of which 22 were prioritized for software standards evaluation. Additional tutorials were developed for 10 tools in the form of reproducible code applied to shared datasets. These resources are available on **PGEforge** (mrc-ide.github.io/PGEforge), a new community resource that serves as a central, open repository for current and future resources for malaria genomic data analysis.

## 54 Introduction

### 55 The central role of data analysis in malaria genomics

In the two decades since sequencing the first *Plasmodium falciparum* genome (Gardner et al., 2002), the malaria research community has devised a wide array of methods aimed at generating insights into malaria epidemiology and transmission dynamics from parasite genetic data. Laboratories can now sequence whole genomes or obtain deep coverage of many genomic targets from small volumes of blood; advanced bioinformatic pipelines can detect minority alleles present in polyclonal infections while removing errors; an array of computational tools have been developed to distill critical insights from the data; and a variety of visualization methods have been created to communicate results to stakeholders. Collectively, these advancements provide the foundation for establishing a virtuous cycle linking public health personnel with laboratory and data scientists (Figure 1). This collaboration is essential for designing meaningful studies, generating and analyzing sequence data, and translating results into actionable information. An integrated approach will be critical when responding to new threats, including the emergence and spread of drug and diagnostic resistance, increased transmission due to insecticide resistance and invasive mosquito species, and importation due to human migration. However, several significant barriers currently hinder the integration of these efforts into a cohesive data generation and analysis ecosystem.
<figure><img src="image_2.png" /></figure>

Figure 1. Model for an ideal genomic epidemiology framework involving public health stakeholders and scientists. This framework outlines the iterative cycle of malaria genomic epidemiology, integrating public health priority questions, data collection and generation, genetic data analysis, and synthesis and interpretation to inform malaria control and elimination policy. The process begins with defining public health questions relevant to national malaria control and elimination programmes, such as whether cases are imported or locally acquired, whether HRP2-based rapid diagnostic tests (RDTs) should continue to be used, or whether frontline antimalarial drugs remain effective. Next, data collection involves appropriate study design and sample types, as well as contextual data, for example demographic information or travel history data. Subsequently, data generation will involve sequencing and bioinformatic processing, as well as epidemiological and clinical data, such as light microscopy and PCR diagnosis of malaria infections. Robust data standards would enable harmonization of downstream analysis of genetic data. This stage includes derivation of key metrics from genetic data, such as allele frequencies and complexity of infection (COI), requiring accessible analysis software. The results are then synthesized and interpreted by integrating genetic, clinical, epidemiological and surveillance data to extract relevant insights and determine appropriate courses of action. Findings are then communicated to public-health decision makers and programme officers, guiding intervention strategies and informing further research questions, thereby continuing the cycle.

Substantial variability in sequencing, bioinformatic methods, and nonstandard data formats, create challenges in harmonizing downstream analysis. This variability, however, should not
impede progress; rather, adopting a principled approach to analysis should enable users to select
from various data generation options while ensuring the comparability of final results. Many
analysis tools have been created over the past 15 years to address the challenges specific to
analyzing Plasmodium genetic data. However, these tools have not been systematically
assessed, and vary widely in their adherence to best practices in software development, for
example FAIR standards (Barker et al., 2022) or those proposed by the Public Health Alliance for
Genomic Epidemiology (PH4GE) (Libuit KG, et al., 2023). Moreover, there are few, if any,
standards for data exchange between them, limiting interoperability. A unified ecosystem that
prioritizes interoperability and accessibility with clear data-sharing guidelines and modular
workflows, as proposed for public health genomic surveillance more broadly (Black et al., 2020),
is essential to maximize the impact of malaria genomic data. Rather than investing resources in
self-contained analysis pipelines, a more sustainable and effective solution would be to create a
collaborative, transparent, and open platform of curated interoperable software to translate
genetic and contextual data into meaningful results. The advantages of harmonization are clearly
visible in other related fields, such as public health genomic surveillance of antimicrobial
resistance, where community-led efforts to evaluate and unify software and data standards have
led to wide uptake and a clear path to improve interoperability of workflows (Mendes et al., 2024).

Previous community efforts to define applications of genetic epidemiological methods for
answering research questions and informing malaria policy have resulted in seven defined use
cases (Dalmat et al., 2019). Implementing workflows for these applications requires going a step
further and outlining the types of analyses needed to translate genetic data to desired results.
Understanding which tools are available to robustly perform which functions would aid the
development of modular analysis workflows.

As a first step in addressing these needs, the Plasmodium Genomic Epidemiology
(PlasmoGenEpi) network convened 18 subject matter experts from 14 institutions to catalogue
and test available software tools, evaluate software standards and usability, improve
documentation, and evaluate next steps in the context of key use cases for malaria public health
and research. This took place at the Reproducibility, Accessibility, Documentation, and
Implementation Software Hackathon in 2023 (RADISH23) - a four-day event hosted by Johns
Hopkins University. This manuscript outlines the main outputs from this event, which center
around the PGEforge website – a new community resource for malaria genomic analysis tools.

## 130 Results

Use cases and analysis workflows for Plasmodium genomic epidemiology

Based on prior publications (Dalmat et al., 2019) and expert discussion to synthesize use cases
based on analysis requirements, we defined eight main use cases for Plasmodium genetic data
informing malaria control and elimination. For each use case, we identified specific functionalities
needed for analysis. For a subset of use cases, we then outlined how these functionalities could
be assembled into analysis workflows to obtain required results from the initial data.
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
1. *Identifying the molecular mechanism/origin of drug and diagnostic resistance:* Studies are initially needed to link parasite genetic variation to phenotypes of drug and diagnostic resistance, for example by sequencing parasites with corresponding data on *in vitro* response to antimalarials or with evidence of clinical failure of drugs or diagnostic tests. Related studies can identify potential polymorphisms of interest via signals of selection and identify how they are evolving and spreading at various temporal and spatial scales.

2. *Monitoring the prevalence/frequency of drug or diagnostic resistance markers:* Once genetic polymorphisms have been associated with resistance phenotypes, they can be used as markers of resistance. Marker prevalence (fraction of infections that contain parasites with the resistance marker) and/or frequency (relative abundance of the resistance marker in the parasite population) can be monitored for surveillance of resistance. Various statistical methods may be needed to estimate prevalence/frequency because polyclonal infections complicate otherwise simple analyses.

3. *Measuring human immune selection on the parasite population:* Identifying areas of the *Plasmodium* genome under immune selection can help identify potential targets for vaccines and related immunologic interventions such as monoclonal antibodies. Related to this, monitoring sequences of such targets can be useful in the context of large-scale implementation to evaluate for selection and monitor for potential escape mutations.

5. *Estimating transmission intensity:* Population level measures derived from parasite genetic data, like other indicators such as parasite prevalence, may be used to estimate malaria transmission intensity for surveillance purposes such as stratification of interventions or to evaluate the effect of interventions.

6. *Estimating the connectivity and movement of parasites between geographically distinct populations:* Identifying whether and to what extent parasites from one geographic area are moving to another can help predict the spread of malaria, as well as the spread of parasites with specific concerning phenotypes such as drug resistance. This in turn can inform planning of the scale and timing of interventions. For example, it may be easier to reduce the burden of malaria in an isolated area compared to one that is more connected to areas of higher transmission. This information may be identifiable by comparing genetic measures between populations.

7. *Classifying malaria cases as locally acquired or imported from another population:* In elimination or pre-elimination settings, understanding if cases are imported is relevant to 1)
181 assessing the feasibility of elimination (i.e. elimination is less feasible in areas with a high number
182 of imported infections that may lead to local transmission) and 2) determining if local elimination
183 has been achieved despite reported cases.

184
185 8. Reconstructing granular patterns of transmission: In areas of very low transmission,
186 characterizing granular details of transmission events — such as chains of transmission between
187 individuals/households, demographic groups at high risk of being infected with or transmitting
188 malaria, hidden reservoirs of residual transmission, contribution of imported cases to local
189 transmission, and the length of sustained transmission chains may help guide and evaluate
190 targeted elimination strategies. This information may be identifiable through studies that sample
191 infected people comprehensively and utilize high resolution genetic data to evaluate relationships
192 between infections.
193

194 Each of these eight use cases requires several analysis functionalities to obtain the desired
195 results from Plasmodium genetic and contextual data. We identified 14 distinct analysis
196 functionalities that are required or useful for the eight use cases, with many functionalities shared
197 by several use cases (Table 1). For example, estimating complexity of infection (COI, the number
198 of genetically distinct clones within an infection) was identified as required or useful for all eight
199 use cases. This makes intuitive sense, since analysis of genetic data from polyclonal infections
200 often needs to be treated differently than that from monoclonal infections. Similarly, estimating
201 phase (assigning alleles to particular clones within a polyclonal sample), population allele
202 frequencies, and within-host relatedness were identified as useful for the majority of use cases.
203 Thus, having a curated set of validated software tools to carry out these functionalities would
204 support a diverse range of analyses. Other functionalities such as estimating copy number
205 variation or multi-locus genotype frequencies were only needed for specific use cases.
206

207 For three example use cases, we outlined how analysis workflows might be assembled from the
208 component functionalities (Figure 2). During the process of outlining the workflows, several
209 observations became noteworthy. First, obtaining seemingly straightforward results such as the
210 frequencies of polymorphisms associated with drug resistance was more nuanced than generally
211 appreciated. Second, there were often several viable paths to obtain a given result, based on
212 features of the data (e.g. monoclonal vs. polyclonal population) and alternative analysis
213 approaches. Third, within a given path, several software tools might interchangeably provide a
214 given functionality, as discussed in the next section. Alternatively, some workflows are not
215 sequential. They estimate key parameter values (e.g., COI, phase, and frequencies) using data
216 on all samples jointly (i.e., they borrow information across samples to make up for relatively
217 uninformative per-sample data) and would fail if polyclonal samples were separated from
218 monoclonal ones. Finally, analysis approaches for some use cases (e.g. estimating transmission
219 intensity) were less mature and although some analysis steps were clear, others remain poorly
220 defined.
221
<figure><img src="image_3.png" /></figure>

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

<caption>Figure 2. Analysis workflow for key use cases – (a) to estimate drug resistance prevalence, raw genetic data is processed through allele calling and subsequent steps vary depending on the complexity of infection (COI) and whether the population is monoclonal or polyclonal (including monoclonal and/or polyclonal infections). If the population is monoclonal, allele prevalence (prev) and allele frequencies (AF) at a single locus can be estimated directly and will be identical. Additionally, multi-locus prev, AF and copy number variation (CNV) or deletions (del) can be estimated directly if relevant. If the population is polyclonal and phasing is possible, the analysis follows the same pathway as for a monoclonal population. If phasing is not possible or not performed, additional steps are required depending on whether unambiguous multilocus genotypes can be obtained and whether the sequencing (seq) method utilized detects CNV or deletions in polyclonal infections; (b) to classify cases as imported or locally acquired raw genetic data is processed through allele calling, followed by estimation of COI. Then estimation of between host relatedness, geographic assignment, transmission networks and structure and/or clustering metrics can be translated into estimates of importation or local transmission. (c) to estimate transmission intensity, raw genetic data is processed through allele calling, COI and AF estimation. Subsequent metric estimation involves measuring the proportion of polyclonal infections in the population, between and within-host relatedness and population diversity, which can all be collectively translated into estimates of transmission. Alternative analysis pathways may be used depending on whether phasing is possible or performed.</caption>

244

## Landscaping of available analysis tools and mapping of tools to functionalities

245

246

247

248

249

250

251

252

253

254

255

To begin building a centralized resource for analysis tools to support the eight use cases, we curated and compared software characteristics of 40 tools (Supplementary Table 1). Twenty two of these tools were found to pass minimum inclusion criteria for evaluation (see Methods), and the characteristics of these tools are shown in Table 2, including implementation type, installation method, platform compatibility, and associated resources (e.g. website, publication or pre-print). It is important to note that this is not an exhaustive list; there are available tools that we did not evaluate.

Among these tools, the majority are implemented as R packages (14 tools) and others as standalone scripts implemented in R (2 tools), C/C++ or Python (3 tools), Perl (1 tool) and a Windows executable file (1 tool). Most tools are platform-independent, capable of running on
multiple operating systems, but one tool is specifically designed for Windows. The installation methods for these tools vary widely: seven require manual installation from source files, six can be cloned and run locally from repositories, six are available via package managers like CRAN or BiocManager, and three require compilation. For some of the tools requiring compilers, efforts have been made to provide 'R wrappers' to decrease installation barriers (e.g. McCOILR, hmmibdr).

We identified the availability of the 14 analysis functionalities (defined in Table 1) in each tool (Table 3). We focused on tools that are actively being used as some of the tools were deemed obsolete or superseded, for example if installation was not possible during our evaluation, or if they have been superseded by newer tools that perform similar functionalities (these reasons are described in Table 2). The most common functionalities are estimating COI, available in eight different tools, estimation of SNP allele frequencies, which is supported by five tools, and estimation of between- and within-host genetic relatedness, which are available in four and two tools, respectively. Given the number of tools that perform similar functionalities, benchmarking them against standardized datasets will be an important next step to enable validation towards a curated list of best practices. There are also several gaps in current tool capabilities. For example, no existing tools are specifically designed to infer malaria transmission networks, and there is limited support for functionalities like phasing and geographic assignment covered by only one tool each.

## Evaluating software standards

Software standards are a critical component in the development and maintenance of reliable, efficient, and interoperable software systems. These standards encompass various aspects of software engineering, including coding practices, documentation, testing, maintenance and accessibility. We categorized software standards into two broad groups: 1) those aimed at end-users, and 2) those aimed at developers (Table 4).

We applied all eight software standards to the 22 tools prioritized from the landscaping exercise. For each software standard, we used a scale from 0 to 2, where 0 indicates that the standard was not met, 1 indicates the standard was partially met (for example if usage instructions exist but are not comprehensive), and 2 indicates the standard was fully met. Overall, user-oriented software standards were generally met more consistently than developer-oriented standards (Figure 3).
<figure><img src="image_4.png" /><figcaption>Figure 3. Evaluation of tools based on software standards for user- and developer-facing criteria.</figcaption></figure>

Most tools provide basic installation instructions and usage guidance, whereas provision of tutorials and test data sets with expected results were not always available. For the developer-oriented software standards, although all tools were open-source and offered reasonably clear channels for software maintenance and issue resolution, very few provide adequate testing, and none achieve high code coverage. This points to a reasonable effort of making tools accessible and open-source for end-users, however there is still room for improvement in providing comprehensive usage resources and testing and documentation.

## Streamlining installation and managing system dependencies

Historically, a significant barrier to the adoption of software tools has been the challenges associated with their installation and the management of system dependencies. While tools may function correctly at the time of publication, differences in operating systems, software versions, and configurations often lead to incompatibilities that hinder successful installation or execution. Hidden system dependencies, in particular, can cause unexpected failures that are difficult to diagnose without advanced knowledge of the underlying software architecture. These obstacles can discourage end-users, especially those with limited technical expertise, reducing the accessibility and impact of the tools.

To address this, we developed the PlasmoGenEpi r-universe (plasmogenepi.r-universe.dev), a dynamic platform hosting R package repositories. This platform features an automated build process that incorporates platform-specific system dependencies, providing pre-compiled R packages to end-users and eliminating the need for local compilation. While this approach significantly improves usability and accessibility, further work on containerization would better
facilitate fully reproducible analysis workflows that can operate seamlessly across diverse computational environments.

## Development of community resources

Community resources are essential for advancing malaria genomics by fostering collaboration, standardization, and accessibility in tool development and data analysis. To address this need, we developed PGEforge (mrc-ide.github.io/PGEforge) as a community-driven platform for end-users and developers of malaria genomic tools. PGEforge serves as a centralized hub for identifying and evaluating analysis tools, hosting example genomic datasets in widely used formats from both WGS and amplicon data and outlining analysis workflows tailored to key research questions. Additionally, it offers comprehensive and fully reproducible tutorials, initially developed for 10 analysis tools identified during RADISH23, that leverage the publicly available datasets hosted on the platform. This resource aims to streamline and enhance malaria genomics research through accessibility, standardization, and reproducibility.

## Discussion

The efforts described here represent an important step towards improving the usability of analysis tools for malaria genomic epidemiology, but substantially more work is needed to ensure useful results can be reliably obtained by a broad range of users. This work includes filling gaps in analysis functionalities, formally evaluating the accuracy of tools in estimating parameters, and creating a broader environment that facilitates flexible assembly of tools into accessible workflows. Mapping out workflows provides valuable insight into gaps currently underserved by existing software tools. Notable gaps include tools for *in silico* phasing of genomic data from polyclonal infections, which were identified as potentially useful across all eight defined use cases. Additionally, specialized tools are needed for specific use cases, such as the classification of outcomes in antimalarial therapeutic efficacy studies.

Where appropriate tools do exist, data on their accuracy are often limited (at best) to an initial evaluation by the original developers. Rigorous benchmarking of tools across different types of data, including assessment of performance in realistic situations that do not satisfy all model assumptions, will allow users and workflow developers to make informed choices on which tools are most appropriate for given applications. Standardized, curated datasets where the truth is known (e.g. from simulations or controlled experiments) along with well-defined performance measures will be an important step in facilitating routine benchmarking. For example, Guo and colleagues undertook comprehensive benchmarking of the accuracy of tools that estimate identity-by-descent using a simulated ground truth to show the impact of marker density on accurate IBD detection (Guo et al., 2024). Further benchmarking work is needed, including updated curation as the available set of tools expands and evolves.

Aspects important for end-to-end workflows, but not addressed during the RADISH23 hackathon, include evaluation of upstream steps of bioinformatic allele calling and downstream steps of
communication and visualization of results. Choosing an appropriate bioinformatic pipeline to obtain allele calls is important since this will determine the type (e.g. SNPs, indels, and/or microhaplotypes) and quality of data used for subsequent analyses. Several pipelines exist for various types of sequencing data, e.g. built around GATK for obtaining SNPs and indels from whole genome sequencing data (Van der Auwera & O'Connor, 2020), or SeekDeep (Hathaway et al., 2018) or DADA2 (Callahan et al., 2016) for obtaining microhaplotypes from targeted sequencing data. However, there have been few efforts to perform systematic benchmarking of these pipelines. As with analysis tools, curated ground truth datasets will be useful in improving (e.g. optimizing tuning parameters), evaluating, and comparing pipelines. At the other end of the workflow, thoughtful consideration of how to best summarize and display results would substantially improve their accessibility and therefore public health impact. Providing several options for users, including summary tables and figures (e.g. https://genremekong.org/tools/grcmalaria-r-package-user-guide), potentially utilizing dynamic interfaces (e.g. (van Wyk et al., 2025), (Battle et al., 2024)), may be a worthwhile investment of resources. Linking all of these components together – allele calling pipelines, analysis tools, and display of results – is somewhat challenging at present. Creating a set of interoperability standards, e.g. agreed upon formats for inputs and outputs at various steps, would greatly facilitate assembly of these steps into complete workflows that start with raw genetic and contextual data and return results in interpretable formats. Similarly, creating example workflows e.g. well-commented scripts with associated documentation and tutorials that connect several disparate tools, can expand access to data analysis for specific use cases.

It will require some upfront effort to optimize, benchmark, and document the many steps required to guide raw sequence data through to meaningful results. However, such an investment is apt to pay large dividends, allowing harmonization of disparate components created via uncoordinated efforts. The end result would be a collaborative, transparent, and open platform of interoperable software, following the successful lead of this approach for other systems (e.g. QIIME, https://qiime2.org/). This model fosters community contributions, promotes healthy competition, and ensures that tools continue to evolve while remaining relevant and applicable. Important considerations to democratize access to such a platform will include documentation in several relevant languages beyond English and flexibility to install workflows locally or run them remotely on cloud-based platforms (e.g. (Data Sciences Platform, Broad Institute, 2024)).

In conclusion, a substantial body of work has been produced on analysis tools for Plasmodium genomic epidemiology. Thoughtful, targeted improvement of these existing tools in the context of an open ecosystem designed for organized growth has the potential to greatly extend their utility. We hope that the resources described here are a useful step in continued efforts towards this goal.
394 Methods

395 Landscaping of available analysis tools

396 We conducted a landscaping exercise of available tools that are commonly applied to
397 *Plasmodium* genetic data. This included identifying, cataloging, and assessing tools. In order to
398 limit the scope of this exercise during RADISH23, we applied the following constraints:

399 • Focus on downstream analysis tools. This includes tools whose primary goal is to extract
400 signal from pre-processed data, but does not include tools that are primarily used within
401 upstream bioinformatic steps, such as variant callers and quality filters.

402 • Focus on *Plasmodium* genetics, including both *P. falciparum* and *P. vivax*. We did not consider applications to mosquito genetics, despite some tools and techniques being applicable to both organisms.

405 • Focus on tools that are specifically engineered for *Plasmodium*. While there are many
406 generic methods that can be applied to any type of genetic data (e.g. F-statistics and other
407 population genetic metrics), we did not seek to evaluate these tools.

408 These constraints were necessary in order to make a meaningful contribution within a narrow
409 timeframe, although we strongly advocate for future events expanding the scope of this exercise.

410

411 Defining software standards for tool evaluation

412 When establishing objective criteria for evaluating software standards, we divided our approach
413 into two categories: user-oriented standards and developer-oriented standards.

414 User-oriented software standards describe a set of best practices and guidelines that prioritize
415 the needs and expectations of end-users. These standards cover interface design, usability,
416 accessibility, and performance, which together create a cohesive and intuitive experience.

417 Although central to commercial software development, user-oriented standards tend to be a blind-
418 spot in academic software development where the incentive structure favors innovation over
419 usability. We identified four standards as critical in promoting a positive user experience (Table
420 4), along with three characteristics that are desirable but not critical (Supplementary Table 2).

421

422 Developer-oriented software standards focus on ensuring that the software development process
423 is efficient, maintainable, and scalable. This is important for the long-term viability of tools, the
424 transparency of methods, and the potential to extend and modify existing frameworks as data and
425 use cases change. We identified four standards as critical in promoting a robust development
426 process (Table 4), along with four characteristics that are desirable but not critical (Supplementary
427 Table 2).

428 Compiling datasets of common genetic data input formats

429 We compiled empirical and simulated datasets of commonly used input formats that could be
430 used as a reference set by the community. In addition, these same datasets were used across all
431 tutorials developed during RADISH23, thereby ensuring results were reproducible and
432 comparable between tools. We identified the following key features to ensure appropriateness
433 and utility for downstream analysis tools:
434

- A subset of monoclonal and polyclonal samples, lab strains, and lab mixtures

435

- Various formats: VCF files, BAM files, tabular haplotypes in .csv

436

- A subset of polyclonal samples with within-host relatedness greater than 0

437

- Some amount of missing data

438

The data input formats derived were whole genome sequences, microhaplotypes, SNP barcodes, and *pfhrp2/3* gene deletion counts. All are freely available at the PGEforge GitHub repository at https://github.com/mrc-ide/PGEforge/tree/main/data.

442

443

### Whole genome sequencing data

444

We derived three VCF files from the MalariaGEN Pf3K project (https://www.malariagen.net/parasite/pf3k), selecting samples from Democratic Republic of Congo (DRC) (n=113), Vietnam (n=97), and laboratory strain mixtures (n=25). Each VCF contains 247,496 high-quality (VQSLOD>6) biallelic SNPs across all fourteen somatic chromosomes. The VCFs are sorted and accompanied by an index file. We provide associated contextual data for the samples, which includes Fws statistics from the MalariaGEN Pf7K dataset (https://www.malariagen.net/sites/default/files/Pf7_fws.txt) (except for the lab mixtures). These are deposited at https://github.com/mrc-ide/PGEforge/tree/main/data/wgs/pf3k.

452

453

We also simulated a dataset of samples with varying COI and within-host relatedness. Briefly, a simulated sample with a given COI, *K*, was created by randomly sampling *K* clonal haplotypes (Fws >0.95, (Manske et al., 2012)) from the Pf3K project (https://www.malariagen.net/parasite/pf3k), assigning these haplotypes to *j* ≤ *K* infectious bites, and if *j* < *K*, simulating meiosis with a constant recombination rate of 13.5kbp/cM (Miles et al., 2016). For each haplotype, proportions were randomly sampled from a symmetric Dirichlet distribution, and for each locus, read count data was simulated from a Beta-binomial distribution given the allelic proportions. Constant sequencing error was simulated in the allele counts at rates (REF to ALT, 10⁻⁵; ALT to REF, 5x10⁻⁴) broadly consistent with observation for the included sites. No variant calling error was simulated. In this way, we produced a VCF file with 40 simulated DRC samples, ranging from COI of one to four, with approximately half of the samples having within-host relatedness. Accompanying contextual data is provided as .csv and .bedfiles, which are deposited at https://github.com/mrc-ide/PGEforge/tree/main/data/wgs/simulated/pf3k-DRCongo-simA.

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

To derive BAM files, we subsetted the VCF calls to only *csp* (PF3D7_0304600), *celTOS* (PF3D7_1133400) and *ama1* (PF3D7_1216600) genes, and provide accompanying contextual data. These are deposited at https://github.com/mrc-ide/PGEforge/tree/main/data/wgs/labisolate_subset.

469

470

471

472

473

### Microhaplotype data

474

475

We provided microhaplotype data from 82 samples from Mozambique and lab strain mixtures from the study by (Tessema et al., 2020). The microhaplotype panel included 100 targets (91 diversity and 9 drug targets). The lab strain mixtures are from various combinations of seven *P. falciparum* lab strains, with some sequenced in replicate at 4 different parasite densities (10, 100,

476

477

478
478 1000, 10000 parasites per µl). The datasets and their accompanying contextual data can be found
479 at https://github.com/mrc-ide/PGEforge/tree/main/data/amplicon.

480
481 We also simulated a Mozambique dataset to create 100 samples with 50 diverse targets sub-
482 selected from the MAD^4HatTeR panel (Aranda-Díaz et al., 2024). Both PCR and Illumina error
483 was simulated using in-house C++ scripts. Starting parasite densities were sampled from previous
484 Mozambique data (Tessema et al., 2020), and PCR was simulated by taking the randomly
485 sampled parasite density and simulating 23 rounds of PCR with a per-base error rate of 3.5e-06
486 for taq PCR rate and simulating chimeras. Illumina error was introduced by creating a Nextseq
487 3D7 reference data error profile. Haplotypes were randomly sampled from a previous
488 Mozambique dataset, which was also used to generate a distribution of COI (da Silva et al., 2023).
489

490 SNP barcodes

491 We derived SNP barcode data by subsetting the Pf3k WGS data from the samples from Vietnam
492 and DRC described above to match the 101-SNP barcode panel from (Chang et al., 2019). The
493 lab strain and lab mixtures dataset are subsetted to the 101-SNP barcode, with the mixtures
494 created from various combinations of *P. falciparum* lab strains. We provide the accompanying
495 contextual data with information on the genotypes that make up the mixtures and the COI.
496

497 We also simulated 101-SNP barcode data for 100 samples (50 from Bangladesh and 50 from
498 Ghana) to include samples with a range of COI, as follows. A simulated sample was created by
499 simulating superinfections by sampling the SNP barcode from each of these countries and
500 selecting COIs based on the observed COIs in each country. Simulations were then created
501 similarly to the details above for the Mozambique dataset but data was sampled from Pf7 dataset
502 (MalariaGEN et al., 2023). We also provide an additional dataset where we removed indels. The
503 datasets are provided at https://github.com/mrc-ide/PGEforge/tree/main/data/snp_barcode.

504
505 pfhrp2/3 gene deletion counts

506 We provide pfhrp2/3 gene deletion count data in tabular .csv format from the study by (Feleke et
507 al., 2021) conducted in twelve sites in Ethiopia. The dataset can be found at
508 https://github.com/mrc-ide/PGEforge/tree/main/data/pfhrp2-3_counts.

509 Tutorial development

510 For each tool, we completed summary documents, installation aids and tutorials. Ten tools were
511 completed during RADISH23, and guidelines were made available for future tutorial development
512 by anyone in the community. These tutorials provide background information on each analysis
513 tool, installation instructions and a fully worked-through analysis tutorial with code showing how
514 an end user could use the tool to analyze data.

515
Data availability

516 The empirical and simulated datasets produced during RADISH23 are freely available on the
517 PGEforge GitHub repository at https://github.com/mrc-ide/PGEforge/tree/main/data. They were
518 derived from publicly available data as described in Methods.
519 Code availability

520 The resources developed during RADISH23 are freely available on the PGEforge website at
521 https://mrc-ide.github.io/PGEforge/ and PGEforge GitHub repository at https://github.com/mrc-ide/PGEforge.

523 References

524 Aranda-Díaz, A., Vickers, E. N., Murie, K., Palmer, B., Hathaway, N., Gerlovina, I., Boene, S.,
525 Garcia-Ulloa, M., Cisteró, P., Katairo, T., Semakuba, F. D., Nsengimaana, B., Gwarinda,
526 H., García-Fernández, C., Da Silva, C., Datta, D., Kiyaga, S., Wiringilimaana, I., Fekele, S.
527 M., ... Greenhouse, B. (2024). Sensitive and modular amplicon sequencing of Plasmodium
528 falciparum diversity and resistance for research and public health. In *bioRxiv* (p.
529 2024.08.22.609145). https://doi.org/10.1101/2024.08.22.609145

530 Barker, M., Chue Hong, N. P., Katz, D. S., Lamprecht, A.-L., Martinez-Ortiz, C., Psomopoulos,
531 F., Harrow, J., Castro, L. J., Gruenpeter, M., Martinez, P. A., & Honeyman, T. (2022).
532 Introducing the FAIR Principles for research software. *Scientific Data*, 9(1), 622.

533 Battle, K. E., N Diop, M., Kong, D., Proctor, J. L., Driano, E. K., Bever, C. A., Amratia, P., Sy,
534 M., Ngom, B., Diallo, I., Sene, D., Wirth, D. F., Volkman, S. K., & Ndiaye, D. (2024,
535 November). *Data integration for decision-making: A malaria data dashboard that merges
536 routine surveillance and genomic research data with modeled outputs for programmatic
537 action in Senegal*. American Society of Tropical Medicine and Hygiene 2024 Annual
538 Meeting, New Orleans, USA. https://www.astmh.org/getmedia/cc92e7ad-8e49-4eb0-83ec-
539 a29e5bba5dc5/ASTMH-2024-Annual-Meeting-Abstract-Book-All-Abstracts.pdf

540 Black, A., MacCannell, D. R., Sibley, T. R., & Bedford, T. (2020). Ten recommendations for
541 supporting open pathogen genomic analysis in public health. *Nature Medicine*, 26(6), 832–
542 841.

543 Callahan, B. J., McMurdie, P. J., Rosen, M. J., Han, A. W., Johnson, A. J. A., & Holmes, S. P.
544 (2016). DADA2: High-resolution sample inference from Illumina amplicon data. *Nature
545 Methods*, 13(7), 581–583.

546 Chang, H.-H., Wesolowski, A., Sinha, I., Jacob, C. G., Mahmud, A., Uddin, D., Zaman, S. I.,
547 Hossain, M. A., Abul Faiz, M., Ghose, A., Sayeed, A. A., Ridwanur Rahman, M., Islam, A.,
548 Karim, M. J., Kamar Rezwan, M., Shamsuzzaman, A. K. M., Jhora, S. T., Aktaruzzaman,
549 M. M., Drury, E., ... Buckee, C. (2019). *Mapping imported malaria in Bangladesh using
550 parasite genetic and human mobility data*. https://doi.org/10.7554/eLife.43481

551 Dalmat, R., Naughton, B., Kwan-Gett, T. S., Slyker, J., & Stuckey, E. M. (2019). Use cases for
552 genetic epidemiology in malaria elimination. *Malaria Journal*, 18(1), 163.

553 da Silva, C., Boene, S., Datta, D., Rovira-Vallbona, E., Aranda-Díaz, A., Cisteró, P., Hathaway,
554 N., Tessema, S., Chidimatembue, A., Matambisso, G., Nhama, A., Macete, E., Pujol, A.,
555 Nhamussua, L., Galatas, B., Guinovart, C., Enosse, S., De Carvalho, E., Rogier, E., ...
556 Mayor, A. (2023). Targeted and whole-genome sequencing reveal a north-south divide in P.
557 falciparum drug resistance markers and genetic structure in Mozambique. *Communications
558 Biology*, 6(1), 619.

559 Data Sciences Platform, Broad Institute. (2024, September 13). *malaria-featured-workspaces/Malaria_Plasmodium_Illumina_Amplicon Workspace*. [Workspace].
560
561 https://publichealth.terra.bio/#workspaces/malaria-featured-

562 workspaces/Malaria_Plasmodium_Illumina_Amplicon

563 Feleke, S. M., Reichert, E. N., Mohammed, H., Brhane, B. G., Mekete, K., Mamo, H., Petros, B.,

564 Solomon, H., Abate, E., Hennelly, C., Denton, M., Keeler, C., Hathaway, N. J., Juliano, J.

565 J., Bailey, J. A., Rogier, E., Cunningham, J., Aydemir, O., & Parr, J. B. (2021). Plasmodium

566 falciparum is evolving to escape malaria rapid diagnostic tests in Ethiopia. *Nature Microbiology*, **6**(10), 1289–1299.

568 Gardner, M. J., Hall, N., Fung, E., White, O., Berriman, M., Hyman, R. W., Carlton, J. M., Pain,

569 A., Nelson, K. E., Bowman, S., Paulsen, I. T., James, K., Eisen, J. A., Rutherford, K.,

570 Salzberg, S. L., Craig, A., Kyes, S., Chan, M.-S., Nene, V., ... Barrell, B. (2002). Genome

571 sequence of the human malaria parasite Plasmodium falciparum. *Nature*, **419**(6906), 498–

572 511.

573 Guo, B., Takala-Harrison, S., & O'Connor, T. D. (2024). Benchmarking and Optimization of

574 Methods for the Detection of Identity-By-Descent in High-Recombining Plasmodium

575 falciparum Genomes. In *bioRxiv* (p. 2024.05.04.592538).

576 https://doi.org/10.1101/2024.05.04.592538

577 Hathaway, N. J., Parobek, C. M., Juliano, J. J., & Bailey, J. A. (2018). SeekDeep: single-base

578 resolution de novo clustering for amplicon deep sequencing. *Nucleic Acids Research,*

579 46(4), e21.

580 Libuit KG, Guthrie J, Ambrosio F, Kapsak C, Unal Gultekin, Holmes J, Wright S, Nguinkal J,

581 Doughty E, Southgate J, O'Cathail C, Carleton H, Kingwara L, Khan W, Baker K, Diallo A,

582 Connor T, Kanwar S, Maturure P, James S, Cuesta I, Dyster V, Gaskin A, Williams C,

583 Smith E, Rokney A, Petkau A, Varona S, Gnimpieba E, Rey S, Macori G, & Mboowa G.

584 (2023, July 14). *Proposed Standards For Public Health Bioinformatics Software*. Public

585 Health Alliance for Genomic Epidemiology (PH4GE).

586 https://pha4ge.org/resource/proposed-standards-for-public-health-bioinformatics-software/

587 MalariaGEN, Abdel Hamid, M. M., Abdelraheem, M. H., Acheampong, D. O., Ahouidi, A., Ali,

588 M., Almagro-Garcia, J., Amambua-Ngwa, A., Amaratunga, C., Amenga-Etego, L.,

589 Andagalu, B., Anderson, T., Andrianaranjaka, V., Aniebo, I., Aninagyei, E., Ansah, F.,

590 Ansah, P. O., Apinjoh, T., Arnaldo, P., ... van der Pluijm, R. W. (2023). Pf7: an open

591 dataset of Plasmodium falciparum genome variation in 20,000 worldwide samples.

592 *Wellcome Open Research*, **8**, 22.

593 Manske, M., Miotto, O., Campino, S., Auburn, S., Almagro-Garcia, J., Maslen, G., O'Brien, J.,

594 Djimde, A., Doumbo, O., Zongo, I., Ouedraogo, J.-B., Michon, P., Mueller, I., Siba, P.,

595 Nzila, A., Borrmann, S., Kiara, S. M., Marsh, K., Jiang, H., ... Kwiatkowski, D. P. (2012).

596 Analysis of Plasmodium falciparum diversity in natural infections by deep sequencing.

597 *Nature*, **487**(7407), 375–379.

598 Meeting report of the technical consultation on the role of parasite and anopheline genetics in

599 malaria surveillance. (n.d.). Retrieved December 13, 2023, from

600 https://www.who.int/publications/m/item/WHO-CDS-GMP-MPAC-2019.17

601 Mendes, I., Griffiths, E., Manuele, A., Fornika, D., Tausch, S. H., Le-Viet, T., Phelan, J.,

602 Meehan, C. J., Raphenya, A. R., Alcock, B., Culp, E., Lorenzo, F., Haim, M. S., Witney, A.,

603 Black, A., Katz, L., Oluniyi, P., Olawoye, I., Timme, R., ... Maguire, F. (2024).

604 hAMRonization: Enhancing antimicrobial resistance prediction using the PHA4GE AMR
605 detection specification and tooling. In *bioRxiv* (p. 2024.03.07.583950).

606 https://doi.org/10.1101/2024.03.07.583950

607 Miles, A., Iqbal, Z., Vauterin, P., Pearson, R., Campino, S., Theron, M., Gould, K., Mead, D.,

608 Drury, E., O'Brien, J., Ruano Rubio, V., MacInnis, B., Mwangi, J., Samarakoon, U.,

609 Ranford-Cartwright, L., Ferdig, M., Hayton, K., Su, X.-Z., Wellems, T., ... Kwiatkowski, D.

610 (2016). Indels, structural variation, and recombination drive genomic diversity in

611 Plasmodium falciparum. *Genome Research*, 26(9), 1288-1299.

612 Tessema, S. K., Hathaway, N. J., Teyssier, N. B., Murphy, M., Chen, A., Aydemir, O., Duarte, E.

613 M., Simone, W., Colborn, J., Saute, F., Crawford, E., Aide, P., Bailey, J. A., & Greenhouse,

614 B. (2020). Sensitive, Highly Multiplexed Sequencing of Microhaplotypes From the

615 Plasmodium falciparum Heterozygome. *The Journal of Infectious Diseases*, 225(7), 1227-

616 1237.

617 Van der Auwera, G. A., & O'Connor, B. D. (2020). *Genomics in the Cloud: Using Docker, GATK,*

618 *and WDL in Terra*. O'Reilly Media.

619 van Wyk, S., Seocharan, I., Mlugu, E. M., Ayuen, D. S., Mategula, D., Makhaza, T., Kiarie, J.,

620 Asua, V., Opigo, J., Mbituyumuremyi, A., Keitany, K. K., Nyandigisi, E. M., Sinarinzi, P.,

621 Baak, P. A. K., Manbul, T. N., Nhiga, S. L., Mwaikambo, S. A., Kassim, M., Sija, S. J., ...

622 Barnes, K. I. (2025). The MARC SE-Africa Dashboard: Joining forces to counteract

623 emerging antimalarial resistance in south and east Africa. In *medRxiv*.

624 https://doi.org/10.1101/2025.01.07.25320158

625 Competing interests

The authors declare no competing interests.

627

628 Funding

629 This work was funded in part by the Bill and Melinda Gates Foundation (INV-031273 to RV; INV-

630 053763 to BG, AW, RV) and NIH/NIAID (U01AI184646 to BG, AW, RV; K24AI144048 to BG).

631 SR-P and RV acknowledge funding from the MRC Centre for Global Infectious Disease Analysis

632 (reference MR/X020258/1), funded by the UK Medical Research Council (MRC). This UK funded

633 award is carried out in the frame of the Global Health EDCTP3 Joint Undertaking. ART is funded

634 by the European Union (project number 101110393). Views and opinions expressed are however

635 those of the author only and do not necessarily reflect those of the European Union or European

636 Research Executive Agency (REA). Neither the European Union nor the granting authority can

637 be held responsible for them.

638
<table><caption>640 Table 1: Mapping use cases to core functionalities</caption><thead><tr><th rowspan="2">Use case</th><th colspan="12">Analysis functionality</th></tr><tr><th>Estimate COI</th><th>Phase</th><th>Estimate allele prevalence</th><th>Estimate allele frequency</th><th>Estimate multi-locus genotype prevalence</th><th>Estimate multi-locus genotype frequency</th><th>Estimate between-host relatedness / genetic distance (e.g. IBD, IBS)</th><th>Identify signs of selection</th><th>Estimate within-host relatedness / genetic distance (e.g. IBD, IBS)</th><th>Infer transmission networks</th><th>Infer geographic assignment</th><th>Calculate simple genetic metrics</th><th>Infer structure / clustering</th><th>Estimate copy number variation / deletions</th></tr></thead><tbody><tr><td>Identifying the molecular mechanism/origin of drug and diagnostic resistance</td><td>X</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Monitoring the prevalence/frequency of drug or diagnostic resistance markers</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td></tr><tr><td>Measuring human immune selection on the parasite population</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Classifying outcomes in therapeutic efficacy studies (TESs) as reinfection, recrudescence or, in the case of P. vivax, relapse</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Estimating transmission intensity</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td></td></tr><tr><td>Estimating the connectivity and movement of parasites between geographically distinct populations</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td></td><td></td></tr><tr><td>Classifying malaria cases as locally acquired or imported from another population</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td></tr><tr><td>Reconstructing granular patterns of transmission</td><td>X</td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>
642 Table 2: Tool landscaping matrix (attached as .xlsx)

643 Table 3: Tools to analysis functionalities

644

<table><thead><tr><th rowspan="2">Tool</th><th colspan="10">Analysis functionality</th></tr><tr><th>Estimate COI</th><th>Phase</th><th>Estimate allele prevalence</th><th>Estimate allele frequency</th><th>Estimate multi-locus genotype prevalence</th><th>Estimate multi-locus genotype frequency</th><th>Estimate between-host relatedness / genetic distance (e.g. IBD, IBS)</th><th>Identify signs of selection</th><th>Estimate within-host relatedness / genetic distance (e.g. IBD, IBS)</th><th>Infer transmission networks</th><th>Infer geographic assignment</th><th>Calculate simple genetic metrics</th><th>Infer structure / clustering</th><th>Estimate copy number variation / deletions</th></tr></thead><tbody><tr><td>FreqEstimationModel</td><td>X</td><td></td><td>X</td><td>X</td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MultiLociBiallelicModel</td><td>X</td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>THEREALMcCOIL</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>coiaf</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>DEploidIBD</td><td>X</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>isoRelate</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>hmmIBD</td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>dcifer</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>moire</td><td>X</td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MALECOT</td><td>X</td><td></td><td></td><td>X</td><td></td><td>X</td><td></td><td></td><td></td><td></td><td>X</td><td></td><td>X</td><td></td></tr><tr><td>DRpower</td><td></td><td></td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>X</td></tr><tr><td>estMOI</td><td>X</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>
646 Table 4: Software standards

647

<table><thead><tr><th>Criteria</th><th>Type</th><th>Notes</th></tr></thead><tbody><tr><td><strong>User-facing</strong></td><td></td><td></td></tr><tr><td>Installation instructions exist</td><td>binary</td><td></td></tr><tr><td>Usage instructions exist</td><td>binary</td><td></td></tr><tr><td>Tutorials exist</td><td>binary</td><td>Must have explanation of inputs, outputs, and test data set in a worked example</td></tr><tr><td>Test data and results available</td><td>binary</td><td></td></tr><tr><td><strong>Developer-facing</strong></td><td></td><td></td></tr><tr><td>Open source</td><td>binary</td><td></td></tr><tr><td>Has software tests</td><td>binary</td><td>Eg, unit tests</td></tr><tr><td>More than 90% code coverage reported</td><td>binary</td><td>Eg, how much of the source code is tested by automated tests</td></tr><tr><td>Clear channels for software maintenance and issues</td><td>binary</td><td>Eg, GitHub issues, author contact information</td></tr></tbody></table>

648
649 Supplementary material

650 Supplementary Table 1: All tools landscaping matrix (attached as .xlsx)

651 Supplementary Table 2: “Nice to have” software standards

652

<table><thead><tr><th>Criteria</th><th>Type</th><th>Notes</th></tr></thead><tbody><tr><td><strong>User-facing</strong></td><td></td><td></td></tr><tr><td>Informative error handling</td><td>binary</td><td></td></tr><tr><td>Multiple languages for tutorials</td><td>binary</td><td></td></tr><tr><td>Uses standard data input formats</td><td>binary</td><td>Eg decreases the need for the user to wrangle data into required input format</td></tr><tr><td><strong>Developer-facing</strong></td><td></td><td></td></tr><tr><td>Minimal dependencies</td><td>binary</td><td></td></tr><tr><td>Computationally efficient</td><td>binary</td><td></td></tr><tr><td>Modular code</td><td>binary</td><td>Eg split into functions</td></tr><tr><td>Well-annotated code</td><td>binary</td><td></td></tr></tbody></table>

653